[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:19:59.910135 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:35583
I20260709 12:19:59.911777 19603 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:19:59.912807 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:19:59.930581 19612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:19:59.930615 19610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:19:59.931819 19603 server_base.cc:1061] running on GCE node
W20260709 12:19:59.930622 19609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:01.151854 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:01.152098 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:01.152362 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599601152330 us; error 0 us; skew 500 ppm
I20260709 12:20:01.153216 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:01.158751 19603 webserver.cc:533] Webserver started at http://127.19.36.254:44645/ using document root <none> and password file <none>
I20260709 12:20:01.159731 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:01.159969 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:01.160434 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:01.166391 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "84d2e33a04034ab3840205a6bd1951fd"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.174642 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:20:01.181232 19618 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.183029 19603 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:20:01.183539 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "84d2e33a04034ab3840205a6bd1951fd"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.183929 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:01.241722 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:01.244032 19603 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:20:01.244724 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:01.319427 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:35583
I20260709 12:20:01.319497 19669 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:35583 every 8 connection(s)
I20260709 12:20:01.327705 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:01.355176 19670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd: Bootstrap starting.
I20260709 12:20:01.363445 19670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:01.365811 19670 log.cc:826] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd: Log is configured to *not* fsync() on all Append() calls
I20260709 12:20:01.372049 19670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd: No bootstrap required, opened a new log
I20260709 12:20:01.393944 19670 raft_consensus.cc:359] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d2e33a04034ab3840205a6bd1951fd" member_type: VOTER }
I20260709 12:20:01.394615 19670 raft_consensus.cc:385] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:01.394896 19670 raft_consensus.cc:740] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d2e33a04034ab3840205a6bd1951fd, State: Initialized, Role: FOLLOWER
I20260709 12:20:01.395978 19670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d2e33a04034ab3840205a6bd1951fd" member_type: VOTER }
I20260709 12:20:01.396520 19670 raft_consensus.cc:399] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:01.396839 19670 raft_consensus.cc:493] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:01.397194 19670 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:01.401654 19670 raft_consensus.cc:515] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d2e33a04034ab3840205a6bd1951fd" member_type: VOTER }
I20260709 12:20:01.402444 19670 leader_election.cc:304] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84d2e33a04034ab3840205a6bd1951fd; no voters: 
I20260709 12:20:01.404377 19670 leader_election.cc:290] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:01.404770 19673 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:01.407157 19673 raft_consensus.cc:697] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 1 LEADER]: Becoming Leader. State: Replica: 84d2e33a04034ab3840205a6bd1951fd, State: Running, Role: LEADER
I20260709 12:20:01.408250 19673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d2e33a04034ab3840205a6bd1951fd" member_type: VOTER }
I20260709 12:20:01.409374 19670 sys_catalog.cc:565] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:01.419399 19675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84d2e33a04034ab3840205a6bd1951fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d2e33a04034ab3840205a6bd1951fd" member_type: VOTER } }
I20260709 12:20:01.420262 19675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:01.420571 19674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84d2e33a04034ab3840205a6bd1951fd. Latest consensus state: current_term: 1 leader_uuid: "84d2e33a04034ab3840205a6bd1951fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d2e33a04034ab3840205a6bd1951fd" member_type: VOTER } }
I20260709 12:20:01.421233 19674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:01.424079 19682 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:01.437803 19682 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:01.440227 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:01.454636 19682 catalog_manager.cc:1383] Generated new cluster ID: f35b495500ea49ae82a936603dde4560
I20260709 12:20:01.454962 19682 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:01.471299 19682 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:01.472873 19682 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:01.488907 19682 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd: Generated new TSK 0
I20260709 12:20:01.489945 19682 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:01.507158 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:01.515955 19692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:01.517531 19693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:01.521158 19695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:01.521359 19603 server_base.cc:1061] running on GCE node
I20260709 12:20:01.522275 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:01.522535 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:01.522810 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599601522790 us; error 0 us; skew 500 ppm
I20260709 12:20:01.523494 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:01.528107 19603 webserver.cc:533] Webserver started at http://127.19.36.193:38819/ using document root <none> and password file <none>
I20260709 12:20:01.528663 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:01.528867 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:01.529182 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:01.530382 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "cf0cb5009aaf467189738c9b165a2614"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.536513 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.005s
I20260709 12:20:01.541208 19700 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.542117 19603 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:01.542433 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "cf0cb5009aaf467189738c9b165a2614"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.542742 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:01.567592 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:01.568984 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:01.571822 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:01.576563 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:01.576781 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.577117 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:01.577283 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.687108 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:33493
I20260709 12:20:01.687175 19762 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:33493 every 8 connection(s)
I20260709 12:20:01.694658 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:01.706216 19767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:01.709604 19768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:01.718634 19770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:01.720013 19603 server_base.cc:1061] running on GCE node
I20260709 12:20:01.721146 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:01.721494 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:01.721719 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599601721702 us; error 0 us; skew 500 ppm
I20260709 12:20:01.722524 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:01.725400 19603 webserver.cc:533] Webserver started at http://127.19.36.194:34907/ using document root <none> and password file <none>
I20260709 12:20:01.725924 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:01.726130 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:01.726466 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:01.727880 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.734222 19763 heartbeater.cc:344] Connected to a master server at 127.19.36.254:35583
I20260709 12:20:01.734539 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:01.734705 19763 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:01.735826 19763 heartbeater.cc:507] Master 127.19.36.254:35583 requested a full tablet report, sending...
I20260709 12:20:01.739254 19635 ts_manager.cc:194] Registered new tserver with Master: cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493)
I20260709 12:20:01.740073 19775 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.741061 19603 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:01.741415 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.741770 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:01.742203 19635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33470
I20260709 12:20:01.783339 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:01.784698 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:01.787343 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:01.790831 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:01.791095 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.791460 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:01.791603 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.896904 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:44731
I20260709 12:20:01.897015 19837 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:44731 every 8 connection(s)
I20260709 12:20:01.903517 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:01.915102 19842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:01.916136 19841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:01.917758 19838 heartbeater.cc:344] Connected to a master server at 127.19.36.254:35583
I20260709 12:20:01.918112 19838 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:01.919219 19838 heartbeater.cc:507] Master 127.19.36.254:35583 requested a full tablet report, sending...
I20260709 12:20:01.922087 19635 ts_manager.cc:194] Registered new tserver with Master: 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:01.924849 19635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33472
I20260709 12:20:01.927947 19603 server_base.cc:1061] running on GCE node
W20260709 12:20:01.930727 19844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:01.931990 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:01.932366 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:01.932601 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599601932583 us; error 0 us; skew 500 ppm
I20260709 12:20:01.933429 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:01.936702 19603 webserver.cc:533] Webserver started at http://127.19.36.195:41329/ using document root <none> and password file <none>
I20260709 12:20:01.937251 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:01.937530 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:01.937832 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:01.939294 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "525fe26e6a9c49e49104ed9b608d87f1"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.944590 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:20:01.948726 19849 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.949689 19603 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:20:01.949987 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "525fe26e6a9c49e49104ed9b608d87f1"
format_stamp: "Formatted at 2026-07-09 12:20:01 on dist-test-slave-hfcn"
I20260709 12:20:01.950264 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:01.966156 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:01.967469 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:01.969103 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:01.972558 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:01.972797 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:01.973040 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:01.973224 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:02.068579 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:36391
I20260709 12:20:02.068682 19911 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:36391 every 8 connection(s)
I20260709 12:20:02.086283 19912 heartbeater.cc:344] Connected to a master server at 127.19.36.254:35583
I20260709 12:20:02.086663 19912 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:02.087582 19912 heartbeater.cc:507] Master 127.19.36.254:35583 requested a full tablet report, sending...
I20260709 12:20:02.089581 19635 ts_manager.cc:194] Registered new tserver with Master: 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:02.090430 19603 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016664493s
I20260709 12:20:02.091156 19635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33478
I20260709 12:20:02.151597 19635 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33490:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:02.156818 19635 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:02.280259 19803 tablet_service.cc:1511] Processing CreateTablet for tablet b18bb7b2fe304ef2832dd06a6050977d (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:20:02.281986 19802 tablet_service.cc:1511] Processing CreateTablet for tablet b2d7377c8cf6469f8fe9812884317e77 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:20:02.284466 19801 tablet_service.cc:1511] Processing CreateTablet for tablet 69272a8bf7fc4d16bde56e636fb60a33 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:20:02.281976 19803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18bb7b2fe304ef2832dd06a6050977d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.291898 19799 tablet_service.cc:1511] Processing CreateTablet for tablet 4dcf46a9de1c4cf599d7ea55da75bde5 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:20:02.290426 19800 tablet_service.cc:1511] Processing CreateTablet for tablet 2870cd5caec145f9a32ffec85190aa39 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:20:02.296358 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69272a8bf7fc4d16bde56e636fb60a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.299355 19877 tablet_service.cc:1511] Processing CreateTablet for tablet b18bb7b2fe304ef2832dd06a6050977d (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:20:02.296505 19876 tablet_service.cc:1511] Processing CreateTablet for tablet b2d7377c8cf6469f8fe9812884317e77 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:20:02.302578 19877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18bb7b2fe304ef2832dd06a6050977d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.295785 19875 tablet_service.cc:1511] Processing CreateTablet for tablet 69272a8bf7fc4d16bde56e636fb60a33 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:20:02.305348 19800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2870cd5caec145f9a32ffec85190aa39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.309546 19799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dcf46a9de1c4cf599d7ea55da75bde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.308638 19798 tablet_service.cc:1511] Processing CreateTablet for tablet 08bec98f13af417c8b66f3248fadf913 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:20:02.316666 19797 tablet_service.cc:1511] Processing CreateTablet for tablet 309a1998660e4f4b8fd5feefc322b9ed (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:20:02.316754 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d7377c8cf6469f8fe9812884317e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.308585 19872 tablet_service.cc:1511] Processing CreateTablet for tablet 08bec98f13af417c8b66f3248fadf913 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:20:02.321739 19876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d7377c8cf6469f8fe9812884317e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.321918 19874 tablet_service.cc:1511] Processing CreateTablet for tablet 2870cd5caec145f9a32ffec85190aa39 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:20:02.319231 19873 tablet_service.cc:1511] Processing CreateTablet for tablet 4dcf46a9de1c4cf599d7ea55da75bde5 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:20:02.323319 19797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309a1998660e4f4b8fd5feefc322b9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.323794 19871 tablet_service.cc:1511] Processing CreateTablet for tablet 309a1998660e4f4b8fd5feefc322b9ed (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:20:02.329622 19796 tablet_service.cc:1511] Processing CreateTablet for tablet 5835dcc7ad0d414393d6d71ebbe9836a (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:20:02.338305 19798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bec98f13af417c8b66f3248fadf913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.342913 19796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5835dcc7ad0d414393d6d71ebbe9836a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.338833 19875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69272a8bf7fc4d16bde56e636fb60a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.357158 19871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309a1998660e4f4b8fd5feefc322b9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.359311 19873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dcf46a9de1c4cf599d7ea55da75bde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.360970 19872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bec98f13af417c8b66f3248fadf913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.362493 19874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2870cd5caec145f9a32ffec85190aa39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.383097 19870 tablet_service.cc:1511] Processing CreateTablet for tablet 5835dcc7ad0d414393d6d71ebbe9836a (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:20:02.384661 19870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5835dcc7ad0d414393d6d71ebbe9836a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.416159 19726 tablet_service.cc:1511] Processing CreateTablet for tablet 69272a8bf7fc4d16bde56e636fb60a33 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:20:02.417516 19726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69272a8bf7fc4d16bde56e636fb60a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.407407 19727 tablet_service.cc:1511] Processing CreateTablet for tablet b2d7377c8cf6469f8fe9812884317e77 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:20:02.419101 19725 tablet_service.cc:1511] Processing CreateTablet for tablet 2870cd5caec145f9a32ffec85190aa39 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:20:02.411568 19728 tablet_service.cc:1511] Processing CreateTablet for tablet b18bb7b2fe304ef2832dd06a6050977d (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:20:02.431028 19725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2870cd5caec145f9a32ffec85190aa39. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:02.438135 19683 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:02.441569 19727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d7377c8cf6469f8fe9812884317e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.441486 19724 tablet_service.cc:1511] Processing CreateTablet for tablet 4dcf46a9de1c4cf599d7ea55da75bde5 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:20:02.444603 19723 tablet_service.cc:1511] Processing CreateTablet for tablet 08bec98f13af417c8b66f3248fadf913 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:20:02.454836 19728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18bb7b2fe304ef2832dd06a6050977d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.457864 19721 tablet_service.cc:1511] Processing CreateTablet for tablet 5835dcc7ad0d414393d6d71ebbe9836a (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:20:02.461663 19722 tablet_service.cc:1511] Processing CreateTablet for tablet 309a1998660e4f4b8fd5feefc322b9ed (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:20:02.472748 19720 tablet_service.cc:1511] Processing CreateTablet for tablet 219f05d76de64095a5416ee10bb3a319 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:20:02.478396 19930 tablet_bootstrap.cc:492] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:02.483899 19723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08bec98f13af417c8b66f3248fadf913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.496858 19870 tablet_service.cc:1511] Processing CreateTablet for tablet 219f05d76de64095a5416ee10bb3a319 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:20:02.498128 19870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219f05d76de64095a5416ee10bb3a319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.477025 19795 tablet_service.cc:1511] Processing CreateTablet for tablet 219f05d76de64095a5416ee10bb3a319 (DEFAULT_TABLE table=test-workload [id=a76e4fd2303041bc933b50c3d030147b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:20:02.500301 19720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219f05d76de64095a5416ee10bb3a319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.503628 19721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5835dcc7ad0d414393d6d71ebbe9836a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.507481 19724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dcf46a9de1c4cf599d7ea55da75bde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.515741 19931 tablet_bootstrap.cc:492] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:02.516407 19795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219f05d76de64095a5416ee10bb3a319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.526484 19722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309a1998660e4f4b8fd5feefc322b9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:02.553049 19930 tablet_bootstrap.cc:654] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.572585 19932 tablet_bootstrap.cc:492] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:02.616612 19932 tablet_bootstrap.cc:654] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.631023 19931 tablet_bootstrap.cc:654] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.632385 19930 tablet_bootstrap.cc:492] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:02.632854 19930 ts_tablet_manager.cc:1403] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.183s	user 0.018s	sys 0.018s
I20260709 12:20:02.643850 19932 tablet_bootstrap.cc:492] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:02.644412 19932 ts_tablet_manager.cc:1403] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.089s	user 0.011s	sys 0.011s
I20260709 12:20:02.644716 19930 raft_consensus.cc:359] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.645555 19930 raft_consensus.cc:385] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.645869 19930 raft_consensus.cc:740] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.646647 19930 consensus_queue.cc:260] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.648361 19932 raft_consensus.cc:359] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.649312 19932 raft_consensus.cc:385] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.649721 19932 raft_consensus.cc:740] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.650882 19932 consensus_queue.cc:260] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.656494 19763 heartbeater.cc:499] Master 127.19.36.254:35583 was elected leader, sending a full tablet report...
I20260709 12:20:02.657984 19932 ts_tablet_manager.cc:1434] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:20:02.660151 19930 ts_tablet_manager.cc:1434] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.027s	user 0.015s	sys 0.004s
I20260709 12:20:02.660451 19912 heartbeater.cc:499] Master 127.19.36.254:35583 was elected leader, sending a full tablet report...
I20260709 12:20:02.661088 19930 tablet_bootstrap.cc:492] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:02.660149 19932 tablet_bootstrap.cc:492] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:02.668030 19930 tablet_bootstrap.cc:654] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.672869 19932 tablet_bootstrap.cc:654] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.675971 19931 tablet_bootstrap.cc:492] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:02.676556 19931 ts_tablet_manager.cc:1403] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.162s	user 0.012s	sys 0.012s
I20260709 12:20:02.681284 19930 tablet_bootstrap.cc:492] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:02.681942 19930 ts_tablet_manager.cc:1403] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 12:20:02.685415 19930 raft_consensus.cc:359] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.686151 19932 tablet_bootstrap.cc:492] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:02.686251 19930 raft_consensus.cc:385] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.686683 19932 ts_tablet_manager.cc:1403] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.027s	user 0.001s	sys 0.018s
I20260709 12:20:02.686638 19930 raft_consensus.cc:740] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.688395 19930 consensus_queue.cc:260] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.690584 19930 ts_tablet_manager.cc:1434] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:20:02.690639 19932 raft_consensus.cc:359] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.691457 19932 raft_consensus.cc:385] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.691692 19930 tablet_bootstrap.cc:492] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:02.691784 19932 raft_consensus.cc:740] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.692611 19932 consensus_queue.cc:260] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.693851 19931 raft_consensus.cc:359] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
W20260709 12:20:02.695586 19764 tablet.cc:2406] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:20:02.695762 19931 raft_consensus.cc:385] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.696110 19931 raft_consensus.cc:740] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.695070 19932 ts_tablet_manager.cc:1434] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:02.696857 19931 consensus_queue.cc:260] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.698170 19932 tablet_bootstrap.cc:492] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:02.699334 19930 tablet_bootstrap.cc:654] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.706034 19932 tablet_bootstrap.cc:654] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.714582 19930 tablet_bootstrap.cc:492] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:02.715180 19930 ts_tablet_manager.cc:1403] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.008s
I20260709 12:20:02.718459 19930 raft_consensus.cc:359] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.719003 19932 tablet_bootstrap.cc:492] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:02.719201 19930 raft_consensus.cc:385] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.719553 19930 raft_consensus.cc:740] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.720129 19932 ts_tablet_manager.cc:1403] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 12:20:02.720774 19838 heartbeater.cc:499] Master 127.19.36.254:35583 was elected leader, sending a full tablet report...
I20260709 12:20:02.720537 19930 consensus_queue.cc:260] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.721755 19931 ts_tablet_manager.cc:1434] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:20:02.722784 19931 tablet_bootstrap.cc:492] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:02.727475 19932 raft_consensus.cc:359] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.728317 19932 raft_consensus.cc:385] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.728680 19932 raft_consensus.cc:740] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.724020 19930 ts_tablet_manager.cc:1434] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:02.730067 19930 tablet_bootstrap.cc:492] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:02.729617 19932 consensus_queue.cc:260] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.732172 19932 ts_tablet_manager.cc:1434] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:20:02.733186 19932 tablet_bootstrap.cc:492] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:02.737228 19931 tablet_bootstrap.cc:654] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.739882 19932 tablet_bootstrap.cc:654] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.737393 19930 tablet_bootstrap.cc:654] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.756085 19930 tablet_bootstrap.cc:492] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:02.756695 19930 ts_tablet_manager.cc:1403] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.008s
I20260709 12:20:02.767225 19932 tablet_bootstrap.cc:492] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:02.767802 19932 ts_tablet_manager.cc:1403] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.024s
I20260709 12:20:02.768564 19930 raft_consensus.cc:359] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.769310 19930 raft_consensus.cc:385] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.769613 19930 raft_consensus.cc:740] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.770455 19930 consensus_queue.cc:260] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.770855 19932 raft_consensus.cc:359] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:02.771910 19932 raft_consensus.cc:385] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.772264 19932 raft_consensus.cc:740] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.772806 19930 ts_tablet_manager.cc:1434] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 12:20:02.773104 19932 consensus_queue.cc:260] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:02.773772 19930 tablet_bootstrap.cc:492] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:02.775621 19932 ts_tablet_manager.cc:1434] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:02.776649 19932 tablet_bootstrap.cc:492] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:02.779004 19931 tablet_bootstrap.cc:492] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:02.779488 19931 ts_tablet_manager.cc:1403] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.008s
I20260709 12:20:02.782724 19931 raft_consensus.cc:359] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.783507 19931 raft_consensus.cc:385] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.783571 19932 tablet_bootstrap.cc:654] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.783874 19931 raft_consensus.cc:740] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.784755 19931 consensus_queue.cc:260] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.787040 19931 ts_tablet_manager.cc:1434] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:20:02.788281 19931 tablet_bootstrap.cc:492] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:02.795254 19932 tablet_bootstrap.cc:492] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:02.795809 19932 ts_tablet_manager.cc:1403] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.003s
I20260709 12:20:02.798779 19932 raft_consensus.cc:359] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.799676 19932 raft_consensus.cc:385] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.800043 19932 raft_consensus.cc:740] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.800438 19931 tablet_bootstrap.cc:654] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.801019 19932 consensus_queue.cc:260] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.805281 19932 ts_tablet_manager.cc:1434] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:20:02.806193 19932 tablet_bootstrap.cc:492] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:02.813719 19930 tablet_bootstrap.cc:654] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.815052 19932 tablet_bootstrap.cc:654] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.819382 19931 tablet_bootstrap.cc:492] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:02.819865 19931 ts_tablet_manager.cc:1403] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 12:20:02.821511 19930 tablet_bootstrap.cc:492] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:02.822027 19930 ts_tablet_manager.cc:1403] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.004s
I20260709 12:20:02.822995 19931 raft_consensus.cc:359] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.823757 19931 raft_consensus.cc:385] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.824052 19931 raft_consensus.cc:740] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.824784 19931 consensus_queue.cc:260] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.827350 19931 ts_tablet_manager.cc:1434] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:02.825263 19930 raft_consensus.cc:359] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:02.828295 19930 raft_consensus.cc:385] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.828645 19930 raft_consensus.cc:740] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.829403 19930 consensus_queue.cc:260] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:02.831877 19937 raft_consensus.cc:493] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.832692 19932 tablet_bootstrap.cc:492] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:02.832331 19937 raft_consensus.cc:515] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.833150 19932 ts_tablet_manager.cc:1403] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.005s
I20260709 12:20:02.835361 19937 leader_election.cc:290] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:02.835901 19932 raft_consensus.cc:359] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.836725 19932 raft_consensus.cc:385] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.837030 19932 raft_consensus.cc:740] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.837754 19932 consensus_queue.cc:260] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.838248 19930 ts_tablet_manager.cc:1434] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:20:02.839457 19930 tablet_bootstrap.cc:492] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:02.846252 19930 tablet_bootstrap.cc:654] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.852711 19932 ts_tablet_manager.cc:1434] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:20:02.853935 19932 tablet_bootstrap.cc:492] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:02.863116 19932 tablet_bootstrap.cc:654] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.870637 19931 tablet_bootstrap.cc:492] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:02.871520 19930 tablet_bootstrap.cc:492] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:02.872082 19930 ts_tablet_manager.cc:1403] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.010s
I20260709 12:20:02.875216 19930 raft_consensus.cc:359] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.876096 19930 raft_consensus.cc:385] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.876449 19930 raft_consensus.cc:740] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.877274 19930 consensus_queue.cc:260] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.877825 19931 tablet_bootstrap.cc:654] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.882683 19930 ts_tablet_manager.cc:1434] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 12:20:02.882611 19813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:02.883944 19930 tablet_bootstrap.cc:492] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:02.884856 19936 raft_consensus.cc:493] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.885458 19936 raft_consensus.cc:515] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
W20260709 12:20:02.888293 19704 leader_election.cc:343] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:02.890841 19936 leader_election.cc:290] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:02.891363 19930 tablet_bootstrap.cc:654] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.899358 19936 raft_consensus.cc:493] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.900573 19936 raft_consensus.cc:515] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.911444 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:02.911494 19936 leader_election.cc:290] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
W20260709 12:20:02.915336 19702 leader_election.cc:343] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:02.916306 19702 leader_election.cc:304] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf0cb5009aaf467189738c9b165a2614; no voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1
I20260709 12:20:02.917568 19937 raft_consensus.cc:2749] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:02.918553 19931 tablet_bootstrap.cc:492] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:02.919060 19931 ts_tablet_manager.cc:1403] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.009s
I20260709 12:20:02.919289 19932 tablet_bootstrap.cc:492] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:02.919723 19932 ts_tablet_manager.cc:1403] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.066s	user 0.022s	sys 0.022s
I20260709 12:20:02.922117 19931 raft_consensus.cc:359] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.922895 19931 raft_consensus.cc:385] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.923242 19931 raft_consensus.cc:740] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.926414 19940 raft_consensus.cc:493] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.924494 19931 consensus_queue.cc:260] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.926903 19940 raft_consensus.cc:515] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.927198 19932 raft_consensus.cc:359] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.928005 19932 raft_consensus.cc:385] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.928373 19932 raft_consensus.cc:740] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.929734 19940 leader_election.cc:290] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:02.932482 19738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:02.934031 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:02.934759 19737 raft_consensus.cc:2468] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 0.
I20260709 12:20:02.936261 19936 raft_consensus.cc:493] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.936836 19936 raft_consensus.cc:515] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
W20260709 12:20:02.937678 19850 leader_election.cc:343] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:02.938416 19850 leader_election.cc:304] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:02.939852 19936 leader_election.cc:290] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:02.940477 19936 raft_consensus.cc:2804] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:02.940908 19936 raft_consensus.cc:493] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.943130 19936 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:02.948408 19813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:02.949214 19813 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 0.
I20260709 12:20:02.949344 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:02.949952 19737 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 0.
I20260709 12:20:02.951355 19850 leader_election.cc:304] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:02.952370 19963 raft_consensus.cc:2804] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:02.952502 19853 leader_election.cc:304] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:02.953009 19812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:02.955843 19936 raft_consensus.cc:515] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.954048 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:02.957085 19811 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 0.
I20260709 12:20:02.957679 19930 tablet_bootstrap.cc:492] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:02.957877 19936 leader_election.cc:290] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:02.958161 19930 ts_tablet_manager.cc:1403] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.008s
I20260709 12:20:02.958391 19936 raft_consensus.cc:2804] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:02.958752 19936 raft_consensus.cc:493] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.959273 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:02.959837 19737 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:02.961817 19930 raft_consensus.cc:359] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.929201 19932 consensus_queue.cc:260] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.966609 19932 ts_tablet_manager.cc:1434] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 12:20:02.968004 19932 tablet_bootstrap.cc:492] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
W20260709 12:20:02.967943 19853 leader_election.cc:343] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:02.966746 19737 raft_consensus.cc:2468] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 1.
I20260709 12:20:02.959079 19936 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:02.970963 19850 leader_election.cc:304] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:02.971999 19963 raft_consensus.cc:493] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.972149 19964 raft_consensus.cc:2804] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:02.975018 19963 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:02.962831 19930 raft_consensus.cc:385] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:02.979408 19930 raft_consensus.cc:740] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:02.981783 19932 tablet_bootstrap.cc:654] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:02.979645 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:02.989126 19931 ts_tablet_manager.cc:1434] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.070s	user 0.009s	sys 0.015s
I20260709 12:20:02.992121 19931 tablet_bootstrap.cc:492] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
W20260709 12:20:02.993132 19853 leader_election.cc:343] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:02.993367 19937 raft_consensus.cc:493] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.993820 19937 raft_consensus.cc:515] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.995955 19937 leader_election.cc:290] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:02.990271 19930 consensus_queue.cc:260] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:02.996716 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:02.997090 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:02.997352 19811 raft_consensus.cc:2468] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 0.
I20260709 12:20:02.997655 19887 raft_consensus.cc:2468] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 0.
I20260709 12:20:02.998672 19704 leader_election.cc:304] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:02.975153 19965 raft_consensus.cc:493] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.000375 19965 raft_consensus.cc:515] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.001370 19937 raft_consensus.cc:2804] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:03.002141 19937 raft_consensus.cc:493] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.002611 19937 raft_consensus.cc:3060] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.003595 19936 raft_consensus.cc:515] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.005748 19936 leader_election.cc:290] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.006589 19738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.009763 19812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:03.007400 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:03.010421 19737 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 0.
W20260709 12:20:03.011089 19853 leader_election.cc:343] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:03.011497 19850 leader_election.cc:343] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:03.012586 19850 leader_election.cc:304] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:03.007637 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:03.013407 19936 raft_consensus.cc:2804] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:03.013810 19936 raft_consensus.cc:493] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.014142 19936 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.014338 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:02.988308 19963 raft_consensus.cc:515] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.013420 19964 raft_consensus.cc:697] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.015336 19887 raft_consensus.cc:2393] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39b94cc5da3d45e8b7d837714a0a39e3 in current term 1: Already voted for candidate 525fe26e6a9c49e49104ed9b608d87f1 in this term.
I20260709 12:20:03.015893 19964 consensus_queue.cc:237] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.001585 19931 tablet_bootstrap.cc:654] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:03.014659 19776 leader_election.cc:343] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:03.019999 19812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.020618 19812 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.020980 19936 raft_consensus.cc:515] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.022192 19777 leader_election.cc:304] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3; no voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.016836 19963 leader_election.cc:290] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.008206 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.008330 19965 leader_election.cc:290] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.017711 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.018282 19937 raft_consensus.cc:515] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.025086 19736 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.024736 19813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.034085 19932 tablet_bootstrap.cc:492] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:03.034504 19932 ts_tablet_manager.cc:1403] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.067s	user 0.010s	sys 0.003s
I20260709 12:20:03.033049 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.035342 19810 raft_consensus.cc:3060] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.037269 19932 raft_consensus.cc:359] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.037920 19932 raft_consensus.cc:385] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.038187 19932 raft_consensus.cc:740] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.032121 19937 leader_election.cc:290] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.028815 19812 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 1.
I20260709 12:20:03.033180 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.040145 19887 raft_consensus.cc:3060] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:03.040352 19853 leader_election.cc:343] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:03.024010 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.041476 19737 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.042410 19736 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 1.
I20260709 12:20:03.023972 19960 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.043576 19850 leader_election.cc:304] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.038869 19932 consensus_queue.cc:260] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.045981 19932 ts_tablet_manager.cc:1434] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:20:03.047055 19932 tablet_bootstrap.cc:492] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614: Bootstrap starting.
I20260709 12:20:03.057305 19810 raft_consensus.cc:2468] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 1.
I20260709 12:20:03.049139 19737 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 1.
I20260709 12:20:03.023165 19936 leader_election.cc:290] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.058450 19704 leader_election.cc:304] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.032749 19930 ts_tablet_manager.cc:1434] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.074s	user 0.011s	sys 0.011s
I20260709 12:20:03.061443 19930 tablet_bootstrap.cc:492] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:03.062738 19850 leader_election.cc:304] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:03.063064 19932 tablet_bootstrap.cc:654] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:03.063685 19967 raft_consensus.cc:2804] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.050591 19960 raft_consensus.cc:2749] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:03.064266 19967 raft_consensus.cc:697] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.054641 19937 raft_consensus.cc:493] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.049543 19634 catalog_manager.cc:5697] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:03.054188 19887 raft_consensus.cc:2468] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 1.
I20260709 12:20:03.049913 19964 raft_consensus.cc:2804] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.053247 19931 tablet_bootstrap.cc:492] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:03.065269 19937 raft_consensus.cc:515] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.066784 19964 raft_consensus.cc:697] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.069031 19931 ts_tablet_manager.cc:1403] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.010s
I20260709 12:20:03.069263 19937 leader_election.cc:290] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.070016 19937 raft_consensus.cc:2804] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.066020 19967 consensus_queue.cc:237] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.071597 19937 raft_consensus.cc:697] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.071270 19964 consensus_queue.cc:237] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.074693 19937 consensus_queue.cc:237] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.077420 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:03.083261 19887 raft_consensus.cc:2468] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 0.
I20260709 12:20:03.091293 19973 raft_consensus.cc:493] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.091702 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:03.099943 19702 leader_election.cc:304] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.102082 19971 raft_consensus.cc:2804] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:03.102644 19971 raft_consensus.cc:493] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.103070 19971 raft_consensus.cc:3060] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.098105 19973 raft_consensus.cc:515] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.116128 19812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:03.120527 19811 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 1.
I20260709 12:20:03.122164 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:03.122960 19887 raft_consensus.cc:2468] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 0.
I20260709 12:20:03.124894 19702 leader_election.cc:304] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
W20260709 12:20:03.130445 19704 leader_election.cc:343] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:03.139483 19930 tablet_bootstrap.cc:654] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:03.120641 19973 leader_election.cc:290] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.144374 19973 raft_consensus.cc:2804] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:03.144831 19973 raft_consensus.cc:493] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.145257 19973 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.139472 19931 raft_consensus.cc:359] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.153643 19971 raft_consensus.cc:515] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.156252 19932 tablet_bootstrap.cc:492] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614: No bootstrap required, opened a new log
I20260709 12:20:03.157354 19932 ts_tablet_manager.cc:1403] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.008s
I20260709 12:20:03.158869 19971 leader_election.cc:290] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.159703 19812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.151458 19931 raft_consensus.cc:385] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.160477 19931 raft_consensus.cc:740] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.161531 19931 consensus_queue.cc:260] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.164258 19812 raft_consensus.cc:3060] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.164884 19932 raft_consensus.cc:359] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.165635 19932 raft_consensus.cc:385] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.165961 19932 raft_consensus.cc:740] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Initialized, Role: FOLLOWER
W20260709 12:20:03.169843 19702 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:36391, user_credentials={real_user=slave}} blocked reactor thread for 45053.6us
I20260709 12:20:03.171548 19930 tablet_bootstrap.cc:492] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:03.171977 19930 ts_tablet_manager.cc:1403] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.111s	user 0.014s	sys 0.008s
I20260709 12:20:03.173071 19932 consensus_queue.cc:260] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.167932 19973 raft_consensus.cc:515] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.174098 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.174676 19887 raft_consensus.cc:3060] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.176163 19973 leader_election.cc:290] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.176023 19930 raft_consensus.cc:359] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.176716 19930 raft_consensus.cc:385] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.177017 19930 raft_consensus.cc:740] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.177659 19812 raft_consensus.cc:2468] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 1.
I20260709 12:20:03.177683 19930 consensus_queue.cc:260] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.180332 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.183247 19930 ts_tablet_manager.cc:1434] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:20:03.184654 19930 tablet_bootstrap.cc:492] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1: Bootstrap starting.
I20260709 12:20:03.182235 19886 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.186692 19813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.187435 19960 raft_consensus.cc:493] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.188431 19932 ts_tablet_manager.cc:1434] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 12:20:03.189185 19931 ts_tablet_manager.cc:1434] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.120s	user 0.005s	sys 0.006s
I20260709 12:20:03.189131 19960 raft_consensus.cc:515] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
W20260709 12:20:03.189908 19704 leader_election.cc:343] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:03.191355 19931 tablet_bootstrap.cc:492] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:03.194531 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309a1998660e4f4b8fd5feefc322b9ed" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:03.193714 19960 leader_election.cc:290] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.195286 19737 raft_consensus.cc:2468] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 0.
I20260709 12:20:03.196861 19930 tablet_bootstrap.cc:654] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:03.196771 19776 leader_election.cc:304] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.195158 19885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309a1998660e4f4b8fd5feefc322b9ed" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:03.197599 19960 raft_consensus.cc:2804] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:03.197887 19885 raft_consensus.cc:2468] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 0.
I20260709 12:20:03.198036 19960 raft_consensus.cc:493] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.197135 19887 raft_consensus.cc:2468] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 1.
I20260709 12:20:03.198467 19886 raft_consensus.cc:2468] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 1.
I20260709 12:20:03.203954 19960 raft_consensus.cc:3060] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.205062 19702 leader_election.cc:304] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:03.207427 19973 raft_consensus.cc:2804] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.208195 19973 raft_consensus.cc:697] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.208398 19930 tablet_bootstrap.cc:492] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1: No bootstrap required, opened a new log
I20260709 12:20:03.208866 19930 ts_tablet_manager.cc:1403] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.002s
I20260709 12:20:03.209066 19973 consensus_queue.cc:237] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.209926 19931 tablet_bootstrap.cc:654] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:03.211730 19702 leader_election.cc:304] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.211660 19930 raft_consensus.cc:359] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.212374 19930 raft_consensus.cc:385] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.212517 19937 raft_consensus.cc:2804] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.212678 19930 raft_consensus.cc:740] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.213156 19937 raft_consensus.cc:697] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.213551 19930 consensus_queue.cc:260] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.214442 19937 consensus_queue.cc:237] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.216150 19930 ts_tablet_manager.cc:1434] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:03.222910 19853 leader_election.cc:304] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.224052 19967 raft_consensus.cc:2804] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.224095 19960 raft_consensus.cc:515] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.224658 19967 raft_consensus.cc:697] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.225489 19967 consensus_queue.cc:237] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.226270 19960 leader_election.cc:290] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.229394 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309a1998660e4f4b8fd5feefc322b9ed" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.230087 19737 raft_consensus.cc:3060] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.235586 19810 raft_consensus.cc:2376] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 1. Re-sending same reply.
I20260709 12:20:03.237416 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "309a1998660e4f4b8fd5feefc322b9ed" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.238215 19887 raft_consensus.cc:3060] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.241142 19931 tablet_bootstrap.cc:492] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:03.241621 19931 ts_tablet_manager.cc:1403] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.051s	user 0.006s	sys 0.010s
I20260709 12:20:03.244655 19931 raft_consensus.cc:359] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.248071 19931 raft_consensus.cc:385] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.248389 19931 raft_consensus.cc:740] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.249186 19931 consensus_queue.cc:260] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.249455 19635 catalog_manager.cc:5697] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.251392 19931 ts_tablet_manager.cc:1434] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:20:03.252319 19931 tablet_bootstrap.cc:492] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:03.258903 19931 tablet_bootstrap.cc:654] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:03.261857 19887 raft_consensus.cc:2468] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 1.
I20260709 12:20:03.266144 19737 raft_consensus.cc:2468] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 1.
I20260709 12:20:03.267498 19777 leader_election.cc:304] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:03.268472 19960 raft_consensus.cc:2804] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.268999 19960 raft_consensus.cc:697] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:03.269805 19960 consensus_queue.cc:237] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.274055 19931 tablet_bootstrap.cc:492] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:03.274529 19931 ts_tablet_manager.cc:1403] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 12:20:03.277395 19931 raft_consensus.cc:359] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.278167 19931 raft_consensus.cc:385] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.278460 19931 raft_consensus.cc:740] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.279304 19931 consensus_queue.cc:260] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.281548 19931 ts_tablet_manager.cc:1434] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:20:03.282413 19931 tablet_bootstrap.cc:492] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:03.273880 19633 catalog_manager.cc:5697] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.286485 19633 catalog_manager.cc:5697] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:03.289312 19931 tablet_bootstrap.cc:654] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:03.289399 19633 catalog_manager.cc:5697] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:03.294293 19931 tablet_bootstrap.cc:492] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:03.294713 19931 ts_tablet_manager.cc:1403] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.013s	user 0.003s	sys 0.007s
I20260709 12:20:03.297188 19931 raft_consensus.cc:359] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.297797 19931 raft_consensus.cc:385] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.298043 19931 raft_consensus.cc:740] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.298648 19931 consensus_queue.cc:260] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.300565 19931 ts_tablet_manager.cc:1434] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:20:03.301455 19931 tablet_bootstrap.cc:492] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3: Bootstrap starting.
I20260709 12:20:03.304384 19632 catalog_manager.cc:5697] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.306350 19632 catalog_manager.cc:5697] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.307008 19931 tablet_bootstrap.cc:654] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:03.312492 19931 tablet_bootstrap.cc:492] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3: No bootstrap required, opened a new log
I20260709 12:20:03.312930 19931 ts_tablet_manager.cc:1403] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:20:03.315408 19931 raft_consensus.cc:359] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.316032 19931 raft_consensus.cc:385] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:03.316288 19931 raft_consensus.cc:740] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Initialized, Role: FOLLOWER
I20260709 12:20:03.317013 19931 consensus_queue.cc:260] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.318948 19931 ts_tablet_manager.cc:1434] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:03.326151 19634 catalog_manager.cc:5697] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.327862 19967 raft_consensus.cc:493] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.328308 19967 raft_consensus.cc:515] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.330217 19967 leader_election.cc:290] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.331061 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:03.331430 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:03.331724 19737 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 0.
I20260709 12:20:03.332037 19810 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 0.
I20260709 12:20:03.333160 19853 leader_election.cc:304] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:03.334582 19967 raft_consensus.cc:2804] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:03.334899 19967 raft_consensus.cc:493] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.335243 19967 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.340360 19967 raft_consensus.cc:515] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.342422 19967 leader_election.cc:290] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.343130 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.343736 19737 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.343897 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.344489 19810 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.349447 19737 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 1.
I20260709 12:20:03.350435 19850 leader_election.cc:304] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.350831 19810 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 1.
I20260709 12:20:03.351091 19967 raft_consensus.cc:2804] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:03.351863 19967 raft_consensus.cc:697] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.352684 19967 consensus_queue.cc:237] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.360071 19634 catalog_manager.cc:5697] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:03.377132 19973 raft_consensus.cc:493] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.377610 19973 raft_consensus.cc:515] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.379351 19973 leader_election.cc:290] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.380102 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" is_pre_election: true
I20260709 12:20:03.380503 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:03.380795 19810 raft_consensus.cc:2393] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf0cb5009aaf467189738c9b165a2614 in current term 1: Already voted for candidate 525fe26e6a9c49e49104ed9b608d87f1 in this term.
I20260709 12:20:03.381955 19702 leader_election.cc:304] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf0cb5009aaf467189738c9b165a2614; no voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1
I20260709 12:20:03.382592 19973 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.386922 19973 raft_consensus.cc:2749] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:03.482465 19887 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
mode: GRACEFUL
new_leader_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:49322
I20260709 12:20:03.483150 19887 raft_consensus.cc:606] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Received request to transfer leadership to TS cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.483095 19960 raft_consensus.cc:493] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.483774 19960 raft_consensus.cc:515] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.485816 19960 leader_election.cc:290] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391), cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493)
I20260709 12:20:03.486552 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:03.487263 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:03.487988 19737 raft_consensus.cc:2393] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39b94cc5da3d45e8b7d837714a0a39e3 in current term 1: Already voted for candidate 525fe26e6a9c49e49104ed9b608d87f1 in this term.
I20260709 12:20:03.488936 19776 leader_election.cc:304] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3; no voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.489471 19887 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
mode: GRACEFUL
new_leader_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:49322
I20260709 12:20:03.489692 19960 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.490058 19887 raft_consensus.cc:606] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Received request to transfer leadership to TS cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.495935 19960 raft_consensus.cc:2749] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:03.496075 19887 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
mode: GRACEFUL
new_leader_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:49322
I20260709 12:20:03.496770 19887 raft_consensus.cc:606] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Received request to transfer leadership to TS cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.503249 19887 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
mode: GRACEFUL
new_leader_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:49322
I20260709 12:20:03.503752 19887 raft_consensus.cc:606] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Received request to transfer leadership to TS cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.503608 19967 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.528201 19736 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:34342
I20260709 12:20:03.528820 19736 raft_consensus.cc:606] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:03.531350 19936 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.537679 19737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:34342
I20260709 12:20:03.538290 19737 raft_consensus.cc:606] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:03.548259 19936 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.549515 19887 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:49322
I20260709 12:20:03.550105 19887 raft_consensus.cc:606] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:03.555721 19738 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
mode: GRACEFUL
new_leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:34342
I20260709 12:20:03.556278 19738 raft_consensus.cc:606] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Received request to transfer leadership to TS 525fe26e6a9c49e49104ed9b608d87f1
I20260709 12:20:03.556713 19936 consensus_queue.cc:1048] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:03.559980 19812 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.572501 19936 consensus_queue.cc:1048] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:03.589365 19964 raft_consensus.cc:993] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: : Instructing follower cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.589855 19936 raft_consensus.cc:1081] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Signalling peer cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.591230 19967 raft_consensus.cc:993] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1: : Instructing follower cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.592592 19736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:34318
I20260709 12:20:03.593259 19736 raft_consensus.cc:493] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:03.593672 19736 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.595041 19960 raft_consensus.cc:493] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.595532 19960 raft_consensus.cc:515] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.597669 19960 leader_election.cc:290] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.598889 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1" is_pre_election: true
I20260709 12:20:03.599759 19887 raft_consensus.cc:2393] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39b94cc5da3d45e8b7d837714a0a39e3 in current term 1: Already voted for candidate cf0cb5009aaf467189738c9b165a2614 in this term.
I20260709 12:20:03.599951 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0cb5009aaf467189738c9b165a2614" is_pre_election: true
I20260709 12:20:03.601260 19736 raft_consensus.cc:515] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.602310 19776 leader_election.cc:304] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3; no voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:03.603516 19738 raft_consensus.cc:1240] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Rejecting Update request from peer 525fe26e6a9c49e49104ed9b608d87f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:03.609752 19936 raft_consensus.cc:1081] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Signalling peer cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.611027 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.612910 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.613330 19737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:34318
I20260709 12:20:03.613560 19810 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.614027 19737 raft_consensus.cc:493] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:03.614199 19960 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:03.614442 19737 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.620350 19810 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.621510 19736 leader_election.cc:290] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Requested vote from peers 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.623488 19704 leader_election.cc:304] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.624450 19973 raft_consensus.cc:2804] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:03.624944 19973 raft_consensus.cc:697] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.625733 19973 consensus_queue.cc:237] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.628803 19737 raft_consensus.cc:515] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.631832 19973 consensus_queue.cc:1048] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.634707 19634 catalog_manager.cc:5697] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 1 to 2, leader changed from 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195) to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.639720 19738 raft_consensus.cc:1240] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Rejecting Update request from peer 525fe26e6a9c49e49104ed9b608d87f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:03.639935 19960 raft_consensus.cc:2749] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:03.645561 20004 consensus_queue.cc:1059] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.646117 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.646771 19811 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.651440 19737 leader_election.cc:290] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.651960 19964 raft_consensus.cc:3055] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:03.652343 19964 raft_consensus.cc:740] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.653169 19964 consensus_queue.cc:260] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:03.654147 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.654711 19886 raft_consensus.cc:3055] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:03.654808 19964 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.655053 19886 raft_consensus.cc:740] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.655977 19886 consensus_queue.cc:260] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
W20260709 12:20:03.659643 19853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:44731, user_credentials={real_user=slave}} blocked reactor thread for 55227.8us
I20260709 12:20:03.661557 19810 raft_consensus.cc:1240] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Rejecting Update request from peer 525fe26e6a9c49e49104ed9b608d87f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:03.664724 19886 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.666600 19971 consensus_queue.cc:1048] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:03.668608 19979 consensus_queue.cc:1048] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:03.670372 19979 consensus_queue.cc:1048] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.679746 20003 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:03.685359 19811 raft_consensus.cc:2468] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.686685 19704 leader_election.cc:304] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.687649 19971 raft_consensus.cc:2804] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:03.688107 19971 raft_consensus.cc:697] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.688834 19979 consensus_queue.cc:1048] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.689232 19886 raft_consensus.cc:2468] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.690307 19971 consensus_queue.cc:237] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.700977 19633 catalog_manager.cc:5697] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 1 to 2, leader changed from 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195) to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.708658 19979 raft_consensus.cc:993] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:03.718087 20005 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.726403 19979 raft_consensus.cc:1081] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:03.730583 19971 raft_consensus.cc:993] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614: : Instructing follower 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:03.731207 19971 raft_consensus.cc:1081] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Signalling peer 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:03.733455 19812 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56434
I20260709 12:20:03.734102 19812 raft_consensus.cc:493] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:03.734854 19885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:49310
I20260709 12:20:03.735486 19885 raft_consensus.cc:493] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:03.736276 19885 raft_consensus.cc:3060] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.735882 19812 raft_consensus.cc:3060] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.754319 19812 raft_consensus.cc:515] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.756908 19812 leader_election.cc:290] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 2 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.751911 19885 raft_consensus.cc:515] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.760094 19885 leader_election.cc:290] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 2 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:03.761278 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.761407 19737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.761956 19810 raft_consensus.cc:3060] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.762526 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219f05d76de64095a5416ee10bb3a319" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.763096 19736 raft_consensus.cc:3055] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:03.763473 19736 raft_consensus.cc:740] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.764245 19736 consensus_queue.cc:260] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.765511 19736 raft_consensus.cc:3060] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.761977 19737 raft_consensus.cc:3055] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:03.767604 19737 raft_consensus.cc:740] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.768395 19737 consensus_queue.cc:260] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.769613 19737 raft_consensus.cc:3060] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.772543 19736 raft_consensus.cc:2468] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 2.
I20260709 12:20:03.773325 19885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dcf46a9de1c4cf599d7ea55da75bde5" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.774228 19936 raft_consensus.cc:993] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1: : Instructing follower cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.774529 19850 leader_election.cc:304] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.774684 19936 raft_consensus.cc:1081] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Signalling peer cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.775316 20005 raft_consensus.cc:2804] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:03.775777 20005 raft_consensus.cc:697] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.774565 19885 raft_consensus.cc:3060] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.776614 20005 consensus_queue.cc:237] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.779486 19738 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:34318
I20260709 12:20:03.780109 19738 raft_consensus.cc:493] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:03.780542 19738 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.781410 19885 raft_consensus.cc:2468] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 2.
I20260709 12:20:03.782174 19810 raft_consensus.cc:2468] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 2.
I20260709 12:20:03.782634 19777 leader_election.cc:304] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:03.783469 19960 raft_consensus.cc:2804] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:03.783948 19960 raft_consensus.cc:697] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:03.784754 19960 consensus_queue.cc:237] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.787010 20005 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:03.788407 19737 raft_consensus.cc:2468] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 2.
I20260709 12:20:03.793751 19887 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.793759 19738 raft_consensus.cc:515] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.797120 19812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.797812 19812 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.801342 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.801946 19887 raft_consensus.cc:3055] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:03.802249 19887 raft_consensus.cc:740] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.803632 19937 consensus_queue.cc:1048] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.803032 19887 consensus_queue.cc:260] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.809890 19887 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.813832 19812 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.814993 19704 leader_election.cc:304] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.815577 19738 leader_election.cc:290] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.815886 19937 raft_consensus.cc:2804] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:03.816357 19937 raft_consensus.cc:697] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.817153 19937 consensus_queue.cc:237] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.817425 19633 catalog_manager.cc:5697] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 1 to 2, leader changed from cf0cb5009aaf467189738c9b165a2614 (127.19.36.193) to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:03.818943 19887 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.836782 19973 raft_consensus.cc:993] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:03.837241 19973 raft_consensus.cc:1081] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:03.841818 19940 consensus_queue.cc:1048] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.843001 19964 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.843518 19634 catalog_manager.cc:5697] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 1 to 2, leader changed from cf0cb5009aaf467189738c9b165a2614 (127.19.36.193) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.847234 19635 catalog_manager.cc:5697] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 1 to 2, leader changed from 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195) to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.838907 19811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56434
I20260709 12:20:03.858327 19811 raft_consensus.cc:493] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:03.858892 19811 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.875607 19964 raft_consensus.cc:993] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1: : Instructing follower cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.876135 20003 raft_consensus.cc:1081] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Signalling peer cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:03.878630 19737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:34318
I20260709 12:20:03.879395 19737 raft_consensus.cc:493] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:03.879890 19737 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.881939 19940 consensus_queue.cc:1048] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:03.889883 19737 raft_consensus.cc:515] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.892488 19811 raft_consensus.cc:515] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.898409 19811 leader_election.cc:290] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 2 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.899859 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:03.900483 19736 raft_consensus.cc:3055] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:03.900813 19736 raft_consensus.cc:740] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.901579 19736 consensus_queue.cc:260] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.902787 19736 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.905061 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.905851 19887 raft_consensus.cc:3055] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:03.906277 19887 raft_consensus.cc:740] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:03.907292 19737 leader_election.cc:290] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:03.907323 19887 consensus_queue.cc:260] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.908658 19887 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.911466 19885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:03.912036 19885 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.918634 19885 raft_consensus.cc:2468] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 2.
I20260709 12:20:03.924656 19777 leader_election.cc:304] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:03.925614 19940 raft_consensus.cc:2804] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:03.926133 19940 raft_consensus.cc:697] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:03.926975 19940 consensus_queue.cc:237] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.932366 19736 raft_consensus.cc:2468] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 2.
I20260709 12:20:03.919139 19887 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.902566 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:03.937305 19811 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:03.943920 19702 leader_election.cc:304] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:03.944996 19937 raft_consensus.cc:2804] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:03.945639 19937 raft_consensus.cc:697] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:03.946592 19811 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 2.
I20260709 12:20:03.946709 19937 consensus_queue.cc:237] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:03.961839 19634 catalog_manager.cc:5697] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 1 to 2, leader changed from cf0cb5009aaf467189738c9b165a2614 (127.19.36.193) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:03.980132 19632 catalog_manager.cc:5697] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 1 to 2, leader changed from 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195) to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:04.078416 19964 raft_consensus.cc:993] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:04.078892 19967 raft_consensus.cc:1081] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:04.080523 19811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56438
I20260709 12:20:04.081050 19811 raft_consensus.cc:493] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:04.081359 19811 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:04.085784 19811 raft_consensus.cc:515] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:04.087428 19811 leader_election.cc:290] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 2 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:04.088574 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:04.088552 19738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:04.089282 19887 raft_consensus.cc:3055] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:04.089327 19738 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:04.089771 19887 raft_consensus.cc:740] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:04.090605 19887 consensus_queue.cc:260] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:04.091827 19887 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:04.096397 19738 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 2.
I20260709 12:20:04.097682 19776 leader_election.cc:304] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:04.098506 20018 raft_consensus.cc:2804] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:04.098560 19887 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 2.
I20260709 12:20:04.099048 20018 raft_consensus.cc:697] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:04.100519 20018 consensus_queue.cc:237] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:04.107627 19632 catalog_manager.cc:5697] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 1 to 2, leader changed from 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:04.114287 19811 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.115909 19937 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:04.128908 19887 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.130199 19971 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.154906 19811 raft_consensus.cc:1275] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.156119 19971 consensus_queue.cc:1048] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:04.163655 19887 raft_consensus.cc:1275] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.165447 20020 consensus_queue.cc:1048] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.187867 19887 raft_consensus.cc:1275] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.189091 20018 consensus_queue.cc:1048] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.197798 19738 raft_consensus.cc:1275] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.204495 20018 consensus_queue.cc:1048] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.209996 19811 raft_consensus.cc:1275] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.211347 19967 consensus_queue.cc:1048] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.223634 19738 raft_consensus.cc:1275] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.225304 20005 consensus_queue.cc:1048] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.273864 19811 raft_consensus.cc:1275] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.275285 19973 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.283632 19887 raft_consensus.cc:1275] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.285241 20020 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.373574 19738 raft_consensus.cc:1275] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.374786 20022 consensus_queue.cc:1048] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.377562 19887 raft_consensus.cc:1275] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.378931 20020 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.383335 19886 raft_consensus.cc:1275] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.384943 20022 consensus_queue.cc:1048] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.396741 19811 raft_consensus.cc:1275] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.398036 20020 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.545509 19886 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.546738 20022 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:04.554761 19738 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:04.557672 20022 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:06.592679 19603 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:20:06.593478 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:06.609831 19811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
mode: GRACEFUL
new_leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:32860
I20260709 12:20:06.610409 19811 raft_consensus.cc:606] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Received request to transfer leadership to TS 525fe26e6a9c49e49104ed9b608d87f1
I20260709 12:20:06.620483 19738 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
mode: GRACEFUL
new_leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:33228
I20260709 12:20:06.621057 19738 raft_consensus.cc:606] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Received request to transfer leadership to TS 525fe26e6a9c49e49104ed9b608d87f1
I20260709 12:20:06.622210 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:06.622700 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:07.020403 20028 raft_consensus.cc:993] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614: : Instructing follower 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:07.020877 20020 raft_consensus.cc:1081] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Signalling peer 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:07.022571 19887 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:49310
I20260709 12:20:07.023059 19887 raft_consensus.cc:493] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:07.023409 19887 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:07.028293 19887 raft_consensus.cc:515] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:07.029914 19887 leader_election.cc:290] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 3 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:07.030766 19738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:07.030956 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:07.031395 19738 raft_consensus.cc:3055] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:07.031482 19811 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:07.031805 19738 raft_consensus.cc:740] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:07.032613 19738 consensus_queue.cc:260] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:07.033803 19738 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:07.038292 19811 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 3.
I20260709 12:20:07.039620 19853 leader_election.cc:304] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:07.040146 19738 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 3.
I20260709 12:20:07.040337 20003 raft_consensus.cc:2804] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:07.041848 20003 raft_consensus.cc:697] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:07.042606 20003 consensus_queue.cc:237] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:07.049309 19634 catalog_manager.cc:5697] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 2 to 3, leader changed from cf0cb5009aaf467189738c9b165a2614 (127.19.36.193) to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:07.113585 19960 raft_consensus.cc:993] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: : Instructing follower 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:07.113996 20035 raft_consensus.cc:1081] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Signalling peer 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:07.115542 19887 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:49318
I20260709 12:20:07.116048 19887 raft_consensus.cc:493] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:07.116389 19887 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:07.121536 19887 raft_consensus.cc:515] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:07.123281 19887 leader_election.cc:290] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 3 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:07.124199 19738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:07.124445 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:07.124848 19738 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:07.125000 19811 raft_consensus.cc:3055] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:07.125366 19811 raft_consensus.cc:740] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:07.126047 19811 consensus_queue.cc:260] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:07.126931 19811 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:07.131233 19738 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 3.
I20260709 12:20:07.132428 19850 leader_election.cc:304] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:07.132551 19811 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 3.
I20260709 12:20:07.133262 20003 raft_consensus.cc:2804] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:07.133837 20003 raft_consensus.cc:697] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:07.134660 20003 consensus_queue.cc:237] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:07.143857 19634 catalog_manager.cc:5697] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 2 to 3, leader changed from 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194) to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:07.479635 19738 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:07.481156 20003 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:07.492233 19811 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:07.495850 20029 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:07.556404 19811 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:07.557682 20039 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:07.566329 19738 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:07.568192 20003 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:07.623697 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:07.630580 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:07.630965 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:08.631902 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:08.636646 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:08.637061 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:09.637948 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:09.642736 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:09.643301 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:10.644483 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:10.649555 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:10.649930 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:11.650827 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:11.656298 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:11.656647 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:12.657518 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:12.662201 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:12.662552 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:13.663436 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:13.668689 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:13.669114 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:14.670075 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:14.674885 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:14.675355 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:15.676184 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:15.680985 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:15.681404 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:16.709326 19603 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:20:16.713639 19886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:49322
I20260709 12:20:16.714349 19886 raft_consensus.cc:606] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:16.719324 19736 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:34342
I20260709 12:20:16.719831 19736 raft_consensus.cc:606] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:16.724824 19736 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:34342
I20260709 12:20:16.725333 19736 raft_consensus.cc:606] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:16.729743 19736 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:34342
I20260709 12:20:16.730293 19736 raft_consensus.cc:606] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:16.742313 19886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
mode: GRACEFUL
new_leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:49322
I20260709 12:20:16.742873 19886 raft_consensus.cc:606] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Received request to transfer leadership to TS 39b94cc5da3d45e8b7d837714a0a39e3
I20260709 12:20:16.874601 20020 raft_consensus.cc:993] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:16.875262 20051 raft_consensus.cc:1081] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:16.876813 19811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56434
I20260709 12:20:16.877260 19811 raft_consensus.cc:493] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:16.877588 19811 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.883239 19811 raft_consensus.cc:515] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.885105 19811 leader_election.cc:290] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 3 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:16.886121 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:16.886291 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:16.886957 19886 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.886937 19736 raft_consensus.cc:3055] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:16.887485 19736 raft_consensus.cc:740] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:16.888355 19736 consensus_queue.cc:260] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.889737 19736 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.892009 20052 raft_consensus.cc:993] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:16.892458 20051 raft_consensus.cc:1081] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:16.893839 19811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56434
I20260709 12:20:16.894265 19811 raft_consensus.cc:493] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:16.894291 19886 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 3.
I20260709 12:20:16.894615 19811 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.895864 19777 leader_election.cc:304] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:16.896801 20035 raft_consensus.cc:2804] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:16.896857 19736 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 3.
I20260709 12:20:16.898038 20035 raft_consensus.cc:697] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:16.898984 20035 consensus_queue.cc:237] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.900609 19811 raft_consensus.cc:515] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.902593 19811 leader_election.cc:290] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 3 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:16.903734 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:16.903833 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69272a8bf7fc4d16bde56e636fb60a33" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:16.904335 19886 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.904433 19736 raft_consensus.cc:3055] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:16.904723 19736 raft_consensus.cc:740] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:16.905243 19736 consensus_queue.cc:260] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.906065 19736 raft_consensus.cc:3060] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.909055 19886 raft_consensus.cc:2468] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 3.
I20260709 12:20:16.910121 19777 leader_election.cc:304] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:16.910795 20035 raft_consensus.cc:2804] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:16.911329 20035 raft_consensus.cc:697] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:16.912047 20035 consensus_queue.cc:237] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.912437 19736 raft_consensus.cc:2468] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 3.
I20260709 12:20:16.912575 19633 catalog_manager.cc:5697] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 2 to 3, leader changed from cf0cb5009aaf467189738c9b165a2614 (127.19.36.193) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:16.929198 19633 catalog_manager.cc:5697] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 2 to 3, leader changed from cf0cb5009aaf467189738c9b165a2614 (127.19.36.193) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:16.967494 20052 raft_consensus.cc:993] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:16.968009 20051 raft_consensus.cc:1081] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:16.969517 19811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56434
I20260709 12:20:16.969944 19811 raft_consensus.cc:493] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:16.970220 19811 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.974129 19811 raft_consensus.cc:515] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.975507 19811 leader_election.cc:290] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 3 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:16.976300 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:16.976641 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:16.976897 19736 raft_consensus.cc:3055] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:16.977180 19886 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.977219 19736 raft_consensus.cc:740] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:16.978075 19736 consensus_queue.cc:260] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.978965 19736 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:16.981731 19886 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 3.
I20260709 12:20:16.982663 19777 leader_election.cc:304] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:16.983361 20035 raft_consensus.cc:2804] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:16.983753 20035 raft_consensus.cc:697] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:16.983918 19736 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 3.
I20260709 12:20:16.984491 20035 consensus_queue.cc:237] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:16.990605 19633 catalog_manager.cc:5697] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 2 to 3, leader changed from cf0cb5009aaf467189738c9b165a2614 (127.19.36.193) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:17.010212 20003 raft_consensus.cc:993] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:17.010622 20003 raft_consensus.cc:1081] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:17.012030 19811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56438
I20260709 12:20:17.012487 19811 raft_consensus.cc:493] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:17.012748 19811 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:17.016644 19811 raft_consensus.cc:515] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:17.017972 19811 leader_election.cc:290] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 4 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:17.018857 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:17.018998 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:17.019407 19736 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:17.019548 19886 raft_consensus.cc:3055] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:17.019902 19886 raft_consensus.cc:740] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:17.020519 19886 consensus_queue.cc:260] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:17.021474 19886 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:17.024245 19736 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 4.
I20260709 12:20:17.025359 19776 leader_election.cc:304] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:17.025987 20035 raft_consensus.cc:2804] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:17.026352 20035 raft_consensus.cc:697] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:17.026592 19886 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 4.
I20260709 12:20:17.027066 20035 consensus_queue.cc:237] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:17.033039 19633 catalog_manager.cc:5697] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 3 to 4, leader changed from 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 4 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:17.161410 20050 raft_consensus.cc:993] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: : Instructing follower 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:17.161934 20050 raft_consensus.cc:1081] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Signalling peer 39b94cc5da3d45e8b7d837714a0a39e3 to start an election
I20260709 12:20:17.163364 19811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
 from {username='slave'} at 127.0.0.1:56438
I20260709 12:20:17.163763 19811 raft_consensus.cc:493] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:17.163985 19811 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:17.168648 19811 raft_consensus.cc:515] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:17.170593 19811 leader_election.cc:290] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 4 election: Requested vote from peers 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391), cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493)
I20260709 12:20:17.171218 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:17.171504 19736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:17.171764 19886 raft_consensus.cc:3055] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:17.172086 19736 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:17.172091 19886 raft_consensus.cc:740] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:17.172931 19886 consensus_queue.cc:260] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:17.173763 19886 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:17.176714 19736 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 4.
I20260709 12:20:17.177706 19776 leader_election.cc:304] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:17.177809 19886 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b94cc5da3d45e8b7d837714a0a39e3 in term 4.
I20260709 12:20:17.178520 20035 raft_consensus.cc:2804] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:17.185398 20035 raft_consensus.cc:697] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Becoming Leader. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:17.186241 20035 consensus_queue.cc:237] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:17.193233 19633 catalog_manager.cc:5697] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 reported cstate change: term changed from 3 to 4, leader changed from 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195) to 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194). New cstate: current_term: 4 leader_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:17.324054 19736 raft_consensus.cc:1275] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:17.325296 20035 consensus_queue.cc:1048] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:17.336967 19886 raft_consensus.cc:1275] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:17.339460 20062 consensus_queue.cc:1048] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:17.438683 19736 raft_consensus.cc:1275] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:17.439896 20062 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:17.449931 19886 raft_consensus.cc:1275] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:17.451079 20062 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:17.486407 19736 raft_consensus.cc:1275] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:17.487666 20062 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:17.491132 19886 raft_consensus.cc:1275] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:17.492549 20063 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:17.581912 19886 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:17.583065 20062 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:17.590659 19736 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:17.592074 20063 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:17.653761 19886 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:17.654969 20063 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:17.663537 19736 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Refusing update from remote peer 39b94cc5da3d45e8b7d837714a0a39e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:17.665251 20062 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:19.774621 19603 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:20:19.775532 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:19.782768 19811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
mode: GRACEFUL
new_leader_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:32860
I20260709 12:20:19.783386 19811 raft_consensus.cc:606] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Received request to transfer leadership to TS cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:19.785826 19811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
mode: GRACEFUL
new_leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:32860
I20260709 12:20:19.786242 19811 raft_consensus.cc:606] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Received request to transfer leadership to TS 525fe26e6a9c49e49104ed9b608d87f1
I20260709 12:20:19.788488 19811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
mode: GRACEFUL
new_leader_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:32860
I20260709 12:20:19.788900 19811 raft_consensus.cc:606] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Received request to transfer leadership to TS cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:19.790946 19811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
mode: GRACEFUL
new_leader_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:32860
I20260709 12:20:19.791317 19811 raft_consensus.cc:606] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Received request to transfer leadership to TS cf0cb5009aaf467189738c9b165a2614
I20260709 12:20:19.793666 19811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77"
dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
mode: GRACEFUL
new_leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:32860
I20260709 12:20:19.794078 19811 raft_consensus.cc:606] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Received request to transfer leadership to TS 525fe26e6a9c49e49104ed9b608d87f1
I20260709 12:20:19.795025 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:19.795584 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:19.811534 20062 raft_consensus.cc:993] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3: : Instructing follower cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:19.811997 20078 raft_consensus.cc:1081] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Signalling peer cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:19.813457 19738 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:34326
I20260709 12:20:19.813911 19738 raft_consensus.cc:493] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:19.814249 19738 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:19.818466 19738 raft_consensus.cc:515] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:19.820487 19738 leader_election.cc:290] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 3 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:19.821569 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:19.821569 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08bec98f13af417c8b66f3248fadf913" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:19.822300 19811 raft_consensus.cc:3055] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:19.822362 19886 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:19.822690 19811 raft_consensus.cc:740] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:19.823402 19811 consensus_queue.cc:260] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:19.824494 19811 raft_consensus.cc:3060] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:19.828567 19886 raft_consensus.cc:2468] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 3.
I20260709 12:20:19.829872 19702 leader_election.cc:304] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:19.830220 19811 raft_consensus.cc:2468] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 3.
I20260709 12:20:19.831467 20080 raft_consensus.cc:2804] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:19.832369 20080 raft_consensus.cc:697] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 3 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:19.833204 20080 consensus_queue.cc:237] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:19.839795 19632 catalog_manager.cc:5697] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 2 to 3, leader changed from 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194) to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 3 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:19.901000 20062 raft_consensus.cc:993] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: : Instructing follower cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:19.901503 20078 raft_consensus.cc:1081] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Signalling peer cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:19.902953 19738 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:34326
I20260709 12:20:19.903470 19738 raft_consensus.cc:493] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:19.903823 19738 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:19.907936 19738 raft_consensus.cc:515] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:19.909360 19738 leader_election.cc:290] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 5 election: Requested vote from peers 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731), 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391)
I20260709 12:20:19.910522 19886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:19.910581 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5835dcc7ad0d414393d6d71ebbe9836a" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:19.911110 19886 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:19.911254 19811 raft_consensus.cc:3055] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:20:19.911667 19811 raft_consensus.cc:740] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:19.912288 19811 consensus_queue.cc:260] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:19.913179 19811 raft_consensus.cc:3060] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:19.915519 19886 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 5.
I20260709 12:20:19.916450 19702 leader_election.cc:304] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:19.917099 20080 raft_consensus.cc:2804] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:20:19.917469 20080 raft_consensus.cc:697] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 5 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:19.917948 19811 raft_consensus.cc:2468] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 5.
I20260709 12:20:19.918218 20080 consensus_queue.cc:237] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:19.924662 19632 catalog_manager.cc:5697] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 4 to 5, leader changed from 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194) to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 5 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:19.985421 20078 raft_consensus.cc:993] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3: : Instructing follower 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:19.986441 20062 raft_consensus.cc:1081] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Signalling peer 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:19.988576 19886 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:49318
I20260709 12:20:19.989729 19886 raft_consensus.cc:493] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:19.990561 19886 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:19.993085 20062 raft_consensus.cc:993] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3: : Instructing follower 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:19.993857 20078 raft_consensus.cc:1081] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Signalling peer 525fe26e6a9c49e49104ed9b608d87f1 to start an election
I20260709 12:20:19.995853 19887 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77"
dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
 from {username='slave'} at 127.0.0.1:49318
I20260709 12:20:19.996454 19887 raft_consensus.cc:493] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:19.996742 19887 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:19.999044 19886 raft_consensus.cc:515] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:20.001228 19887 raft_consensus.cc:515] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:20.002628 19886 leader_election.cc:290] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 4 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:20.003211 19738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:20.003665 19887 leader_election.cc:290] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 4 election: Requested vote from peers cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:20.004070 19738 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:20.004037 19811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2870cd5caec145f9a32ffec85190aa39" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:20.004616 19811 raft_consensus.cc:3055] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:20.004946 19811 raft_consensus.cc:740] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:20.005820 19811 consensus_queue.cc:260] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:20.006724 19811 raft_consensus.cc:3060] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:20.009912 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:20.010646 19810 raft_consensus.cc:3055] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:20.011018 19810 raft_consensus.cc:740] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:20.011314 19738 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 4.
I20260709 12:20:20.012187 19810 consensus_queue.cc:260] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:20.013692 19810 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:20.015151 19850 leader_election.cc:304] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:20.015269 19811 raft_consensus.cc:2468] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 4.
I20260709 12:20:20.016305 20079 raft_consensus.cc:2804] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:20.016589 19738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d7377c8cf6469f8fe9812884317e77" candidate_uuid: "525fe26e6a9c49e49104ed9b608d87f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
I20260709 12:20:20.017320 19738 raft_consensus.cc:3060] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:20.017894 20079 raft_consensus.cc:697] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:20.019054 20079 consensus_queue.cc:237] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:20.021085 19810 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 4.
I20260709 12:20:20.022320 19853 leader_election.cc:304] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b94cc5da3d45e8b7d837714a0a39e3, 525fe26e6a9c49e49104ed9b608d87f1; no voters: 
I20260709 12:20:20.023473 20090 raft_consensus.cc:2804] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:20.024727 19738 raft_consensus.cc:2468] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 525fe26e6a9c49e49104ed9b608d87f1 in term 4.
I20260709 12:20:20.026842 20090 raft_consensus.cc:697] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 LEADER]: Becoming Leader. State: Replica: 525fe26e6a9c49e49104ed9b608d87f1, State: Running, Role: LEADER
I20260709 12:20:20.027843 20090 consensus_queue.cc:237] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } }
I20260709 12:20:20.030242 19632 catalog_manager.cc:5697] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 3 to 4, leader changed from 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194) to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 4 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:20.044289 19632 catalog_manager.cc:5697] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 reported cstate change: term changed from 3 to 4, leader changed from 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194) to 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195). New cstate: current_term: 4 leader_uuid: "525fe26e6a9c49e49104ed9b608d87f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:20.234637 20089 raft_consensus.cc:993] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3: : Instructing follower cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:20.235206 20078 raft_consensus.cc:1081] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Signalling peer cf0cb5009aaf467189738c9b165a2614 to start an election
I20260709 12:20:20.236606 19738 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d"
dest_uuid: "cf0cb5009aaf467189738c9b165a2614"
 from {username='slave'} at 127.0.0.1:34326
I20260709 12:20:20.237064 19738 raft_consensus.cc:493] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:20.237344 19738 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:20.241405 19738 raft_consensus.cc:515] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:20.243839 19738 leader_election.cc:290] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 5 election: Requested vote from peers 525fe26e6a9c49e49104ed9b608d87f1 (127.19.36.195:36391), 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731)
I20260709 12:20:20.244163 19887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "525fe26e6a9c49e49104ed9b608d87f1"
I20260709 12:20:20.244742 19887 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:20.244618 19810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18bb7b2fe304ef2832dd06a6050977d" candidate_uuid: "cf0cb5009aaf467189738c9b165a2614" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "39b94cc5da3d45e8b7d837714a0a39e3"
I20260709 12:20:20.245412 19810 raft_consensus.cc:3055] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:20:20.245759 19810 raft_consensus.cc:740] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 39b94cc5da3d45e8b7d837714a0a39e3, State: Running, Role: LEADER
I20260709 12:20:20.246608 19810 consensus_queue.cc:260] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:20.247546 19810 raft_consensus.cc:3060] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:20.249341 19887 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 5.
I20260709 12:20:20.250424 19702 leader_election.cc:304] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 525fe26e6a9c49e49104ed9b608d87f1, cf0cb5009aaf467189738c9b165a2614; no voters: 
I20260709 12:20:20.251474 20080 raft_consensus.cc:2804] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:20:20.251933 20080 raft_consensus.cc:697] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 5 LEADER]: Becoming Leader. State: Replica: cf0cb5009aaf467189738c9b165a2614, State: Running, Role: LEADER
I20260709 12:20:20.253118 19810 raft_consensus.cc:2468] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf0cb5009aaf467189738c9b165a2614 in term 5.
I20260709 12:20:20.253051 20080 consensus_queue.cc:237] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } }
I20260709 12:20:20.260689 19632 catalog_manager.cc:5697] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 reported cstate change: term changed from 4 to 5, leader changed from 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194) to cf0cb5009aaf467189738c9b165a2614 (127.19.36.193). New cstate: current_term: 5 leader_uuid: "cf0cb5009aaf467189738c9b165a2614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:20.266467 19810 raft_consensus.cc:1275] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:20.267926 20080 consensus_queue.cc:1048] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:20.278267 19887 raft_consensus.cc:1275] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:20.279876 20080 consensus_queue.cc:1048] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:20.308225 19887 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:20:20.309274 20080 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:20:20.318894 19810 raft_consensus.cc:1275] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:20:20.320353 20080 consensus_queue.cc:1048] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:20:20.449198 19738 raft_consensus.cc:1275] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:20.450369 20090 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:20.459826 19810 raft_consensus.cc:1275] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:20.461412 20092 consensus_queue.cc:1048] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:20.577755 19810 raft_consensus.cc:1275] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:20.578976 20092 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:20.587013 19738 raft_consensus.cc:1275] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Refusing update from remote peer 525fe26e6a9c49e49104ed9b608d87f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:20.588928 20079 consensus_queue.cc:1048] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0cb5009aaf467189738c9b165a2614" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:20.717677 19810 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:20:20.718997 20080 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b94cc5da3d45e8b7d837714a0a39e3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:20:20.728581 19887 raft_consensus.cc:1275] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Refusing update from remote peer cf0cb5009aaf467189738c9b165a2614: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:20:20.730319 20097 consensus_queue.cc:1048] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "525fe26e6a9c49e49104ed9b608d87f1" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:20:20.796933 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:20.801869 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:20.802251 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:21.803393 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:21.808970 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:21.809417 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:22.810546 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:22.815723 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:22.816110 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:23.817066 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:23.821769 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:23.822223 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:24.823309 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:24.828253 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:24.828760 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:25.829896 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:25.834918 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:25.835418 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:26.836455 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:26.842149 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:26.842525 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:27.843609 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:27.848690 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:27.849054 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:28.850080 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:28.855054 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:28.855633 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:29.883468 19603 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:20:29.884021 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
I20260709 12:20:29.909307 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:29.910171 19603 tablet_replica.cc:333] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.911358 19603 raft_consensus.cc:2243] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:29.911989 19603 raft_consensus.cc:2272] T b2d7377c8cf6469f8fe9812884317e77 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:29.912253 19850 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:29.914392 19603 tablet_replica.cc:333] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.915063 19603 raft_consensus.cc:2243] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:29.915256 19850 consensus_peers.cc:597] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 -> Peer cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493): Couldn't send request to peer cf0cb5009aaf467189738c9b165a2614. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:29.915625 19603 raft_consensus.cc:2272] T 309a1998660e4f4b8fd5feefc322b9ed P cf0cb5009aaf467189738c9b165a2614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:29.917366 19603 tablet_replica.cc:333] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.917881 19603 raft_consensus.cc:2243] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:29.918572 19603 raft_consensus.cc:2272] T 08bec98f13af417c8b66f3248fadf913 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:29.920258 19603 tablet_replica.cc:333] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.920796 19603 raft_consensus.cc:2243] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:20:29.921494 19603 raft_consensus.cc:2272] T 5835dcc7ad0d414393d6d71ebbe9836a P cf0cb5009aaf467189738c9b165a2614 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:29.923352 19603 tablet_replica.cc:333] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.923830 19603 raft_consensus.cc:2243] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:20:29.924474 19603 raft_consensus.cc:2272] T b18bb7b2fe304ef2832dd06a6050977d P cf0cb5009aaf467189738c9b165a2614 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:29.926056 19603 tablet_replica.cc:333] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.926559 19603 raft_consensus.cc:2243] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:29.926967 19603 raft_consensus.cc:2272] T 69272a8bf7fc4d16bde56e636fb60a33 P cf0cb5009aaf467189738c9b165a2614 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:29.928567 19603 tablet_replica.cc:333] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.929065 19603 raft_consensus.cc:2243] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:29.929508 19603 raft_consensus.cc:2272] T 4dcf46a9de1c4cf599d7ea55da75bde5 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:29.931794 19603 tablet_replica.cc:333] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.932415 19603 raft_consensus.cc:2243] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:29.932940 19603 raft_consensus.cc:2272] T 219f05d76de64095a5416ee10bb3a319 P cf0cb5009aaf467189738c9b165a2614 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:29.934593 19603 tablet_replica.cc:333] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614: stopping tablet replica
I20260709 12:20:29.935141 19603 raft_consensus.cc:2243] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:29.935570 19603 raft_consensus.cc:2272] T 2870cd5caec145f9a32ffec85190aa39 P cf0cb5009aaf467189738c9b165a2614 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:30.029371 19776 consensus_peers.cc:597] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 -> Peer cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493): Couldn't send request to peer cf0cb5009aaf467189738c9b165a2614. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:33493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:30.038008 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
W20260709 12:20:30.049690 19850 consensus_peers.cc:597] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 -> Peer cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493): Couldn't send request to peer cf0cb5009aaf467189738c9b165a2614. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:33493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:30.058174 19776 consensus_peers.cc:597] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 -> Peer cf0cb5009aaf467189738c9b165a2614 (127.19.36.193:33493): Couldn't send request to peer cf0cb5009aaf467189738c9b165a2614. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:33493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:30.068272 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
I20260709 12:20:30.103897 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:30.104753 19603 tablet_replica.cc:333] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.105769 19603 raft_consensus.cc:2243] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.106478 19603 raft_consensus.cc:2272] T 2870cd5caec145f9a32ffec85190aa39 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.108567 19603 tablet_replica.cc:333] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.109153 19603 raft_consensus.cc:2243] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.109694 19603 raft_consensus.cc:2272] T 08bec98f13af417c8b66f3248fadf913 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.111642 19603 tablet_replica.cc:333] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.112490 19603 raft_consensus.cc:2243] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:30.113572 19603 raft_consensus.cc:2272] T 69272a8bf7fc4d16bde56e636fb60a33 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.115336 19603 tablet_replica.cc:333] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.115844 19603 raft_consensus.cc:2243] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.116328 19603 raft_consensus.cc:2272] T b18bb7b2fe304ef2832dd06a6050977d P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.117991 19603 tablet_replica.cc:333] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.118492 19603 raft_consensus.cc:2243] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.118980 19603 raft_consensus.cc:2272] T 219f05d76de64095a5416ee10bb3a319 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.120755 19603 tablet_replica.cc:333] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.121237 19603 raft_consensus.cc:2243] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:30.122162 19603 raft_consensus.cc:2272] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.123899 19603 tablet_replica.cc:333] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.124475 19603 raft_consensus.cc:2243] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.125607 19603 raft_consensus.cc:2272] T 5835dcc7ad0d414393d6d71ebbe9836a P 39b94cc5da3d45e8b7d837714a0a39e3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.127161 19603 tablet_replica.cc:333] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.127705 19603 raft_consensus.cc:2243] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.128201 19603 raft_consensus.cc:2272] T b2d7377c8cf6469f8fe9812884317e77 P 39b94cc5da3d45e8b7d837714a0a39e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.129907 19603 tablet_replica.cc:333] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3: stopping tablet replica
I20260709 12:20:30.130360 19603 raft_consensus.cc:2243] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:30.131256 19603 raft_consensus.cc:2272] T 309a1998660e4f4b8fd5feefc322b9ed P 39b94cc5da3d45e8b7d837714a0a39e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.161032 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
W20260709 12:20:30.179224 19853 consensus_peers.cc:597] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 -> Peer 39b94cc5da3d45e8b7d837714a0a39e3 (127.19.36.194:44731): Couldn't send request to peer 39b94cc5da3d45e8b7d837714a0a39e3. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:44731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:30.182114 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
I20260709 12:20:30.206496 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:30.207301 19603 tablet_replica.cc:333] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.207971 19603 raft_consensus.cc:2243] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:30.208829 19603 raft_consensus.cc:2272] T 2870cd5caec145f9a32ffec85190aa39 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.210727 19603 tablet_replica.cc:333] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.211299 19603 raft_consensus.cc:2243] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.211748 19603 raft_consensus.cc:2272] T 309a1998660e4f4b8fd5feefc322b9ed P 525fe26e6a9c49e49104ed9b608d87f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.213488 19603 tablet_replica.cc:333] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.213994 19603 raft_consensus.cc:2243] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:30.214999 19603 raft_consensus.cc:2272] T b2d7377c8cf6469f8fe9812884317e77 P 525fe26e6a9c49e49104ed9b608d87f1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.216874 19603 tablet_replica.cc:333] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.217294 19603 raft_consensus.cc:2243] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.217731 19603 raft_consensus.cc:2272] T 08bec98f13af417c8b66f3248fadf913 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.219321 19603 tablet_replica.cc:333] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.219753 19603 raft_consensus.cc:2243] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.220180 19603 raft_consensus.cc:2272] T 69272a8bf7fc4d16bde56e636fb60a33 P 525fe26e6a9c49e49104ed9b608d87f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.221707 19603 tablet_replica.cc:333] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.222184 19603 raft_consensus.cc:2243] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.222747 19603 raft_consensus.cc:2272] T b18bb7b2fe304ef2832dd06a6050977d P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.224507 19603 tablet_replica.cc:333] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.224959 19603 raft_consensus.cc:2243] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:30.225871 19603 raft_consensus.cc:2272] T 219f05d76de64095a5416ee10bb3a319 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.227523 19603 tablet_replica.cc:333] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.228027 19603 raft_consensus.cc:2243] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.228441 19603 raft_consensus.cc:2272] T 4dcf46a9de1c4cf599d7ea55da75bde5 P 525fe26e6a9c49e49104ed9b608d87f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.230015 19603 tablet_replica.cc:333] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1: stopping tablet replica
I20260709 12:20:30.230427 19603 raft_consensus.cc:2243] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:30.230888 19603 raft_consensus.cc:2272] T 5835dcc7ad0d414393d6d71ebbe9836a P 525fe26e6a9c49e49104ed9b608d87f1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.258024 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
I20260709 12:20:30.276832 19603 master.cc:562] Master@127.19.36.254:35583 shutting down...
I20260709 12:20:30.300197 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:30.300915 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:30.301266 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P 84d2e33a04034ab3840205a6bd1951fd: stopping tablet replica
I20260709 12:20:30.321316 19603 master.cc:584] Master@127.19.36.254:35583 shutdown complete.
39b94cc5da3d45e8b7d837714a0a39e3  4
525fe26e6a9c49e49104ed9b608d87f1  1
cf0cb5009aaf467189738c9b165a2614  4
39b94cc5da3d45e8b7d837714a0a39e3  3
525fe26e6a9c49e49104ed9b608d87f1  3
cf0cb5009aaf467189738c9b165a2614  3
39b94cc5da3d45e8b7d837714a0a39e3  8
525fe26e6a9c49e49104ed9b608d87f1  1
cf0cb5009aaf467189738c9b165a2614  0
39b94cc5da3d45e8b7d837714a0a39e3  3
525fe26e6a9c49e49104ed9b608d87f1  3
cf0cb5009aaf467189738c9b165a2614  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (30456 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:20:30.363662 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:37435
I20260709 12:20:30.364831 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:30.370548 20116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:30.370873 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:30.373955 20119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:30.374079 19603 server_base.cc:1061] running on GCE node
I20260709 12:20:30.375680 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:30.375922 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:30.376102 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599630376084 us; error 0 us; skew 500 ppm
I20260709 12:20:30.376677 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:30.379297 19603 webserver.cc:533] Webserver started at http://127.19.36.254:43043/ using document root <none> and password file <none>
I20260709 12:20:30.379819 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:30.380033 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:30.380410 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:30.381587 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "c91235ff0c5a409a8e42f8d25fce2d01"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.386087 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:20:30.389385 20124 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.390267 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:20:30.390563 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "c91235ff0c5a409a8e42f8d25fce2d01"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.390815 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:30.408654 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:30.409826 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:30.448335 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:37435
I20260709 12:20:30.448429 20175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:37435 every 8 connection(s)
I20260709 12:20:30.452410 20176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.461843 20176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01: Bootstrap starting.
I20260709 12:20:30.466476 20176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.470582 20176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01: No bootstrap required, opened a new log
I20260709 12:20:30.472790 20176 raft_consensus.cc:359] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" member_type: VOTER }
I20260709 12:20:30.473205 20176 raft_consensus.cc:385] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.473382 20176 raft_consensus.cc:740] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91235ff0c5a409a8e42f8d25fce2d01, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.473860 20176 consensus_queue.cc:260] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" member_type: VOTER }
I20260709 12:20:30.474278 20176 raft_consensus.cc:399] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:30.474491 20176 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:30.474725 20176 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.478605 20176 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" member_type: VOTER }
I20260709 12:20:30.479086 20176 leader_election.cc:304] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c91235ff0c5a409a8e42f8d25fce2d01; no voters: 
I20260709 12:20:30.480348 20176 leader_election.cc:290] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:30.480860 20179 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.482180 20179 raft_consensus.cc:697] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 1 LEADER]: Becoming Leader. State: Replica: c91235ff0c5a409a8e42f8d25fce2d01, State: Running, Role: LEADER
I20260709 12:20:30.482913 20179 consensus_queue.cc:237] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" member_type: VOTER }
I20260709 12:20:30.483606 20176 sys_catalog.cc:565] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:30.489184 20181 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91235ff0c5a409a8e42f8d25fce2d01. Latest consensus state: current_term: 1 leader_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" member_type: VOTER } }
I20260709 12:20:30.489930 20181 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:30.494362 20180 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c91235ff0c5a409a8e42f8d25fce2d01" member_type: VOTER } }
I20260709 12:20:30.494998 20180 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:30.503317 20185 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:30.508361 20185 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:30.510485 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:30.516993 20185 catalog_manager.cc:1383] Generated new cluster ID: d476cc354b87440d907395028dc72ab5
I20260709 12:20:30.517356 20185 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:30.537849 20185 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:30.539073 20185 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:30.554946 20185 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01: Generated new TSK 0
I20260709 12:20:30.555543 20185 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:30.576821 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:30.583040 20197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:30.583915 20198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:30.585117 20200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:30.585644 19603 server_base.cc:1061] running on GCE node
I20260709 12:20:30.586575 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:30.586783 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:30.586972 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599630586959 us; error 0 us; skew 500 ppm
I20260709 12:20:30.587589 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:30.589777 19603 webserver.cc:533] Webserver started at http://127.19.36.193:38937/ using document root <none> and password file <none>
I20260709 12:20:30.590274 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:30.590507 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:30.590803 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:30.591953 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.596370 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:30.599529 20205 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.600270 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:20:30.600570 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.600865 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:30.624014 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:30.625255 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:30.626714 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:30.628931 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:30.629159 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.629398 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:30.629588 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.671295 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:33379
I20260709 12:20:30.671386 20267 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:33379 every 8 connection(s)
I20260709 12:20:30.676750 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:30.684087 20272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:30.685577 20273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:30.687788 20275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:30.688421 19603 server_base.cc:1061] running on GCE node
I20260709 12:20:30.688989 20268 heartbeater.cc:344] Connected to a master server at 127.19.36.254:37435
I20260709 12:20:30.689388 20268 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:30.689553 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:30.689816 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:30.690001 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599630689970 us; error 0 us; skew 500 ppm
I20260709 12:20:30.690254 20268 heartbeater.cc:507] Master 127.19.36.254:37435 requested a full tablet report, sending...
I20260709 12:20:30.690639 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:30.692816 20141 ts_manager.cc:194] Registered new tserver with Master: 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379)
I20260709 12:20:30.693612 19603 webserver.cc:533] Webserver started at http://127.19.36.194:34097/ using document root <none> and password file <none>
I20260709 12:20:30.694300 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:30.694540 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:30.694841 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:30.695313 20141 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42582
I20260709 12:20:30.696377 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "de1ed702f9174763a40084a7c4dd7922"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.700811 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:20:30.704150 20280 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.704946 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:20:30.705247 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "de1ed702f9174763a40084a7c4dd7922"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.705518 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:30.717161 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:30.718448 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:30.719921 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:30.722069 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:30.722239 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.722522 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:30.722685 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.760938 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:38751
I20260709 12:20:30.761031 20343 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:38751 every 8 connection(s)
I20260709 12:20:30.765779 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:30.772970 20347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:30.775059 20348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:30.776412 19603 server_base.cc:1061] running on GCE node
W20260709 12:20:30.776986 20350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:30.778024 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:30.778290 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:30.778502 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599630778484 us; error 0 us; skew 500 ppm
I20260709 12:20:30.779111 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:30.779137 20344 heartbeater.cc:344] Connected to a master server at 127.19.36.254:37435
I20260709 12:20:30.779634 20344 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:30.780568 20344 heartbeater.cc:507] Master 127.19.36.254:37435 requested a full tablet report, sending...
I20260709 12:20:30.782023 19603 webserver.cc:533] Webserver started at http://127.19.36.195:42881/ using document root <none> and password file <none>
I20260709 12:20:30.782603 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:30.782794 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:30.782877 20141 ts_manager.cc:194] Registered new tserver with Master: de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:30.783080 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:30.784299 20141 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42594
I20260709 12:20:30.784302 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "0e3df13b1d27427f92024ccb926c88a7"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.789009 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.003s
I20260709 12:20:30.792490 20355 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.793282 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20260709 12:20:30.793603 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "0e3df13b1d27427f92024ccb926c88a7"
format_stamp: "Formatted at 2026-07-09 12:20:30 on dist-test-slave-hfcn"
I20260709 12:20:30.793853 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:30.811419 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:30.812615 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:30.813952 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:30.816314 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:30.816507 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.816828 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:30.816988 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:30.855513 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:44345
I20260709 12:20:30.855615 20417 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:44345 every 8 connection(s)
I20260709 12:20:30.875783 20418 heartbeater.cc:344] Connected to a master server at 127.19.36.254:37435
I20260709 12:20:30.876186 20418 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:30.876928 20418 heartbeater.cc:507] Master 127.19.36.254:37435 requested a full tablet report, sending...
I20260709 12:20:30.878798 20141 ts_manager.cc:194] Registered new tserver with Master: 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:30.879065 19603 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020516163s
I20260709 12:20:30.880215 20141 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42608
I20260709 12:20:30.905169 20141 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42622:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:30.908164 20141 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:30.989156 20233 tablet_service.cc:1511] Processing CreateTablet for tablet ccd84b3af63b457babd4365151fcdedd (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:30.990617 20233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd84b3af63b457babd4365151fcdedd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.993115 20232 tablet_service.cc:1511] Processing CreateTablet for tablet 9dbca0c06910494d9f02427793ceaef9 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:30.993458 20231 tablet_service.cc:1511] Processing CreateTablet for tablet 29a5ffe3fb714214a1388435b5621d0a (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:30.994573 20232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dbca0c06910494d9f02427793ceaef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.994897 20382 tablet_service.cc:1511] Processing CreateTablet for tablet 9dbca0c06910494d9f02427793ceaef9 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:30.995883 20231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a5ffe3fb714214a1388435b5621d0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.996313 20382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dbca0c06910494d9f02427793ceaef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.001770 20230 tablet_service.cc:1511] Processing CreateTablet for tablet df15adc2714c46f496be1c16d7d8bde5 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:31.003659 20229 tablet_service.cc:1511] Processing CreateTablet for tablet 32e6cbe46a0946898963a7ab5296704b (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:31.004338 20383 tablet_service.cc:1511] Processing CreateTablet for tablet ccd84b3af63b457babd4365151fcdedd (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:31.001682 20380 tablet_service.cc:1511] Processing CreateTablet for tablet df15adc2714c46f496be1c16d7d8bde5 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:31.005704 20383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd84b3af63b457babd4365151fcdedd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.006312 20380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df15adc2714c46f496be1c16d7d8bde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.000305 20381 tablet_service.cc:1511] Processing CreateTablet for tablet 29a5ffe3fb714214a1388435b5621d0a (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:31.008534 20381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a5ffe3fb714214a1388435b5621d0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.003119 20230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df15adc2714c46f496be1c16d7d8bde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.017086 20229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32e6cbe46a0946898963a7ab5296704b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.017138 20228 tablet_service.cc:1511] Processing CreateTablet for tablet 9da02bc50d1d4a33be9a722da51a1925 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:31.019610 20379 tablet_service.cc:1511] Processing CreateTablet for tablet 32e6cbe46a0946898963a7ab5296704b (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:31.021019 20379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32e6cbe46a0946898963a7ab5296704b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.024436 20378 tablet_service.cc:1511] Processing CreateTablet for tablet 9da02bc50d1d4a33be9a722da51a1925 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:31.025820 20378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da02bc50d1d4a33be9a722da51a1925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.032604 20308 tablet_service.cc:1511] Processing CreateTablet for tablet ccd84b3af63b457babd4365151fcdedd (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:31.034020 20308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd84b3af63b457babd4365151fcdedd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.036427 20307 tablet_service.cc:1511] Processing CreateTablet for tablet 9dbca0c06910494d9f02427793ceaef9 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:31.037851 20307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dbca0c06910494d9f02427793ceaef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.040298 20306 tablet_service.cc:1511] Processing CreateTablet for tablet 29a5ffe3fb714214a1388435b5621d0a (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:31.040565 20227 tablet_service.cc:1511] Processing CreateTablet for tablet f98f6e010897428790d76aaac51a5d45 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:31.041704 20306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a5ffe3fb714214a1388435b5621d0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.044685 20305 tablet_service.cc:1511] Processing CreateTablet for tablet df15adc2714c46f496be1c16d7d8bde5 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:31.046499 20305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df15adc2714c46f496be1c16d7d8bde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.048861 20304 tablet_service.cc:1511] Processing CreateTablet for tablet 32e6cbe46a0946898963a7ab5296704b (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:31.050685 20304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32e6cbe46a0946898963a7ab5296704b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.049472 20383 tablet_service.cc:1511] Processing CreateTablet for tablet f98f6e010897428790d76aaac51a5d45 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:31.052884 20226 tablet_service.cc:1511] Processing CreateTablet for tablet c095105d0efd4980bdd0df8e117918d7 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:31.052814 20303 tablet_service.cc:1511] Processing CreateTablet for tablet 9da02bc50d1d4a33be9a722da51a1925 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:31.054767 20303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da02bc50d1d4a33be9a722da51a1925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.056327 20228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da02bc50d1d4a33be9a722da51a1925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.058158 20227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f98f6e010897428790d76aaac51a5d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.064569 20434 tablet_bootstrap.cc:492] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.065030 20226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c095105d0efd4980bdd0df8e117918d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.063468 20383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f98f6e010897428790d76aaac51a5d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.064155 20302 tablet_service.cc:1511] Processing CreateTablet for tablet f98f6e010897428790d76aaac51a5d45 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:31.072002 20302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f98f6e010897428790d76aaac51a5d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.084048 20301 tablet_service.cc:1511] Processing CreateTablet for tablet c095105d0efd4980bdd0df8e117918d7 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:31.085914 20301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c095105d0efd4980bdd0df8e117918d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.089506 20300 tablet_service.cc:1511] Processing CreateTablet for tablet 36d1a4b62d9043048cd90c028120460f (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:31.092387 20381 tablet_service.cc:1511] Processing CreateTablet for tablet c095105d0efd4980bdd0df8e117918d7 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:31.093760 20381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c095105d0efd4980bdd0df8e117918d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.104511 20300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d1a4b62d9043048cd90c028120460f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.110888 20434 tablet_bootstrap.cc:654] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.128897 20299 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4b80faf89f4f9fac6d49cb1173a095 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:31.129511 20381 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4b80faf89f4f9fac6d49cb1173a095 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:31.130324 20299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4b80faf89f4f9fac6d49cb1173a095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.130713 20381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4b80faf89f4f9fac6d49cb1173a095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.116582 20233 tablet_service.cc:1511] Processing CreateTablet for tablet 36d1a4b62d9043048cd90c028120460f (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:31.117688 20380 tablet_service.cc:1511] Processing CreateTablet for tablet 36d1a4b62d9043048cd90c028120460f (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:31.140445 20380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d1a4b62d9043048cd90c028120460f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.143903 20233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d1a4b62d9043048cd90c028120460f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.120344 20226 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4b80faf89f4f9fac6d49cb1173a095 (DEFAULT_TABLE table=test-workload [id=a2fae13df4e24c23b2c52515827f0a41]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:31.149766 20226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4b80faf89f4f9fac6d49cb1173a095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:31.150626 20434 tablet_bootstrap.cc:492] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.151113 20434 ts_tablet_manager.cc:1403] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.087s	user 0.005s	sys 0.007s
I20260709 12:20:31.154028 20434 raft_consensus.cc:359] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.154727 20434 raft_consensus.cc:385] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.155014 20434 raft_consensus.cc:740] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.159533 20434 consensus_queue.cc:260] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.176559 20433 tablet_bootstrap.cc:492] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.193521 20434 ts_tablet_manager.cc:1434] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.042s	user 0.011s	sys 0.003s
I20260709 12:20:31.194254 20268 heartbeater.cc:499] Master 127.19.36.254:37435 was elected leader, sending a full tablet report...
I20260709 12:20:31.194542 20434 tablet_bootstrap.cc:492] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.206225 20436 tablet_bootstrap.cc:492] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.211741 20434 tablet_bootstrap.cc:654] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.212699 20436 tablet_bootstrap.cc:654] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.221944 20434 tablet_bootstrap.cc:492] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.223671 20434 ts_tablet_manager.cc:1403] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.004s
I20260709 12:20:31.225709 20434 raft_consensus.cc:359] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.226261 20434 raft_consensus.cc:385] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.226526 20434 raft_consensus.cc:740] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.227468 20433 tablet_bootstrap.cc:654] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.227154 20434 consensus_queue.cc:260] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.229399 20434 ts_tablet_manager.cc:1434] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:20:31.230453 20434 tablet_bootstrap.cc:492] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.236819 20434 tablet_bootstrap.cc:654] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.240415 20433 tablet_bootstrap.cc:492] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.241025 20433 ts_tablet_manager.cc:1403] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.012s
I20260709 12:20:31.244431 20433 raft_consensus.cc:359] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.244966 20433 raft_consensus.cc:385] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.245224 20433 raft_consensus.cc:740] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.245465 20436 tablet_bootstrap.cc:492] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.245718 20434 tablet_bootstrap.cc:492] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.245926 20436 ts_tablet_manager.cc:1403] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.017s
I20260709 12:20:31.246119 20434 ts_tablet_manager.cc:1403] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.008s
I20260709 12:20:31.245872 20433 consensus_queue.cc:260] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.248265 20418 heartbeater.cc:499] Master 127.19.36.254:37435 was elected leader, sending a full tablet report...
I20260709 12:20:31.248785 20433 ts_tablet_manager.cc:1434] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:20:31.248951 20434 raft_consensus.cc:359] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.249158 20436 raft_consensus.cc:359] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.249718 20433 tablet_bootstrap.cc:492] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.249706 20434 raft_consensus.cc:385] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.249840 20436 raft_consensus.cc:385] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.250290 20436 raft_consensus.cc:740] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.250257 20434 raft_consensus.cc:740] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.251080 20436 consensus_queue.cc:260] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.253806 20344 heartbeater.cc:499] Master 127.19.36.254:37435 was elected leader, sending a full tablet report...
I20260709 12:20:31.251080 20434 consensus_queue.cc:260] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.257187 20433 tablet_bootstrap.cc:654] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.254293 20436 ts_tablet_manager.cc:1434] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:31.260740 20436 tablet_bootstrap.cc:492] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.261405 20434 ts_tablet_manager.cc:1434] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:20:31.262346 20434 tablet_bootstrap.cc:492] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.264519 20437 raft_consensus.cc:493] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.265027 20437 raft_consensus.cc:515] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.267035 20436 tablet_bootstrap.cc:654] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.269671 20437 leader_election.cc:290] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.278750 20436 tablet_bootstrap.cc:492] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.279353 20436 ts_tablet_manager.cc:1403] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 12:20:31.280591 20434 tablet_bootstrap.cc:654] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.282408 20436 raft_consensus.cc:359] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.283239 20436 raft_consensus.cc:385] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.283624 20436 raft_consensus.cc:740] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.286818 20433 tablet_bootstrap.cc:492] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.287328 20433 ts_tablet_manager.cc:1403] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.000s
I20260709 12:20:31.290138 20433 raft_consensus.cc:359] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.289860 20436 consensus_queue.cc:260] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.290901 20433 raft_consensus.cc:385] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.292923 20436 ts_tablet_manager.cc:1434] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 12:20:31.301306 20433 raft_consensus.cc:740] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.302068 20436 tablet_bootstrap.cc:492] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.302227 20433 consensus_queue.cc:260] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.304479 20433 ts_tablet_manager.cc:1434] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 12:20:31.305385 20433 tablet_bootstrap.cc:492] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.308782 20436 tablet_bootstrap.cc:654] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.296505 20434 tablet_bootstrap.cc:492] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.326822 20434 ts_tablet_manager.cc:1403] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.003s
I20260709 12:20:31.328003 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.328613 20393 raft_consensus.cc:2468] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.329746 20206 leader_election.cc:304] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:31.312199 20433 tablet_bootstrap.cc:654] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.309199 20437 raft_consensus.cc:493] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.330883 20434 raft_consensus.cc:359] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.331638 20450 raft_consensus.cc:2804] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.331532 20437 raft_consensus.cc:515] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.331954 20434 raft_consensus.cc:385] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.332247 20450 raft_consensus.cc:493] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.332417 20434 raft_consensus.cc:740] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.332763 20450 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.333738 20437 leader_election.cc:290] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.325620 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.333439 20434 consensus_queue.cc:260] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.336210 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.336442 20434 ts_tablet_manager.cc:1434] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:20:31.336813 20393 raft_consensus.cc:2468] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.337666 20434 tablet_bootstrap.cc:492] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.338083 20206 leader_election.cc:304] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:31.338544 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.339016 20437 raft_consensus.cc:2804] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.339442 20437 raft_consensus.cc:493] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.339838 20437 raft_consensus.cc:3060] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:31.346364 20207 leader_election.cc:343] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.346604 20434 tablet_bootstrap.cc:654] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:31.347400 20207 leader_election.cc:343] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.351975 20437 raft_consensus.cc:515] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.355302 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.355871 20393 raft_consensus.cc:3060] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.358271 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
W20260709 12:20:31.359709 20207 leader_election.cc:343] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.361685 20393 raft_consensus.cc:2468] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 1.
I20260709 12:20:31.361742 20434 tablet_bootstrap.cc:492] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.361737 20433 tablet_bootstrap.cc:492] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.362421 20433 ts_tablet_manager.cc:1403] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.006s
I20260709 12:20:31.362864 20206 leader_election.cc:304] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:31.363408 20437 leader_election.cc:290] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.363633 20434 ts_tablet_manager.cc:1403] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.009s
I20260709 12:20:31.363992 20437 raft_consensus.cc:2804] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.364588 20437 raft_consensus.cc:697] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:31.365206 20433 raft_consensus.cc:359] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.365950 20433 raft_consensus.cc:385] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.366268 20433 raft_consensus.cc:740] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.365666 20437 consensus_queue.cc:237] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.362694 20450 raft_consensus.cc:515] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.367013 20434 raft_consensus.cc:359] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.367003 20433 consensus_queue.cc:260] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.368031 20434 raft_consensus.cc:385] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.368410 20434 raft_consensus.cc:740] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.369693 20433 ts_tablet_manager.cc:1434] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:20:31.369264 20434 consensus_queue.cc:260] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.370795 20433 tablet_bootstrap.cc:492] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.370882 20436 tablet_bootstrap.cc:492] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.379642 20436 ts_tablet_manager.cc:1403] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.078s	user 0.023s	sys 0.031s
I20260709 12:20:31.384897 20436 raft_consensus.cc:359] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.385880 20436 raft_consensus.cc:385] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.386261 20436 raft_consensus.cc:740] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.387365 20436 consensus_queue.cc:260] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.390317 20436 ts_tablet_manager.cc:1434] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:20:31.391607 20436 tablet_bootstrap.cc:492] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.395969 20433 tablet_bootstrap.cc:654] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.399678 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
W20260709 12:20:31.401541 20207 leader_election.cc:343] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.403388 20436 tablet_bootstrap.cc:654] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.403223 20450 leader_election.cc:290] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
W20260709 12:20:31.409196 20206 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:44345, user_credentials={real_user=slave}} blocked reactor thread for 46413.7us
I20260709 12:20:31.411108 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.411432 20434 ts_tablet_manager.cc:1434] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.027s
I20260709 12:20:31.411882 20393 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.412292 20433 tablet_bootstrap.cc:492] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.412679 20434 tablet_bootstrap.cc:492] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.412762 20433 ts_tablet_manager.cc:1403] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.004s
I20260709 12:20:31.415663 20433 raft_consensus.cc:359] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.416383 20433 raft_consensus.cc:385] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.416699 20433 raft_consensus.cc:740] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.417562 20433 consensus_queue.cc:260] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.419764 20434 tablet_bootstrap.cc:654] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.421240 20393 raft_consensus.cc:2468] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 1.
I20260709 12:20:31.422673 20433 ts_tablet_manager.cc:1434] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 12:20:31.423022 20206 leader_election.cc:304] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:31.423683 20433 tablet_bootstrap.cc:492] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.424232 20454 raft_consensus.cc:2804] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.424887 20454 raft_consensus.cc:697] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:31.425944 20454 consensus_queue.cc:237] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.430364 20433 tablet_bootstrap.cc:654] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.441152 20436 tablet_bootstrap.cc:492] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.441841 20436 ts_tablet_manager.cc:1403] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.012s
I20260709 12:20:31.442162 20139 catalog_manager.cc:5697] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 1 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.445653 20436 raft_consensus.cc:359] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.446669 20436 raft_consensus.cc:385] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.446977 20436 raft_consensus.cc:740] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.447883 20436 consensus_queue.cc:260] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.449797 20434 tablet_bootstrap.cc:492] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.450284 20436 ts_tablet_manager.cc:1434] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:20:31.450476 20434 ts_tablet_manager.cc:1403] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.038s	user 0.025s	sys 0.000s
I20260709 12:20:31.451272 20436 tablet_bootstrap.cc:492] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.454532 20434 raft_consensus.cc:359] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.455538 20434 raft_consensus.cc:385] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.455988 20434 raft_consensus.cc:740] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.457397 20436 tablet_bootstrap.cc:654] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.457159 20434 consensus_queue.cc:260] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.460443 20434 ts_tablet_manager.cc:1434] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.007s
I20260709 12:20:31.461442 20434 tablet_bootstrap.cc:492] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.463594 20443 raft_consensus.cc:493] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.464080 20443 raft_consensus.cc:515] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.463786 20139 catalog_manager.cc:5697] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 1 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.466787 20443 leader_election.cc:290] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.468036 20434 tablet_bootstrap.cc:654] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.469868 20436 tablet_bootstrap.cc:492] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.470323 20436 ts_tablet_manager.cc:1403] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.006s
I20260709 12:20:31.473048 20436 raft_consensus.cc:359] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.473740 20436 raft_consensus.cc:385] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.474037 20436 raft_consensus.cc:740] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.474735 20436 consensus_queue.cc:260] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.477020 20436 ts_tablet_manager.cc:1434] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:20:31.477957 20436 tablet_bootstrap.cc:492] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.487963 20434 tablet_bootstrap.cc:492] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.488476 20434 ts_tablet_manager.cc:1403] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 12:20:31.491291 20434 raft_consensus.cc:359] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.492003 20434 raft_consensus.cc:385] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.492311 20434 raft_consensus.cc:740] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.493036 20434 consensus_queue.cc:260] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.496727 20436 tablet_bootstrap.cc:654] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.497359 20433 tablet_bootstrap.cc:492] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.497973 20433 ts_tablet_manager.cc:1403] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.011s
I20260709 12:20:31.495286 20434 ts_tablet_manager.cc:1434] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:20:31.501708 20434 tablet_bootstrap.cc:492] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
W20260709 12:20:31.505688 20189 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:31.501580 20433 raft_consensus.cc:359] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.507125 20433 raft_consensus.cc:385] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.507635 20433 raft_consensus.cc:740] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.509989 20433 consensus_queue.cc:260] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.512085 20434 tablet_bootstrap.cc:654] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.526033 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
W20260709 12:20:31.528532 20281 leader_election.cc:343] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.530367 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.531087 20433 ts_tablet_manager.cc:1434] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.004s
I20260709 12:20:31.531208 20436 tablet_bootstrap.cc:492] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.531749 20436 ts_tablet_manager.cc:1403] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.007s
W20260709 12:20:31.532121 20281 leader_election.cc:343] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.532301 20433 tablet_bootstrap.cc:492] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.535426 20434 tablet_bootstrap.cc:492] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.534977 20436 raft_consensus.cc:359] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.535804 20436 raft_consensus.cc:385] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.535951 20434 ts_tablet_manager.cc:1403] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.006s
I20260709 12:20:31.536140 20436 raft_consensus.cc:740] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.536931 20437 raft_consensus.cc:493] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.536967 20436 consensus_queue.cc:260] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.537598 20437 raft_consensus.cc:515] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.539021 20434 raft_consensus.cc:359] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.539521 20436 ts_tablet_manager.cc:1434] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:20:31.539700 20434 raft_consensus.cc:385] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.539964 20434 raft_consensus.cc:740] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.540500 20436 tablet_bootstrap.cc:492] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.540616 20434 consensus_queue.cc:260] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.542526 20434 ts_tablet_manager.cc:1434] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:20:31.543210 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.543848 20393 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.544714 20437 leader_election.cc:290] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.547159 20436 tablet_bootstrap.cc:654] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.547650 20433 tablet_bootstrap.cc:654] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.549182 20206 leader_election.cc:304] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:31.549804 20281 leader_election.cc:304] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de1ed702f9174763a40084a7c4dd7922; no voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:31.552026 20443 raft_consensus.cc:2749] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.552712 20443 raft_consensus.cc:493] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.553174 20443 raft_consensus.cc:515] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.550804 20437 raft_consensus.cc:2804] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.554391 20437 raft_consensus.cc:493] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.554804 20437 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.556957 20443 leader_election.cc:290] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.559832 20434 tablet_bootstrap.cc:492] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.567116 20434 tablet_bootstrap.cc:654] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.568038 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.568097 20437 raft_consensus.cc:515] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.569084 20319 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.570997 20469 raft_consensus.cc:493] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.571374 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.571949 20433 tablet_bootstrap.cc:492] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.572263 20243 raft_consensus.cc:2393] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de1ed702f9174763a40084a7c4dd7922 in current term 1: Already voted for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in this term.
I20260709 12:20:31.571506 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.571682 20469 raft_consensus.cc:515] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.573289 20433 ts_tablet_manager.cc:1403] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.007s
I20260709 12:20:31.573372 20393 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 0.
I20260709 12:20:31.575573 20469 leader_election.cc:290] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.576124 20436 tablet_bootstrap.cc:492] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.576610 20436 ts_tablet_manager.cc:1403] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.017s
I20260709 12:20:31.576428 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:31.577003 20319 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.577983 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.578603 20243 raft_consensus.cc:2468] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 0.
I20260709 12:20:31.579079 20437 leader_election.cc:290] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.578408 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.579967 20393 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.580734 20281 leader_election.cc:304] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.581135 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.582435 20433 raft_consensus.cc:359] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.584458 20281 leader_election.cc:304] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.585316 20433 raft_consensus.cc:385] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.586891 20433 raft_consensus.cc:740] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.587018 20393 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 1.
I20260709 12:20:31.586417 20319 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 1.
I20260709 12:20:31.588413 20206 leader_election.cc:304] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:31.588610 20469 raft_consensus.cc:2764] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:20:31.589553 20437 raft_consensus.cc:2804] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.590102 20437 raft_consensus.cc:697] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:31.591064 20437 consensus_queue.cc:237] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.592186 20443 raft_consensus.cc:2804] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.592547 20443 raft_consensus.cc:493] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.592870 20443 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:31.594125 20281 leader_election.cc:343] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.595453 20433 consensus_queue.cc:260] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.597640 20436 raft_consensus.cc:359] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.598532 20436 raft_consensus.cc:385] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.598850 20436 raft_consensus.cc:740] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.599761 20436 consensus_queue.cc:260] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.600523 20433 ts_tablet_manager.cc:1434] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.027s	user 0.002s	sys 0.005s
I20260709 12:20:31.596088 20434 tablet_bootstrap.cc:492] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.601511 20434 ts_tablet_manager.cc:1403] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.021s
I20260709 12:20:31.601655 20433 tablet_bootstrap.cc:492] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.604408 20434 raft_consensus.cc:359] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.605108 20434 raft_consensus.cc:385] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.605396 20434 raft_consensus.cc:740] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.606408 20434 consensus_queue.cc:260] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.608886 20434 ts_tablet_manager.cc:1434] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:31.609351 20433 tablet_bootstrap.cc:654] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.609856 20434 tablet_bootstrap.cc:492] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a: Bootstrap starting.
I20260709 12:20:31.618108 20443 raft_consensus.cc:515] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.624689 20443 leader_election.cc:290] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.621877 20139 catalog_manager.cc:5697] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 1 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.625775 20434 tablet_bootstrap.cc:654] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.625905 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.625945 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:31.627177 20243 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:31.632157 20281 leader_election.cc:343] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.635509 20243 raft_consensus.cc:2468] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 1.
I20260709 12:20:31.636749 20281 leader_election.cc:304] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 0e3df13b1d27427f92024ccb926c88a7
I20260709 12:20:31.638085 20472 raft_consensus.cc:2804] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.638525 20436 ts_tablet_manager.cc:1434] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.052s	user 0.012s	sys 0.006s
I20260709 12:20:31.638671 20472 raft_consensus.cc:697] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Becoming Leader. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:31.639721 20436 tablet_bootstrap.cc:492] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.639801 20443 raft_consensus.cc:493] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.639829 20472 consensus_queue.cc:237] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.640535 20443 raft_consensus.cc:515] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.643599 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.644207 20243 raft_consensus.cc:2468] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 0.
I20260709 12:20:31.644361 20433 tablet_bootstrap.cc:492] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.644790 20443 leader_election.cc:290] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.644948 20433 ts_tablet_manager.cc:1403] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.014s
I20260709 12:20:31.645924 20281 leader_election.cc:304] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.647245 20436 tablet_bootstrap.cc:654] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.647900 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.648206 20433 raft_consensus.cc:359] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.649026 20433 raft_consensus.cc:385] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.649374 20433 raft_consensus.cc:740] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
W20260709 12:20:31.649848 20281 leader_election.cc:343] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.650981 20443 raft_consensus.cc:2804] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.651540 20443 raft_consensus.cc:493] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.652009 20443 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.656661 20434 tablet_bootstrap.cc:492] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a: No bootstrap required, opened a new log
I20260709 12:20:31.657275 20434 ts_tablet_manager.cc:1403] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.007s
I20260709 12:20:31.659444 20436 tablet_bootstrap.cc:492] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.659952 20436 ts_tablet_manager.cc:1403] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 12:20:31.661986 20434 raft_consensus.cc:359] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.661623 20139 catalog_manager.cc:5697] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 reported cstate change: term changed from 0 to 1, leader changed from <none> to de1ed702f9174763a40084a7c4dd7922 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "de1ed702f9174763a40084a7c4dd7922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.662760 20434 raft_consensus.cc:385] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.662480 20436 raft_consensus.cc:359] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.663085 20434 raft_consensus.cc:740] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.663224 20436 raft_consensus.cc:385] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.656683 20433 consensus_queue.cc:260] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.663625 20436 raft_consensus.cc:740] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.662286 20443 raft_consensus.cc:515] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.665555 20433 ts_tablet_manager.cc:1434] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:20:31.665853 20443 leader_election.cc:290] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.666617 20436 consensus_queue.cc:260] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.668279 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:31.668197 20434 consensus_queue.cc:260] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.668639 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.666581 20433 tablet_bootstrap.cc:492] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
W20260709 12:20:31.672753 20281 leader_election.cc:343] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.668924 20243 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.674103 20434 ts_tablet_manager.cc:1434] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:20:31.668922 20436 ts_tablet_manager.cc:1434] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:20:31.676050 20436 tablet_bootstrap.cc:492] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.676751 20433 tablet_bootstrap.cc:654] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.682768 20436 tablet_bootstrap.cc:654] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.685626 20243 raft_consensus.cc:2468] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 1.
I20260709 12:20:31.686934 20281 leader_election.cc:304] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 0e3df13b1d27427f92024ccb926c88a7
I20260709 12:20:31.688109 20443 raft_consensus.cc:2804] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.688700 20443 raft_consensus.cc:697] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Becoming Leader. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:31.689440 20433 tablet_bootstrap.cc:492] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.689770 20443 consensus_queue.cc:237] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.690068 20433 ts_tablet_manager.cc:1403] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.001s
I20260709 12:20:31.693279 20433 raft_consensus.cc:359] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.694205 20433 raft_consensus.cc:385] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.694602 20433 raft_consensus.cc:740] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.695526 20433 consensus_queue.cc:260] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.698594 20433 ts_tablet_manager.cc:1434] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:31.699869 20433 tablet_bootstrap.cc:492] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.700788 20472 raft_consensus.cc:493] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.701378 20472 raft_consensus.cc:515] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.704005 20472 leader_election.cc:290] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.704435 20472 raft_consensus.cc:493] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.705009 20472 raft_consensus.cc:515] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.705291 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.707223 20472 leader_election.cc:290] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.707381 20437 raft_consensus.cc:493] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.707604 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.707957 20437 raft_consensus.cc:515] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.708453 20393 raft_consensus.cc:2393] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de1ed702f9174763a40084a7c4dd7922 in current term 1: Already voted for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in this term.
I20260709 12:20:31.708324 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.708966 20243 raft_consensus.cc:2468] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 0.
I20260709 12:20:31.710096 20281 leader_election.cc:304] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.710208 20437 leader_election.cc:290] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.712431 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
W20260709 12:20:31.713758 20206 leader_election.cc:343] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.714059 20433 tablet_bootstrap.cc:654] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.714532 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.715157 20393 raft_consensus.cc:2468] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 0.
I20260709 12:20:31.716516 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.717841 20472 raft_consensus.cc:2804] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.717896 20319 raft_consensus.cc:2468] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.718295 20472 raft_consensus.cc:493] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.716501 20281 leader_election.cc:304] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de1ed702f9174763a40084a7c4dd7922; no voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:31.718644 20472 raft_consensus.cc:3060] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.719141 20469 raft_consensus.cc:3060] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.724223 20207 leader_election.cc:304] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 0e3df13b1d27427f92024ccb926c88a7
I20260709 12:20:31.725425 20442 raft_consensus.cc:493] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.725878 20442 raft_consensus.cc:515] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.726105 20472 raft_consensus.cc:515] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.727641 20469 raft_consensus.cc:2749] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.728048 20472 leader_election.cc:290] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.728471 20442 leader_election.cc:290] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:31.736861 20454 raft_consensus.cc:493] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.737495 20454 raft_consensus.cc:515] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.741473 20454 leader_election.cc:290] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.735611 20437 raft_consensus.cc:2804] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.742460 20437 raft_consensus.cc:493] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.743026 20437 raft_consensus.cc:3060] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.756070 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:31.756712 20243 raft_consensus.cc:3060] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.757387 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.761085 20140 catalog_manager.cc:5697] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 reported cstate change: term changed from 0 to 1, leader changed from <none> to de1ed702f9174763a40084a7c4dd7922 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "de1ed702f9174763a40084a7c4dd7922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.763691 20243 raft_consensus.cc:2468] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 1.
W20260709 12:20:31.763859 20207 leader_election.cc:343] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.765026 20281 leader_election.cc:304] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.765997 20443 raft_consensus.cc:2804] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.766486 20443 raft_consensus.cc:697] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Becoming Leader. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:31.768573 20443 consensus_queue.cc:237] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.769124 20436 tablet_bootstrap.cc:492] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.769691 20436 ts_tablet_manager.cc:1403] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.094s	user 0.030s	sys 0.052s
I20260709 12:20:31.774256 20437 raft_consensus.cc:515] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.776026 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.777151 20393 raft_consensus.cc:2468] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.778842 20436 raft_consensus.cc:359] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.779363 20206 leader_election.cc:304] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:31.779634 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:31.779762 20436 raft_consensus.cc:385] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.780256 20436 raft_consensus.cc:740] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.780272 20319 raft_consensus.cc:3060] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.781268 20436 consensus_queue.cc:260] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.780616 20454 raft_consensus.cc:2804] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.783362 20454 raft_consensus.cc:493] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.783779 20454 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.784327 20436 ts_tablet_manager.cc:1434] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 12:20:31.785501 20436 tablet_bootstrap.cc:492] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922: Bootstrap starting.
I20260709 12:20:31.786697 20469 raft_consensus.cc:493] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.787302 20469 raft_consensus.cc:515] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.790997 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.791117 20469 leader_election.cc:290] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.793560 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.793733 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.794560 20392 raft_consensus.cc:2393] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de1ed702f9174763a40084a7c4dd7922 in current term 1: Already voted for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in this term.
I20260709 12:20:31.794569 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.795823 20393 raft_consensus.cc:3060] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.797150 20319 raft_consensus.cc:2468] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 1.
I20260709 12:20:31.798362 20207 leader_election.cc:304] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.799297 20453 raft_consensus.cc:2804] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.799845 20436 tablet_bootstrap.cc:654] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.799768 20453 raft_consensus.cc:697] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:31.800812 20453 consensus_queue.cc:237] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.803277 20281 leader_election.cc:304] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de1ed702f9174763a40084a7c4dd7922; no voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:31.805158 20469 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.805617 20454 raft_consensus.cc:515] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.806907 20243 raft_consensus.cc:2393] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e3df13b1d27427f92024ccb926c88a7 in current term 1: Already voted for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in this term.
I20260709 12:20:31.812968 20454 leader_election.cc:290] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.814772 20469 raft_consensus.cc:2749] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.815629 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.816882 20392 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.817768 20437 leader_election.cc:290] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.818701 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
W20260709 12:20:31.820122 20207 leader_election.cc:343] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.815932 20393 raft_consensus.cc:2468] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 1.
I20260709 12:20:31.824918 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
W20260709 12:20:31.826195 20206 leader_election.cc:343] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.829299 20437 consensus_queue.cc:1048] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.830205 20140 catalog_manager.cc:5697] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 reported cstate change: term changed from 0 to 1, leader changed from <none> to de1ed702f9174763a40084a7c4dd7922 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "de1ed702f9174763a40084a7c4dd7922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.831898 20433 tablet_bootstrap.cc:492] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.832376 20433 ts_tablet_manager.cc:1403] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.133s	user 0.034s	sys 0.023s
I20260709 12:20:31.832861 20442 raft_consensus.cc:493] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.833397 20442 raft_consensus.cc:515] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.833750 20392 raft_consensus.cc:2468] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 1.
I20260709 12:20:31.835465 20442 leader_election.cc:290] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:31.832387 20437 raft_consensus.cc:493] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.836458 20437 raft_consensus.cc:515] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.838222 20436 tablet_bootstrap.cc:492] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922: No bootstrap required, opened a new log
I20260709 12:20:31.838476 20437 leader_election.cc:290] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:31.838688 20436 ts_tablet_manager.cc:1403] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922: Time spent bootstrapping tablet: real 0.053s	user 0.024s	sys 0.016s
I20260709 12:20:31.839715 20316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.840343 20316 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.841209 20436 raft_consensus.cc:359] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.841563 20207 leader_election.cc:304] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.841874 20436 raft_consensus.cc:385] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.840808 20141 catalog_manager.cc:5697] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 1 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.842128 20436 raft_consensus.cc:740] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.842427 20437 raft_consensus.cc:2804] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.842784 20437 raft_consensus.cc:493] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.842859 20436 consensus_queue.cc:260] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.843233 20437 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.843261 20319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.844180 20319 raft_consensus.cc:2468] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 0.
I20260709 12:20:31.844930 20436 ts_tablet_manager.cc:1434] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:20:31.845750 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.846591 20317 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 0.
I20260709 12:20:31.846843 20206 leader_election.cc:304] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:31.846959 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.846917 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.847434 20357 leader_election.cc:304] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:31.847801 20433 raft_consensus.cc:359] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.848457 20433 raft_consensus.cc:385] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.848731 20442 raft_consensus.cc:2764] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:20:31.847682 20392 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 0.
I20260709 12:20:31.851024 20453 raft_consensus.cc:2804] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.851094 20357 leader_election.cc:304] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.851617 20453 raft_consensus.cc:697] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:31.852095 20442 raft_consensus.cc:2804] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.852479 20442 raft_consensus.cc:493] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.852478 20453 consensus_queue.cc:237] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.853056 20442 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.860949 20442 raft_consensus.cc:515] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.848800 20433 raft_consensus.cc:740] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.864130 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:31.864437 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:31.864655 20433 consensus_queue.cc:260] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.865073 20317 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.867462 20437 raft_consensus.cc:515] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.872370 20243 raft_consensus.cc:2393] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e3df13b1d27427f92024ccb926c88a7 in current term 1: Already voted for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in this term.
I20260709 12:20:31.873714 20317 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 1.
I20260709 12:20:31.874974 20357 leader_election.cc:304] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:31.878073 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:31.879171 20392 raft_consensus.cc:2393] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c3f0782fda4f7c94211aadc5c3cd8a in current term 1: Already voted for candidate 0e3df13b1d27427f92024ccb926c88a7 in this term.
I20260709 12:20:31.880285 20442 leader_election.cc:290] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:31.884438 20437 leader_election.cc:290] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Requested vote from peers 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:31.884621 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:31.886415 20317 raft_consensus.cc:2393] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c3f0782fda4f7c94211aadc5c3cd8a in current term 1: Already voted for candidate 0e3df13b1d27427f92024ccb926c88a7 in this term.
I20260709 12:20:31.887544 20207 leader_election.cc:304] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:31.888440 20437 raft_consensus.cc:2749] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.885252 20442 raft_consensus.cc:2804] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.890193 20442 raft_consensus.cc:697] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 LEADER]: Becoming Leader. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:31.891109 20442 consensus_queue.cc:237] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:31.894477 20490 raft_consensus.cc:493] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.898070 20433 ts_tablet_manager.cc:1434] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.000s
I20260709 12:20:31.898836 20437 raft_consensus.cc:493] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.899102 20433 tablet_bootstrap.cc:492] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7: Bootstrap starting.
I20260709 12:20:31.895005 20490 raft_consensus.cc:515] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.907718 20433 tablet_bootstrap.cc:654] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.907649 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4b80faf89f4f9fac6d49cb1173a095" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:31.908489 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4b80faf89f4f9fac6d49cb1173a095" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.909310 20317 raft_consensus.cc:2468] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 0.
I20260709 12:20:31.911451 20454 consensus_queue.cc:1048] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.911803 20357 leader_election.cc:304] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.912858 20491 raft_consensus.cc:2804] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.913223 20491 raft_consensus.cc:493] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.913558 20491 raft_consensus.cc:3060] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.918205 20490 leader_election.cc:290] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:31.920935 20433 tablet_bootstrap.cc:492] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7: No bootstrap required, opened a new log
I20260709 12:20:31.899614 20437 raft_consensus.cc:515] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.921391 20433 ts_tablet_manager.cc:1403] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.001s
I20260709 12:20:31.921746 20491 raft_consensus.cc:515] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.923027 20437 leader_election.cc:290] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:31.923442 20243 raft_consensus.cc:2468] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 0.
I20260709 12:20:31.924131 20433 raft_consensus.cc:359] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.924777 20433 raft_consensus.cc:385] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.925029 20433 raft_consensus.cc:740] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.925779 20433 consensus_queue.cc:260] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.927747 20433 ts_tablet_manager.cc:1434] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:20:31.927995 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4b80faf89f4f9fac6d49cb1173a095" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:31.928076 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4b80faf89f4f9fac6d49cb1173a095" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3df13b1d27427f92024ccb926c88a7" is_pre_election: true
I20260709 12:20:31.928632 20317 raft_consensus.cc:3060] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.929136 20393 raft_consensus.cc:2393] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c3f0782fda4f7c94211aadc5c3cd8a in current term 1: Already voted for candidate 0e3df13b1d27427f92024ccb926c88a7 in this term.
I20260709 12:20:31.931331 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4b80faf89f4f9fac6d49cb1173a095" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:31.932073 20318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4b80faf89f4f9fac6d49cb1173a095" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:31.928068 20491 leader_election.cc:290] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:31.922262 20140 catalog_manager.cc:5697] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "0e3df13b1d27427f92024ccb926c88a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.934309 20207 leader_election.cc:304] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:31.933171 20243 raft_consensus.cc:3060] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.935791 20317 raft_consensus.cc:2468] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 1.
I20260709 12:20:31.937776 20139 catalog_manager.cc:5697] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 1 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.942202 20243 raft_consensus.cc:2468] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 1.
I20260709 12:20:31.944043 20437 raft_consensus.cc:2749] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.944104 20357 leader_election.cc:304] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:31.945012 20491 raft_consensus.cc:2804] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.945566 20491 raft_consensus.cc:697] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 LEADER]: Becoming Leader. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:31.946393 20491 consensus_queue.cc:237] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:31.953020 20437 consensus_queue.cc:1048] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.962217 20139 catalog_manager.cc:5697] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "0e3df13b1d27427f92024ccb926c88a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.968350 20454 consensus_queue.cc:1048] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.015729 20454 consensus_queue.cc:1048] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.026640 20453 consensus_queue.cc:1048] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.093426 20491 raft_consensus.cc:493] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.093953 20491 raft_consensus.cc:515] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.095877 20491 leader_election.cc:290] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:32.096566 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:32.096911 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:32.097201 20243 raft_consensus.cc:2393] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e3df13b1d27427f92024ccb926c88a7 in current term 1: Already voted for candidate de1ed702f9174763a40084a7c4dd7922 in this term.
I20260709 12:20:32.098338 20356 leader_election.cc:304] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7; no voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:32.099287 20491 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.104040 20491 raft_consensus.cc:2749] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.154630 20392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd"
dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
mode: GRACEFUL
new_leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:34664
I20260709 12:20:32.155285 20392 raft_consensus.cc:606] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 LEADER]: Received request to transfer leadership to TS 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:32.176168 20317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
mode: GRACEFUL
new_leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:42696
I20260709 12:20:32.176728 20317 raft_consensus.cc:606] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Received request to transfer leadership to TS 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:32.195719 20243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:37160
I20260709 12:20:32.196211 20243 raft_consensus.cc:606] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Received request to transfer leadership to TS de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:32.201838 20243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:37160
I20260709 12:20:32.201851 20443 consensus_queue.cc:1048] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.202399 20243 raft_consensus.cc:606] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Received request to transfer leadership to TS de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:32.214048 20469 consensus_queue.cc:1048] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.222030 20472 raft_consensus.cc:993] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922: : Instructing follower 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:32.222556 20469 raft_consensus.cc:1081] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Signalling peer 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:32.224184 20243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:37136
I20260709 12:20:32.224725 20243 raft_consensus.cc:493] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:32.225071 20243 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.229614 20243 raft_consensus.cc:515] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.231731 20243 leader_election.cc:290] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:32.232089 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:32.232300 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:32.232663 20317 raft_consensus.cc:3055] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:32.232869 20392 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.233048 20317 raft_consensus.cc:740] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:32.233940 20317 consensus_queue.cc:260] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.234880 20317 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.238391 20392 raft_consensus.cc:2468] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:32.239403 20206 leader_election.cc:304] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:32.239796 20317 raft_consensus.cc:2468] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:32.240262 20453 raft_consensus.cc:2804] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:32.240718 20453 raft_consensus.cc:697] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:32.241410 20453 consensus_queue.cc:237] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.247325 20453 consensus_queue.cc:1048] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.247735 20140 catalog_manager.cc:5697] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 1 to 2, leader changed from de1ed702f9174763a40084a7c4dd7922 (127.19.36.194) to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 2 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.252214 20469 consensus_queue.cc:1048] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.261036 20317 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.265223 20491 raft_consensus.cc:493] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.265733 20491 raft_consensus.cc:515] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.267232 20392 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.268151 20491 leader_election.cc:290] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:32.272222 20454 consensus_queue.cc:1048] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.270208 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:32.269235 20318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:32.277208 20356 leader_election.cc:304] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7; no voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:32.278322 20491 raft_consensus.cc:2749] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.280849 20454 raft_consensus.cc:993] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:32.279420 20469 consensus_queue.cc:1048] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:20:32.282022 20454 raft_consensus.cc:1081] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Signalling peer de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:32.283862 20318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:42680
I20260709 12:20:32.284433 20318 raft_consensus.cc:493] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:32.284955 20318 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.289898 20491 raft_consensus.cc:493] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.290400 20491 raft_consensus.cc:515] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.291916 20318 raft_consensus.cc:515] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.292532 20491 leader_election.cc:290] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:32.293957 20243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" is_pre_election: true
I20260709 12:20:32.294684 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:32.295256 20242 raft_consensus.cc:3055] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:32.295491 20242 raft_consensus.cc:740] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:32.296000 20242 consensus_queue.cc:260] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.297000 20242 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.297500 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:32.298044 20393 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.297920 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de1ed702f9174763a40084a7c4dd7922" is_pre_election: true
I20260709 12:20:32.299525 20317 raft_consensus.cc:2393] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e3df13b1d27427f92024ccb926c88a7 in current term 1: Already voted for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in this term.
I20260709 12:20:32.300614 20357 leader_election.cc:304] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7; no voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:32.301470 20491 raft_consensus.cc:3060] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.303903 20242 raft_consensus.cc:2468] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 2.
I20260709 12:20:32.294122 20318 leader_election.cc:290] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 2 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:32.305649 20393 raft_consensus.cc:2468] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 2.
I20260709 12:20:32.308418 20281 leader_election.cc:304] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:32.309198 20469 raft_consensus.cc:2804] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:32.309751 20469 raft_consensus.cc:697] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 LEADER]: Becoming Leader. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:32.310201 20491 raft_consensus.cc:2749] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.310559 20469 consensus_queue.cc:237] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.312410 20472 consensus_queue.cc:1048] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.317301 20140 catalog_manager.cc:5697] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 reported cstate change: term changed from 1 to 2, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to de1ed702f9174763a40084a7c4dd7922 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "de1ed702f9174763a40084a7c4dd7922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.323982 20469 consensus_queue.cc:1048] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.362743 20454 consensus_queue.cc:1048] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.369289 20437 raft_consensus.cc:993] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:32.369768 20437 raft_consensus.cc:1081] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Signalling peer de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:32.372316 20317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:42680
I20260709 12:20:32.373543 20317 raft_consensus.cc:493] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:32.373966 20317 raft_consensus.cc:3060] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.375722 20454 consensus_queue.cc:1048] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.380920 20317 raft_consensus.cc:515] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.382872 20317 leader_election.cc:290] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 2 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:32.384478 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:32.385103 20393 raft_consensus.cc:3060] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.385221 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d1a4b62d9043048cd90c028120460f" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:32.385866 20242 raft_consensus.cc:3055] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:32.386214 20242 raft_consensus.cc:740] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:32.387075 20242 consensus_queue.cc:260] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.388348 20242 raft_consensus.cc:3060] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.391114 20393 raft_consensus.cc:2468] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 2.
I20260709 12:20:32.392410 20281 leader_election.cc:304] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:32.393293 20469 raft_consensus.cc:2804] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:32.393838 20469 raft_consensus.cc:697] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 2 LEADER]: Becoming Leader. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:32.394383 20242 raft_consensus.cc:2468] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 2.
I20260709 12:20:32.394703 20469 consensus_queue.cc:237] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:32.401135 20491 consensus_queue.cc:1048] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.401391 20140 catalog_manager.cc:5697] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 reported cstate change: term changed from 1 to 2, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to de1ed702f9174763a40084a7c4dd7922 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "de1ed702f9174763a40084a7c4dd7922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.411612 20442 consensus_queue.cc:1048] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.418723 20494 raft_consensus.cc:993] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7: : Instructing follower 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:32.419205 20494 raft_consensus.cc:1081] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 LEADER]: Signalling peer 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:32.421039 20242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:37152
I20260709 12:20:32.421502 20242 raft_consensus.cc:493] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:32.421815 20242 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.426678 20242 raft_consensus.cc:515] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:32.428428 20242 leader_election.cc:290] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Requested vote from peers 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:32.429229 20393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:32.429661 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:32.429826 20393 raft_consensus.cc:3055] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:32.430198 20317 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.430470 20393 raft_consensus.cc:740] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:32.431341 20393 consensus_queue.cc:260] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:32.432248 20393 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:32.436465 20317 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:32.437918 20207 leader_election.cc:304] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:32.438830 20454 raft_consensus.cc:2804] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:32.439327 20454 raft_consensus.cc:697] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:32.439960 20494 consensus_queue.cc:1048] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.440106 20454 consensus_queue.cc:237] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:32.441256 20393 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:32.451490 20140 catalog_manager.cc:5697] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 1 to 2, leader changed from 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195) to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 2 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.459734 20491 consensus_queue.cc:1048] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.665005 20318 raft_consensus.cc:1275] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.666250 20453 consensus_queue.cc:1048] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:32.674908 20393 raft_consensus.cc:1275] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.676388 20454 consensus_queue.cc:1048] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:32.798086 20393 raft_consensus.cc:1275] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.799371 20469 consensus_queue.cc:1048] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:32.807073 20242 raft_consensus.cc:1275] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.808909 20469 consensus_queue.cc:1048] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:32.849768 20393 raft_consensus.cc:1275] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.850898 20469 consensus_queue.cc:1048] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:32.858655 20242 raft_consensus.cc:1275] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.860033 20469 consensus_queue.cc:1048] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:32.879830 20393 raft_consensus.cc:1275] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.881678 20453 consensus_queue.cc:1048] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:32.889756 20318 raft_consensus.cc:1275] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:32.894222 20454 consensus_queue.cc:1048] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:35.240425 19603 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:20:35.241230 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:35.258817 20242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:37174
I20260709 12:20:35.259387 20242 raft_consensus.cc:606] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Received request to transfer leadership to TS 0e3df13b1d27427f92024ccb926c88a7
I20260709 12:20:35.261706 20242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:37174
I20260709 12:20:35.262176 20242 raft_consensus.cc:606] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Received request to transfer leadership to TS 0e3df13b1d27427f92024ccb926c88a7
I20260709 12:20:35.263154 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:35.263688 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:35.381301 20437 raft_consensus.cc:993] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:35.381805 20523 raft_consensus.cc:1081] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Signalling peer 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:35.383529 20393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a"
dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:34642
I20260709 12:20:35.384020 20393 raft_consensus.cc:493] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:35.384377 20393 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.388711 20393 raft_consensus.cc:515] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:35.390216 20393 leader_election.cc:290] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 2 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:35.391010 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:35.391403 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:35.391640 20242 raft_consensus.cc:3055] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:35.391911 20317 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.392001 20242 raft_consensus.cc:740] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:35.392913 20242 consensus_queue.cc:260] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:35.393850 20242 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.396682 20317 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 2.
I20260709 12:20:35.397768 20357 leader_election.cc:304] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:35.398464 20490 raft_consensus.cc:2804] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:35.399075 20242 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 2.
I20260709 12:20:35.399471 20490 raft_consensus.cc:697] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Becoming Leader. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:35.400338 20490 consensus_queue.cc:237] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:35.407701 20140 catalog_manager.cc:5697] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 reported cstate change: term changed from 1 to 2, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "0e3df13b1d27427f92024ccb926c88a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:35.700052 20437 raft_consensus.cc:993] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:35.700464 20523 raft_consensus.cc:1081] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Signalling peer 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:35.701802 20393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9"
dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:34642
I20260709 12:20:35.702227 20393 raft_consensus.cc:493] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:35.702528 20393 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.706845 20393 raft_consensus.cc:515] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:35.708431 20393 leader_election.cc:290] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 2 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:35.709306 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:35.709471 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:35.709818 20242 raft_consensus.cc:3055] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:35.709996 20317 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.710153 20242 raft_consensus.cc:740] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:35.710955 20242 consensus_queue.cc:260] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:35.711925 20242 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.715256 20317 raft_consensus.cc:2468] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 2.
I20260709 12:20:35.716257 20357 leader_election.cc:304] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:35.716939 20490 raft_consensus.cc:2804] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:35.716917 20242 raft_consensus.cc:2468] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 2.
I20260709 12:20:35.717551 20490 raft_consensus.cc:697] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Becoming Leader. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:35.718384 20490 consensus_queue.cc:237] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:35.725955 20140 catalog_manager.cc:5697] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 reported cstate change: term changed from 1 to 2, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "0e3df13b1d27427f92024ccb926c88a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:35.837153 20242 raft_consensus.cc:1275] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:35.838341 20490 consensus_queue.cc:1048] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:35.847445 20317 raft_consensus.cc:1275] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:35.849520 20524 consensus_queue.cc:1048] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:36.193152 20242 raft_consensus.cc:1275] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:36.194303 20524 consensus_queue.cc:1048] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:36.202704 20317 raft_consensus.cc:1275] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:36.204509 20490 consensus_queue.cc:1048] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:36.264694 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:36.270289 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:36.270627 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:37.271544 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:37.276749 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:37.277120 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:38.277921 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:38.283202 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:38.283689 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:39.284500 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:39.290158 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:39.290665 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:40.292191 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:40.300261 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:40.300746 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:41.301874 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:41.308231 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:41.308653 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:42.309547 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:42.314740 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:42.315135 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:43.316084 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:43.322013 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:43.322479 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:44.323561 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:44.328787 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:44.329213 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:45.360119 19603 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:20:45.367689 20392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9"
dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
mode: GRACEFUL
new_leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:34664
I20260709 12:20:45.368376 20392 raft_consensus.cc:606] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Received request to transfer leadership to TS 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:45.373327 20392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a"
dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
mode: GRACEFUL
new_leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:34664
I20260709 12:20:45.373834 20392 raft_consensus.cc:606] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Received request to transfer leadership to TS 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:45.375384 20490 raft_consensus.cc:993] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7: : Instructing follower 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.375918 20547 raft_consensus.cc:1081] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Signalling peer 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.377703 20242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:37152
I20260709 12:20:45.378232 20242 raft_consensus.cc:493] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.378660 20242 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.385525 20317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
mode: GRACEFUL
new_leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:42696
I20260709 12:20:45.386067 20317 raft_consensus.cc:606] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Received request to transfer leadership to TS 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:45.385758 20242 raft_consensus.cc:515] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.388886 20242 leader_election.cc:290] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 3 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:45.390013 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:45.390583 20392 raft_consensus.cc:3055] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:45.390888 20392 raft_consensus.cc:740] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:45.388877 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a5ffe3fb714214a1388435b5621d0a" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:45.391456 20317 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.391628 20392 consensus_queue.cc:260] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.392829 20392 raft_consensus.cc:3060] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.396591 20317 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 3.
I20260709 12:20:45.397893 20207 leader_election.cc:304] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:45.398041 20392 raft_consensus.cc:2468] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 3.
I20260709 12:20:45.398842 20543 raft_consensus.cc:2804] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:45.399384 20317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
mode: GRACEFUL
new_leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:42696
I20260709 12:20:45.399865 20317 raft_consensus.cc:606] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Received request to transfer leadership to TS 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:45.399976 20543 raft_consensus.cc:697] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:45.401122 20543 consensus_queue.cc:237] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.405512 20317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
mode: GRACEFUL
new_leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:42696
I20260709 12:20:45.406093 20317 raft_consensus.cc:606] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 LEADER]: Received request to transfer leadership to TS 38c3f0782fda4f7c94211aadc5c3cd8a
I20260709 12:20:45.410012 20140 catalog_manager.cc:5697] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 2 to 3, leader changed from 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195) to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 3 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.449656 20544 raft_consensus.cc:993] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922: : Instructing follower 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.450163 20546 raft_consensus.cc:1081] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Signalling peer 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.451611 20242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:37136
I20260709 12:20:45.452101 20242 raft_consensus.cc:493] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.452382 20242 raft_consensus.cc:3060] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.456842 20242 raft_consensus.cc:515] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.458223 20242 leader_election.cc:290] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:45.459029 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:45.459239 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da02bc50d1d4a33be9a722da51a1925" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:45.459600 20317 raft_consensus.cc:3055] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:45.459654 20392 raft_consensus.cc:3060] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.460021 20317 raft_consensus.cc:740] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:45.460827 20317 consensus_queue.cc:260] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.461970 20317 raft_consensus.cc:3060] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.465919 20392 raft_consensus.cc:2468] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:45.467049 20206 leader_election.cc:304] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:45.467762 20543 raft_consensus.cc:2804] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:45.467772 20317 raft_consensus.cc:2468] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:45.468343 20543 raft_consensus.cc:697] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:45.469187 20543 consensus_queue.cc:237] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.475718 20140 catalog_manager.cc:5697] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 1 to 2, leader changed from de1ed702f9174763a40084a7c4dd7922 (127.19.36.194) to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 2 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.534595 20490 raft_consensus.cc:993] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7: : Instructing follower 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.535015 20547 raft_consensus.cc:1081] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Signalling peer 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.536442 20242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:37152
I20260709 12:20:45.536871 20242 raft_consensus.cc:493] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.537161 20242 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.541302 20242 raft_consensus.cc:515] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.542652 20242 leader_election.cc:290] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 3 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:45.543447 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:45.543710 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbca0c06910494d9f02427793ceaef9" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:45.543980 20317 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.544189 20392 raft_consensus.cc:3055] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:45.544498 20392 raft_consensus.cc:740] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:45.545290 20392 consensus_queue.cc:260] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.546473 20392 raft_consensus.cc:3060] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.550365 20317 raft_consensus.cc:2468] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 3.
I20260709 12:20:45.551429 20207 leader_election.cc:304] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c3f0782fda4f7c94211aadc5c3cd8a, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:45.552064 20543 raft_consensus.cc:2804] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:45.552527 20543 raft_consensus.cc:697] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:45.552554 20392 raft_consensus.cc:2468] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 3.
I20260709 12:20:45.553498 20543 consensus_queue.cc:237] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.561154 20140 catalog_manager.cc:5697] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 2 to 3, leader changed from 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195) to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 3 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.605216 20544 raft_consensus.cc:993] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922: : Instructing follower 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.605777 20546 raft_consensus.cc:1081] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 LEADER]: Signalling peer 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.607374 20242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:37136
I20260709 12:20:45.607816 20242 raft_consensus.cc:493] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.608058 20242 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.612210 20242 raft_consensus.cc:515] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.613534 20242 leader_election.cc:290] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 3 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:45.614377 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:45.614602 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c095105d0efd4980bdd0df8e117918d7" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:45.614959 20317 raft_consensus.cc:3055] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:45.615139 20392 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.615314 20317 raft_consensus.cc:740] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:45.616119 20317 consensus_queue.cc:260] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.616992 20317 raft_consensus.cc:3060] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.619683 20392 raft_consensus.cc:2468] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 3.
I20260709 12:20:45.620568 20206 leader_election.cc:304] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:45.621336 20543 raft_consensus.cc:2804] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:45.621599 20317 raft_consensus.cc:2468] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 3.
I20260709 12:20:45.621898 20543 raft_consensus.cc:697] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:45.622750 20543 consensus_queue.cc:237] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.629298 20140 catalog_manager.cc:5697] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 2 to 3, leader changed from de1ed702f9174763a40084a7c4dd7922 (127.19.36.194) to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 3 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.745813 20544 raft_consensus.cc:993] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922: : Instructing follower 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.746299 20546 raft_consensus.cc:1081] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Signalling peer 38c3f0782fda4f7c94211aadc5c3cd8a to start an election
I20260709 12:20:45.747785 20242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
 from {username='slave'} at 127.0.0.1:37136
I20260709 12:20:45.748265 20242 raft_consensus.cc:493] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.748554 20242 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.752791 20242 raft_consensus.cc:515] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.754215 20242 leader_election.cc:290] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Requested vote from peers de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:45.755053 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:45.755300 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:45.755602 20317 raft_consensus.cc:3055] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:45.755825 20392 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.755919 20317 raft_consensus.cc:740] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:45.756664 20317 consensus_queue.cc:260] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.757522 20317 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.761474 20392 raft_consensus.cc:2468] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:45.761686 20317 raft_consensus.cc:2468] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c3f0782fda4f7c94211aadc5c3cd8a in term 2.
I20260709 12:20:45.762539 20206 leader_election.cc:304] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, 38c3f0782fda4f7c94211aadc5c3cd8a; no voters: 
I20260709 12:20:45.763217 20543 raft_consensus.cc:2804] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:45.763628 20543 raft_consensus.cc:697] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Becoming Leader. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:45.764276 20543 consensus_queue.cc:237] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:45.770741 20140 catalog_manager.cc:5697] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a reported cstate change: term changed from 1 to 2, leader changed from de1ed702f9174763a40084a7c4dd7922 (127.19.36.194) to 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193). New cstate: current_term: 2 leader_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.916740 20392 raft_consensus.cc:1275] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:45.918169 20543 consensus_queue.cc:1048] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:45.933333 20317 raft_consensus.cc:1275] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:45.934999 20543 consensus_queue.cc:1048] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:46.019373 20392 raft_consensus.cc:1275] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:46.020663 20543 consensus_queue.cc:1048] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:46.025086 20317 raft_consensus.cc:1275] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:46.026316 20543 consensus_queue.cc:1048] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:46.033427 20318 raft_consensus.cc:1275] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:46.037704 20561 consensus_queue.cc:1048] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:46.042111 20392 raft_consensus.cc:1275] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:46.043916 20565 consensus_queue.cc:1048] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:46.105597 20392 raft_consensus.cc:1275] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:46.106735 20548 consensus_queue.cc:1048] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:46.115447 20317 raft_consensus.cc:1275] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:46.118018 20543 consensus_queue.cc:1048] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:46.252074 20317 raft_consensus.cc:1275] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:46.253340 20543 consensus_queue.cc:1048] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:46.263561 20392 raft_consensus.cc:1275] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Refusing update from remote peer 38c3f0782fda4f7c94211aadc5c3cd8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:46.264973 20543 consensus_queue.cc:1048] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:48.443075 19603 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:20:48.443764 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:48.451048 20242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:37174
I20260709 12:20:48.451705 20242 raft_consensus.cc:606] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Received request to transfer leadership to TS de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:48.454470 20242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:37174
I20260709 12:20:48.455010 20242 raft_consensus.cc:606] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Received request to transfer leadership to TS de1ed702f9174763a40084a7c4dd7922
I20260709 12:20:48.457255 20242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:37174
I20260709 12:20:48.457739 20242 raft_consensus.cc:606] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Received request to transfer leadership to TS 0e3df13b1d27427f92024ccb926c88a7
I20260709 12:20:48.458254 20574 raft_consensus.cc:993] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:48.458787 20573 raft_consensus.cc:1081] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Signalling peer de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:48.460206 20317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:42680
I20260709 12:20:48.460172 20242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45"
dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
mode: GRACEFUL
new_leader_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:37174
I20260709 12:20:48.460767 20317 raft_consensus.cc:493] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.460788 20242 raft_consensus.cc:606] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Received request to transfer leadership to TS 0e3df13b1d27427f92024ccb926c88a7
I20260709 12:20:48.461133 20317 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.461946 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:48.462395 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:48.465989 20317 raft_consensus.cc:515] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.468027 20317 leader_election.cc:290] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 3 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345)
I20260709 12:20:48.468755 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:48.468946 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32e6cbe46a0946898963a7ab5296704b" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:48.469314 20242 raft_consensus.cc:3055] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:48.469492 20392 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.469609 20242 raft_consensus.cc:740] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:48.470345 20242 consensus_queue.cc:260] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.471174 20242 raft_consensus.cc:3060] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.473847 20392 raft_consensus.cc:2468] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 3.
I20260709 12:20:48.475003 20281 leader_election.cc:304] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:48.475677 20242 raft_consensus.cc:2468] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 3.
I20260709 12:20:48.475824 20546 raft_consensus.cc:2804] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:48.477063 20546 raft_consensus.cc:697] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 3 LEADER]: Becoming Leader. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:48.477999 20546 consensus_queue.cc:237] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.485288 20139 catalog_manager.cc:5697] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 reported cstate change: term changed from 2 to 3, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to de1ed702f9174763a40084a7c4dd7922 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "de1ed702f9174763a40084a7c4dd7922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.738888 20574 raft_consensus.cc:993] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:48.739544 20573 raft_consensus.cc:1081] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Signalling peer 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:48.741241 20392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45"
dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:34642
I20260709 12:20:48.741791 20392 raft_consensus.cc:493] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.742136 20392 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.746598 20392 raft_consensus.cc:515] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.748492 20392 leader_election.cc:290] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 3 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:48.749498 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:48.749444 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f98f6e010897428790d76aaac51a5d45" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:48.750156 20317 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.750240 20242 raft_consensus.cc:3055] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:48.750617 20242 raft_consensus.cc:740] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:48.751417 20242 consensus_queue.cc:260] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.752470 20242 raft_consensus.cc:3060] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.753687 20573 raft_consensus.cc:993] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:48.754189 20574 raft_consensus.cc:1081] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Signalling peer 0e3df13b1d27427f92024ccb926c88a7 to start an election
I20260709 12:20:48.755851 20392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5"
dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
 from {username='slave'} at 127.0.0.1:34642
I20260709 12:20:48.755999 20317 raft_consensus.cc:2468] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 3.
I20260709 12:20:48.756364 20392 raft_consensus.cc:493] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.756747 20392 raft_consensus.cc:3060] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:48.757171 20357 leader_election.cc:304] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:48.758009 20547 raft_consensus.cc:2804] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:48.758729 20242 raft_consensus.cc:2468] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 3.
I20260709 12:20:48.759413 20547 raft_consensus.cc:697] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 LEADER]: Becoming Leader. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:48.760352 20547 consensus_queue.cc:237] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.762992 20392 raft_consensus.cc:515] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.765412 20392 leader_election.cc:290] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 2 election: Requested vote from peers 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379), de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751)
I20260709 12:20:48.766280 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:48.766934 20242 raft_consensus.cc:3055] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:48.767145 20317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df15adc2714c46f496be1c16d7d8bde5" candidate_uuid: "0e3df13b1d27427f92024ccb926c88a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
I20260709 12:20:48.767329 20242 raft_consensus.cc:740] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:48.767683 20317 raft_consensus.cc:3060] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:48.768114 20242 consensus_queue.cc:260] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.769074 20242 raft_consensus.cc:3060] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:48.768851 20139 catalog_manager.cc:5697] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 reported cstate change: term changed from 2 to 3, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "0e3df13b1d27427f92024ccb926c88a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.773041 20317 raft_consensus.cc:2468] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 2.
I20260709 12:20:48.773990 20357 leader_election.cc:304] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:48.774757 20547 raft_consensus.cc:2804] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:48.774979 20242 raft_consensus.cc:2468] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e3df13b1d27427f92024ccb926c88a7 in term 2.
I20260709 12:20:48.775319 20547 raft_consensus.cc:697] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Becoming Leader. State: Replica: 0e3df13b1d27427f92024ccb926c88a7, State: Running, Role: LEADER
I20260709 12:20:48.776160 20547 consensus_queue.cc:237] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } }
I20260709 12:20:48.783797 20139 catalog_manager.cc:5697] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 reported cstate change: term changed from 1 to 2, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "0e3df13b1d27427f92024ccb926c88a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.849195 20573 raft_consensus.cc:993] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a: : Instructing follower de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:48.849629 20574 raft_consensus.cc:1081] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Signalling peer de1ed702f9174763a40084a7c4dd7922 to start an election
I20260709 12:20:48.850987 20317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd"
dest_uuid: "de1ed702f9174763a40084a7c4dd7922"
 from {username='slave'} at 127.0.0.1:42680
I20260709 12:20:48.851444 20317 raft_consensus.cc:493] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.851715 20317 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.855782 20317 raft_consensus.cc:515] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:48.857142 20317 leader_election.cc:290] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 3 election: Requested vote from peers 0e3df13b1d27427f92024ccb926c88a7 (127.19.36.195:44345), 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379)
I20260709 12:20:48.857954 20392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e3df13b1d27427f92024ccb926c88a7"
I20260709 12:20:48.858173 20242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd84b3af63b457babd4365151fcdedd" candidate_uuid: "de1ed702f9174763a40084a7c4dd7922" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a"
I20260709 12:20:48.858570 20392 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.858698 20242 raft_consensus.cc:3055] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:48.859081 20242 raft_consensus.cc:740] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c3f0782fda4f7c94211aadc5c3cd8a, State: Running, Role: LEADER
I20260709 12:20:48.859866 20242 consensus_queue.cc:260] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:48.860836 20242 raft_consensus.cc:3060] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:48.860872 20391 raft_consensus.cc:1275] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:48.862242 20546 consensus_queue.cc:1048] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:48.864459 20392 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 3.
I20260709 12:20:48.865402 20281 leader_election.cc:304] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3df13b1d27427f92024ccb926c88a7, de1ed702f9174763a40084a7c4dd7922; no voters: 
I20260709 12:20:48.866163 20546 raft_consensus.cc:2804] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:48.866693 20546 raft_consensus.cc:697] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 3 LEADER]: Becoming Leader. State: Replica: de1ed702f9174763a40084a7c4dd7922, State: Running, Role: LEADER
I20260709 12:20:48.867070 20242 raft_consensus.cc:2468] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1ed702f9174763a40084a7c4dd7922 in term 3.
I20260709 12:20:48.867630 20546 consensus_queue.cc:237] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } }
I20260709 12:20:48.876579 20139 catalog_manager.cc:5697] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 reported cstate change: term changed from 2 to 3, leader changed from 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193) to de1ed702f9174763a40084a7c4dd7922 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "de1ed702f9174763a40084a7c4dd7922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.880704 20242 raft_consensus.cc:1275] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:48.882189 20546 consensus_queue.cc:1048] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:49.185619 20317 raft_consensus.cc:1275] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:49.186990 20547 consensus_queue.cc:1048] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:49.195413 20242 raft_consensus.cc:1275] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:49.196762 20547 consensus_queue.cc:1048] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:49.252355 20242 raft_consensus.cc:1275] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:49.253607 20547 consensus_queue.cc:1048] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:49.261664 20317 raft_consensus.cc:1275] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Refusing update from remote peer 0e3df13b1d27427f92024ccb926c88a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:49.263957 20547 consensus_queue.cc:1048] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de1ed702f9174763a40084a7c4dd7922" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:49.276424 20391 raft_consensus.cc:1275] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:49.277653 20575 consensus_queue.cc:1048] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3df13b1d27427f92024ccb926c88a7" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 44345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:49.286324 20242 raft_consensus.cc:1275] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Refusing update from remote peer de1ed702f9174763a40084a7c4dd7922: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:49.288292 20546 consensus_queue.cc:1048] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c3f0782fda4f7c94211aadc5c3cd8a" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:49.463395 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:49.469228 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:49.469743 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:50.471022 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:50.478314 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:50.478816 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:51.479933 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:51.485570 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:51.486001 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:52.487025 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:52.493232 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:52.493772 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:53.494771 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:53.500231 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:53.500717 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:54.501683 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:54.507553 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:54.507995 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:55.508848 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:55.514046 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:55.514483 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:56.515578 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:56.521003 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:56.521464 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:57.522434 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:57.527732 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:57.528163 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:58.559582 19603 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:20:58.560144 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
I20260709 12:20:58.596179 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:58.596983 19603 tablet_replica.cc:333] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.597692 19603 raft_consensus.cc:2243] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.598621 19603 raft_consensus.cc:2272] T 9da02bc50d1d4a33be9a722da51a1925 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.600723 19603 tablet_replica.cc:333] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.601353 19603 raft_consensus.cc:2243] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.601836 19603 raft_consensus.cc:2272] T f98f6e010897428790d76aaac51a5d45 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.604203 19603 tablet_replica.cc:333] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.604869 19603 raft_consensus.cc:2243] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.605237 19603 raft_consensus.cc:2272] T 0b4b80faf89f4f9fac6d49cb1173a095 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.607268 19603 tablet_replica.cc:333] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.607960 19603 raft_consensus.cc:2243] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.608675 19603 raft_consensus.cc:2272] T c095105d0efd4980bdd0df8e117918d7 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.610858 19603 tablet_replica.cc:333] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.611732 19603 raft_consensus.cc:2243] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.612293 19603 raft_consensus.cc:2272] T 32e6cbe46a0946898963a7ab5296704b P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.614034 19603 tablet_replica.cc:333] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.614519 19603 raft_consensus.cc:2243] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.614949 19603 raft_consensus.cc:2272] T ccd84b3af63b457babd4365151fcdedd P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.616616 19603 tablet_replica.cc:333] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.617136 19603 raft_consensus.cc:2243] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.617906 19603 raft_consensus.cc:2272] T 29a5ffe3fb714214a1388435b5621d0a P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.619923 19603 tablet_replica.cc:333] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.620515 19603 raft_consensus.cc:2243] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.621260 19603 raft_consensus.cc:2272] T 9dbca0c06910494d9f02427793ceaef9 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.622939 19603 tablet_replica.cc:333] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.623464 19603 raft_consensus.cc:2243] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.623910 19603 raft_consensus.cc:2272] T 36d1a4b62d9043048cd90c028120460f P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.625569 19603 tablet_replica.cc:333] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a: stopping tablet replica
I20260709 12:20:58.626008 19603 raft_consensus.cc:2243] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.626408 19603 raft_consensus.cc:2272] T df15adc2714c46f496be1c16d7d8bde5 P 38c3f0782fda4f7c94211aadc5c3cd8a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.656994 20356 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.193:33379: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260709 12:20:58.658612 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
W20260709 12:20:58.661356 20356 consensus_peers.cc:597] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 -> Peer 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379): Couldn't send request to peer 38c3f0782fda4f7c94211aadc5c3cd8a. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:33379: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.678787 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
W20260709 12:20:58.691165 20356 consensus_peers.cc:597] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 -> Peer 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379): Couldn't send request to peer 38c3f0782fda4f7c94211aadc5c3cd8a. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:33379: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.707466 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:58.708084 19603 tablet_replica.cc:333] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.708756 19603 raft_consensus.cc:2243] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.709275 19603 raft_consensus.cc:2272] T c095105d0efd4980bdd0df8e117918d7 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.711702 19603 tablet_replica.cc:333] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.712374 19603 raft_consensus.cc:2243] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.712795 19603 raft_consensus.cc:2272] T 0b4b80faf89f4f9fac6d49cb1173a095 P de1ed702f9174763a40084a7c4dd7922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.714543 19603 tablet_replica.cc:333] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.715003 19603 raft_consensus.cc:2243] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.715806 19603 raft_consensus.cc:2272] T ccd84b3af63b457babd4365151fcdedd P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.717516 19603 tablet_replica.cc:333] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.718008 19603 raft_consensus.cc:2243] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.718395 19603 raft_consensus.cc:2272] T 9da02bc50d1d4a33be9a722da51a1925 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.720253 19603 tablet_replica.cc:333] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.720703 19603 raft_consensus.cc:2243] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.721052 19603 raft_consensus.cc:2272] T 9dbca0c06910494d9f02427793ceaef9 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.722720 19603 tablet_replica.cc:333] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.723140 19603 raft_consensus.cc:2243] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.723608 19603 raft_consensus.cc:2272] T 29a5ffe3fb714214a1388435b5621d0a P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.725175 19603 tablet_replica.cc:333] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.725593 19603 raft_consensus.cc:2243] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.726312 19603 raft_consensus.cc:2272] T 36d1a4b62d9043048cd90c028120460f P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.727986 19603 tablet_replica.cc:333] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.728406 19603 raft_consensus.cc:2243] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.728775 19603 raft_consensus.cc:2272] T df15adc2714c46f496be1c16d7d8bde5 P de1ed702f9174763a40084a7c4dd7922 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.730329 19603 tablet_replica.cc:333] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.730770 19603 raft_consensus.cc:2243] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.731160 19603 raft_consensus.cc:2272] T f98f6e010897428790d76aaac51a5d45 P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.732959 19603 tablet_replica.cc:333] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922: stopping tablet replica
I20260709 12:20:58.733347 19603 raft_consensus.cc:2243] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.734055 19603 raft_consensus.cc:2272] T 32e6cbe46a0946898963a7ab5296704b P de1ed702f9174763a40084a7c4dd7922 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.762310 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
W20260709 12:20:58.778968 20356 consensus_peers.cc:597] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 -> Peer 38c3f0782fda4f7c94211aadc5c3cd8a (127.19.36.193:33379): Couldn't send request to peer 38c3f0782fda4f7c94211aadc5c3cd8a. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:33379: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.783844 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
I20260709 12:20:58.811725 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:58.812351 19603 tablet_replica.cc:333] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.812997 19603 raft_consensus.cc:2243] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.813827 19603 raft_consensus.cc:2272] T f98f6e010897428790d76aaac51a5d45 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.815714 19603 tablet_replica.cc:333] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.816295 19603 raft_consensus.cc:2243] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.816763 19603 raft_consensus.cc:2272] T 32e6cbe46a0946898963a7ab5296704b P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.818441 19603 tablet_replica.cc:333] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.818955 19603 raft_consensus.cc:2243] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.819420 19603 raft_consensus.cc:2272] T 9da02bc50d1d4a33be9a722da51a1925 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.821002 19603 tablet_replica.cc:333] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.821476 19603 raft_consensus.cc:2243] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.822218 19603 raft_consensus.cc:2272] T 0b4b80faf89f4f9fac6d49cb1173a095 P 0e3df13b1d27427f92024ccb926c88a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.823985 19603 tablet_replica.cc:333] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.824496 19603 raft_consensus.cc:2243] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.824937 19603 raft_consensus.cc:2272] T c095105d0efd4980bdd0df8e117918d7 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.826514 19603 tablet_replica.cc:333] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.826995 19603 raft_consensus.cc:2243] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.827713 19603 raft_consensus.cc:2272] T 9dbca0c06910494d9f02427793ceaef9 P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.830230 19603 tablet_replica.cc:333] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.830955 19603 raft_consensus.cc:2243] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.831550 19603 raft_consensus.cc:2272] T 29a5ffe3fb714214a1388435b5621d0a P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.832912 20357 consensus_peers.cc:597] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 -> Peer de1ed702f9174763a40084a7c4dd7922 (127.19.36.194:38751): Couldn't send request to peer de1ed702f9174763a40084a7c4dd7922. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:38751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.833709 19603 tablet_replica.cc:333] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.834229 19603 raft_consensus.cc:2243] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.834661 19603 raft_consensus.cc:2272] T 36d1a4b62d9043048cd90c028120460f P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.836410 19603 tablet_replica.cc:333] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.836871 19603 raft_consensus.cc:2243] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.837581 19603 raft_consensus.cc:2272] T df15adc2714c46f496be1c16d7d8bde5 P 0e3df13b1d27427f92024ccb926c88a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.839278 19603 tablet_replica.cc:333] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7: stopping tablet replica
I20260709 12:20:58.839711 19603 raft_consensus.cc:2243] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.840133 19603 raft_consensus.cc:2272] T ccd84b3af63b457babd4365151fcdedd P 0e3df13b1d27427f92024ccb926c88a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.867969 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
I20260709 12:20:58.888870 19603 master.cc:562] Master@127.19.36.254:37435 shutting down...
I20260709 12:20:59.037683 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:59.038347 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:59.038704 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P c91235ff0c5a409a8e42f8d25fce2d01: stopping tablet replica
I20260709 12:20:59.058452 19603 master.cc:584] Master@127.19.36.254:37435 shutdown complete.
0e3df13b1d27427f92024ccb926c88a7  1
38c3f0782fda4f7c94211aadc5c3cd8a  5
de1ed702f9174763a40084a7c4dd7922  4
0e3df13b1d27427f92024ccb926c88a7  3
38c3f0782fda4f7c94211aadc5c3cd8a  3
de1ed702f9174763a40084a7c4dd7922  4
0e3df13b1d27427f92024ccb926c88a7  1
38c3f0782fda4f7c94211aadc5c3cd8a  8
de1ed702f9174763a40084a7c4dd7922  1
0e3df13b1d27427f92024ccb926c88a7  3
38c3f0782fda4f7c94211aadc5c3cd8a  4
de1ed702f9174763a40084a7c4dd7922  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28741 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:20:59.104471 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:46039
I20260709 12:20:59.105607 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.111310 20609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.111318 20608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.112692 19603 server_base.cc:1061] running on GCE node
W20260709 12:20:59.113087 20611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.114042 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:59.114285 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.114401 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599659114388 us; error 0 us; skew 500 ppm
I20260709 12:20:59.114850 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.117229 19603 webserver.cc:533] Webserver started at http://127.19.36.254:34841/ using document root <none> and password file <none>
I20260709 12:20:59.117657 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.117807 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.118038 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.119139 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "9f77a9352772433bb5809bf72fd4f0ee"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.124109 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:59.127404 20616 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.128161 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:20:59.128468 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "9f77a9352772433bb5809bf72fd4f0ee"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.128777 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.140130 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.141357 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.182158 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:46039
I20260709 12:20:59.182447 20667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:46039 every 8 connection(s)
I20260709 12:20:59.186995 20668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.197281 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Bootstrap starting.
I20260709 12:20:59.202009 20668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.206324 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: No bootstrap required, opened a new log
I20260709 12:20:59.208544 20668 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f77a9352772433bb5809bf72fd4f0ee" member_type: VOTER }
I20260709 12:20:59.209031 20668 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.209261 20668 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f77a9352772433bb5809bf72fd4f0ee, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.209884 20668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f77a9352772433bb5809bf72fd4f0ee" member_type: VOTER }
I20260709 12:20:59.210356 20668 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:59.210597 20668 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:59.210872 20668 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.215780 20668 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f77a9352772433bb5809bf72fd4f0ee" member_type: VOTER }
I20260709 12:20:59.216640 20668 leader_election.cc:304] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f77a9352772433bb5809bf72fd4f0ee; no voters: 
I20260709 12:20:59.217880 20668 leader_election.cc:290] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:59.218290 20671 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:59.219774 20671 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 1 LEADER]: Becoming Leader. State: Replica: 9f77a9352772433bb5809bf72fd4f0ee, State: Running, Role: LEADER
I20260709 12:20:59.220767 20671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f77a9352772433bb5809bf72fd4f0ee" member_type: VOTER }
I20260709 12:20:59.221261 20668 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:59.226809 20672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f77a9352772433bb5809bf72fd4f0ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f77a9352772433bb5809bf72fd4f0ee" member_type: VOTER } }
I20260709 12:20:59.227010 20673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f77a9352772433bb5809bf72fd4f0ee. Latest consensus state: current_term: 1 leader_uuid: "9f77a9352772433bb5809bf72fd4f0ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f77a9352772433bb5809bf72fd4f0ee" member_type: VOTER } }
I20260709 12:20:59.227784 20672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:59.227842 20673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:59.237567 20679 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:59.246749 20679 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:59.247786 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:59.256610 20679 catalog_manager.cc:1383] Generated new cluster ID: 1749297288e34cb6bf44f64def4a2931
I20260709 12:20:59.256930 20679 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:59.273609 20679 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:59.274966 20679 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:59.289613 20679 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Generated new TSK 0
I20260709 12:20:59.290323 20679 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:59.314301 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.320441 20689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.321609 20690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.324503 19603 server_base.cc:1061] running on GCE node
W20260709 12:20:59.324432 20692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.325665 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:59.325855 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.325976 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599659325965 us; error 0 us; skew 500 ppm
I20260709 12:20:59.326437 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.328794 19603 webserver.cc:533] Webserver started at http://127.19.36.193:37409/ using document root <none> and password file <none>
I20260709 12:20:59.329210 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.329360 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.329571 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.330644 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "49323f66286f4798b268f216bb62b3be"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.335016 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 12:20:59.338186 20697 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.338960 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:20:59.339268 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "49323f66286f4798b268f216bb62b3be"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.339534 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.352095 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.353488 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.355043 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:59.357383 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:59.357637 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.357892 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:59.358088 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.400897 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:34333
I20260709 12:20:59.401027 20759 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:34333 every 8 connection(s)
I20260709 12:20:59.406054 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.413373 20764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.414170 20765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.417575 20767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.419476 19603 server_base.cc:1061] running on GCE node
I20260709 12:20:59.420912 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:59.421212 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.421443 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599659421426 us; error 0 us; skew 500 ppm
I20260709 12:20:59.421514 20760 heartbeater.cc:344] Connected to a master server at 127.19.36.254:46039
I20260709 12:20:59.421983 20760 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.422287 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.423048 20760 heartbeater.cc:507] Master 127.19.36.254:46039 requested a full tablet report, sending...
I20260709 12:20:59.425275 19603 webserver.cc:533] Webserver started at http://127.19.36.194:34523/ using document root <none> and password file <none>
I20260709 12:20:59.425796 20633 ts_manager.cc:194] Registered new tserver with Master: 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
I20260709 12:20:59.426002 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.426290 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.426651 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.428344 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "657bc62d20374e539b23641715f22745"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.428473 20633 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36964
I20260709 12:20:59.434459 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:59.438431 20772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.439443 19603 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:20:59.439720 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "657bc62d20374e539b23641715f22745"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.439983 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.461050 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.462352 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.464022 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:59.466387 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:59.466647 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.466914 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:59.467114 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.509624 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:43085
I20260709 12:20:59.509714 20834 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:43085 every 8 connection(s)
I20260709 12:20:59.514678 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.523379 20838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.525141 20839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.528717 20835 heartbeater.cc:344] Connected to a master server at 127.19.36.254:46039
I20260709 12:20:59.529238 20835 heartbeater.cc:461] Registering TS with master...
W20260709 12:20:59.530376 20841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.530406 19603 server_base.cc:1061] running on GCE node
I20260709 12:20:59.530447 20835 heartbeater.cc:507] Master 127.19.36.254:46039 requested a full tablet report, sending...
I20260709 12:20:59.531819 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:59.532078 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.532323 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599659532304 us; error 0 us; skew 500 ppm
I20260709 12:20:59.533181 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.533371 20633 ts_manager.cc:194] Registered new tserver with Master: 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:20:59.535454 20633 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36968
I20260709 12:20:59.536258 19603 webserver.cc:533] Webserver started at http://127.19.36.195:36979/ using document root <none> and password file <none>
I20260709 12:20:59.536952 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.537206 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.537570 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.538928 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "116023b4c0c94bfea737fc238bda3838"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.543817 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:59.547125 20846 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.547955 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:20:59.548285 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "116023b4c0c94bfea737fc238bda3838"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-hfcn"
I20260709 12:20:59.548606 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.570267 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.571569 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.573086 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:59.575502 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:59.575744 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.576000 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:59.576182 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.615163 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:45387
I20260709 12:20:59.615317 20908 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:45387 every 8 connection(s)
I20260709 12:20:59.629133 20909 heartbeater.cc:344] Connected to a master server at 127.19.36.254:46039
I20260709 12:20:59.629503 20909 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.630240 20909 heartbeater.cc:507] Master 127.19.36.254:46039 requested a full tablet report, sending...
I20260709 12:20:59.632270 20633 ts_manager.cc:194] Registered new tserver with Master: 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:20:59.632774 19603 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014360326s
I20260709 12:20:59.633556 20633 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36978
I20260709 12:20:59.659332 20633 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36982:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:59.666181 20633 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:59.883177 20874 tablet_service.cc:1511] Processing CreateTablet for tablet d82740a2c094456a91cf0e411dd72159 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:59.884713 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82740a2c094456a91cf0e411dd72159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.884491 20870 tablet_service.cc:1511] Processing CreateTablet for tablet 6a458432a57843e4af30e5d9af5dfcfe (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:59.885839 20870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a458432a57843e4af30e5d9af5dfcfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.886389 20867 tablet_service.cc:1511] Processing CreateTablet for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:59.886807 20873 tablet_service.cc:1511] Processing CreateTablet for tablet 62396122903848d4a49e6d637c5c8b49 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:59.887702 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.889096 20872 tablet_service.cc:1511] Processing CreateTablet for tablet 0f517f5d84b3437eb2fd2ae144fc6380 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:59.891595 20871 tablet_service.cc:1511] Processing CreateTablet for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:59.892338 20868 tablet_service.cc:1511] Processing CreateTablet for tablet ec190d02960a4e88af0807817b789e6c (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:59.893092 20872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f517f5d84b3437eb2fd2ae144fc6380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.894235 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 75e78bb14a7646c58f2f948b2d33b2cf (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:59.894513 20869 tablet_service.cc:1511] Processing CreateTablet for tablet d5c70f45aef146f4bcb47f1498221b99 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:59.895610 20871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.893093 20866 tablet_service.cc:1511] Processing CreateTablet for tablet c0b73d72c3ae4b17abbf64b777ddb8c2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:59.896217 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec190d02960a4e88af0807817b789e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.896795 20866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b73d72c3ae4b17abbf64b777ddb8c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.897400 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5c70f45aef146f4bcb47f1498221b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.898039 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e78bb14a7646c58f2f948b2d33b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.898627 20873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62396122903848d4a49e6d637c5c8b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.927867 20924 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:20:59.930505 20865 tablet_service.cc:1511] Processing CreateTablet for tablet ba4e3bf73fae46119cf926d5af0782a6 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:59.936395 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4e3bf73fae46119cf926d5af0782a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.952633 20924 tablet_bootstrap.cc:654] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.951300 20867 tablet_service.cc:1511] Processing CreateTablet for tablet 38d64dd1339c4abfb4ef05669402e801 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:59.955600 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d64dd1339c4abfb4ef05669402e801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.950754 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:59.956672 20868 tablet_service.cc:1511] Processing CreateTablet for tablet d5374e2302374917828294b0a4e3313a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:59.957226 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.957922 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5374e2302374917828294b0a4e3313a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.959820 20725 tablet_service.cc:1511] Processing CreateTablet for tablet d82740a2c094456a91cf0e411dd72159 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:59.961308 20725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82740a2c094456a91cf0e411dd72159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.966585 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 0a4e4474d2fd45438665993211ae862a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:59.971594 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a4e4474d2fd45438665993211ae862a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.970504 20924 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:20:59.972240 20924 ts_tablet_manager.cc:1403] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.000s
I20260709 12:20:59.975135 20924 raft_consensus.cc:359] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:20:59.975992 20924 raft_consensus.cc:385] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.976363 20924 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.977182 20924 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:20:59.988360 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd857e30c3240c5a4fdd11bc0602c00 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:59.989091 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 62396122903848d4a49e6d637c5c8b49 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:59.989764 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd857e30c3240c5a4fdd11bc0602c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.990399 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62396122903848d4a49e6d637c5c8b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.992483 20723 tablet_service.cc:1511] Processing CreateTablet for tablet 0f517f5d84b3437eb2fd2ae144fc6380 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:59.994395 20720 tablet_service.cc:1511] Processing CreateTablet for tablet d5c70f45aef146f4bcb47f1498221b99 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:59.994779 20722 tablet_service.cc:1511] Processing CreateTablet for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:59.996767 20719 tablet_service.cc:1511] Processing CreateTablet for tablet ec190d02960a4e88af0807817b789e6c (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:59.996992 20721 tablet_service.cc:1511] Processing CreateTablet for tablet 6a458432a57843e4af30e5d9af5dfcfe (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:59.999177 20718 tablet_service.cc:1511] Processing CreateTablet for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:00.001286 20717 tablet_service.cc:1511] Processing CreateTablet for tablet c0b73d72c3ae4b17abbf64b777ddb8c2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:00.003378 20716 tablet_service.cc:1511] Processing CreateTablet for tablet 75e78bb14a7646c58f2f948b2d33b2cf (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:00.005908 20723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f517f5d84b3437eb2fd2ae144fc6380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.010305 20867 tablet_service.cc:1511] Processing CreateTablet for tablet 3199c6fc2e3441849ac7dd78244a3a18 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:00.012040 20722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.016248 20718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.018303 20866 tablet_service.cc:1511] Processing CreateTablet for tablet c5ef54d60e4646b68d43519ad3fffa6d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:00.019587 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a458432a57843e4af30e5d9af5dfcfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.015491 20868 tablet_service.cc:1511] Processing CreateTablet for tablet 4415902e5b30489691f67f4a635305b4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:00.011664 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3199c6fc2e3441849ac7dd78244a3a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.026432 20866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ef54d60e4646b68d43519ad3fffa6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.028792 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4415902e5b30489691f67f4a635305b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.037914 20873 tablet_service.cc:1511] Processing CreateTablet for tablet 981a4d0731e940b79eccfb33d1928fb4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:00.039309 20873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981a4d0731e940b79eccfb33d1928fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.048697 20795 tablet_service.cc:1511] Processing CreateTablet for tablet d5c70f45aef146f4bcb47f1498221b99 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:00.049437 20798 tablet_service.cc:1511] Processing CreateTablet for tablet 0f517f5d84b3437eb2fd2ae144fc6380 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:00.050119 20795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5c70f45aef146f4bcb47f1498221b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.050725 20798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f517f5d84b3437eb2fd2ae144fc6380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.053028 20799 tablet_service.cc:1511] Processing CreateTablet for tablet 62396122903848d4a49e6d637c5c8b49 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:00.053594 20800 tablet_service.cc:1511] Processing CreateTablet for tablet d82740a2c094456a91cf0e411dd72159 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:00.054050 20792 tablet_service.cc:1511] Processing CreateTablet for tablet c0b73d72c3ae4b17abbf64b777ddb8c2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:00.053028 20794 tablet_service.cc:1511] Processing CreateTablet for tablet ec190d02960a4e88af0807817b789e6c (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:00.054545 20799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62396122903848d4a49e6d637c5c8b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.055841 20796 tablet_service.cc:1511] Processing CreateTablet for tablet 6a458432a57843e4af30e5d9af5dfcfe (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:00.057734 20797 tablet_service.cc:1511] Processing CreateTablet for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:00.053056 20793 tablet_service.cc:1511] Processing CreateTablet for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:00.060003 20800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82740a2c094456a91cf0e411dd72159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.064172 20716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e78bb14a7646c58f2f948b2d33b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.072846 20791 tablet_service.cc:1511] Processing CreateTablet for tablet 75e78bb14a7646c58f2f948b2d33b2cf (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:00.074424 20791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e78bb14a7646c58f2f948b2d33b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.075405 20872 tablet_service.cc:1511] Processing CreateTablet for tablet a32323ce10e140bb895085f97c97fa46 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:00.076084 20794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec190d02960a4e88af0807817b789e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.076680 20872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a32323ce10e140bb895085f97c97fa46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.079372 20793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.080159 20871 tablet_service.cc:1511] Processing CreateTablet for tablet b7040eacc899473abc05a913812e82a4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:00.081449 20871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7040eacc899473abc05a913812e82a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.084278 20792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b73d72c3ae4b17abbf64b777ddb8c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.087500 20717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b73d72c3ae4b17abbf64b777ddb8c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.081455 20723 tablet_service.cc:1511] Processing CreateTablet for tablet 38d64dd1339c4abfb4ef05669402e801 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:00.088506 20723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d64dd1339c4abfb4ef05669402e801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.088506 20797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.090256 20720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5c70f45aef146f4bcb47f1498221b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.091094 20796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a458432a57843e4af30e5d9af5dfcfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.075112 20725 tablet_service.cc:1511] Processing CreateTablet for tablet ba4e3bf73fae46119cf926d5af0782a6 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:00.092077 20725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4e3bf73fae46119cf926d5af0782a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.097234 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:00.098174 20909 heartbeater.cc:499] Master 127.19.36.254:46039 was elected leader, sending a full tablet report...
I20260709 12:21:00.111683 20719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec190d02960a4e88af0807817b789e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.125396 20722 tablet_service.cc:1511] Processing CreateTablet for tablet d5374e2302374917828294b0a4e3313a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:00.126776 20722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5374e2302374917828294b0a4e3313a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.127496 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.132714 20868 tablet_service.cc:1511] Processing CreateTablet for tablet 26ced453445642b09bda798a2963748b (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:00.134526 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ced453445642b09bda798a2963748b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.138863 20867 tablet_service.cc:1511] Processing CreateTablet for tablet e36a8df2d02f44ca965324882aba3ef2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:00.140261 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36a8df2d02f44ca965324882aba3ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.160642 20799 tablet_service.cc:1511] Processing CreateTablet for tablet ba4e3bf73fae46119cf926d5af0782a6 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:00.153016 20924 ts_tablet_manager.cc:1434] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.180s	user 0.020s	sys 0.049s
I20260709 12:21:00.168269 20924 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:00.169520 20799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4e3bf73fae46119cf926d5af0782a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.169409 20795 tablet_service.cc:1511] Processing CreateTablet for tablet 38d64dd1339c4abfb4ef05669402e801 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:00.170742 20795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d64dd1339c4abfb4ef05669402e801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.171831 20870 tablet_service.cc:1511] Processing CreateTablet for tablet 761ea542665e419c8a63392e3014efa7 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:00.173265 20870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761ea542665e419c8a63392e3014efa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.177456 20924 tablet_bootstrap.cc:654] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.179646 20874 tablet_service.cc:1511] Processing CreateTablet for tablet a48bec319402467881543ac686fb8f0c (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:00.187564 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48bec319402467881543ac686fb8f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.188433 20795 tablet_service.cc:1511] Processing CreateTablet for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:00.189924 20795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.181142 20927 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:00.200539 20793 tablet_service.cc:1511] Processing CreateTablet for tablet d5374e2302374917828294b0a4e3313a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:00.202001 20793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5374e2302374917828294b0a4e3313a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.198776 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 37c593b286274096a8c431979866be8a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:00.203219 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c593b286274096a8c431979866be8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.204294 20794 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd857e30c3240c5a4fdd11bc0602c00 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:00.206235 20871 tablet_service.cc:1511] Processing CreateTablet for tablet be08f375c22d436fa6e99ac7bd7f6c93 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:00.207504 20871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be08f375c22d436fa6e99ac7bd7f6c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.208410 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 8f381d2ab1764a739c2d1b3d41e65181 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:00.209750 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f381d2ab1764a739c2d1b3d41e65181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.210248 20794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd857e30c3240c5a4fdd11bc0602c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.217188 20795 tablet_service.cc:1511] Processing CreateTablet for tablet 0a4e4474d2fd45438665993211ae862a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:00.217725 20797 tablet_service.cc:1511] Processing CreateTablet for tablet 3199c6fc2e3441849ac7dd78244a3a18 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:00.218793 20795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a4e4474d2fd45438665993211ae862a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.219166 20927 tablet_bootstrap.cc:654] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.221786 20796 tablet_service.cc:1511] Processing CreateTablet for tablet 4415902e5b30489691f67f4a635305b4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:00.222193 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 626209a224524080801f3a9ad7307fbe (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:00.223538 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626209a224524080801f3a9ad7307fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.233470 20929 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:00.235090 20924 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:00.243675 20924 ts_tablet_manager.cc:1403] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.076s	user 0.005s	sys 0.010s
I20260709 12:21:00.245553 20867 tablet_service.cc:1511] Processing CreateTablet for tablet c6fe576036c04c149240cd6107285eca (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:00.253896 20927 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:00.254393 20927 ts_tablet_manager.cc:1403] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.074s	user 0.017s	sys 0.005s
I20260709 12:21:00.257361 20927 raft_consensus.cc:359] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.258038 20927 raft_consensus.cc:385] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.247146 20924 raft_consensus.cc:359] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.260365 20924 raft_consensus.cc:385] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.260659 20924 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.261346 20924 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.261855 20871 tablet_service.cc:1511] Processing CreateTablet for tablet d46401ecb9b443118647ee361081a01e (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:00.263520 20924 ts_tablet_manager.cc:1434] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.005s
I20260709 12:21:00.264447 20924 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:00.249886 20874 tablet_service.cc:1511] Processing CreateTablet for tablet 70cdcdfe95804895979397c927fae46f (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:00.265777 20927 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.265879 20873 tablet_service.cc:1511] Processing CreateTablet for tablet 1be2fc0a60b1472a9f537a9cd3bcc133 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:00.266561 20927 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.266976 20868 tablet_service.cc:1511] Processing CreateTablet for tablet 942b8c65bd1e42a99f971b099d0e19e6 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:00.259490 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fe576036c04c149240cd6107285eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.259469 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 986267de598f431fb25840d9d741436d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:00.268275 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70cdcdfe95804895979397c927fae46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.269003 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986267de598f431fb25840d9d741436d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.273288 20927 ts_tablet_manager.cc:1434] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.003s
I20260709 12:21:00.274119 20873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1be2fc0a60b1472a9f537a9cd3bcc133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.274339 20927 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:00.281325 20927 tablet_bootstrap.cc:654] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.280972 20871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46401ecb9b443118647ee361081a01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.247056 20799 tablet_service.cc:1511] Processing CreateTablet for tablet a32323ce10e140bb895085f97c97fa46 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:00.360457 20760 heartbeater.cc:499] Master 127.19.36.254:46039 was elected leader, sending a full tablet report...
I20260709 12:21:00.291074 20924 tablet_bootstrap.cc:654] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.397444 20720 tablet_service.cc:1511] Processing CreateTablet for tablet 4415902e5b30489691f67f4a635305b4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:00.430449 20720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4415902e5b30489691f67f4a635305b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.397746 20796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4415902e5b30489691f67f4a635305b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:00.316938 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:00.382599 20722 tablet_service.cc:1511] Processing CreateTablet for tablet 3199c6fc2e3441849ac7dd78244a3a18 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:00.433269 20722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3199c6fc2e3441849ac7dd78244a3a18. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:00.306859 20618 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 176148us
I20260709 12:21:00.436173 20927 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:00.436849 20927 ts_tablet_manager.cc:1403] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.163s	user 0.015s	sys 0.005s
I20260709 12:21:00.398506 20798 tablet_service.cc:1511] Processing CreateTablet for tablet c5ef54d60e4646b68d43519ad3fffa6d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:00.439463 20926 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.440328 20926 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.441121 20798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ef54d60e4646b68d43519ad3fffa6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.442687 20725 tablet_service.cc:1511] Processing CreateTablet for tablet c5ef54d60e4646b68d43519ad3fffa6d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
W20260709 12:21:00.400656 20910 tablet.cc:2406] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:00.449594 20926 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:00.444015 20725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ef54d60e4646b68d43519ad3fffa6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.370858 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd857e30c3240c5a4fdd11bc0602c00 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:00.463778 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942b8c65bd1e42a99f971b099d0e19e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.382962 20792 tablet_service.cc:1511] Processing CreateTablet for tablet 981a4d0731e940b79eccfb33d1928fb4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:00.334295 20718 tablet_service.cc:1511] Processing CreateTablet for tablet 0a4e4474d2fd45438665993211ae862a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:00.480322 20799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a32323ce10e140bb895085f97c97fa46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.481258 20797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3199c6fc2e3441849ac7dd78244a3a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.490226 20929 tablet_bootstrap.cc:654] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.527204 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 6dc725b7c88e47f693e3a16879798246 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:00.528638 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dc725b7c88e47f693e3a16879798246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.491240 20723 tablet_service.cc:1511] Processing CreateTablet for tablet a32323ce10e140bb895085f97c97fa46 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:00.503998 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd857e30c3240c5a4fdd11bc0602c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.529349 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 210839e60f364f8cab9f53ff440c461d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:00.530774 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210839e60f364f8cab9f53ff440c461d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.496532 20924 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:00.531635 20924 ts_tablet_manager.cc:1403] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.267s	user 0.011s	sys 0.007s
I20260709 12:21:00.534878 20924 raft_consensus.cc:359] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.535723 20924 raft_consensus.cc:385] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.536075 20924 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.536952 20924 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.508706 20716 tablet_service.cc:1511] Processing CreateTablet for tablet 981a4d0731e940b79eccfb33d1928fb4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:00.538784 20835 heartbeater.cc:499] Master 127.19.36.254:46039 was elected leader, sending a full tablet report...
I20260709 12:21:00.469275 20927 raft_consensus.cc:359] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:00.539369 20927 raft_consensus.cc:385] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.539757 20927 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.527385 20874 tablet_service.cc:1511] Processing CreateTablet for tablet f15b5f1ac1444b63bad60a89fa9f8a29 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:00.541576 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15b5f1ac1444b63bad60a89fa9f8a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.546764 20791 tablet_service.cc:1511] Processing CreateTablet for tablet 761ea542665e419c8a63392e3014efa7 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:00.547595 20723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a32323ce10e140bb895085f97c97fa46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.549695 20794 tablet_service.cc:1511] Processing CreateTablet for tablet b7040eacc899473abc05a913812e82a4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:00.550184 20800 tablet_service.cc:1511] Processing CreateTablet for tablet 26ced453445642b09bda798a2963748b (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:00.523311 20792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981a4d0731e940b79eccfb33d1928fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.544667 20721 tablet_service.cc:1511] Processing CreateTablet for tablet 761ea542665e419c8a63392e3014efa7 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:00.556567 20718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a4e4474d2fd45438665993211ae862a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.563760 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761ea542665e419c8a63392e3014efa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.561650 20927 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:00.573639 20927 ts_tablet_manager.cc:1434] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.136s	user 0.008s	sys 0.000s
I20260709 12:21:00.574697 20927 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:00.578807 20924 ts_tablet_manager.cc:1434] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.002s
I20260709 12:21:00.580019 20924 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:00.582396 20793 tablet_service.cc:1511] Processing CreateTablet for tablet e36a8df2d02f44ca965324882aba3ef2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:00.583798 20720 tablet_service.cc:1511] Processing CreateTablet for tablet b7040eacc899473abc05a913812e82a4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:00.585098 20720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7040eacc899473abc05a913812e82a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.560510 20800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ced453445642b09bda798a2963748b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.592404 20791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761ea542665e419c8a63392e3014efa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.597183 20873 tablet_service.cc:1511] Processing CreateTablet for tablet 13cfeabebc54415b87ab9ce1d8a4caab (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:00.597616 20870 tablet_service.cc:1511] Processing CreateTablet for tablet 87775c62d4bc48d3b1831b4bdeff2870 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:00.598704 20873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13cfeabebc54415b87ab9ce1d8a4caab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.599400 20870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87775c62d4bc48d3b1831b4bdeff2870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.597509 20721 tablet_service.cc:1511] Processing CreateTablet for tablet 26ced453445642b09bda798a2963748b (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:00.599990 20717 tablet_service.cc:1511] Processing CreateTablet for tablet e36a8df2d02f44ca965324882aba3ef2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:00.601045 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ced453445642b09bda798a2963748b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.608122 20795 tablet_service.cc:1511] Processing CreateTablet for tablet a48bec319402467881543ac686fb8f0c (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:00.621704 20794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7040eacc899473abc05a913812e82a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.629875 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:00.631929 20849 leader_election.cc:343] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.634609 20717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36a8df2d02f44ca965324882aba3ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.634235 20872 tablet_service.cc:1511] Processing CreateTablet for tablet c3046bc775e64931b4a1ff963cccf0e2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:00.636283 20872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3046bc775e64931b4a1ff963cccf0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.639935 20721 tablet_service.cc:1511] Processing CreateTablet for tablet a48bec319402467881543ac686fb8f0c (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:00.641026 20866 tablet_service.cc:1511] Processing CreateTablet for tablet d5e3c520e78241879a779a912c66a4ba (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:00.643359 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48bec319402467881543ac686fb8f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.643489 20866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5e3c520e78241879a779a912c66a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.643467 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 37c593b286274096a8c431979866be8a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:00.655273 20871 tablet_service.cc:1511] Processing CreateTablet for tablet a9c17d2fc98f481597ce7b97e8f98c43 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:00.638754 20874 tablet_service.cc:1511] Processing CreateTablet for tablet 72e60e5ddf96445ab5e284dbeed45c81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:00.642266 20869 tablet_service.cc:1511] Processing CreateTablet for tablet ff0a54292b924069aae5a0cd6c1366d1 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:00.656821 20871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c17d2fc98f481597ce7b97e8f98c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.655714 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c593b286274096a8c431979866be8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.653357 20867 tablet_service.cc:1511] Processing CreateTablet for tablet 3efcc9a1f14042de975fa0b662cad24d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:00.662653 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff0a54292b924069aae5a0cd6c1366d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.664556 20719 tablet_service.cc:1511] Processing CreateTablet for tablet be08f375c22d436fa6e99ac7bd7f6c93 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:00.666023 20719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be08f375c22d436fa6e99ac7bd7f6c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.670290 20722 tablet_service.cc:1511] Processing CreateTablet for tablet 8f381d2ab1764a739c2d1b3d41e65181 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:00.672005 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efcc9a1f14042de975fa0b662cad24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.682401 20927 tablet_bootstrap.cc:654] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.699568 20924 tablet_bootstrap.cc:654] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.706493 20867 tablet_service.cc:1511] Processing CreateTablet for tablet c00fffbb02ff4bc2a974d8a5b179a97d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:00.709697 20867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c00fffbb02ff4bc2a974d8a5b179a97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.713951 20795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48bec319402467881543ac686fb8f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.713600 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:00.715557 20849 leader_election.cc:343] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.716029 20849 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:00.717188 20926 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.721187 20799 tablet_service.cc:1511] Processing CreateTablet for tablet be08f375c22d436fa6e99ac7bd7f6c93 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:00.722436 20793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36a8df2d02f44ca965324882aba3ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.721194 20797 tablet_service.cc:1511] Processing CreateTablet for tablet d46401ecb9b443118647ee361081a01e (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:00.724017 20797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46401ecb9b443118647ee361081a01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.721282 20796 tablet_service.cc:1511] Processing CreateTablet for tablet 37c593b286274096a8c431979866be8a (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:00.728005 20796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c593b286274096a8c431979866be8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.751519 20716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981a4d0731e940b79eccfb33d1928fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.752730 20719 tablet_service.cc:1511] Processing CreateTablet for tablet 626209a224524080801f3a9ad7307fbe (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:00.758719 20799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be08f375c22d436fa6e99ac7bd7f6c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.759498 20721 tablet_service.cc:1511] Processing CreateTablet for tablet c6fe576036c04c149240cd6107285eca (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:00.761066 20719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626209a224524080801f3a9ad7307fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.763741 20722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f381d2ab1764a739c2d1b3d41e65181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.764775 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fe576036c04c149240cd6107285eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.772614 20797 tablet_service.cc:1511] Processing CreateTablet for tablet 8f381d2ab1764a739c2d1b3d41e65181 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:00.780652 20800 tablet_service.cc:1511] Processing CreateTablet for tablet 626209a224524080801f3a9ad7307fbe (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:00.757210 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e60e5ddf96445ab5e284dbeed45c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.783859 20932 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.784394 20932 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.807893 20868 tablet_service.cc:1511] Processing CreateTablet for tablet 335e9cdc3f6745ab9b44f08b6e8b2118 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:00.809697 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 335e9cdc3f6745ab9b44f08b6e8b2118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.795918 20932 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:00.834646 20792 tablet_service.cc:1511] Processing CreateTablet for tablet c6fe576036c04c149240cd6107285eca (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:00.844861 20723 tablet_service.cc:1511] Processing CreateTablet for tablet 986267de598f431fb25840d9d741436d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:00.845569 20927 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:00.846237 20723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986267de598f431fb25840d9d741436d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.846371 20927 ts_tablet_manager.cc:1403] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.272s	user 0.022s	sys 0.041s
I20260709 12:21:00.841446 20797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f381d2ab1764a739c2d1b3d41e65181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.852682 20927 raft_consensus.cc:359] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.856712 20929 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:00.864967 20932 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.864756 20794 tablet_service.cc:1511] Processing CreateTablet for tablet 986267de598f431fb25840d9d741436d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:00.865422 20932 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:00.867565 20932 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:00.868317 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 70cdcdfe95804895979397c927fae46f (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:00.869627 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70cdcdfe95804895979397c927fae46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.870879 20798 tablet_service.cc:1511] Processing CreateTablet for tablet 1be2fc0a60b1472a9f537a9cd3bcc133 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:00.856775 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 54cd23b6f74a477a99461098dd4f236d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:00.874858 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cd23b6f74a477a99461098dd4f236d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.876279 20720 tablet_service.cc:1511] Processing CreateTablet for tablet 1be2fc0a60b1472a9f537a9cd3bcc133 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:00.882680 20926 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.883247 20926 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.884934 20791 tablet_service.cc:1511] Processing CreateTablet for tablet 70cdcdfe95804895979397c927fae46f (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:00.885502 20926 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:00.888465 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:00.890095 20849 leader_election.cc:343] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.891113 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 0b31661191c14b44ac964555853900f1 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:00.857744 20927 raft_consensus.cc:385] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.892045 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:00.892339 20927 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
W20260709 12:21:00.894392 20849 leader_election.cc:343] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.894934 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:00.895121 20717 tablet_service.cc:1511] Processing CreateTablet for tablet d46401ecb9b443118647ee361081a01e (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:00.897820 20926 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.898948 20927 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.901603 20927 ts_tablet_manager.cc:1434] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 12:21:00.903859 20792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fe576036c04c149240cd6107285eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.904491 20794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986267de598f431fb25840d9d741436d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.906349 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:00.907222 20800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626209a224524080801f3a9ad7307fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.907588 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:00.909066 20929 ts_tablet_manager.cc:1403] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.676s	user 0.067s	sys 0.068s
I20260709 12:21:00.909614 20791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70cdcdfe95804895979397c927fae46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.911944 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b31661191c14b44ac964555853900f1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:00.909555 20700 leader_election.cc:343] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:00.914377 20700 leader_election.cc:343] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.928494 20796 tablet_service.cc:1511] Processing CreateTablet for tablet 942b8c65bd1e42a99f971b099d0e19e6 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:00.936250 20929 raft_consensus.cc:359] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:00.937279 20929 raft_consensus.cc:385] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.937502 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:00.937716 20929 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.939215 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:00.942562 20699 leader_election.cc:343] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.943022 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:21:00.944200 20699 leader_election.cc:343] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.944605 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:00.945544 20932 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.948483 20873 tablet_service.cc:1511] Processing CreateTablet for tablet 3894130cfc2549b5a8c99d056675ef94 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:00.949167 20932 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.949942 20865 tablet_service.cc:1511] Processing CreateTablet for tablet c1303aa66bdb4474b6bcb604461e74a5 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:00.950011 20873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3894130cfc2549b5a8c99d056675ef94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.951617 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1303aa66bdb4474b6bcb604461e74a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.949983 20800 tablet_service.cc:1511] Processing CreateTablet for tablet 6dc725b7c88e47f693e3a16879798246 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:00.953536 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 942b8c65bd1e42a99f971b099d0e19e6 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:00.954799 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942b8c65bd1e42a99f971b099d0e19e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.961071 20720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1be2fc0a60b1472a9f537a9cd3bcc133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.963536 20798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1be2fc0a60b1472a9f537a9cd3bcc133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.967762 20717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46401ecb9b443118647ee361081a01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.969110 20871 tablet_service.cc:1511] Processing CreateTablet for tablet 2b098f0a68194f18be24dc4f3742a4ab (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:00.970415 20871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b098f0a68194f18be24dc4f3742a4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.970292 20725 tablet_service.cc:1511] Processing CreateTablet for tablet 6dc725b7c88e47f693e3a16879798246 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:00.971540 20725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dc725b7c88e47f693e3a16879798246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.973860 20794 tablet_service.cc:1511] Processing CreateTablet for tablet a9c17d2fc98f481597ce7b97e8f98c43 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:00.978683 20800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dc725b7c88e47f693e3a16879798246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.980041 20796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942b8c65bd1e42a99f971b099d0e19e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.981436 20791 tablet_service.cc:1511] Processing CreateTablet for tablet f15b5f1ac1444b63bad60a89fa9f8a29 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:00.984246 20927 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:00.984881 20795 tablet_service.cc:1511] Processing CreateTablet for tablet 210839e60f364f8cab9f53ff440c461d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:00.993662 20718 tablet_service.cc:1511] Processing CreateTablet for tablet 210839e60f364f8cab9f53ff440c461d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:00.994983 20718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210839e60f364f8cab9f53ff440c461d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.000990 20716 tablet_service.cc:1511] Processing CreateTablet for tablet 13cfeabebc54415b87ab9ce1d8a4caab (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:01.002318 20716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13cfeabebc54415b87ab9ce1d8a4caab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.003286 20795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210839e60f364f8cab9f53ff440c461d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.003931 20794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c17d2fc98f481597ce7b97e8f98c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.006747 20927 tablet_bootstrap.cc:654] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.985860 20722 tablet_service.cc:1511] Processing CreateTablet for tablet f15b5f1ac1444b63bad60a89fa9f8a29 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:01.019364 20722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15b5f1ac1444b63bad60a89fa9f8a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.999501 20929 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.014940 20791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15b5f1ac1444b63bad60a89fa9f8a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.016224 20719 tablet_service.cc:1511] Processing CreateTablet for tablet a9c17d2fc98f481597ce7b97e8f98c43 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:01.025898 20719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c17d2fc98f481597ce7b97e8f98c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.022894 20794 tablet_service.cc:1511] Processing CreateTablet for tablet 87775c62d4bc48d3b1831b4bdeff2870 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:01.027472 20794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87775c62d4bc48d3b1831b4bdeff2870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.025791 20945 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.028002 20945 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.033416 20924 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.025535 20792 tablet_service.cc:1511] Processing CreateTablet for tablet 72e60e5ddf96445ab5e284dbeed45c81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:01.034269 20924 ts_tablet_manager.cc:1403] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.455s	user 0.016s	sys 0.020s
I20260709 12:21:01.027226 20724 tablet_service.cc:1511] Processing CreateTablet for tablet 87775c62d4bc48d3b1831b4bdeff2870 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:01.035013 20792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e60e5ddf96445ab5e284dbeed45c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.035447 20724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87775c62d4bc48d3b1831b4bdeff2870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.017567 20799 tablet_service.cc:1511] Processing CreateTablet for tablet 13cfeabebc54415b87ab9ce1d8a4caab (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:01.040748 20865 tablet_service.cc:1511] Processing CreateTablet for tablet 7ccae903e8484f8d8c3af847e2c4c3d4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:01.041664 20865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ccae903e8484f8d8c3af847e2c4c3d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.046552 20721 tablet_service.cc:1511] Processing CreateTablet for tablet 72e60e5ddf96445ab5e284dbeed45c81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:01.058846 20716 tablet_service.cc:1511] Processing CreateTablet for tablet ff0a54292b924069aae5a0cd6c1366d1 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:01.062502 20924 raft_consensus.cc:359] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.029532 20793 tablet_service.cc:1511] Processing CreateTablet for tablet ff0a54292b924069aae5a0cd6c1366d1 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:01.073643 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e60e5ddf96445ab5e284dbeed45c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.075079 20945 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.075332 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:01.076738 20699 leader_election.cc:343] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.079809 20799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13cfeabebc54415b87ab9ce1d8a4caab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.079592 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:01.081669 20700 leader_election.cc:343] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.082228 20700 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:01.083711 20945 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:01.085784 20869 tablet_service.cc:1511] Processing CreateTablet for tablet ad57e68a6a114847ac252579307658f7 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:01.087393 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad57e68a6a114847ac252579307658f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.090646 20723 tablet_service.cc:1511] Processing CreateTablet for tablet 3efcc9a1f14042de975fa0b662cad24d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:01.091948 20723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efcc9a1f14042de975fa0b662cad24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.095867 20793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff0a54292b924069aae5a0cd6c1366d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.101435 20797 tablet_service.cc:1511] Processing CreateTablet for tablet 3efcc9a1f14042de975fa0b662cad24d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:01.102811 20797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efcc9a1f14042de975fa0b662cad24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.118045 20721 tablet_service.cc:1511] Processing CreateTablet for tablet d5e3c520e78241879a779a912c66a4ba (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:01.103642 20716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff0a54292b924069aae5a0cd6c1366d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.119464 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5e3c520e78241879a779a912c66a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.125725 20723 tablet_service.cc:1511] Processing CreateTablet for tablet c00fffbb02ff4bc2a974d8a5b179a97d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:01.106288 20717 tablet_service.cc:1511] Processing CreateTablet for tablet c3046bc775e64931b4a1ff963cccf0e2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:01.127115 20723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c00fffbb02ff4bc2a974d8a5b179a97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.135666 20717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3046bc775e64931b4a1ff963cccf0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.139438 20926 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.128345 20924 raft_consensus.cc:385] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.139986 20926 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.143028 20927 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:01.143085 20926 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:01.144881 20927 ts_tablet_manager.cc:1403] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.033s
I20260709 12:21:01.144869 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.146442 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.148022 20927 raft_consensus.cc:359] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:01.148442 20849 leader_election.cc:343] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.148749 20927 raft_consensus.cc:385] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.149083 20927 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
W20260709 12:21:01.149225 20849 leader_election.cc:343] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.149744 20849 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:01.151057 20929 ts_tablet_manager.cc:1434] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.242s	user 0.026s	sys 0.026s
I20260709 12:21:01.151288 20926 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:01.152145 20792 tablet_service.cc:1511] Processing CreateTablet for tablet c3046bc775e64931b4a1ff963cccf0e2 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:01.152901 20869 tablet_service.cc:1511] Processing CreateTablet for tablet 1047135f71834dbab6b869eb3c081b81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:01.153648 20792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3046bc775e64931b4a1ff963cccf0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.153564 20720 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37260 (request call id 33) took 1016 ms. Trace:
I20260709 12:21:01.154102 20929 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.154392 20924 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.157017 20924 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.154740 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047135f71834dbab6b869eb3c081b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.155409 20720 rpcz_store.cc:276] 0709 12:21:00.136985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:00.875297 (+738312us) service_pool.cc:224] Handling call
0709 12:21:00.960581 (+ 85284us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:21:00.960711 (+   130us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:21:01.048947 (+ 88236us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:01.153016 (+104069us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:01.153500 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16256}
I20260709 12:21:01.163582 20929 tablet_bootstrap.cc:654] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.177561 20927 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.179230 20792 tablet_service.cc:1511] Processing CreateTablet for tablet d5e3c520e78241879a779a912c66a4ba (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:01.189836 20793 tablet_service.cc:1511] Processing CreateTablet for tablet c00fffbb02ff4bc2a974d8a5b179a97d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:01.194334 20926 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.204298 20926 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.163746 20720 tablet_service.cc:1511] Processing CreateTablet for tablet 54cd23b6f74a477a99461098dd4f236d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:01.216969 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.207155 20926 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:01.219782 20849 leader_election.cc:343] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.221789 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.222082 20792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5e3c520e78241879a779a912c66a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:01.223560 20849 leader_election.cc:343] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.223978 20849 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:01.224431 20795 tablet_service.cc:1511] Processing CreateTablet for tablet 0b31661191c14b44ac964555853900f1 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:01.225026 20926 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:01.225744 20795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b31661191c14b44ac964555853900f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.227850 20799 tablet_service.cc:1511] Processing CreateTablet for tablet 335e9cdc3f6745ab9b44f08b6e8b2118 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:01.229192 20799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 335e9cdc3f6745ab9b44f08b6e8b2118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.236089 20924 ts_tablet_manager.cc:1434] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.201s	user 0.002s	sys 0.006s
I20260709 12:21:01.237738 20924 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.249116 20720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cd23b6f74a477a99461098dd4f236d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.249882 20927 ts_tablet_manager.cc:1434] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.000s
I20260709 12:21:01.250545 20718 tablet_service.cc:1511] Processing CreateTablet for tablet 0b31661191c14b44ac964555853900f1 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:01.252121 20718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b31661191c14b44ac964555853900f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.256242 20793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c00fffbb02ff4bc2a974d8a5b179a97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.257673 20800 tablet_service.cc:1511] Processing CreateTablet for tablet 54cd23b6f74a477a99461098dd4f236d (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:01.258941 20800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cd23b6f74a477a99461098dd4f236d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.260710 20927 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:01.260850 20725 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37260 (request call id 36) took 1121 ms. Trace:
I20260709 12:21:01.267592 20725 rpcz_store.cc:276] 0709 12:21:00.137815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:00.969373 (+831558us) service_pool.cc:224] Handling call
0709 12:21:00.971093 (+  1720us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:21:00.971177 (+    84us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:21:01.088267 (+117090us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:01.215907 (+127640us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:01.259166 (+ 43259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:01.268262 20946 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.268731 20946 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.269619 20725 tablet_service.cc:1511] Processing CreateTablet for tablet 3894130cfc2549b5a8c99d056675ef94 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:01.270035 20927 tablet_bootstrap.cc:654] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.270922 20725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3894130cfc2549b5a8c99d056675ef94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.271456 20946 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.292300 20929 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:01.293354 20929 ts_tablet_manager.cc:1403] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.139s	user 0.020s	sys 0.025s
I20260709 12:21:01.296347 20929 raft_consensus.cc:359] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.297081 20929 raft_consensus.cc:385] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.297374 20929 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.305435 20798 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33466 (request call id 33) took 1156 ms. Trace:
I20260709 12:21:01.305879 20798 rpcz_store.cc:276] 0709 12:21:00.148960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:00.869917 (+720957us) service_pool.cc:224] Handling call
0709 12:21:00.871702 (+  1785us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:21:00.871824 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:21:01.131399 (+259575us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:01.304915 (+173516us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:01.305368 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:01.307528 20929 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.310269 20798 tablet_service.cc:1511] Processing CreateTablet for tablet c1303aa66bdb4474b6bcb604461e74a5 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:01.311616 20798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1303aa66bdb4474b6bcb604461e74a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.311986 20722 tablet_service.cc:1511] Processing CreateTablet for tablet 335e9cdc3f6745ab9b44f08b6e8b2118 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:01.313297 20722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 335e9cdc3f6745ab9b44f08b6e8b2118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.319154 20929 ts_tablet_manager.cc:1434] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 12:21:01.322225 20873 tablet_service.cc:1511] Processing CreateTablet for tablet 8f3454624b7c40c6934257c77f2174de (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:01.322225 20869 tablet_service.cc:1511] Processing CreateTablet for tablet f61efc9c6d5f43ac84cfe1def41de8a0 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:01.325085 20873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f3454624b7c40c6934257c77f2174de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.327704 20869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f61efc9c6d5f43ac84cfe1def41de8a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.350451 20929 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.366881 20719 tablet_service.cc:1511] Processing CreateTablet for tablet 2b098f0a68194f18be24dc4f3742a4ab (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:01.368458 20719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b098f0a68194f18be24dc4f3742a4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.372489 20924 tablet_bootstrap.cc:654] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.375735 20716 tablet_service.cc:1511] Processing CreateTablet for tablet c1303aa66bdb4474b6bcb604461e74a5 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:01.377301 20716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1303aa66bdb4474b6bcb604461e74a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.388381 20929 tablet_bootstrap.cc:654] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.401188 20718 tablet_service.cc:1511] Processing CreateTablet for tablet 7ccae903e8484f8d8c3af847e2c4c3d4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:01.404861 20721 tablet_service.cc:1511] Processing CreateTablet for tablet ad57e68a6a114847ac252579307658f7 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:01.405233 20791 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33466 (request call id 37) took 1040 ms. Trace:
I20260709 12:21:01.403029 20718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ccae903e8484f8d8c3af847e2c4c3d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.405638 20791 rpcz_store.cc:276] 0709 12:21:00.364535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:00.980499 (+615964us) service_pool.cc:224] Handling call
0709 12:21:00.998229 (+ 17730us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:21:01.014564 (+ 16335us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:21:01.116702 (+102138us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:01.392785 (+276083us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:01.405171 (+ 12386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:01.416378 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad57e68a6a114847ac252579307658f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.416164 20799 tablet_service.cc:1511] Processing CreateTablet for tablet 3894130cfc2549b5a8c99d056675ef94 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:01.417213 20796 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33466 (request call id 35) took 1258 ms. Trace:
I20260709 12:21:01.417719 20799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3894130cfc2549b5a8c99d056675ef94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.417586 20796 rpcz_store.cc:276] 0709 12:21:00.187380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:00.927460 (+740080us) service_pool.cc:224] Handling call
0709 12:21:00.929382 (+  1922us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:21:00.929481 (+    99us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:21:01.259638 (+330157us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:01.416704 (+157066us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:01.417155 (+   451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:01.421362 20796 tablet_service.cc:1511] Processing CreateTablet for tablet 7ccae903e8484f8d8c3af847e2c4c3d4 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:01.422623 20796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ccae903e8484f8d8c3af847e2c4c3d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.411572 20794 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33466 (request call id 41) took 1029 ms. Trace:
I20260709 12:21:01.423501 20794 rpcz_store.cc:276] 0709 12:21:00.382582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:01.021965 (+639383us) service_pool.cc:224] Handling call
0709 12:21:01.026967 (+  5002us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:21:01.027053 (+    86us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:21:01.178513 (+151460us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:01.399345 (+220832us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:01.410479 (+ 11134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:01.425419 20794 tablet_service.cc:1511] Processing CreateTablet for tablet ad57e68a6a114847ac252579307658f7 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:01.426682 20794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad57e68a6a114847ac252579307658f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.404906 20720 tablet_service.cc:1511] Processing CreateTablet for tablet 1047135f71834dbab6b869eb3c081b81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:01.432238 20800 tablet_service.cc:1511] Processing CreateTablet for tablet 1047135f71834dbab6b869eb3c081b81 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 12:21:01.434123 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:01.439560 20796 tablet_service.cc:1511] Processing CreateTablet for tablet f61efc9c6d5f43ac84cfe1def41de8a0 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:01.449082 20720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047135f71834dbab6b869eb3c081b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.454497 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:01.456363 20774 leader_election.cc:343] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.456698 20725 tablet_service.cc:1511] Processing CreateTablet for tablet f61efc9c6d5f43ac84cfe1def41de8a0 (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:01.460566 20721 tablet_service.cc:1511] Processing CreateTablet for tablet 8f3454624b7c40c6934257c77f2174de (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:01.464217 20927 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:01.464721 20927 ts_tablet_manager.cc:1403] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.204s	user 0.011s	sys 0.010s
I20260709 12:21:01.463454 20725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f61efc9c6d5f43ac84cfe1def41de8a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.466743 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.467414 20735 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:01.468498 20775 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:01.468873 20796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f61efc9c6d5f43ac84cfe1def41de8a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.470045 20946 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.470418 20946 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.470737 20946 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.468022 20927 raft_consensus.cc:359] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.472747 20927 raft_consensus.cc:385] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.473089 20927 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.473989 20927 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.475816 20797 tablet_service.cc:1511] Processing CreateTablet for tablet 2b098f0a68194f18be24dc4f3742a4ab (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:01.477159 20797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b098f0a68194f18be24dc4f3742a4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.440354 20791 tablet_service.cc:1511] Processing CreateTablet for tablet 8f3454624b7c40c6934257c77f2174de (DEFAULT_TABLE table=test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:01.479873 20721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f3454624b7c40c6934257c77f2174de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.480106 20791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f3454624b7c40c6934257c77f2174de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.483773 20800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047135f71834dbab6b869eb3c081b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.458684 20724 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37260 (request call id 41) took 1077 ms. Trace:
I20260709 12:21:01.487588 20724 rpcz_store.cc:276] 0709 12:21:00.380977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:01.026316 (+645339us) service_pool.cc:224] Handling call
0709 12:21:01.034924 (+  8608us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:21:01.035018 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:21:01.249616 (+214598us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:01.395140 (+145524us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:01.458288 (+ 63148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:01.498417 20946 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.505877 20929 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:01.506423 20929 ts_tablet_manager.cc:1403] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.186s	user 0.015s	sys 0.013s
I20260709 12:21:01.507704 20927 ts_tablet_manager.cc:1434] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:21:01.509256 20929 raft_consensus.cc:359] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.509986 20929 raft_consensus.cc:385] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.509917 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:01.510403 20929 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.510603 20735 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.512732 20924 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.512264 20929 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.513219 20924 ts_tablet_manager.cc:1403] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.276s	user 0.026s	sys 0.029s
I20260709 12:21:01.518851 20735 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:01.520251 20775 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:01.526312 20946 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.530995 20946 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:01.531489 20946 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:01.532362 20946 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.526228 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:01.544740 20774 leader_election.cc:343] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.561887 20927 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:01.562487 20929 ts_tablet_manager.cc:1434] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.056s	user 0.028s	sys 0.020s
I20260709 12:21:01.563622 20929 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.571956 20924 raft_consensus.cc:359] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.572773 20924 raft_consensus.cc:385] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.573083 20924 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.574235 20927 tablet_bootstrap.cc:654] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.574038 20924 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.577029 20924 ts_tablet_manager.cc:1434] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 12:21:01.578140 20924 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.584342 20946 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.584843 20946 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.586869 20946 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.592232 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:01.593851 20929 tablet_bootstrap.cc:654] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:01.593963 20774 leader_election.cc:343] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.594722 20924 tablet_bootstrap.cc:654] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:01.582943 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62782us
I20260709 12:21:01.599782 20927 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:01.600466 20927 ts_tablet_manager.cc:1403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.004s
I20260709 12:21:01.600730 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.601728 20735 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:01.605089 20927 raft_consensus.cc:359] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.606364 20927 raft_consensus.cc:385] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.606895 20927 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.608250 20927 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.612244 20927 ts_tablet_manager.cc:1434] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:21:01.613956 20927 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:01.624101 20927 tablet_bootstrap.cc:654] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.628839 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:01.630924 20929 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:01.631494 20929 ts_tablet_manager.cc:1403] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.068s	user 0.024s	sys 0.019s
I20260709 12:21:01.633157 20957 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.633564 20957 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.633901 20957 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.634513 20929 raft_consensus.cc:359] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.635124 20924 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.635308 20929 raft_consensus.cc:385] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.635718 20929 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.635890 20924 ts_tablet_manager.cc:1403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.058s	user 0.036s	sys 0.012s
I20260709 12:21:01.636441 20929 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.638989 20929 ts_tablet_manager.cc:1434] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:01.640012 20929 tablet_bootstrap.cc:492] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.640394 20924 raft_consensus.cc:359] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.642261 20924 raft_consensus.cc:385] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.642683 20924 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.643477 20632 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.644013 20924 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.650487 20929 tablet_bootstrap.cc:654] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.652964 20924 ts_tablet_manager.cc:1434] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 12:21:01.654666 20924 tablet_bootstrap.cc:492] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.650823 20957 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.662367 20929 tablet_bootstrap.cc:492] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:01.667078 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:01.668962 20929 ts_tablet_manager.cc:1403] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.010s
I20260709 12:21:01.669122 20735 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.672245 20929 raft_consensus.cc:359] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.673002 20929 raft_consensus.cc:385] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.673301 20929 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.674078 20929 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.665167 20924 tablet_bootstrap.cc:654] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.676646 20929 ts_tablet_manager.cc:1434] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:21:01.677016 20735 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:01.677860 20929 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.677779 20945 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.678267 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:01.678452 20945 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.679057 20946 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:01.666194 20957 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.679590 20946 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:01.680351 20946 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.681833 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.667943 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:01.683048 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:01.683825 20884 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
W20260709 12:21:01.683902 20774 leader_election.cc:343] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.685190 20945 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.685326 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:21:01.686213 20700 leader_election.cc:343] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.686653 20945 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.687233 20945 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.687781 20945 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.690806 20929 tablet_bootstrap.cc:654] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.697538 20927 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:01.698072 20927 ts_tablet_manager.cc:1403] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.084s	user 0.031s	sys 0.030s
I20260709 12:21:01.698690 20945 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.701109 20945 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.702313 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:01.702435 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:01.703248 20884 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.704123 20924 tablet_bootstrap.cc:492] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.704663 20924 ts_tablet_manager.cc:1403] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.009s
W20260709 12:21:01.704540 20700 leader_election.cc:343] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.708320 20924 raft_consensus.cc:359] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.709165 20924 raft_consensus.cc:385] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.709488 20924 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.710309 20924 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.711958 20884 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:01.712622 20924 ts_tablet_manager.cc:1434] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.002s
I20260709 12:21:01.713363 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:01.713762 20924 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.714695 20927 raft_consensus.cc:359] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.715539 20927 raft_consensus.cc:385] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.715873 20927 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.716701 20945 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:01.718289 20927 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.723763 20924 tablet_bootstrap.cc:654] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.725190 20929 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:01.725380 20946 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.725736 20929 ts_tablet_manager.cc:1403] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.015s
I20260709 12:21:01.725903 20946 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.728298 20946 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.729133 20929 raft_consensus.cc:359] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.729898 20929 raft_consensus.cc:385] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.730206 20929 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.730441 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:01.731135 20884 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:01.731065 20929 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.731884 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.732566 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:01.733500 20946 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.733896 20946 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:01.734037 20775 leader_election.cc:343] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.734295 20946 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.734063 20929 ts_tablet_manager.cc:1434] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:01.736936 20929 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.737561 20945 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:01.736979 20924 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.738322 20924 ts_tablet_manager.cc:1403] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.006s
I20260709 12:21:01.738521 20945 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.740435 20632 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.744469 20929 tablet_bootstrap.cc:654] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.744429 20946 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.745398 20924 raft_consensus.cc:359] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.746138 20924 raft_consensus.cc:385] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.746438 20924 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.742206 20927 ts_tablet_manager.cc:1434] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.044s	user 0.011s	sys 0.000s
I20260709 12:21:01.747285 20924 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.748099 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:01.749634 20775 leader_election.cc:343] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.749840 20924 ts_tablet_manager.cc:1434] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:21:01.751045 20924 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.752188 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:01.752828 20884 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.757339 20924 tablet_bootstrap.cc:654] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.761564 20946 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.762570 20884 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:01.764415 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:01.766400 20946 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:01.767624 20946 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:01.768889 20946 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.748365 20927 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:01.785382 20927 tablet_bootstrap.cc:654] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.789793 20924 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.790063 20929 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:01.790321 20924 ts_tablet_manager.cc:1403] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.039s	user 0.027s	sys 0.010s
I20260709 12:21:01.790691 20929 ts_tablet_manager.cc:1403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.024s
I20260709 12:21:01.791963 20632 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.794507 20929 raft_consensus.cc:359] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.793293 20924 raft_consensus.cc:359] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.795645 20929 raft_consensus.cc:385] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.795841 20924 raft_consensus.cc:385] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.796108 20929 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.796238 20924 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.797379 20929 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.799081 20924 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.800644 20929 ts_tablet_manager.cc:1434] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:21:01.802008 20929 tablet_bootstrap.cc:492] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.806779 20949 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.807426 20949 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.809710 20949 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:01.810199 20929 tablet_bootstrap.cc:654] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.810256 20949 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.810811 20949 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.812901 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.813884 20735 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:01.815497 20927 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:01.815351 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.816118 20927 ts_tablet_manager.cc:1403] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.005s
W20260709 12:21:01.818020 20849 leader_election.cc:343] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.819664 20927 raft_consensus.cc:359] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.820578 20927 raft_consensus.cc:385] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.820922 20927 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.821754 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.823444 20924 ts_tablet_manager.cc:1434] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 12:21:01.823547 20946 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.823856 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.824507 20924 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.823998 20946 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.826300 20946 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.828728 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.818818 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:01.831881 20924 tablet_bootstrap.cc:654] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.830075 20633 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.812944 20949 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:01.839644 20949 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.840449 20949 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.845281 20924 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.845903 20924 ts_tablet_manager.cc:1403] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.006s
I20260709 12:21:01.849382 20949 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:01.849432 20924 raft_consensus.cc:359] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.850503 20924 raft_consensus.cc:385] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.850961 20924 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.851925 20924 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.854604 20924 ts_tablet_manager.cc:1434] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:01.855065 20949 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.855541 20949 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.855861 20924 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.855943 20949 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.860677 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.863461 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.864164 20924 tablet_bootstrap.cc:654] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:01.865048 20849 leader_election.cc:343] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:01.865828 20849 leader_election.cc:343] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:01.866515 20849 leader_election.cc:343] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.866993 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:01.867966 20946 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:01.868212 20849 leader_election.cc:343] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.868693 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:01.868609 20946 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.872643 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:01.874361 20775 leader_election.cc:343] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.875455 20946 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.876240 20924 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.876849 20924 ts_tablet_manager.cc:1403] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 12:21:01.876945 20979 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:01.875422 20981 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:01.877396 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:01.878243 20884 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:01.821781 20927 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.879652 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:01.880332 20884 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:01.880566 20924 raft_consensus.cc:359] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.881505 20924 raft_consensus.cc:385] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.881592 20774 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:01.881958 20924 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.882557 20946 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.882897 20924 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.884446 20927 ts_tablet_manager.cc:1434] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 12:21:01.884960 20924 ts_tablet_manager.cc:1434] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:01.885450 20927 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:01.885840 20924 tablet_bootstrap.cc:492] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.886998 20774 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:01.888422 20956 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.888871 20956 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.889360 20956 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.890118 20946 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:01.892889 20924 tablet_bootstrap.cc:654] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.899722 20949 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.902172 20949 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:01.903262 20946 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.903821 20946 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.904489 20929 tablet_bootstrap.cc:492] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:01.905014 20929 ts_tablet_manager.cc:1403] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.103s	user 0.021s	sys 0.050s
I20260709 12:21:01.907169 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.908057 20735 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:01.908241 20929 raft_consensus.cc:359] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.909104 20929 raft_consensus.cc:385] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.909161 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:01.909557 20929 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.909919 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:01.910356 20929 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.910971 20957 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.911377 20957 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.911710 20957 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.909917 20734 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.913035 20927 tablet_bootstrap.cc:654] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.914654 20946 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.911708 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:01.920954 20957 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:01.922338 20849 leader_election.cc:343] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.923462 20957 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.924216 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:01.924847 20735 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.920493 20734 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:01.926198 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:01.926805 20924 tablet_bootstrap.cc:492] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:01.927091 20949 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:01.930001 20949 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:01.929910 20924 ts_tablet_manager.cc:1403] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.008s
I20260709 12:21:01.930944 20949 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.931480 20929 ts_tablet_manager.cc:1434] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.003s
I20260709 12:21:01.933410 20924 raft_consensus.cc:359] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.934257 20924 raft_consensus.cc:385] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.934442 20929 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:01.934547 20924 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.935411 20924 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.936390 20927 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:01.936830 20927 ts_tablet_manager.cc:1403] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.000s
I20260709 12:21:01.937675 20924 ts_tablet_manager.cc:1434] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:01.938586 20924 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:01.939422 20927 raft_consensus.cc:359] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.941988 20735 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:01.942822 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:01.945158 20924 tablet_bootstrap.cc:654] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.940114 20927 raft_consensus.cc:385] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:01.945442 20884 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:01.945537 20927 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:01.944413 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:01.946748 20883 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.946508 20927 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.944537 20981 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.950901 20981 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.953874 20981 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:01.956004 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:01.957309 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.957947 20809 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:01.959239 20927 ts_tablet_manager.cc:1434] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:21:01.960179 20927 tablet_bootstrap.cc:492] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
W20260709 12:21:01.964588 20849 leader_election.cc:343] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.965354 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:01.966120 20927 tablet_bootstrap.cc:654] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.944857 20929 tablet_bootstrap.cc:654] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:01.973459 20945 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.977063 20945 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.974100 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:01.945027 20956 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.967924 20981 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.978430 20981 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.978819 20981 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.978760 20985 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:01.979628 20985 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:01.981187 20945 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.983439 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:01.984105 20809 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:01.985566 20956 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.975750 20883 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:01.985797 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:01.987305 20945 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:01.987656 20945 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.988008 20945 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:01.980530 20985 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.992084 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:01.994706 20775 leader_election.cc:343] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:01.994782 20945 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:01.997246 20945 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:01.996845 20633 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:02.001012 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.001940 20809 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.003288 20924 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.003732 20924 ts_tablet_manager.cc:1403] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.065s	user 0.020s	sys 0.018s
I20260709 12:21:02.016599 20632 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.019541 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.020345 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.019052 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.020419 20884 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:02.021358 20883 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.022588 20981 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.027951 20981 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.029408 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:02.029935 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.030663 20810 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:02.031059 20849 leader_election.cc:343] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.042580 20924 raft_consensus.cc:359] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.043156 20924 raft_consensus.cc:385] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.043443 20924 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.044344 20810 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:02.044219 20924 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.044677 20927 tablet_bootstrap.cc:492] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:02.045153 20927 ts_tablet_manager.cc:1403] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.085s	user 0.022s	sys 0.010s
I20260709 12:21:02.031088 20883 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:02.045620 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:02.046814 20774 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:02.047469 20981 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.047942 20981 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:02.048535 20927 raft_consensus.cc:359] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.049207 20927 raft_consensus.cc:385] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.048890 20981 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.049495 20927 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.050127 20927 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.047968 20985 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.051698 20985 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:02.052018 20809 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:02.052577 20985 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.053352 20929 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.053769 20929 ts_tablet_manager.cc:1403] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.120s	user 0.015s	sys 0.021s
I20260709 12:21:02.054356 20927 ts_tablet_manager.cc:1434] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:21:02.055655 20927 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:02.056557 20929 raft_consensus.cc:359] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.057199 20929 raft_consensus.cc:385] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.057475 20929 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.059720 20924 ts_tablet_manager.cc:1434] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 12:21:02.053298 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:02.064539 20945 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.064220 20929 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.065267 20945 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:02.066936 20945 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.067752 20929 ts_tablet_manager.cc:1434] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:21:02.070909 20929 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:02.075042 20924 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.084333 20633 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.090451 20956 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.092507 20956 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.090509 20929 tablet_bootstrap.cc:654] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.093497 20631 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:02.110474 20924 tablet_bootstrap.cc:654] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.115047 20929 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.115868 20929 ts_tablet_manager.cc:1403] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.007s
I20260709 12:21:02.118845 20929 raft_consensus.cc:359] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.119817 20929 raft_consensus.cc:385] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.120260 20929 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.128659 20929 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.129091 20981 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.131289 20981 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.132417 20929 ts_tablet_manager.cc:1434] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.002s
I20260709 12:21:02.133652 20929 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:02.134702 20981 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.130962 20949 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.135548 20927 tablet_bootstrap.cc:654] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.135753 20949 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.143843 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.144207 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.144513 20734 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:02.144846 20810 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:02.145061 20949 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.138216 20981 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.146008 20981 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.147984 20981 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.148944 20883 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.150631 20924 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.151118 20924 ts_tablet_manager.cc:1403] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.079s	user 0.016s	sys 0.015s
I20260709 12:21:02.153043 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:02.153450 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.153852 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.156136 20884 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.167835 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.157644 20981 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.171615 20981 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.172039 20981 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.176539 20946 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:02.172870 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.176649 20924 raft_consensus.cc:359] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.177770 20924 raft_consensus.cc:385] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.178066 20924 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.178383 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.178738 20924 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.179551 20949 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.180811 20924 ts_tablet_manager.cc:1434] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:21:02.181787 20924 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.184440 20631 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.185793 20946 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.196740 20924 tablet_bootstrap.cc:654] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.197414 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.196053 20981 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.198204 20929 tablet_bootstrap.cc:654] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.201464 20949 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.203755 20981 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.207993 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.208592 20809 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.210698 20924 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.211309 20924 ts_tablet_manager.cc:1403] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.001s
I20260709 12:21:02.213740 20927 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:02.214357 20927 ts_tablet_manager.cc:1403] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.159s	user 0.012s	sys 0.004s
I20260709 12:21:02.214075 20924 raft_consensus.cc:359] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.214880 20924 raft_consensus.cc:385] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.215232 20924 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.215838 20809 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:02.216095 20924 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.217063 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:02.217825 20949 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.218367 20949 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:02.217898 20927 raft_consensus.cc:359] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.218951 20924 ts_tablet_manager.cc:1434] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:02.218905 20927 raft_consensus.cc:385] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.219617 20927 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.219309 20949 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.220381 20924 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.220698 20927 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.223845 20927 ts_tablet_manager.cc:1434] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:21:02.225029 20927 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:02.227988 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:02.228618 20735 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.269500 20945 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.280421 20956 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.283227 20927 tablet_bootstrap.cc:654] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.290138 20972 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.295559 20927 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:02.296064 20927 ts_tablet_manager.cc:1403] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.071s	user 0.026s	sys 0.009s
I20260709 12:21:02.296718 20735 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:02.297308 20733 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.302594 20924 tablet_bootstrap.cc:654] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.303099 20927 raft_consensus.cc:359] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.307883 20927 raft_consensus.cc:385] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.308261 20927 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.308530 20946 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:02.309013 20927 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.310060 20929 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.310644 20929 ts_tablet_manager.cc:1403] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.177s	user 0.027s	sys 0.031s
I20260709 12:21:02.313612 20929 raft_consensus.cc:359] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.314575 20929 raft_consensus.cc:385] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.315001 20929 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.316730 20929 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.320139 20929 ts_tablet_manager.cc:1434] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:02.321593 20929 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:02.314363 20927 ts_tablet_manager.cc:1434] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:21:02.323081 20927 tablet_bootstrap.cc:492] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:02.324723 20924 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.325188 20924 ts_tablet_manager.cc:1403] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.105s	user 0.013s	sys 0.011s
I20260709 12:21:02.330194 20927 tablet_bootstrap.cc:654] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.331473 20924 raft_consensus.cc:359] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:02.332168 20924 raft_consensus.cc:385] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.332502 20929 tablet_bootstrap.cc:654] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.332470 20924 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.333534 20924 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:02.335644 20924 ts_tablet_manager.cc:1434] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:21:02.336923 20924 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.344050 20924 tablet_bootstrap.cc:654] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.344470 20929 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.344970 20929 ts_tablet_manager.cc:1403] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.004s
I20260709 12:21:02.348083 20929 raft_consensus.cc:359] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.349030 20929 raft_consensus.cc:385] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.348265 20631 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:02.349387 20929 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.350265 20949 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.350843 20949 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.352182 20972 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.352638 20972 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.353215 20949 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.354470 20927 tablet_bootstrap.cc:492] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:02.354794 20972 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.354969 20927 ts_tablet_manager.cc:1403] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.032s	user 0.003s	sys 0.014s
I20260709 12:21:02.355211 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.356369 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.356997 20884 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:02.357694 20929 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.358026 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.357889 20927 raft_consensus.cc:359] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.358836 20927 raft_consensus.cc:385] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.359163 20927 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.359851 20929 ts_tablet_manager.cc:1434] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:21:02.359736 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:21:02.357067 20849 leader_election.cc:343] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.360013 20927 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.360771 20929 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
W20260709 12:21:02.360883 20849 leader_election.cc:343] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.361305 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.361392 20849 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.362047 20809 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:02.362618 20949 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.362726 20927 ts_tablet_manager.cc:1434] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:21:02.363862 20927 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:02.361305 20945 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.364250 20945 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.364727 20945 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.372893 20945 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.376040 20945 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.376720 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.377096 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.377357 20809 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.377650 20884 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.380280 20927 tablet_bootstrap.cc:654] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.383652 20924 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.384101 20924 ts_tablet_manager.cc:1403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.017s
I20260709 12:21:02.384970 20809 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:02.384977 20884 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:02.386852 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:02.386824 20924 raft_consensus.cc:359] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.387645 20924 raft_consensus.cc:385] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.387934 20924 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.388098 20945 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.388844 20929 tablet_bootstrap.cc:654] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.388572 20924 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.389104 20945 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:02.390818 20924 ts_tablet_manager.cc:1434] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.005s
I20260709 12:21:02.390098 20945 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.391844 20924 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.397933 20924 tablet_bootstrap.cc:654] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.399493 20949 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.406105 20946 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.412670 20927 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:02.413167 20927 ts_tablet_manager.cc:1403] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.006s
I20260709 12:21:02.406126 20949 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.414274 20949 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:02.423633 20949 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.420504 20972 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.429029 20972 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.420020 20956 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.430841 20972 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:02.422869 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:02.431773 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.431846 20956 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.433568 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.434207 20734 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:02.435578 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.431879 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.436805 20884 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:02.437321 20927 raft_consensus.cc:359] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.437986 20927 raft_consensus.cc:385] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.431988 20957 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:21:02.438259 20809 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:02.435698 20956 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.438256 20927 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.443231 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.444406 20734 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:02.445823 20775 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:02.446838 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:02.441933 20927 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.452044 20957 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.453966 20957 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.457322 20699 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:02.521215 20927 ts_tablet_manager.cc:1434] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.000s
I20260709 12:21:02.523418 20927 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:02.518990 20981 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.526430 20945 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.526834 20945 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.503898 20956 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.457536 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.527179 20945 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:02.496003 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50290.2us
I20260709 12:21:02.539239 20924 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.539816 20924 ts_tablet_manager.cc:1403] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.148s	user 0.040s	sys 0.058s
W20260709 12:21:02.525663 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 82227us
I20260709 12:21:02.452541 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:02.531495 20956 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.538836 20884 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:02.547897 20956 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.543534 20946 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.544009 20979 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.527992 20981 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:02.544378 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101557us
I20260709 12:21:02.545719 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.549980 20979 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.551791 20979 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.551913 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.542688 20957 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.561830 20981 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.557298 20735 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:02.557916 20949 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.569720 20924 raft_consensus.cc:359] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.570554 20924 raft_consensus.cc:385] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.570811 20924 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.571862 20735 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.557799 20882 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:02.574286 20924 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:02.572780 20849 leader_election.cc:343] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.581689 20631 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.577257 20774 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:02.573748 20949 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.584260 20985 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.584399 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.584704 20985 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.585137 20985 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.585012 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.585747 20949 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.586036 20734 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
W20260709 12:21:02.586920 20775 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:02.582222 20945 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.589812 20927 tablet_bootstrap.cc:654] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.593603 20924 ts_tablet_manager.cc:1434] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.002s
I20260709 12:21:02.594664 20924 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.598066 20972 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.598555 20972 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.599144 20956 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.599858 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.600648 20810 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 1: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:02.602298 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.602870 20882 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.603682 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.605410 20972 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.605840 20945 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.608681 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.609282 20810 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:02.610507 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.612804 20699 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.613878 20972 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.607856 20956 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.610672 20985 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.619729 20809 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 1: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:02.619829 20929 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.620307 20929 ts_tablet_manager.cc:1403] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.260s	user 0.013s	sys 0.005s
I20260709 12:21:02.613826 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:02.623229 20929 raft_consensus.cc:359] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:02.623942 20929 raft_consensus.cc:385] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.624214 20929 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.625350 20945 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:02.625803 20945 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.626276 20945 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:02.624560 20979 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:02.625250 20929 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:02.627123 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.628237 20979 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.628523 20929 ts_tablet_manager.cc:1434] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:02.630420 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.631549 20972 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:02.624768 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:02.632763 20735 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:02.633502 20883 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.635165 20882 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:02.622620 20849 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:02.637869 21026 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.638362 21026 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.638398 20979 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.638715 21026 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.641624 20924 tablet_bootstrap.cc:654] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:02.637820 20849 leader_election.cc:343] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.636850 20929 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:02.644279 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:02.647086 20956 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.647765 20956 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:02.648564 20956 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.650753 21026 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.653539 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:02.654129 21026 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.654229 20733 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.661391 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.662647 20884 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:02.658133 20945 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.663278 20985 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.668797 20972 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:02.669548 20849 leader_election.cc:343] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.669461 21026 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:02.670109 20849 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.670898 20979 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.672930 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:02.673825 20734 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.679047 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.681588 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.682438 20810 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 1: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:02.683282 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:02.676442 20732 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:21:02.687027 20849 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.687687 20808 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.703925 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.704288 20979 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.714493 20979 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.676664 20945 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.723306 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.724023 20979 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.725119 20734 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:02.721336 20949 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.728894 20924 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.729431 20924 ts_tablet_manager.cc:1403] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.135s	user 0.012s	sys 0.008s
I20260709 12:21:02.732780 20929 tablet_bootstrap.cc:654] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.739323 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.740814 21029 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:02.741215 20924 raft_consensus.cc:359] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.741899 20924 raft_consensus.cc:385] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.742204 20924 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.747704 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.742820 20924 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.749455 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:02.751130 20949 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.751662 20949 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.753413 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.753701 20949 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.753722 20927 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:02.755123 20927 ts_tablet_manager.cc:1403] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.233s	user 0.032s	sys 0.032s
I20260709 12:21:02.755450 20979 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.756347 20979 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:02.757328 20927 raft_consensus.cc:359] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.757921 20927 raft_consensus.cc:385] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.758117 20927 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.757611 20979 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } }
I20260709 12:21:02.756356 20949 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:02.760663 20924 ts_tablet_manager.cc:1434] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:21:02.759745 20927 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.761256 21026 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:02.761660 21026 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:02.761782 20924 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.763159 20927 ts_tablet_manager.cc:1434] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:21:02.762588 21026 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.764046 20927 tablet_bootstrap.cc:492] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:02.763912 21026 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:02.766065 20981 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:02.768810 20733 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:02.770434 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:02.771519 20972 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.772058 20972 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.774773 20927 tablet_bootstrap.cc:654] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.775588 20972 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.776105 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:02.777462 20700 leader_election.cc:343] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.772028 20956 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:02.778638 20849 leader_election.cc:343] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.779273 20956 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:02.784729 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.784821 20808 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:02.790517 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.784619 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.799889 20884 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:02.802050 20699 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:02.804751 21029 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.805167 21029 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.805542 21029 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.808815 20924 tablet_bootstrap.cc:654] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.809298 21026 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:02.808769 20956 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:02.809450 20849 leader_election.cc:343] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.814877 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:21:02.817428 20849 leader_election.cc:343] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.818406 20929 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.818888 20929 ts_tablet_manager.cc:1403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.182s	user 0.007s	sys 0.011s
I20260709 12:21:02.822165 20945 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.822696 20945 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.825067 20927 tablet_bootstrap.cc:492] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:02.825366 20945 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.825541 20927 ts_tablet_manager.cc:1403] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.022s
I20260709 12:21:02.826572 20929 raft_consensus.cc:359] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.827620 20929 raft_consensus.cc:385] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.827977 20929 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.828864 20929 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.831566 20929 ts_tablet_manager.cc:1434] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:21:02.832499 20929 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:02.832453 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.832190 20630 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.835968 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:02.835845 20849 leader_election.cc:343] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.821830 21026 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.837234 20849 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.840354 20699 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.841382 20972 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.844920 21029 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.846910 21029 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.849121 20946 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.851016 20946 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.853145 20946 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.853578 21026 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.861389 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.863376 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:02.864403 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.865227 20882 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.865700 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:02.866334 20883 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
W20260709 12:21:02.867552 20700 leader_election.cc:343] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.867583 20774 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:02.873148 20775 leader_election.cc:343] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.875152 20946 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:02.884296 20946 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.884452 20929 tablet_bootstrap.cc:654] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.884644 20927 raft_consensus.cc:359] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.885752 20927 raft_consensus.cc:385] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.886554 21026 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.887085 21026 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.890640 21026 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.884732 20946 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.893216 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.894856 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.886148 20927 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
W20260709 12:21:02.897189 20849 leader_election.cc:343] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:02.897820 20849 leader_election.cc:343] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.897405 20633 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.898511 20849 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.899307 21026 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.906471 20946 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.908049 20924 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.908615 20924 ts_tablet_manager.cc:1403] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.147s	user 0.012s	sys 0.013s
I20260709 12:21:02.909948 20946 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:02.907621 20927 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.910650 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:02.912377 20927 ts_tablet_manager.cc:1434] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.087s	user 0.015s	sys 0.001s
I20260709 12:21:02.913272 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:02.913902 20883 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:02.916211 20924 raft_consensus.cc:359] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.916923 20924 raft_consensus.cc:385] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.917186 20924 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.917794 20924 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.919406 20924 ts_tablet_manager.cc:1434] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:02.919951 21026 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.920372 20924 tablet_bootstrap.cc:492] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.920467 21026 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:02.922049 20775 leader_election.cc:343] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.922626 21026 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:02.924237 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:02.926488 20883 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:02.927135 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:02.927790 20774 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:21:02.928748 20849 leader_election.cc:343] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.929019 20956 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:02.929503 20849 leader_election.cc:343] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:02.929558 20956 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:02.930275 20924 tablet_bootstrap.cc:654] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.930064 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:02.931011 21026 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:02.930783 20956 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.936764 20882 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:02.937417 20631 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.938143 20699 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:02.941672 21043 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:02.939888 20924 tablet_bootstrap.cc:492] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.942596 20924 ts_tablet_manager.cc:1403] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.005s
I20260709 12:21:02.942466 21043 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:02.943820 21043 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.946475 20929 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.947007 20929 ts_tablet_manager.cc:1403] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.115s	user 0.019s	sys 0.000s
I20260709 12:21:02.948640 20924 raft_consensus.cc:359] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.949375 20924 raft_consensus.cc:385] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.949698 20924 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.950441 20924 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.950461 20929 raft_consensus.cc:359] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.951593 20929 raft_consensus.cc:385] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.952000 20929 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.953080 20924 ts_tablet_manager.cc:1434] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.002s
I20260709 12:21:02.952879 20929 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.954110 20924 tablet_bootstrap.cc:492] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.956207 20929 ts_tablet_manager.cc:1434] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:21:02.957587 20929 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:02.960351 20924 tablet_bootstrap.cc:654] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.966339 20929 tablet_bootstrap.cc:654] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.969867 20927 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:02.972527 20924 tablet_bootstrap.cc:492] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.973426 20924 ts_tablet_manager.cc:1403] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:21:02.976085 20924 raft_consensus.cc:359] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.976738 20924 raft_consensus.cc:385] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.977001 20924 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.977718 20924 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.980027 20924 ts_tablet_manager.cc:1434] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:21:02.980988 20924 tablet_bootstrap.cc:492] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:02.986994 20924 tablet_bootstrap.cc:654] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.988720 20633 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.991120 20927 tablet_bootstrap.cc:654] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:02.993170 20929 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:02.993772 20929 ts_tablet_manager.cc:1403] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.015s
I20260709 12:21:02.994170 20924 tablet_bootstrap.cc:492] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:02.994786 20924 ts_tablet_manager.cc:1403] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 12:21:02.997956 20924 raft_consensus.cc:359] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:02.998637 20924 raft_consensus.cc:385] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:02.999243 20924 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:02.998524 20929 raft_consensus.cc:359] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.000309 20929 raft_consensus.cc:385] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.000720 20929 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.001565 20929 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.001713 20631 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:03.003865 20927 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.004197 20929 ts_tablet_manager.cc:1434] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 12:21:03.004413 20927 ts_tablet_manager.cc:1403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.002s
I20260709 12:21:02.999985 20924 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.005352 20929 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:03.007050 20924 ts_tablet_manager.cc:1434] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:21:03.008001 20924 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.007627 20927 raft_consensus.cc:359] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.008407 20927 raft_consensus.cc:385] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.008742 20927 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.009654 20927 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.012677 20929 tablet_bootstrap.cc:654] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.014364 20924 tablet_bootstrap.cc:654] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.014444 20927 ts_tablet_manager.cc:1434] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:21:03.015278 20956 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.015534 20927 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.015798 20956 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.018133 20956 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.019342 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.020067 20883 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:03.021485 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.015318 21043 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:21:03.021569 20774 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:03.027360 20956 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.027817 20956 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.028198 20956 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.029542 21029 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:03.027354 20775 leader_election.cc:343] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.031924 20924 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.032379 20924 ts_tablet_manager.cc:1403] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.009s
I20260709 12:21:03.035219 20924 raft_consensus.cc:359] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.035856 20924 raft_consensus.cc:385] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.036162 20924 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.037038 20924 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.039335 20924 ts_tablet_manager.cc:1434] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:03.040302 20924 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.041666 20733 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.048532 20929 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.049074 20929 ts_tablet_manager.cc:1403] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.001s
I20260709 12:21:03.051214 20956 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.052469 20929 raft_consensus.cc:359] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.053182 20929 raft_consensus.cc:385] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.053470 20929 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.054145 20929 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.056368 20929 ts_tablet_manager.cc:1434] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:03.056465 20927 tablet_bootstrap.cc:654] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.056806 21029 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.057397 20929 tablet_bootstrap.cc:492] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:03.057044 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:03.058893 21029 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.064414 21029 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.065822 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.066704 20882 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 1: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:21:03.067237 20775 leader_election.cc:343] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.059631 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:03.070636 20883 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.057471 20956 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.071903 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.069991 21026 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.074095 20700 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:03.077101 21026 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.079296 21026 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.080416 21029 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.081739 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:03.084487 20849 leader_election.cc:343] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.086778 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:03.088724 20849 leader_election.cc:343] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.089226 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:03.090456 21026 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.091924 21029 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.092097 20883 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:03.093501 20924 tablet_bootstrap.cc:654] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.093462 20774 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:03.098475 20956 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.096371 20929 tablet_bootstrap.cc:654] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.100473 20956 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:03.101411 20956 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.111097 20924 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.111662 20924 ts_tablet_manager.cc:1403] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.002s
I20260709 12:21:03.114367 20924 raft_consensus.cc:359] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.116993 20924 raft_consensus.cc:385] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.117470 20927 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.117974 20927 ts_tablet_manager.cc:1403] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.103s	user 0.017s	sys 0.001s
I20260709 12:21:03.119055 20631 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:03.120321 20924 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.120630 20927 raft_consensus.cc:359] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.121294 20927 raft_consensus.cc:385] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.121538 20927 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.121886 21029 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.122157 20927 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.122498 21029 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.121711 20924 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.124734 20924 ts_tablet_manager.cc:1434] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:21:03.125695 20924 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.126452 21029 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.127247 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.127875 20927 ts_tablet_manager.cc:1434] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
W20260709 12:21:03.128501 20699 leader_election.cc:343] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.128804 20927 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.129977 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.131309 20929 tablet_bootstrap.cc:492] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.131767 20929 ts_tablet_manager.cc:1403] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.075s	user 0.018s	sys 0.002s
I20260709 12:21:03.135322 20981 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.135136 20929 raft_consensus.cc:359] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.135892 20929 raft_consensus.cc:385] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.136230 20929 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.135803 20981 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:03.137310 20700 leader_election.cc:343] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.137786 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:03.138183 20981 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.140116 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.140261 20929 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.141624 20927 tablet_bootstrap.cc:654] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:03.141521 20849 leader_election.cc:343] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.142562 20929 ts_tablet_manager.cc:1434] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:21:03.142771 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.144001 20929 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
W20260709 12:21:03.144619 20849 leader_election.cc:343] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.145087 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:03.145850 21026 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.148609 21029 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.149118 20924 tablet_bootstrap.cc:654] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.154982 20929 tablet_bootstrap.cc:654] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.156843 20981 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:03.196470 20956 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:03.208772 20927 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.209352 20927 ts_tablet_manager.cc:1403] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.081s	user 0.021s	sys 0.017s
I20260709 12:21:03.213431 20981 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.214107 20981 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.218303 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:03.220201 20849 leader_election.cc:343] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.221434 20981 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.221828 20946 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:03.223716 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:03.225731 20849 leader_election.cc:343] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.226368 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:03.227516 21026 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.212612 20927 raft_consensus.cc:359] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.229600 20927 raft_consensus.cc:385] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.230131 20927 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.231360 20927 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.235025 20927 ts_tablet_manager.cc:1434] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:21:03.236009 20927 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.237818 20956 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.238309 20956 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.242025 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:03.244197 20775 leader_election.cc:343] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.251721 20924 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.252511 20924 ts_tablet_manager.cc:1403] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.127s	user 0.009s	sys 0.008s
I20260709 12:21:03.256048 20956 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.257092 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:03.258214 20774 leader_election.cc:343] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.258672 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:03.259859 20956 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.259454 20924 raft_consensus.cc:359] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.262147 20924 raft_consensus.cc:385] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.262573 20924 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.263454 20927 tablet_bootstrap.cc:654] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.263370 20924 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.264350 20946 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:03.265079 20929 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.265537 20929 ts_tablet_manager.cc:1403] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.122s	user 0.050s	sys 0.034s
I20260709 12:21:03.268442 20929 raft_consensus.cc:359] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.269169 20929 raft_consensus.cc:385] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.269466 20929 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.270177 20929 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.272302 20929 ts_tablet_manager.cc:1434] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:03.274252 20929 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:03.278301 20924 ts_tablet_manager.cc:1434] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:21:03.279400 20924 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.283706 20979 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:03.290403 20924 tablet_bootstrap.cc:654] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.294471 20956 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:03.307720 20927 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.307969 20956 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.308261 20927 ts_tablet_manager.cc:1403] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.072s	user 0.008s	sys 0.013s
I20260709 12:21:03.308380 20929 tablet_bootstrap.cc:654] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.308504 20956 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.312327 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.313010 20733 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:03.314210 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:03.315060 21025 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.315474 21025 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.315837 21025 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.311983 20927 raft_consensus.cc:359] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.316715 20927 raft_consensus.cc:385] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.317018 20927 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.319054 20927 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.322981 20927 ts_tablet_manager.cc:1434] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:21:03.323983 20927 tablet_bootstrap.cc:492] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.334906 20924 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.336036 20924 ts_tablet_manager.cc:1403] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.057s	user 0.019s	sys 0.013s
I20260709 12:21:03.336122 20927 tablet_bootstrap.cc:654] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.338699 20924 raft_consensus.cc:359] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.339430 20924 raft_consensus.cc:385] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.339742 20924 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.340453 20924 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.344844 20985 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:03.348982 21025 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.352052 20924 ts_tablet_manager.cc:1434] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.001s
I20260709 12:21:03.361825 20924 tablet_bootstrap.cc:492] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.361948 20927 tablet_bootstrap.cc:492] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.368925 20927 ts_tablet_manager.cc:1403] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.003s
I20260709 12:21:03.369830 20924 tablet_bootstrap.cc:654] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.370781 20979 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.371541 20979 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.372274 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:03.372953 20732 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.375700 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.376681 20735 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:03.378538 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:03.379920 20949 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.380487 20949 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.380973 20949 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.384624 21025 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.385610 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:03.386574 20882 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.387988 20979 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.390350 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:03.391976 20849 leader_election.cc:343] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.393417 20956 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.393635 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.400063 20985 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:03.407306 20732 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:03.408924 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:03.410022 20985 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.410820 20985 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:03.411871 20985 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.421666 20946 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.422067 20927 raft_consensus.cc:359] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.422284 20946 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.426476 20946 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.426811 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.432030 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.434432 20882 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:03.434585 20810 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.423125 20927 raft_consensus.cc:385] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.430341 20949 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.439450 20949 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.439777 20775 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:03.441103 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:03.442523 20849 leader_election.cc:343] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.443392 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:03.444008 20734 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.447343 21025 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.436805 20927 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.449360 20927 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.464804 20972 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:03.465139 20927 ts_tablet_manager.cc:1434] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.002s
I20260709 12:21:03.466470 20927 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.478158 20927 tablet_bootstrap.cc:654] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.483562 20929 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.484041 20929 ts_tablet_manager.cc:1403] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.210s	user 0.055s	sys 0.075s
I20260709 12:21:03.483966 20630 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:03.485584 20924 tablet_bootstrap.cc:492] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.486032 20924 ts_tablet_manager.cc:1403] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.125s	user 0.027s	sys 0.038s
I20260709 12:21:03.486963 21029 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.487453 21029 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.487571 20734 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:03.489203 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:03.490674 20949 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.491317 20949 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:03.486739 20929 raft_consensus.cc:359] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.492169 20929 raft_consensus.cc:385] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.492380 20927 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.491827 21029 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.492496 20929 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.492276 20949 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.496368 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.496474 20929 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.497342 20883 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:03.498899 20929 ts_tablet_manager.cc:1434] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:21:03.497711 21029 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:03.499861 20929 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:03.488538 20924 raft_consensus.cc:359] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.493732 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.498701 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:03.503367 20924 raft_consensus.cc:385] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.503688 20924 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.493258 20927 ts_tablet_manager.cc:1403] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.008s
I20260709 12:21:03.504313 20924 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:03.504462 20700 leader_election.cc:343] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.506425 20924 ts_tablet_manager.cc:1434] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:21:03.507357 20924 tablet_bootstrap.cc:492] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.508059 21029 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.508447 21029 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.508769 21029 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.507486 20927 raft_consensus.cc:359] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.513554 20927 raft_consensus.cc:385] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.513867 20927 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.514952 20808 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.514716 20927 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.517261 20929 tablet_bootstrap.cc:654] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.513563 20924 tablet_bootstrap.cc:654] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.524883 20927 ts_tablet_manager.cc:1434] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:21:03.526044 20927 tablet_bootstrap.cc:492] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.536937 20949 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.537500 20949 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.538709 20979 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.539170 20979 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.541188 20979 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.532855 21029 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.543558 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.544221 20924 tablet_bootstrap.cc:492] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.544665 20924 ts_tablet_manager.cc:1403] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.001s
I20260709 12:21:03.544953 20949 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:03.545584 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:03.543089 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.547313 20924 raft_consensus.cc:359] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.548045 20924 raft_consensus.cc:385] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.548312 20924 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
W20260709 12:21:03.548681 20849 leader_election.cc:343] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.549026 20924 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.547307 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:03.551385 20849 leader_election.cc:343] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.551802 20924 ts_tablet_manager.cc:1434] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:21:03.552652 20924 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.544154 20807 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:03.554243 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.553620 20629 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:03.555784 20849 leader_election.cc:343] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.556277 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:03.557920 20981 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.558518 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:03.558650 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:03.559115 20924 tablet_bootstrap.cc:654] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.559113 20883 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.561997 20981 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.562995 20981 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.563375 20981 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.563501 20927 tablet_bootstrap.cc:654] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.564608 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:03.566174 21029 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.567677 20924 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.568148 20924 ts_tablet_manager.cc:1403] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.005s
I20260709 12:21:03.570693 20924 raft_consensus.cc:359] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.571364 20924 raft_consensus.cc:385] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.571633 20924 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.572328 20924 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.573628 20927 tablet_bootstrap.cc:492] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.574091 20927 ts_tablet_manager.cc:1403] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.000s
I20260709 12:21:03.574399 20924 ts_tablet_manager.cc:1434] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
W20260709 12:21:03.575179 20700 leader_election.cc:343] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.575330 20924 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.576952 20927 raft_consensus.cc:359] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.577662 20927 raft_consensus.cc:385] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.577934 20927 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.578716 20927 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.580806 20927 ts_tablet_manager.cc:1434] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:03.581693 20927 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.581848 20924 tablet_bootstrap.cc:654] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.594492 20927 tablet_bootstrap.cc:654] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.605147 20929 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.605661 20929 ts_tablet_manager.cc:1403] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.106s	user 0.007s	sys 0.006s
I20260709 12:21:03.605486 20981 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.611778 20929 raft_consensus.cc:359] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.612267 20929 raft_consensus.cc:385] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.612511 20929 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.616230 20929 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.627141 20929 ts_tablet_manager.cc:1434] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:21:03.628957 20929 tablet_bootstrap.cc:492] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:03.625880 20981 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.630607 20883 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:03.641574 20972 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.643625 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:03.644086 20972 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.640929 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:03.645300 21029 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.643517 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:03.645287 21084 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:21:03.646103 20924 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.645956 20733 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.647122 20924 ts_tablet_manager.cc:1403] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.016s
I20260709 12:21:03.646447 21029 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:03.648129 20809 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:03.646872 20775 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:03.650470 21029 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.656984 20972 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.658115 20733 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:03.659377 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.660044 20882 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:03.660264 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:03.661091 20949 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.661558 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:03.661572 20949 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:03.662703 20945 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.663179 20945 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.662654 20949 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.663537 20945 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.665192 20809 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:03.668329 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.669955 21025 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:03.670573 20924 raft_consensus.cc:359] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:03.670280 20700 leader_election.cc:343] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.671252 20924 raft_consensus.cc:385] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.671533 20924 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.672464 20924 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.675617 20924 ts_tablet_manager.cc:1434] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:21:03.676448 20924 tablet_bootstrap.cc:492] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.681303 20929 tablet_bootstrap.cc:654] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.685667 20945 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.696461 20945 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.697670 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:03.697670 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:03.698559 20883 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:03.699324 20700 leader_election.cc:343] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.701695 20927 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.702219 20927 ts_tablet_manager.cc:1403] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.121s	user 0.016s	sys 0.001s
I20260709 12:21:03.701586 20631 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:03.702726 20945 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.703250 20945 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.708792 20929 tablet_bootstrap.cc:492] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.709278 20929 ts_tablet_manager.cc:1403] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.080s	user 0.007s	sys 0.004s
I20260709 12:21:03.712908 20949 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.713569 20949 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.729985 20929 raft_consensus.cc:359] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.730849 20929 raft_consensus.cc:385] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.731132 20929 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.731742 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.731822 20929 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:03.733537 20849 leader_election.cc:343] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.734090 20929 ts_tablet_manager.cc:1434] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.015s
I20260709 12:21:03.735031 20929 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:03.739285 20927 raft_consensus.cc:359] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.739997 20927 raft_consensus.cc:385] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.740826 20927 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.741595 20946 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.742409 20946 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.743403 20927 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.745143 20883 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:03.745687 20949 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.747140 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.747416 20629 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:03.749696 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.750366 20884 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:03.751629 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:03.756721 20927 ts_tablet_manager.cc:1434] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 12:21:03.757687 20927 tablet_bootstrap.cc:492] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.759833 20972 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.760268 20972 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.760602 20972 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:03.763885 20849 leader_election.cc:343] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.764401 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:03.768009 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:03.769264 21029 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.769754 21029 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:03.770721 20949 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.770581 21029 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.756665 20945 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.775023 20929 tablet_bootstrap.cc:654] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.780083 20949 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.780861 20949 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.784323 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.785202 20949 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.785223 20733 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:03.786478 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.787683 20807 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:03.789582 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:03.790371 20946 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.793097 20949 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.796051 20949 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.796934 20949 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.798538 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.799353 20883 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:03.799842 20927 tablet_bootstrap.cc:654] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.803033 20972 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.804755 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:03.805936 20700 leader_election.cc:343] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.806622 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:03.807224 20924 tablet_bootstrap.cc:654] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.807279 20883 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.811651 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.813071 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:03.808914 20972 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:03.822371 20700 leader_election.cc:343] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.823342 20775 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:03.824261 20956 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.821105 20631 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:03.828562 20927 tablet_bootstrap.cc:492] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.829037 20927 ts_tablet_manager.cc:1403] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.007s
I20260709 12:21:03.832331 20949 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.833333 20956 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.834702 20949 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.835224 20883 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:03.836560 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:03.837339 20807 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.839851 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:03.839807 20927 raft_consensus.cc:359] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.841485 20972 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.841661 20927 raft_consensus.cc:385] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.842005 20972 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:03.842109 20927 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.842854 20972 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.843007 20927 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.846292 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:03.846447 20927 ts_tablet_manager.cc:1434] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 12:21:03.846902 20733 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.847886 20927 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.849944 20945 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.850493 20945 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.852824 20945 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.853010 20929 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.854669 20929 ts_tablet_manager.cc:1403] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.120s	user 0.030s	sys 0.003s
I20260709 12:21:03.854635 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.855021 20927 tablet_bootstrap.cc:654] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.855379 20883 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:03.856726 20699 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:03.857473 20945 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.857888 20945 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.858259 20945 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.858702 20733 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:03.861207 20807 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:03.864866 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:03.865635 20949 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.865525 20929 raft_consensus.cc:359] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.866216 20929 raft_consensus.cc:385] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.866626 20929 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.866178 20949 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:03.867803 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.867937 20949 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.868193 20945 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:03.869079 20700 leader_election.cc:343] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.869566 20927 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.870013 20927 ts_tablet_manager.cc:1403] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.003s
I20260709 12:21:03.871343 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:03.872674 20700 leader_election.cc:343] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.872547 20929 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.872704 20927 raft_consensus.cc:359] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.873569 20927 raft_consensus.cc:385] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.873883 20927 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.874526 20927 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.875217 20945 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.874835 20929 ts_tablet_manager.cc:1434] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:21:03.874856 20946 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:03.876758 20929 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:03.880281 20927 ts_tablet_manager.cc:1434] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:21:03.882895 20927 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:03.883324 20929 tablet_bootstrap.cc:654] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.884424 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:03.885111 20882 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.888644 20927 tablet_bootstrap.cc:654] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.885716 20946 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.891319 20946 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.892882 20631 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:03.902220 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:03.907245 20775 leader_election.cc:343] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.910809 20882 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:03.932608 20699 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:03.934069 20972 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:03.934690 20972 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:03.935784 20972 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.972865 20924 tablet_bootstrap.cc:492] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:03.973974 20929 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:03.976730 20924 ts_tablet_manager.cc:1403] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.300s	user 0.040s	sys 0.014s
I20260709 12:21:03.976763 20929 ts_tablet_manager.cc:1403] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.100s	user 0.019s	sys 0.023s
I20260709 12:21:03.941772 20629 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:03.950424 20956 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:21:03.963166 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:03.961241 20946 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:03.979561 20924 raft_consensus.cc:359] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.980283 20924 raft_consensus.cc:385] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.980566 20924 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
W20260709 12:21:03.980966 20774 leader_election.cc:343] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.981227 20924 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.981612 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:03.982194 20979 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:03.983064 20979 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.984652 20927 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:03.985081 20979 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:03.985476 20924 ts_tablet_manager.cc:1434] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:03.986361 20924 tablet_bootstrap.cc:492] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:03.987126 20929 raft_consensus.cc:359] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.988095 20929 raft_consensus.cc:385] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.988261 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:03.988457 20929 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.987569 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:03.989679 20732 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:03.991149 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:21:03.992062 20849 leader_election.cc:343] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:03.991645 20929 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.992959 20924 tablet_bootstrap.cc:654] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:03.993021 20979 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:03.993438 20979 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:03.993791 20979 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:03.994580 20929 ts_tablet_manager.cc:1434] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.005s
I20260709 12:21:03.985113 20927 ts_tablet_manager.cc:1403] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.051s
I20260709 12:21:03.996279 20956 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:03.997864 20927 raft_consensus.cc:359] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:03.998543 20927 raft_consensus.cc:385] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:03.998848 20927 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:03.999552 20927 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.001096 20927 ts_tablet_manager.cc:1434] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:04.002046 20927 tablet_bootstrap.cc:492] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
W20260709 12:21:03.975956 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 43483.8us
I20260709 12:21:04.006685 20929 tablet_bootstrap.cc:492] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:04.009285 20927 tablet_bootstrap.cc:654] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.008592 20979 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.012807 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:04.013617 20732 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.029034 20979 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.030119 20631 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.032279 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:04.035730 20849 leader_election.cc:343] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.051026 20732 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
W20260709 12:21:04.051647 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:04.053315 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:04.057518 20972 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.058338 20949 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.058825 20981 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:04.059451 20981 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:04.059041 20949 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.065419 20949 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.060842 20981 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.095564 20924 tablet_bootstrap.cc:492] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:04.096326 20924 ts_tablet_manager.cc:1403] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.110s	user 0.021s	sys 0.038s
I20260709 12:21:04.089478 20949 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.100073 20949 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.102216 20949 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.102687 20949 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.108626 20927 tablet_bootstrap.cc:492] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:04.109108 20927 ts_tablet_manager.cc:1403] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.107s	user 0.012s	sys 0.007s
I20260709 12:21:04.115226 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.116429 20945 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.119285 20929 tablet_bootstrap.cc:654] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.103109 20949 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.127357 20949 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.132126 20927 raft_consensus.cc:359] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.132694 20927 raft_consensus.cc:385] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.132863 20927 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.137995 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.144291 20924 raft_consensus.cc:359] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.145313 20924 raft_consensus.cc:385] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.145680 20924 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.146800 20924 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.147680 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.137225 20927 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.148927 20924 ts_tablet_manager.cc:1434] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 12:21:04.149747 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.150158 20927 ts_tablet_manager.cc:1434] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:21:04.149845 20924 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:04.150429 20732 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:04.152592 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.153252 20927 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:04.155625 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:04.157228 20849 leader_election.cc:343] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.157931 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:04.158694 20949 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:21:04.158823 20849 leader_election.cc:343] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.159137 20949 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.159641 20949 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:04.162473 20631 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:04.171386 20849 leader_election.cc:343] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.171990 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:21:04.175539 20849 leader_election.cc:343] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.176141 20849 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:04.175592 20981 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:04.176925 20979 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:04.177809 20979 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.178232 20979 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.181592 20972 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.182308 20972 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.180375 20979 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.185949 21043 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.186738 21043 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.186005 20956 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:04.191270 20956 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.194065 20927 tablet_bootstrap.cc:654] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.200623 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.200762 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.201411 20732 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:04.201758 20929 tablet_bootstrap.cc:492] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:04.202325 20972 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.202229 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.202207 20929 ts_tablet_manager.cc:1403] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.196s	user 0.023s	sys 0.017s
I20260709 12:21:04.203081 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.198876 20956 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.194881 21043 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:04.205977 20700 leader_election.cc:343] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:04.206578 20700 leader_election.cc:343] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.205737 20979 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:04.206815 20849 leader_election.cc:343] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.208343 20929 raft_consensus.cc:359] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.209015 20929 raft_consensus.cc:385] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.209270 20929 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.210012 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.209904 20929 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.211155 20884 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:04.211524 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.213375 20883 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:04.204465 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.216348 20929 ts_tablet_manager.cc:1434] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:21:04.217062 20929 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:04.219913 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.221432 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:04.222937 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:04.223757 20972 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:04.224128 20972 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.224262 21043 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:04.224483 20972 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.224833 21043 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.225266 21043 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.225859 20979 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.231312 20972 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:04.232297 20849 leader_election.cc:343] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.232757 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:04.233945 20972 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.234126 20979 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:04.234853 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:04.236481 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:04.237062 20884 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.238444 21043 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.241540 20929 tablet_bootstrap.cc:654] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.242251 21043 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.244171 20884 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:04.245286 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:04.246098 21043 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:04.246833 21043 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:21:04.247223 20700 leader_election.cc:343] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.247816 21043 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.251837 20924 tablet_bootstrap.cc:654] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.250793 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:04.252313 20809 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.255002 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:04.255616 20884 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.259562 20949 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.263995 20883 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
I20260709 12:21:04.266999 20774 leader_election.cc:400] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
I20260709 12:21:04.267311 20956 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.267344 20774 leader_election.cc:403] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:21:04.267799 20956 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.261304 20929 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:04.268069 20979 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.268810 20929 ts_tablet_manager.cc:1403] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.052s	user 0.005s	sys 0.008s
I20260709 12:21:04.269841 20956 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.270331 20956 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:04.261837 21029 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:21:04.273793 20927 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:04.273664 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.274245 20927 ts_tablet_manager.cc:1403] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.123s	user 0.011s	sys 0.001s
I20260709 12:21:04.274370 20732 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:04.276329 20775 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:04.276836 20927 raft_consensus.cc:359] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.277549 20927 raft_consensus.cc:385] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.277774 21094 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:04.278360 21094 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.278781 21094 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.271497 20929 raft_consensus.cc:359] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.279359 20929 raft_consensus.cc:385] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.279673 20929 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.280936 20949 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.282768 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:04.284976 20956 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
I20260709 12:21:04.287717 20929 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.277837 20927 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.285079 20884 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:04.291627 20929 ts_tablet_manager.cc:1434] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:21:04.292153 20927 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.307013 20927 ts_tablet_manager.cc:1434] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.002s
I20260709 12:21:04.305552 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.308216 20927 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:04.308560 20884 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:04.311527 20929 tablet_bootstrap.cc:492] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Bootstrap starting.
W20260709 12:21:04.311578 20700 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:04.307511 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:04.316694 20979 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:04.321417 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:04.322552 20732 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:04.323640 21029 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:04.324256 21029 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:04.325196 21029 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.325538 21043 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.331930 20949 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.317333 21094 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:04.335340 21120 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:21:04.336721 20806 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.334950 21094 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.340229 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:04.340873 20882 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.344967 20808 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:04.347290 20946 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.347424 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:04.348083 20733 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.347873 20946 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.350251 20946 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.353142 20630 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.357300 20924 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:04.357743 20924 ts_tablet_manager.cc:1403] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.208s	user 0.012s	sys 0.019s
I20260709 12:21:04.360774 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:04.362118 20949 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:04.362591 20949 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:04.363402 20949 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.371343 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.372320 20881 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:04.373715 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.391876 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:04.405095 21094 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.409147 20882 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:04.412413 20809 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:04.436288 20924 raft_consensus.cc:359] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.437068 20924 raft_consensus.cc:385] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.437322 20924 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.438943 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:04.439864 20946 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:04.440384 20946 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:04.441365 20924 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.443099 20924 ts_tablet_manager.cc:1434] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.085s	user 0.005s	sys 0.003s
I20260709 12:21:04.444104 20924 tablet_bootstrap.cc:492] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:04.433399 20733 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:04.457098 20929 tablet_bootstrap.cc:654] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.457538 20946 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:04.513025 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74818.7us
I20260709 12:21:04.520316 20927 tablet_bootstrap.cc:654] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.521665 20972 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.525396 20972 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.527647 20972 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.530112 20732 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:04.531479 21094 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:04.533023 21125 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.069s	user 0.000s	sys 0.004s
I20260709 12:21:04.530754 21029 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:04.533634 20972 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:04.536887 21094 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.539561 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.544793 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.545383 20972 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:04.549335 20699 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:21:04.551597 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:04.555399 20630 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:04.537374 21094 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.559130 21094 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.552910 20972 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:04.585045 20924 tablet_bootstrap.cc:654] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.592617 21026 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:04.601092 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:04.607909 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:04.616086 20929 tablet_bootstrap.cc:492] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:04.616607 20929 ts_tablet_manager.cc:1403] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.305s	user 0.011s	sys 0.012s
I20260709 12:21:04.619263 20929 raft_consensus.cc:359] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.625667 20929 raft_consensus.cc:385] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.626034 20929 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.624531 20630 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.626802 20929 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.630820 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.633420 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:04.632470 20929 ts_tablet_manager.cc:1434] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.002s
I20260709 12:21:04.637989 20929 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:04.651928 21094 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:04.654321 20924 tablet_bootstrap.cc:492] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:04.654801 20924 ts_tablet_manager.cc:1403] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.211s	user 0.006s	sys 0.009s
I20260709 12:21:04.661918 20927 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:04.662434 20927 ts_tablet_manager.cc:1403] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.354s	user 0.034s	sys 0.026s
I20260709 12:21:04.665174 20927 raft_consensus.cc:359] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.665890 20927 raft_consensus.cc:385] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.666169 20927 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.666106 20924 raft_consensus.cc:359] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.666765 20924 raft_consensus.cc:385] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.667013 20924 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.667083 20927 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.667650 20924 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.669495 20929 tablet_bootstrap.cc:654] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.674510 20924 ts_tablet_manager.cc:1434] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:21:04.675545 20924 tablet_bootstrap.cc:492] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:04.696995 20927 ts_tablet_manager.cc:1434] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.004s
I20260709 12:21:04.698046 20927 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
W20260709 12:21:04.696995 20700 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:04.702556 20927 tablet_bootstrap.cc:654] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.706701 20929 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:04.706333 20631 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.707130 20929 ts_tablet_manager.cc:1403] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.001s
I20260709 12:21:04.707942 20927 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:04.708369 20927 ts_tablet_manager.cc:1403] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:21:04.709767 20924 tablet_bootstrap.cc:654] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.710469 20929 raft_consensus.cc:359] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.711269 20929 raft_consensus.cc:385] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.711313 20927 raft_consensus.cc:359] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.711715 20929 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.712051 20927 raft_consensus.cc:385] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.712363 20927 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.712981 20927 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.712771 20929 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.715566 20927 ts_tablet_manager.cc:1434] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:04.716501 20927 tablet_bootstrap.cc:492] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:04.717162 20929 ts_tablet_manager.cc:1434] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:21:04.718391 20929 tablet_bootstrap.cc:492] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:04.720676 20924 tablet_bootstrap.cc:492] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:04.721161 20924 ts_tablet_manager.cc:1403] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.000s
I20260709 12:21:04.722224 20927 tablet_bootstrap.cc:654] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.724273 20929 tablet_bootstrap.cc:654] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.723784 20924 raft_consensus.cc:359] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.731391 20924 raft_consensus.cc:385] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.731747 20924 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.732975 21094 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.733448 21094 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.734174 20927 tablet_bootstrap.cc:492] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:04.734637 20927 ts_tablet_manager.cc:1403] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:21:04.735579 21094 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.737061 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.737244 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.737900 20927 raft_consensus.cc:359] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.738880 20927 raft_consensus.cc:385] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.739159 20927 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.739096 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:04.740274 21094 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:04.739931 20927 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.732514 20924 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.743139 20924 ts_tablet_manager.cc:1434] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:21:04.743710 20927 ts_tablet_manager.cc:1434] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:21:04.744094 20924 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:04.744581 20927 tablet_bootstrap.cc:492] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:04.749212 20929 tablet_bootstrap.cc:492] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:04.749734 20929 ts_tablet_manager.cc:1403] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 12:21:04.751497 20927 tablet_bootstrap.cc:654] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.752286 20929 raft_consensus.cc:359] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.752983 20929 raft_consensus.cc:385] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.753263 20929 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.753978 20929 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.756103 20929 ts_tablet_manager.cc:1434] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:04.756286 20927 tablet_bootstrap.cc:492] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:04.756738 20927 ts_tablet_manager.cc:1403] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:21:04.756893 20929 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:04.759352 20927 raft_consensus.cc:359] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.760030 20927 raft_consensus.cc:385] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.760308 20927 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.760922 20927 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.761861 20733 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.762890 20927 ts_tablet_manager.cc:1434] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:04.763293 20924 tablet_bootstrap.cc:654] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.763764 20927 tablet_bootstrap.cc:492] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:04.766953 21127 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.767524 21127 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.769007 20929 tablet_bootstrap.cc:654] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.769209 20927 tablet_bootstrap.cc:654] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.769690 21127 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.773093 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.773908 20735 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:04.777273 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:04.778786 21094 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:04.779269 21094 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.779686 21094 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.787179 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:04.788062 20883 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:04.796640 20809 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:04.798700 21026 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:04.813851 21094 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.819833 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:04.820676 20732 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.822861 21026 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.823427 21026 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.825609 21026 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.826393 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:04.827617 20735 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:04.830830 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:04.831876 21026 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:04.832377 21026 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.832806 21026 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.832653 21043 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:04.836834 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:04.839566 20849 leader_election.cc:343] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.852310 21026 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.854838 21043 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.855479 21043 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.858912 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:04.860886 20700 leader_election.cc:343] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.863703 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:04.865651 20699 leader_election.cc:343] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.866240 20735 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:04.866218 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:04.869414 21043 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.869481 21138 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:04.871115 21118 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:04.881170 21138 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:04.882685 21029 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:04.885721 21118 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.886435 21118 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.890848 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:04.892459 20849 leader_election.cc:343] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.894222 20806 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.901675 21118 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.902757 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.903604 20809 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:04.905043 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:04.908776 21118 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:04.909264 21118 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.909674 21118 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.916280 20732 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:04.918046 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:04.919493 21127 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:04.920672 21127 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:04.921844 21127 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.946576 21094 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:04.948112 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:04.949918 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:04.950639 20882 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.955472 21025 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.957635 20882 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:04.949057 21026 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:04.959945 20849 leader_election.cc:343] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.963913 21094 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:04.963515 20924 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:04.971370 20924 ts_tablet_manager.cc:1403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.227s	user 0.062s	sys 0.034s
I20260709 12:21:04.982074 20924 raft_consensus.cc:359] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.982916 20924 raft_consensus.cc:385] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.983284 20924 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.984009 20924 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.986027 20924 ts_tablet_manager.cc:1434] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 12:21:04.986150 20927 tablet_bootstrap.cc:492] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:04.986649 20927 ts_tablet_manager.cc:1403] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.223s	user 0.051s	sys 0.095s
I20260709 12:21:04.987048 20924 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
W20260709 12:21:04.964476 21151 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:21:04.963558 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:04.989151 20927 raft_consensus.cc:359] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.967247 20929 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:04.989828 20733 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:04.970623 20981 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.990136 20929 ts_tablet_manager.cc:1403] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.233s	user 0.031s	sys 0.062s
I20260709 12:21:04.990583 20981 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.992854 20929 raft_consensus.cc:359] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.993549 20929 raft_consensus.cc:385] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.993822 20929 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:04.994109 20924 tablet_bootstrap.cc:654] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:04.994545 20981 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:04.994469 20929 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:04.996666 20929 ts_tablet_manager.cc:1434] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:04.996582 20733 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:04.997144 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:04.989809 20927 raft_consensus.cc:385] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:04.997808 20929 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:04.997897 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:21:05.001915 20849 leader_election.cc:343] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.002720 20981 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:05.003407 20981 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:05.004433 20981 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.011029 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:05.015141 20849 leader_election.cc:343] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:04.999132 20927 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.015702 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.019412 20927 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.025588 20927 ts_tablet_manager.cc:1434] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:21:05.027082 20927 tablet_bootstrap.cc:492] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.019160 21026 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.068382 20946 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.068887 20946 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.070433 20929 tablet_bootstrap.cc:654] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.070749 20946 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.071557 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.072206 21118 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.077585 20924 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.078040 20924 ts_tablet_manager.cc:1403] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.091s	user 0.031s	sys 0.019s
I20260709 12:21:05.078150 20806 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.081554 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:05.073272 20633 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:05.082942 20849 leader_election.cc:343] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.084614 20924 raft_consensus.cc:359] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.085314 20924 raft_consensus.cc:385] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.085602 20924 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.086344 20924 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.089469 20924 ts_tablet_manager.cc:1434] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:05.090451 20924 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.094641 20927 tablet_bootstrap.cc:654] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.095352 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.096097 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:05.107548 20810 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.110496 20981 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.111040 20981 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.113083 20981 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.111517 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:05.117288 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.117655 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:05.119086 20849 leader_election.cc:343] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:05.119710 20849 leader_election.cc:343] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.119714 20810 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:05.120261 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.121176 20981 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.121608 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:05.122788 20981 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:05.123279 20981 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:05.101409 20924 tablet_bootstrap.cc:654] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.123946 20981 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.125828 20927 tablet_bootstrap.cc:492] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.126313 20927 ts_tablet_manager.cc:1403] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.000s
I20260709 12:21:05.127722 21118 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.131493 20927 raft_consensus.cc:359] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.132232 20927 raft_consensus.cc:385] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.132542 20927 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.131893 20631 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:05.135025 21127 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.136139 20929 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.136606 20929 ts_tablet_manager.cc:1403] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.139s	user 0.028s	sys 0.032s
I20260709 12:21:05.139122 20929 raft_consensus.cc:359] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.139850 20929 raft_consensus.cc:385] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.140110 20929 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.142035 20927 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.143108 20924 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.143582 20924 ts_tablet_manager.cc:1403] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.005s
I20260709 12:21:05.144574 20927 ts_tablet_manager.cc:1434] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:21:05.145422 20927 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.145771 20929 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.146026 20924 raft_consensus.cc:359] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.147137 20924 raft_consensus.cc:385] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.147446 20924 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.148079 20924 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.149048 20929 ts_tablet_manager.cc:1434] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:05.150028 20929 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.152026 20927 tablet_bootstrap.cc:654] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.155982 20929 tablet_bootstrap.cc:654] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.158493 21029 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.159021 21029 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.161015 20924 ts_tablet_manager.cc:1434] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:21:05.161214 21029 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.167086 20924 tablet_bootstrap.cc:492] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.168043 20929 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.168540 20929 ts_tablet_manager.cc:1403] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 12:21:05.168655 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.169363 20884 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:05.170722 20699 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:05.171159 20929 raft_consensus.cc:359] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.171901 20929 raft_consensus.cc:385] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.172173 20929 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.172294 21043 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.172694 21043 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.173475 20924 tablet_bootstrap.cc:654] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.174809 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.173030 20929 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:05.180576 20700 leader_election.cc:343] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.186338 20929 ts_tablet_manager.cc:1434] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.003s
I20260709 12:21:05.187393 20929 tablet_bootstrap.cc:492] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.173038 21043 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.193019 20924 tablet_bootstrap.cc:492] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.193553 20924 ts_tablet_manager.cc:1403] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.012s
I20260709 12:21:05.193056 20629 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:05.196244 20924 raft_consensus.cc:359] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.198342 20924 raft_consensus.cc:385] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.198694 20924 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.196440 21043 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.196987 20929 tablet_bootstrap.cc:654] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.197803 20981 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.199635 20981 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.201465 20924 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.202127 20981 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.202626 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.202626 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:05.204093 20849 leader_election.cc:343] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.204136 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:05.204572 20700 leader_election.cc:343] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.199074 20927 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.205536 20927 ts_tablet_manager.cc:1403] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.004s
I20260709 12:21:05.207036 20924 ts_tablet_manager.cc:1434] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:21:05.208029 20924 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.208705 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:05.209261 20884 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:05.211855 20849 leader_election.cc:343] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.212415 20849 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.213271 20981 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.218247 20924 tablet_bootstrap.cc:654] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.224159 20924 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.224560 20929 tablet_bootstrap.cc:492] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.224677 20924 ts_tablet_manager.cc:1403] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 12:21:05.225046 20929 ts_tablet_manager.cc:1403] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.012s
I20260709 12:21:05.228052 20929 raft_consensus.cc:359] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.228695 20929 raft_consensus.cc:385] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.228971 20929 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.229714 20929 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.231577 20946 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.231698 20929 ts_tablet_manager.cc:1434] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:21:05.232162 21043 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.232014 20946 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.233306 20929 tablet_bootstrap.cc:492] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.234179 20946 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.234778 20884 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:05.236720 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.237394 20881 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:05.237326 20927 raft_consensus.cc:359] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.238147 20927 raft_consensus.cc:385] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.238452 20927 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.239461 20699 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:05.240087 20929 tablet_bootstrap.cc:654] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.241014 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:05.242002 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.242622 20732 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:05.239064 20927 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.245805 21025 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.246352 21025 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.248488 21025 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.248984 21025 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.249318 21025 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.249666 21025 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.249330 20927 ts_tablet_manager.cc:1434] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.005s
I20260709 12:21:05.252036 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.252871 20731 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:05.254688 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.256253 21043 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:05.259660 20924 raft_consensus.cc:359] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.260149 21043 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:05.260465 20924 raft_consensus.cc:385] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.260882 20924 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.261617 20924 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.252038 20929 tablet_bootstrap.cc:492] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.263592 20929 ts_tablet_manager.cc:1403] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.003s
I20260709 12:21:05.265360 21025 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.266956 20924 ts_tablet_manager.cc:1434] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.001s
I20260709 12:21:05.267674 21025 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.268289 20924 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.269212 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:05.261098 21043 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.272840 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:05.272655 20929 raft_consensus.cc:359] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.273241 20927 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.273378 20929 raft_consensus.cc:385] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.273382 20731 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.273747 20929 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.271061 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:05.274389 20929 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.274626 20946 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.278879 20929 ts_tablet_manager.cc:1434] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 12:21:05.281366 20929 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.281314 20806 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.277153 21025 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.292789 20946 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.294850 20924 tablet_bootstrap.cc:654] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.276968 20881 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.289426 21025 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.299970 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:05.301606 20775 leader_election.cc:343] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.314549 21025 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.319916 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:05.321341 20774 leader_election.cc:343] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.321568 20927 tablet_bootstrap.cc:654] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.321827 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:05.323103 21025 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.327633 20929 tablet_bootstrap.cc:654] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.331115 20731 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:05.332319 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:05.333266 21025 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:05.333784 21025 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:05.337134 21025 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.333853 20881 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:05.346522 20972 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.347290 20924 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.347069 20972 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.347862 20924 ts_tablet_manager.cc:1403] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.080s	user 0.021s	sys 0.018s
I20260709 12:21:05.349290 20972 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.350797 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.352473 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:05.352802 20699 leader_election.cc:343] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.353430 20927 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.353880 20927 ts_tablet_manager.cc:1403] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.081s	user 0.008s	sys 0.004s
W20260709 12:21:05.354661 20700 leader_election.cc:343] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.355311 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.356851 21025 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.357797 20924 raft_consensus.cc:359] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.358559 20924 raft_consensus.cc:385] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.358896 20924 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.359520 20972 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.359675 20924 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.362174 20924 ts_tablet_manager.cc:1434] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 12:21:05.363503 20924 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.358711 20631 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:05.362499 20927 raft_consensus.cc:359] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.366797 20927 raft_consensus.cc:385] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.367257 20927 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.368367 20927 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.371953 20927 ts_tablet_manager.cc:1434] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.002s
I20260709 12:21:05.373312 20927 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.382668 20927 tablet_bootstrap.cc:654] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.391842 20929 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.392446 20929 ts_tablet_manager.cc:1403] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.008s
I20260709 12:21:05.398512 20929 raft_consensus.cc:359] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.399302 20929 raft_consensus.cc:385] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.399600 20929 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.400228 20929 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.403785 20924 tablet_bootstrap.cc:654] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.405903 20929 ts_tablet_manager.cc:1434] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:21:05.406911 20929 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.407609 21025 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.411324 20981 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.411818 20981 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.417603 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.418864 20929 tablet_bootstrap.cc:654] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:05.419026 20849 leader_election.cc:343] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.419997 20927 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.420495 20927 ts_tablet_manager.cc:1403] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.000s
I20260709 12:21:05.425029 20927 raft_consensus.cc:359] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.425891 20927 raft_consensus.cc:385] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.426255 20927 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.426788 20981 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.427453 20927 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.424149 20631 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:05.428627 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.429895 20927 ts_tablet_manager.cc:1434] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
W20260709 12:21:05.430667 20849 leader_election.cc:343] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.431113 20849 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.431788 20981 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.441562 20927 tablet_bootstrap.cc:492] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.448415 20929 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.448941 20929 ts_tablet_manager.cc:1403] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.005s
I20260709 12:21:05.450119 20927 tablet_bootstrap.cc:654] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.451573 20929 raft_consensus.cc:359] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.452288 20929 raft_consensus.cc:385] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.452572 20929 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.454972 20924 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.455498 20924 ts_tablet_manager.cc:1403] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.092s	user 0.016s	sys 0.005s
I20260709 12:21:05.460466 20927 tablet_bootstrap.cc:492] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.460980 20927 ts_tablet_manager.cc:1403] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.003s
I20260709 12:21:05.464306 20927 raft_consensus.cc:359] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.465009 20927 raft_consensus.cc:385] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.465286 20927 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.465293 20924 raft_consensus.cc:359] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.466034 20924 raft_consensus.cc:385] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.466337 20924 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.466161 20927 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.467497 21043 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.467979 21043 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.468297 20927 ts_tablet_manager.cc:1434] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:05.468843 20946 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.469269 20927 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.469254 20946 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.469823 21043 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.470517 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.471972 20946 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.466979 20924 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.475142 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.475560 20927 tablet_bootstrap.cc:654] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.477874 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:05.480149 20699 leader_election.cc:343] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.480535 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.482481 20774 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:05.483925 21127 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.475018 20924 ts_tablet_manager.cc:1434] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:21:05.485376 20924 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.486676 20927 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.487147 20927 ts_tablet_manager.cc:1403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.010s
I20260709 12:21:05.453434 20929 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.488041 20806 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:05.489890 21026 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.490355 20927 raft_consensus.cc:359] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.491060 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:05.491051 20927 raft_consensus.cc:385] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.492372 20927 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.493034 20927 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.493927 21029 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.494275 21029 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.494596 21029 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.495108 20927 ts_tablet_manager.cc:1434] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.001s
I20260709 12:21:05.496094 20927 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.502140 20927 tablet_bootstrap.cc:654] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.505064 20949 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.507655 21029 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.517233 20972 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.517800 20972 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.523886 20929 ts_tablet_manager.cc:1434] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.001s
I20260709 12:21:05.526958 20972 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.530534 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.527340 21026 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.531615 21026 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.533893 21026 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.538373 20927 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.539032 20927 ts_tablet_manager.cc:1403] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.020s
I20260709 12:21:05.539155 20929 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.541649 20927 raft_consensus.cc:359] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.542351 20927 raft_consensus.cc:385] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.542610 20927 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.543236 20927 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.545308 20927 ts_tablet_manager.cc:1434] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:05.546166 20927 tablet_bootstrap.cc:492] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.546891 21029 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.546764 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.547650 20806 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:05.548081 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:05.548710 20810 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.551927 20924 tablet_bootstrap.cc:654] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.553826 20700 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.554605 20927 tablet_bootstrap.cc:654] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.556483 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:05.559827 20929 tablet_bootstrap.cc:654] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.561883 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.562606 20735 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
W20260709 12:21:05.562983 20699 leader_election.cc:343] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.556584 21026 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.584343 21026 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.556530 21029 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.578032 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.599164 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:05.602602 20924 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.602881 20949 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.603109 20924 ts_tablet_manager.cc:1403] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.118s	user 0.018s	sys 0.003s
I20260709 12:21:05.603364 20949 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.603528 21026 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.603775 20949 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.605288 20810 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:05.605803 20924 raft_consensus.cc:359] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.606477 20924 raft_consensus.cc:385] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.606758 20924 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.607403 20924 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.616955 21118 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.643118 20924 ts_tablet_manager.cc:1434] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.017s
I20260709 12:21:05.644565 20924 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.648901 21029 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.654299 20949 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.615101 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:05.655977 20972 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:05.638973 21026 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.657488 20972 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:05.658007 21026 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.658761 20972 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.660238 21026 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.636238 21118 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.662755 21118 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.666384 20810 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.667995 20949 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:05.678630 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70312.9us
I20260709 12:21:05.680964 20927 tablet_bootstrap.cc:492] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.681479 20927 ts_tablet_manager.cc:1403] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.019s
W20260709 12:21:05.683894 20849 leader_election.cc:343] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.685235 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:05.686745 20849 leader_election.cc:343] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.687537 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.692346 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:05.692934 20929 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.693382 20929 ts_tablet_manager.cc:1403] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.154s	user 0.018s	sys 0.020s
W20260709 12:21:05.694335 20849 leader_election.cc:343] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.694916 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.696256 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:05.696936 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.696849 20929 raft_consensus.cc:359] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.697736 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.687574 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.692355 20807 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.701314 20927 raft_consensus.cc:359] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.710608 20927 raft_consensus.cc:385] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.710955 20927 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
W20260709 12:21:05.705117 20849 leader_election.cc:343] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.704484 20929 raft_consensus.cc:385] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.711697 20929 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.711922 20927 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.717669 20929 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.709829 20732 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.727070 20929 ts_tablet_manager.cc:1434] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.006s
I20260709 12:21:05.727183 20927 ts_tablet_manager.cc:1434] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.003s
I20260709 12:21:05.711329 20730 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:05.728514 20927 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.728466 20929 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
W20260709 12:21:05.726727 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:05.735729 21043 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.737092 20946 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.737560 20946 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.740567 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.742791 20946 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.744910 20924 tablet_bootstrap.cc:654] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.744768 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.745683 20884 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:05.748236 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:05.750576 20849 leader_election.cc:343] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:05.751266 20849 leader_election.cc:343] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.751641 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.747728 20732 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:05.755107 20929 tablet_bootstrap.cc:654] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.757370 20927 tablet_bootstrap.cc:654] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.761574 21026 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.766670 20946 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:05.769480 20946 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:05.769603 20924 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.769757 20946 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:05.770138 20924 ts_tablet_manager.cc:1403] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.126s	user 0.005s	sys 0.007s
I20260709 12:21:05.770337 20929 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.770764 20929 ts_tablet_manager.cc:1403] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.005s
I20260709 12:21:05.771142 21138 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.771610 21138 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.765822 20631 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:05.772840 20924 raft_consensus.cc:359] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.773594 21138 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.773780 20924 raft_consensus.cc:385] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.774058 20924 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.775337 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.775967 20884 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:05.775947 20924 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.777096 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.779959 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:05.781158 20972 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.781517 20972 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.781852 20972 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.782436 20924 ts_tablet_manager.cc:1434] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:21:05.782176 20946 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.783409 20924 tablet_bootstrap.cc:492] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:05.787388 20945 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.788878 20805 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:05.787393 20700 leader_election.cc:343] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.795847 20972 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.799749 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:05.801663 20700 leader_election.cc:343] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.802465 20946 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.763206 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:05.802968 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:05.803383 21026 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.803622 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:05.803772 21026 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.804026 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:05.804319 21026 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.803927 21192 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:05.804857 20981 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:05.805171 20849 leader_election.cc:343] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.805332 20981 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:05.805605 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:05.806293 20981 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.803653 20884 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.807551 20949 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.806296 21191 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:05.809069 20929 raft_consensus.cc:359] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.791287 20924 tablet_bootstrap.cc:654] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.816231 20972 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.809684 20929 raft_consensus.cc:385] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.817315 20929 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.818149 20929 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.820246 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:05.820597 20929 ts_tablet_manager.cc:1434] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.005s
I20260709 12:21:05.821512 20929 tablet_bootstrap.cc:492] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.831312 21138 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:05.833863 21029 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.834347 21029 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.836508 20775 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:05.837307 20929 tablet_bootstrap.cc:654] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.836659 21029 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.837277 20946 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:05.841234 20927 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:05.841724 20927 ts_tablet_manager.cc:1403] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.113s	user 0.016s	sys 0.000s
I20260709 12:21:05.843156 20945 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:05.844228 20945 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:05.844426 20927 raft_consensus.cc:359] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.845080 20927 raft_consensus.cc:385] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.845356 20927 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.845194 20945 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.846127 20945 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:05.846010 20927 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.847762 21026 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.850611 20929 tablet_bootstrap.cc:492] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:05.851073 20929 ts_tablet_manager.cc:1403] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.006s
I20260709 12:21:05.850752 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.851991 20884 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:05.852242 20879 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:05.852949 21026 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.854246 20929 raft_consensus.cc:359] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.855142 20927 ts_tablet_manager.cc:1434] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:21:05.856110 20927 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:05.858299 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.858493 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:05.859944 21138 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:05.860839 20699 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:05.854895 20929 raft_consensus.cc:385] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.862075 20929 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.862828 20929 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.863651 21026 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.864117 21026 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.866106 21026 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.866883 20929 ts_tablet_manager.cc:1434] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
W20260709 12:21:05.864395 20700 leader_election.cc:343] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.862221 21138 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:05.867259 20972 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.869421 20929 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:05.886456 20927 tablet_bootstrap.cc:654] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.891223 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:05.891983 20810 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:05.878067 20972 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.894209 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:05.895025 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:05.894093 20972 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.895498 20730 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.900027 20949 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.900436 20949 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.900753 20949 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.893292 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:05.910388 20972 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.912568 20972 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.912580 21029 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:05.913784 21029 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.915342 21029 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:05.913156 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:05.919548 20849 leader_election.cc:343] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.921263 20949 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.892733 21138 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.923550 20949 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:05.924580 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:05.925226 20849 leader_election.cc:343] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:05.926350 20849 leader_election.cc:343] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:05.927673 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:05.928311 20805 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.931998 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:05.932226 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:05.932672 20884 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:05.932880 20879 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.941268 20929 tablet_bootstrap.cc:654] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:05.944443 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:05.945837 20924 tablet_bootstrap.cc:492] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:05.946605 20924 ts_tablet_manager.cc:1403] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.163s	user 0.013s	sys 0.043s
I20260709 12:21:05.944931 20633 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:05.971553 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 109921us
I20260709 12:21:05.948261 20945 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:05.978421 20945 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:05.978734 20945 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:05.945482 20730 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:05.983348 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:05.984639 20949 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:05.985090 20949 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:05.984912 20945 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.985188 20924 raft_consensus.cc:359] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.985816 20924 raft_consensus.cc:385] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:05.986095 20924 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.985883 20949 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.986783 20924 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:05.972234 20879 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:05.950047 21127 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:21:05.991429 20699 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:06.001107 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:06.001609 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:05.996302 21138 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.007612 21138 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.009646 21138 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.010186 20945 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.010361 21191 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.001156 21029 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:06.018591 20924 ts_tablet_manager.cc:1434] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:21:06.019954 20924 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:06.020735 20949 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.020989 20927 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:06.021502 20927 ts_tablet_manager.cc:1403] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.166s	user 0.006s	sys 0.005s
I20260709 12:21:06.021238 20949 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.019805 21029 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:06.029721 20949 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:06.014223 20805 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:06.035629 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:06.036134 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:06.025867 21029 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.038585 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:06.045442 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:06.046077 20809 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 1: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:06.047384 20981 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:06.047888 20981 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:06.048954 20981 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.051154 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:06.055845 21192 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.060330 20927 raft_consensus.cc:359] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.014084 21191 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.061189 20927 raft_consensus.cc:385] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.063458 20927 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.064502 20927 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.070060 20927 ts_tablet_manager.cc:1434] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.003s
I20260709 12:21:06.067054 20633 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:06.071916 20929 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
W20260709 12:21:06.073551 20700 leader_election.cc:343] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.076014 20927 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:06.076503 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:06.086248 21191 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:06.108800 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:06.110776 20849 leader_election.cc:343] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.124891 21127 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.125609 21127 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.129002 20946 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:06.131035 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:06.131699 20734 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
W20260709 12:21:06.121191 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47666.1us
W20260709 12:21:06.132486 20700 leader_election.cc:343] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.132916 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:06.134613 20929 ts_tablet_manager.cc:1403] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.265s	user 0.042s	sys 0.051s
I20260709 12:21:06.133713 20946 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:06.138787 20946 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:06.141238 20946 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:06.133901 21209 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:21:06.147171 20945 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.133253 21127 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.147651 20945 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:06.135708 20700 leader_election.cc:343] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.153090 20945 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.164539 20946 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.152244 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:06.170975 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:06.171656 20734 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.172406 20927 tablet_bootstrap.cc:654] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.173324 20929 raft_consensus.cc:359] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.174021 20929 raft_consensus.cc:385] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.174295 20929 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.174901 20929 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.170221 20946 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.179319 20629 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:06.189940 20929 ts_tablet_manager.cc:1434] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 12:21:06.211380 21192 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:06.211726 20946 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.213250 20946 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:06.166688 20700 leader_election.cc:343] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.211382 20929 tablet_bootstrap.cc:492] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:06.228191 20924 tablet_bootstrap.cc:654] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:06.230360 20849 leader_election.cc:343] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.230888 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:06.232301 20946 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.229566 20734 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:06.233287 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:06.233881 20879 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.233973 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:06.234961 20929 tablet_bootstrap.cc:654] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.235312 21213 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:06.237818 21213 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:21:06.226011 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63072.2us
I20260709 12:21:06.232473 21026 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:06.242884 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.249363 21213 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.250581 20879 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:06.254185 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:06.254786 20882 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.256524 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.257321 20880 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:06.235697 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.260408 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:06.261392 20882 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:06.230319 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:06.277565 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.278252 20883 raft_consensus.cc:2376] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1. Re-sending same reply.
W20260709 12:21:06.282606 20699 leader_election.cc:343] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.282928 20924 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:06.283401 20924 ts_tablet_manager.cc:1403] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.264s	user 0.020s	sys 0.011s
W20260709 12:21:06.285975 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56435.2us
I20260709 12:21:06.283664 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:06.284067 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:06.297251 20945 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:06.297785 20945 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:06.291172 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:06.298542 20945 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.302232 20805 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:06.308911 20929 tablet_bootstrap.cc:492] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:06.308068 20629 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.309428 20929 ts_tablet_manager.cc:1403] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.116s	user 0.019s	sys 0.010s
I20260709 12:21:06.311335 20629 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.310215 21127 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.319327 20924 raft_consensus.cc:359] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.321954 20924 raft_consensus.cc:385] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.322353 20924 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.329891 20929 raft_consensus.cc:359] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.330981 20929 raft_consensus.cc:385] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.331881 20929 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.310953 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:06.340170 21138 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:06.340591 21138 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:06.340940 21138 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.348711 20924 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.349931 20929 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.355914 21029 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:06.361418 20929 ts_tablet_manager.cc:1434] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.002s
I20260709 12:21:06.362474 20929 tablet_bootstrap.cc:492] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:06.364583 21127 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:06.372097 20924 ts_tablet_manager.cc:1434] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.000s
I20260709 12:21:06.372953 20924 tablet_bootstrap.cc:492] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:06.388543 20927 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:06.389153 20927 ts_tablet_manager.cc:1403] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.313s	user 0.017s	sys 0.011s
I20260709 12:21:06.351881 21138 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.385886 20805 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.391552 21026 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:06.393563 21138 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.388247 20633 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.395895 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:06.396500 20883 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.405198 21026 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:06.413599 20883 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:06.416921 21025 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.417434 21025 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:06.380730 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82835us
I20260709 12:21:06.423995 21025 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.424314 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:06.431361 20927 raft_consensus.cc:359] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.432339 20927 raft_consensus.cc:385] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.432678 20927 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.433467 20927 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.441685 20927 ts_tablet_manager.cc:1434] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.001s
I20260709 12:21:06.439359 21138 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:06.443086 20927 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:06.442602 21138 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:06.437094 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.435878 21220 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:21:06.460597 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:06.462029 21138 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.468609 20775 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:21:06.540233 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71744.5us
I20260709 12:21:06.540071 21192 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:06.540098 21025 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:06.526944 20924 tablet_bootstrap.cc:654] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.553610 20929 tablet_bootstrap.cc:654] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.562760 20927 tablet_bootstrap.cc:654] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.570621 20972 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.571475 20972 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.573522 21234 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:06.575145 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.576618 20929 tablet_bootstrap.cc:492] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
W20260709 12:21:06.576642 20699 leader_election.cc:343] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.577199 20929 ts_tablet_manager.cc:1403] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.215s	user 0.028s	sys 0.016s
I20260709 12:21:06.590253 20924 tablet_bootstrap.cc:492] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:06.590823 20924 ts_tablet_manager.cc:1403] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.218s	user 0.011s	sys 0.001s
W20260709 12:21:06.591390 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.591701 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132604us
I20260709 12:21:06.593029 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:06.585867 20929 raft_consensus.cc:359] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.595950 20929 raft_consensus.cc:385] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.596310 20929 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.597115 20929 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.597828 21229 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:06.598021 20972 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.598232 21229 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:06.599774 21229 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.603164 20929 ts_tablet_manager.cc:1434] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.026s	user 0.002s	sys 0.004s
I20260709 12:21:06.609355 21229 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.611892 20929 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:06.612947 21229 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.617944 20929 tablet_bootstrap.cc:654] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.631247 20929 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:06.630180 20633 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.631875 20929 ts_tablet_manager.cc:1403] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 12:21:06.632468 20633 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.634869 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:06.635555 20880 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.643617 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:06.644363 21213 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.644932 21213 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.645812 20929 raft_consensus.cc:359] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.646466 20929 raft_consensus.cc:385] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.648342 21213 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.648491 20929 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.650866 20924 raft_consensus.cc:359] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.651556 20924 raft_consensus.cc:385] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.651808 20924 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.652364 20924 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.656993 20929 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.658566 20929 ts_tablet_manager.cc:1434] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.001s
I20260709 12:21:06.658411 20880 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:06.659555 20929 tablet_bootstrap.cc:492] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:06.662569 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.663347 20880 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:06.664314 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:06.664978 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:06.665555 20805 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.665871 20972 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:06.666591 20972 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:06.667497 20972 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.668715 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:06.669113 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:06.669284 20807 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.669972 21213 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:06.670353 21213 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:06.670753 21213 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.676041 20807 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:06.681960 21228 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:06.684237 21127 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:06.685832 20700 leader_election.cc:343] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.687646 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:06.688313 20735 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:06.689368 20924 ts_tablet_manager.cc:1434] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.000s
I20260709 12:21:06.690363 20924 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
W20260709 12:21:06.694582 20849 consensus_peers.cc:597] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:06.698171 21193 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:06.701037 20700 leader_election.cc:343] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.701545 20700 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:06.706291 20927 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:06.708586 20927 ts_tablet_manager.cc:1403] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.266s	user 0.014s	sys 0.002s
I20260709 12:21:06.709985 21138 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:06.718065 21214 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:21:06.710068 21228 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:21:06.730969 20927 raft_consensus.cc:359] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.731827 20927 raft_consensus.cc:385] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.732110 20924 tablet_bootstrap.cc:654] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.732122 20927 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.737617 21230 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:06.738317 20927 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.747277 20929 tablet_bootstrap.cc:654] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.749501 20629 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.758729 20927 ts_tablet_manager.cc:1434] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 12:21:06.759621 20927 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:06.773300 21213 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.792343 20946 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.792977 20946 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:06.824277 20700 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:06.827495 20946 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.829927 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:06.832088 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:06.832368 20880 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.832809 20734 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:06.830291 21213 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.832192 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:06.832391 21243 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:21:06.840451 20927 tablet_bootstrap.cc:654] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.849797 20929 tablet_bootstrap.cc:492] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:06.850364 20929 ts_tablet_manager.cc:1403] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.191s	user 0.013s	sys 0.012s
I20260709 12:21:06.852649 20929 raft_consensus.cc:359] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.853368 20929 raft_consensus.cc:385] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.853654 20929 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.854297 20929 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.855875 20929 ts_tablet_manager.cc:1434] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:06.856585 20929 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
W20260709 12:21:06.860916 20774 leader_election.cc:343] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.863653 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:06.869949 20775 leader_election.cc:343] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.870357 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:06.873090 21229 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:06.873181 21213 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:06.873641 21229 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.881886 20880 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:06.883627 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:06.884336 21214 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:06.885843 21214 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:06.900023 21229 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:06.897859 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:06.886673 21214 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.911650 20924 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
W20260709 12:21:06.910295 20699 leader_election.cc:343] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.912200 20924 ts_tablet_manager.cc:1403] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.222s	user 0.018s	sys 0.007s
I20260709 12:21:06.910154 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:06.913350 21234 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:06.914084 20700 leader_election.cc:343] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:06.914613 20700 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:06.916329 21229 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:06.914448 20734 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:06.941265 20924 raft_consensus.cc:359] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.941951 20924 raft_consensus.cc:385] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.942225 20924 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.968132 20927 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:06.968880 20927 ts_tablet_manager.cc:1403] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.209s	user 0.012s	sys 0.011s
I20260709 12:21:06.979421 20924 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.981700 20924 ts_tablet_manager.cc:1434] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 12:21:06.982640 20924 tablet_bootstrap.cc:492] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:06.987608 20927 raft_consensus.cc:359] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.988358 20927 raft_consensus.cc:385] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:06.988648 20927 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.989289 20927 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:06.991575 20927 ts_tablet_manager.cc:1434] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 12:21:06.992182 20929 tablet_bootstrap.cc:654] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:06.995903 20927 tablet_bootstrap.cc:492] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:07.012622 20927 tablet_bootstrap.cc:654] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.015735 20924 tablet_bootstrap.cc:654] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.019308 20929 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:07.019769 20929 ts_tablet_manager.cc:1403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.163s	user 0.021s	sys 0.010s
I20260709 12:21:07.022439 20929 raft_consensus.cc:359] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.023089 20929 raft_consensus.cc:385] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.024159 20929 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.026758 20924 tablet_bootstrap.cc:492] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:07.027242 20924 ts_tablet_manager.cc:1403] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.004s
I20260709 12:21:07.029692 20927 tablet_bootstrap.cc:492] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:07.030157 20927 ts_tablet_manager.cc:1403] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.004s
I20260709 12:21:07.025002 20929 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:07.031361 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:07.032981 20929 ts_tablet_manager.cc:1434] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:21:07.033922 20929 tablet_bootstrap.cc:492] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:07.034893 20924 raft_consensus.cc:359] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.035506 20924 raft_consensus.cc:385] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.035773 20924 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.036434 20924 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.038506 20924 ts_tablet_manager.cc:1434] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.000s
I20260709 12:21:07.039243 20629 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:07.040580 20929 tablet_bootstrap.cc:654] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.040259 20927 raft_consensus.cc:359] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.041108 20927 raft_consensus.cc:385] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.041376 20927 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.041983 20927 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.043983 20927 ts_tablet_manager.cc:1434] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:21:07.047233 20924 tablet_bootstrap.cc:492] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:07.050622 20927 tablet_bootstrap.cc:492] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:07.053980 20924 tablet_bootstrap.cc:654] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.051067 20929 tablet_bootstrap.cc:492] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:07.060423 20929 ts_tablet_manager.cc:1403] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.003s
I20260709 12:21:07.068027 20924 tablet_bootstrap.cc:492] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:07.068521 20924 ts_tablet_manager.cc:1403] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.006s
I20260709 12:21:07.068760 20929 raft_consensus.cc:359] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.069512 20929 raft_consensus.cc:385] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.069806 20929 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.070461 20929 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.071023 20927 tablet_bootstrap.cc:654] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.071096 20924 raft_consensus.cc:359] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.072458 20924 raft_consensus.cc:385] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.072649 20929 ts_tablet_manager.cc:1434] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:07.072741 20924 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.073776 20929 tablet_bootstrap.cc:492] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:07.073719 20924 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.079651 21138 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:07.080075 20929 tablet_bootstrap.cc:654] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.112635 21214 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.113351 21214 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.163172 21214 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.163606 20929 tablet_bootstrap.cc:492] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:07.164157 20929 ts_tablet_manager.cc:1403] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.091s	user 0.027s	sys 0.043s
I20260709 12:21:07.165485 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.169222 20929 raft_consensus.cc:359] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.170383 20929 raft_consensus.cc:385] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.165660 20810 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.170727 20929 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.171562 20929 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.173513 20924 ts_tablet_manager.cc:1434] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.105s	user 0.006s	sys 0.019s
I20260709 12:21:07.174412 20924 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:07.174914 20927 tablet_bootstrap.cc:492] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:07.175391 20927 ts_tablet_manager.cc:1403] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.125s	user 0.011s	sys 0.020s
I20260709 12:21:07.175796 20929 ts_tablet_manager.cc:1434] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:07.177032 20929 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:07.178270 20927 raft_consensus.cc:359] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.179212 20981 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.184988 20981 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.181735 20927 raft_consensus.cc:385] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.201720 20927 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.190635 20981 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.197700 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.235805 21127 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.236627 21127 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.339735 21229 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.269941 20949 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.345486 21229 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.345741 20949 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.336849 20927 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.348189 20981 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.348629 20949 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.348779 20981 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.276971 21234 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.073s
I20260709 12:21:07.349872 20927 ts_tablet_manager.cc:1434] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.174s	user 0.032s	sys 0.058s
I20260709 12:21:07.350265 21229 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.317258 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.350966 20927 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:07.352974 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.353719 20981 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.356667 21026 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:07.323781 21258 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:21:07.357537 20927 tablet_bootstrap.cc:654] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.357138 21026 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.358316 20924 tablet_bootstrap.cc:654] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.358788 21228 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:07.359211 21026 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.236827 20731 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:07.360325 21259 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:07.338490 20929 tablet_bootstrap.cc:654] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.358356 21193 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:07.362445 21193 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.364655 21193 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.364770 21127 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.376220 20927 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:07.376266 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.376717 20927 ts_tablet_manager.cc:1403] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.006s
I20260709 12:21:07.378077 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.378162 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.378762 20734 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:07.379379 20927 raft_consensus.cc:359] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.379609 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.380064 20927 raft_consensus.cc:385] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.380334 20733 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:07.380817 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:21:07.381654 20849 leader_election.cc:343] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.380375 20927 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
W20260709 12:21:07.382408 20849 leader_election.cc:343] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.382970 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:07.382833 20927 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.383719 20949 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:07.384050 20949 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:07.384305 21193 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:07.384853 21193 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:07.384375 20949 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.385247 21193 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.385203 20927 ts_tablet_manager.cc:1434] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:07.388026 20927 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:07.388393 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.389886 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.391718 20949 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.392608 21193 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.393572 20949 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.385233 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:07.394071 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.394766 21193 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.395429 20810 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:07.395754 21026 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:07.396541 21026 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:07.394888 20927 tablet_bootstrap.cc:654] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.396929 21026 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.395427 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.397711 20805 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
W20260709 12:21:07.401566 20849 leader_election.cc:343] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:07.402649 20849 leader_election.cc:343] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.403039 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:07.405076 21026 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.405712 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:07.406944 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:07.407612 20735 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.407932 20804 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.407812 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.413189 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.414620 20735 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:07.416265 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:07.416648 21026 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.417093 20732 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.420730 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:07.424141 20929 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:07.424650 20929 ts_tablet_manager.cc:1403] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.248s	user 0.011s	sys 0.002s
I20260709 12:21:07.426980 20981 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:07.429718 21228 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.430042 20929 raft_consensus.cc:359] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.430234 21228 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.430756 20929 raft_consensus.cc:385] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:07.429857 20700 leader_election.cc:343] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.431056 20929 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.431747 20929 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.432683 21228 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.433542 20927 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
W20260709 12:21:07.433744 20849 leader_election.cc:343] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.434020 20929 ts_tablet_manager.cc:1434] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:21:07.434098 20927 ts_tablet_manager.cc:1403] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.000s
I20260709 12:21:07.435068 20929 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:07.435484 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.436441 20804 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:07.438352 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:07.442171 20929 tablet_bootstrap.cc:654] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.444413 21259 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:07.444614 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.444862 21259 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:07.445233 21259 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.445428 20884 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:07.446893 20924 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:07.447412 20924 ts_tablet_manager.cc:1403] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.273s	user 0.012s	sys 0.006s
I20260709 12:21:07.452080 20927 raft_consensus.cc:359] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.452927 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:07.453370 21127 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.453804 20981 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:07.454242 20981 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:07.453939 21127 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.454654 20981 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.456287 21127 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.457609 20732 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:07.453227 21259 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.452868 20927 raft_consensus.cc:385] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.472527 20927 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.473394 20927 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.461892 20929 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:07.461944 20981 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.475536 20929 ts_tablet_manager.cc:1403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.009s
I20260709 12:21:07.477010 20981 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:07.463335 20849 leader_election.cc:343] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:07.477831 20849 leader_election.cc:343] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.477836 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.478554 20879 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
I20260709 12:21:07.478741 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:07.479857 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:07.480170 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:07.480727 20804 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.481514 20949 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:07.481971 20949 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:07.482671 20949 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.468381 20924 raft_consensus.cc:359] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.477131 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:07.485616 20924 raft_consensus.cc:385] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.485925 20924 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.485848 20929 raft_consensus.cc:359] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.486950 21025 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:07.486706 20924 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.488505 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:07.489084 20924 ts_tablet_manager.cc:1434] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 12:21:07.489354 20807 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:07.490244 20924 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:07.490698 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:07.490998 20810 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:07.492969 20849 leader_election.cc:343] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.495317 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:07.495890 20809 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.486702 20929 raft_consensus.cc:385] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.496896 20929 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.497584 20929 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.500460 21260 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:07.500988 21260 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:07.501232 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:07.502223 20927 ts_tablet_manager.cc:1434] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.000s
I20260709 12:21:07.501895 21260 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.503046 20809 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:07.503317 20927 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:07.504841 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:07.501080 20924 tablet_bootstrap.cc:654] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.499881 20804 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:07.518234 20929 ts_tablet_manager.cc:1434] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.003s
I20260709 12:21:07.519237 20929 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:07.508996 21259 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.510484 21193 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:07.530233 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:07.533588 20699 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:07.534444 21228 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:07.535660 21193 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:07.536638 21193 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.531165 21259 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:07.552165 21259 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:07.587697 21138 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:07.595945 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69173.6us
I20260709 12:21:07.595853 21138 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.601176 21138 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.606076 21259 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.641441 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:07.643059 20700 leader_election.cc:343] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.643615 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.644822 20732 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:07.628214 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:07.658001 21228 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.687932 20946 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.688468 20946 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.678390 20876 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.680217 21228 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.695775 20946 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.695931 21228 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.697603 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:07.700011 20700 leader_election.cc:343] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.701270 20735 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:07.718376 20946 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:07.721611 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51074.8us
I20260709 12:21:07.693639 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.733208 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.791251 20927 tablet_bootstrap.cc:654] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:07.793143 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.816926 20929 tablet_bootstrap.cc:654] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:07.817955 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129292us
I20260709 12:21:07.819008 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:07.820981 20946 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.822184 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.822842 20735 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:07.824458 20924 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:07.824882 20924 ts_tablet_manager.cc:1403] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.335s	user 0.022s	sys 0.024s
I20260709 12:21:07.832716 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:07.834256 20981 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.820338 21025 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:21:07.834873 20981 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:07.835837 20981 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.846112 21230 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.852643 20946 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:07.860021 21026 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.860671 21026 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.865604 21026 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.849763 21230 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.873006 20924 raft_consensus.cc:359] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.873911 20924 raft_consensus.cc:385] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.874253 20924 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:07.875063 20924 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.880635 20924 ts_tablet_manager.cc:1434] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 12:21:07.887125 21230 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:07.889415 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:07.890000 20733 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:07.892899 21260 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.893271 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.893488 21260 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.895350 20924 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:07.909731 20775 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:07.930190 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:07.931434 21279 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:07.932463 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:07.933446 21026 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:07.933732 21026 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:07.933995 21026 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:07.931871 21279 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:07.937681 21279 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:07.945333 21026 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:07.946973 20699 leader_election.cc:343] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.947288 21026 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:07.947432 20699 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:07.948724 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:07.951382 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.952543 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:07.953297 20880 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:07.953595 20876 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:07.959429 20927 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:07.960103 20927 ts_tablet_manager.cc:1403] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.457s	user 0.011s	sys 0.007s
I20260709 12:21:07.962249 20929 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:07.962860 20929 ts_tablet_manager.cc:1403] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.444s	user 0.014s	sys 0.009s
W20260709 12:21:07.965035 20699 leader_election.cc:343] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.965500 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:07.966852 21138 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:07.967514 21138 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:07.962920 21214 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:07.984107 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44007.5us
I20260709 12:21:07.991231 20929 raft_consensus.cc:359] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:07.992028 20929 raft_consensus.cc:385] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:07.992390 20929 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
W20260709 12:21:07.995687 20849 leader_election.cc:343] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:07.986231 21271 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:07.998986 20927 raft_consensus.cc:359] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.000932 20927 raft_consensus.cc:385] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.001206 20927 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.002596 20927 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.005651 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:08.005517 20929 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.006219 20729 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.007026 21260 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:08.001593 21283 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:21:08.079358 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:08.091435 20927 ts_tablet_manager.cc:1434] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.131s	user 0.007s	sys 0.001s
I20260709 12:21:08.091749 20929 ts_tablet_manager.cc:1434] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.002s
W20260709 12:21:08.092694 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 111497us
I20260709 12:21:08.106472 20927 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:08.089855 21271 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.183810 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:08.247601 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52018.6us
W20260709 12:21:08.254016 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74631.9us
I20260709 12:21:08.260799 20924 tablet_bootstrap.cc:654] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:08.268671 21291 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:21:08.280673 20729 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:08.261550 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:08.285972 21271 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:08.293605 20927 tablet_bootstrap.cc:654] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.304320 21299 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:08.272737 21279 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.311208 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:08.305094 21299 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.330803 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:08.340436 20929 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:08.341028 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:08.339622 21279 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:08.339648 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:08.369935 20631 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:08.375537 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:08.394942 20924 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:08.395613 20924 ts_tablet_manager.cc:1403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.500s	user 0.024s	sys 0.040s
I20260709 12:21:08.404153 20924 raft_consensus.cc:359] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.405192 20924 raft_consensus.cc:385] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.405481 20924 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.406323 20924 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.408392 20949 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:08.412153 21278 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:08.417052 20929 tablet_bootstrap.cc:654] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:08.422960 20774 leader_election.cc:343] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.421065 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:08.436714 20949 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.442576 20949 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:08.435585 20733 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.417335 20924 ts_tablet_manager.cc:1434] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
W20260709 12:21:08.433352 20849 leader_election.cc:343] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:08.446277 20849 leader_election.cc:343] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:08.448328 20849 leader_election.cc:343] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.448724 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:08.451411 20949 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:08.431777 21299 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:08.454362 20924 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:08.460052 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:08.461318 20981 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:08.461994 20981 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:08.462991 20981 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.466459 21260 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:08.469616 21260 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:08.470959 20810 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:08.473567 20924 tablet_bootstrap.cc:654] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:08.473479 20849 consensus_peers.cc:597] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.485137 20924 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:08.484927 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:08.487540 20924 ts_tablet_manager.cc:1403] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.003s
I20260709 12:21:08.488564 20730 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.489934 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:08.490540 20731 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:08.490648 21299 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:08.491532 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:08.492357 20734 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:08.492887 20806 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.497641 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:08.498164 20731 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:08.502172 20927 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:08.502779 20927 ts_tablet_manager.cc:1403] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.411s	user 0.020s	sys 0.010s
I20260709 12:21:08.505448 20927 raft_consensus.cc:359] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.506162 20927 raft_consensus.cc:385] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.506439 20927 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.510975 20924 raft_consensus.cc:359] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.511703 20924 raft_consensus.cc:385] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.511991 20924 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.512521 20981 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:08.513067 20981 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.514824 20981 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:08.507094 20927 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.517346 20927 ts_tablet_manager.cc:1434] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:21:08.512584 20924 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:08.518553 20700 leader_election.cc:343] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.518990 20700 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:08.519618 20924 ts_tablet_manager.cc:1434] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:21:08.519972 21278 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:08.520553 20924 tablet_bootstrap.cc:492] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:08.523612 20927 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:08.530406 20927 tablet_bootstrap.cc:654] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.536051 20929 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:08.536620 20929 ts_tablet_manager.cc:1403] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.196s	user 0.008s	sys 0.005s
I20260709 12:21:08.539362 20929 raft_consensus.cc:359] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.540109 20929 raft_consensus.cc:385] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.543610 20700 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:08.544301 20924 tablet_bootstrap.cc:654] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.540405 20929 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.548249 20929 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.550590 20929 ts_tablet_manager.cc:1434] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:21:08.552732 20929 tablet_bootstrap.cc:492] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:08.518656 20629 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.556679 20927 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:08.557039 21234 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:08.557197 20927 ts_tablet_manager.cc:1403] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.003s
I20260709 12:21:08.560467 20927 raft_consensus.cc:359] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.561050 20927 raft_consensus.cc:385] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.556703 20629 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.561214 20927 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.562024 20629 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.563906 21260 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:08.565156 20949 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:08.565480 20949 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:08.565768 20929 tablet_bootstrap.cc:654] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.566226 20949 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:08.567525 20775 leader_election.cc:343] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.567950 20775 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:21:08.568307 20849 leader_election.cc:343] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.568651 21260 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:08.569370 20924 tablet_bootstrap.cc:492] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:08.569823 20924 ts_tablet_manager.cc:1403] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 12:21:08.570024 21193 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:08.570565 21214 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:08.572206 20949 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:08.572409 20924 raft_consensus.cc:359] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.573118 20924 raft_consensus.cc:385] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.573400 20924 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.574095 20924 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.575304 20775 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:08.576049 21214 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:08.576244 20924 ts_tablet_manager.cc:1434] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:08.576619 21214 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:08.577242 20924 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:08.577461 21214 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.579442 20927 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.581066 20927 ts_tablet_manager.cc:1434] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.000s
I20260709 12:21:08.570014 20981 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:21:08.583575 20927 tablet_bootstrap.cc:492] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:08.588539 21289 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:08.589605 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:08.589082 21289 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.590346 20810 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:08.593358 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:08.600265 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:08.595119 21289 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:08.603958 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:08.605926 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:08.607611 21214 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:08.608181 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:08.609297 20981 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:08.609635 20981 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:08.610008 20981 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.633175 20929 tablet_bootstrap.cc:492] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:08.633896 20929 ts_tablet_manager.cc:1403] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.081s	user 0.009s	sys 0.005s
I20260709 12:21:08.644395 20929 raft_consensus.cc:359] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.648422 20929 raft_consensus.cc:385] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.649223 20929 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
W20260709 12:21:08.654773 20849 leader_election.cc:343] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.658943 20929 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.662341 20981 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.665318 21193 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:08.665906 21193 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.666862 20929 ts_tablet_manager.cc:1434] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 12:21:08.667883 20929 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:08.668001 21193 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:08.668867 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:08.670876 20633 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:08.684669 20981 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:08.686961 21323 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:08.687429 21323 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:08.687747 21323 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.695137 20927 tablet_bootstrap.cc:654] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.719295 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:08.720041 20804 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.706401 21234 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:08.711370 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:08.742511 21234 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.771533 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:08.772614 20733 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:08.773317 20924 tablet_bootstrap.cc:654] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:08.774624 20849 leader_election.cc:343] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.779912 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:08.778024 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:08.781389 20981 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:08.782666 20981 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:08.783098 20981 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.776285 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:08.778468 21234 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:08.777719 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:08.789042 20700 leader_election.cc:343] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:08.792182 20699 leader_election.cc:343] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.792624 20699 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:08.786649 20734 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.795011 21278 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:08.806260 20981 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.816635 20981 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:08.825266 21323 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.828199 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:08.822655 20804 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
W20260709 12:21:08.837374 20849 leader_election.cc:343] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.838127 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:08.838929 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:08.839218 20981 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:08.839589 20729 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.839917 20981 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:08.840955 20981 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.837558 20929 tablet_bootstrap.cc:654] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.870069 20629 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.857323 20734 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:08.884003 21234 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:08.894462 20927 tablet_bootstrap.cc:492] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:08.887324 21323 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:08.894972 20927 ts_tablet_manager.cc:1403] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.312s	user 0.027s	sys 0.022s
I20260709 12:21:08.897974 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:08.898622 20808 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:08.900668 20924 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:08.901158 20924 ts_tablet_manager.cc:1403] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.324s	user 0.009s	sys 0.010s
I20260709 12:21:08.902827 20927 raft_consensus.cc:359] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.903524 20927 raft_consensus.cc:385] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.903854 20927 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.903460 20924 raft_consensus.cc:359] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.904197 20924 raft_consensus.cc:385] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.904449 20924 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.904489 20927 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.906117 20808 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:08.909173 20927 ts_tablet_manager.cc:1434] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 12:21:08.909449 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:08.910189 21323 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:08.910699 21323 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:08.912771 21323 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.894471 21234 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:08.914362 21325 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:21:08.905094 20924 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.918112 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:08.923049 20924 ts_tablet_manager.cc:1434] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:21:08.924327 20924 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
W20260709 12:21:08.925251 20849 leader_election.cc:343] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.928201 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:08.929039 20927 tablet_bootstrap.cc:492] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
W20260709 12:21:08.929687 20700 leader_election.cc:343] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:08.934504 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.937697 21234 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:08.942397 20729 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:08.943748 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:08.944607 20929 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
W20260709 12:21:08.944932 20699 leader_election.cc:343] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:08.945072 20929 ts_tablet_manager.cc:1403] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.277s	user 0.011s	sys 0.016s
I20260709 12:21:08.945345 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:08.946250 21234 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:08.950444 20924 tablet_bootstrap.cc:654] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.950552 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:08.952386 20927 tablet_bootstrap.cc:654] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:08.953130 20929 raft_consensus.cc:359] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.954522 21193 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:08.966792 21193 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:08.967726 21193 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.971136 20929 raft_consensus.cc:385] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:08.971762 20929 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:08.972443 20929 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:08.974025 20929 ts_tablet_manager.cc:1434] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.029s	user 0.000s	sys 0.004s
I20260709 12:21:08.974923 20929 tablet_bootstrap.cc:492] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:08.963426 20927 tablet_bootstrap.cc:492] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:08.985761 20729 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:08.995297 21323 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:08.986292 20927 ts_tablet_manager.cc:1403] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.005s
I20260709 12:21:08.997526 21295 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.999715 20927 raft_consensus.cc:359] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.000569 20927 raft_consensus.cc:385] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.000666 20924 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:09.000857 20927 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.001191 20924 ts_tablet_manager.cc:1403] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.077s	user 0.029s	sys 0.019s
I20260709 12:21:09.001658 20927 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.004392 20927 ts_tablet_manager.cc:1434] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:09.005498 20927 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:09.011493 20924 raft_consensus.cc:359] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.012254 20927 tablet_bootstrap.cc:654] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.012276 20924 raft_consensus.cc:385] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.012832 20924 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.015260 20629 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.017266 20924 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.017737 20629 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.020311 20924 ts_tablet_manager.cc:1434] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 12:21:09.022136 20924 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: Bootstrap starting.
I20260709 12:21:09.024914 21234 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.025591 21234 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.032763 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:09.037807 21234 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.040732 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.043418 20882 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:21:09.043807 20882 pending_rounds.cc:85] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 1
I20260709 12:21:09.044103 20882 pending_rounds.cc:107] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:21:09.044417 20882 raft_consensus.cc:2889] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:09.045244 20924 tablet_bootstrap.cc:654] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.046855 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:09.048413 20927 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:09.048849 20927 ts_tablet_manager.cc:1403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.008s
I20260709 12:21:09.051395 21277 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:09.051328 20927 raft_consensus.cc:359] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.052933 21324 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.053543 21324 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.055886 21127 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:09.056279 21324 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:09.058079 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.058841 20731 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:09.061144 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:09.061724 20810 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:09.052083 20927 raft_consensus.cc:385] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.063174 20927 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.064368 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:09.065963 20927 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.067512 21026 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:09.067932 21026 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:09.068594 21026 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.068980 20927 ts_tablet_manager.cc:1434] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:21:09.069931 20927 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:09.076174 20927 tablet_bootstrap.cc:654] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.111482 20929 tablet_bootstrap.cc:654] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.101994 21026 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.115533 21234 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.116060 21234 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.107402 20629 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.118099 21234 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.120304 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:09.120002 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.121729 20876 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 0.
W20260709 12:21:09.121969 20700 leader_election.cc:343] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:09.124680 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:09.125346 21234 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:09.125690 21234 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:09.126015 21234 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.134682 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:09.135332 20732 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.135844 21234 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.139434 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:09.141004 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:09.141793 20810 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:09.143409 20700 leader_election.cc:343] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:09.143646 20927 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:09.144325 20927 ts_tablet_manager.cc:1403] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.075s	user 0.031s	sys 0.017s
I20260709 12:21:09.165741 20927 raft_consensus.cc:359] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.166477 20927 raft_consensus.cc:385] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.166752 20927 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.167399 20927 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.168917 20927 ts_tablet_manager.cc:1434] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.004s
I20260709 12:21:09.169857 20927 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Bootstrap starting.
I20260709 12:21:09.175606 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:09.177187 20876 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.120232 21026 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:09.119350 21277 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.187482 21277 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.194104 21234 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.196044 21277 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.199306 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:09.200500 20804 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:09.212663 20929 tablet_bootstrap.cc:492] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:09.213292 20929 ts_tablet_manager.cc:1403] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.239s	user 0.016s	sys 0.017s
I20260709 12:21:09.218117 20927 tablet_bootstrap.cc:654] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.222258 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.222968 20876 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:09.224472 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:09.225024 20929 raft_consensus.cc:359] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.225672 20929 raft_consensus.cc:385] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.225535 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:09.225963 20929 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.226619 21234 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.228223 21259 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:09.228701 21259 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:09.229507 21259 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.239040 20927 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: No bootstrap required, opened a new log
I20260709 12:21:09.239558 20927 ts_tablet_manager.cc:1403] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.003s
I20260709 12:21:09.239603 21234 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:09.227515 20929 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.249317 20929 ts_tablet_manager.cc:1434] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.002s
I20260709 12:21:09.250267 20929 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:09.246737 20927 raft_consensus.cc:359] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.254071 20927 raft_consensus.cc:385] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.254347 20927 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.256099 20927 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.261250 20927 ts_tablet_manager.cc:1434] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:21:09.272840 20929 tablet_bootstrap.cc:654] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.274713 21214 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.275419 21214 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.281190 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.282879 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.283735 20876 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:09.284968 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:09.285795 21025 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.293915 21293 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.283071 21214 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.294127 21025 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:09.294492 21293 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.300391 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.301111 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.302342 20732 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:09.304864 20774 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:09.305965 21293 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.307045 21293 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:09.307371 20810 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:09.320686 20929 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:09.321398 20929 ts_tablet_manager.cc:1403] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.000s
I20260709 12:21:09.322057 20981 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.322597 20981 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.324959 20981 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:09.325176 20929 raft_consensus.cc:359] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.325853 20929 raft_consensus.cc:385] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.326117 20929 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.326713 20929 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.334003 20929 ts_tablet_manager.cc:1434] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:09.340994 20929 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:09.348861 20929 tablet_bootstrap.cc:654] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.357574 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:09.374233 20981 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:09.374893 20981 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:09.375942 20981 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.386703 20734 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.388710 20929 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:09.389302 20929 ts_tablet_manager.cc:1403] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.048s	user 0.022s	sys 0.022s
I20260709 12:21:09.392553 20929 raft_consensus.cc:359] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.393579 20929 raft_consensus.cc:385] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.393981 20929 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.394901 20929 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.397039 20929 ts_tablet_manager.cc:1434] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:21:09.402803 20929 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:09.404919 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:09.407510 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.408161 20729 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:09.411172 20810 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.412817 21293 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.413501 20929 tablet_bootstrap.cc:654] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.413352 21293 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.415506 21293 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.417166 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.417343 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.417968 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:09.418035 20729 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:09.420886 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:09.423079 21127 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:09.423574 21127 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:09.423759 20924 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: No bootstrap required, opened a new log
I20260709 12:21:09.424034 21127 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.424269 20924 ts_tablet_manager.cc:1403] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: Time spent bootstrapping tablet: real 0.402s	user 0.150s	sys 0.143s
I20260709 12:21:09.426873 20924 raft_consensus.cc:359] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.427544 20924 raft_consensus.cc:385] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.427809 20924 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.428766 20924 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.441058 20924 ts_tablet_manager.cc:1434] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:21:09.446817 21127 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.462766 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:09.463543 20734 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.463636 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:09.464469 20882 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:09.471053 20681 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:09.522531 21127 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.527402 20981 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:09.528251 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55289.4us
I20260709 12:21:09.529767 20629 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 0 to 1, leader changed from <none> to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.535539 21026 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.537102 21026 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.539646 21026 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:09.540217 20849 leader_election.cc:343] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:09.546927 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:09.547660 21026 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:09.547988 21026 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:09.548255 21026 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.558878 20981 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:09.563323 20929 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:09.563891 20929 ts_tablet_manager.cc:1403] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.161s	user 0.025s	sys 0.070s
I20260709 12:21:09.558729 21026 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.566591 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:09.567176 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:09.567479 20929 raft_consensus.cc:359] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.567734 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:09.568174 20929 raft_consensus.cc:385] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.568471 20929 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.568549 21214 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:09.569192 21214 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:09.569147 20929 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.572145 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.572856 20731 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 0.
I20260709 12:21:09.570065 21214 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.568553 21026 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:09.575834 20929 ts_tablet_manager.cc:1434] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:09.576812 20929 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:09.576556 21323 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:09.582532 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:09.583122 20732 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.583511 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:09.585841 21026 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:09.586055 20849 consensus_peers.cc:597] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.596822 21026 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.600919 21127 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.600360 20629 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.601425 21127 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.603526 21127 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.605005 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:09.609407 20849 leader_election.cc:343] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:09.616132 20849 leader_election.cc:343] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:09.616839 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:09.619133 20981 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:09.619537 20981 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:09.619868 20981 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.623029 20732 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:09.631661 21026 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:09.635416 20981 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.616869 21260 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:21:09.619402 21293 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.644338 21293 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.647619 20981 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:09.651793 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.663443 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:09.669750 21193 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:09.673870 21193 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:09.674367 20734 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:09.674706 21193 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.679517 20775 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:09.680004 21293 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.680300 21127 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:09.681758 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.682420 20730 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:09.683528 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.683662 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:09.684167 20876 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 0.
I20260709 12:21:09.684420 21127 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:09.684762 21127 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:09.685068 21127 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.694095 21127 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.704769 20629 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.707854 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:09.718540 20730 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.718883 20929 tablet_bootstrap.cc:654] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.723157 21127 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:09.724229 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57877.5us
I20260709 12:21:09.725831 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.728740 21278 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:09.765673 20730 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:09.779428 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:09.780226 20810 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 1: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:09.781121 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:09.781720 20876 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.787699 20929 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:09.788178 20929 ts_tablet_manager.cc:1403] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.212s	user 0.015s	sys 0.027s
I20260709 12:21:09.791339 20876 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
W20260709 12:21:09.779598 21357 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:21:09.791356 20929 raft_consensus.cc:359] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.793692 20929 raft_consensus.cc:385] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.793968 20929 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.779464 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:09.796449 21293 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:09.797135 21293 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:09.797960 21293 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.799382 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:09.799990 20730 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.795439 20929 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.803706 20929 ts_tablet_manager.cc:1434] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:21:09.806084 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:09.806697 20804 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.807757 20730 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:09.808946 20929 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Bootstrap starting.
I20260709 12:21:09.814395 20804 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:09.820643 20929 tablet_bootstrap.cc:654] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:09.853379 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:09.855260 20929 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: No bootstrap required, opened a new log
I20260709 12:21:09.855571 21354 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:09.855748 20929 ts_tablet_manager.cc:1403] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.012s
I20260709 12:21:09.856081 21354 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:09.856981 21354 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.858422 20929 raft_consensus.cc:359] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.859084 20929 raft_consensus.cc:385] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:09.859357 20929 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Initialized, Role: FOLLOWER
I20260709 12:21:09.860081 20929 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.863022 20929 ts_tablet_manager.cc:1434] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:09.863775 21260 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:09.866537 21138 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:09.872417 20849 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:09.873206 21356 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:09.875759 20631 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.921854 20805 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.923383 21356 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:09.932245 20629 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.950665 20981 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:09.966928 21293 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:09.967382 21293 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:09.969228 21293 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:09.969794 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:09.970551 20729 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 1: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:09.970489 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:09.973335 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:09.974656 21293 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:09.985316 20633 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.986409 21293 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:10.093124 21323 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:10.115454 21026 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:10.122006 21127 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:10.148643 21127 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:10.337667 21127 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:10.348846 21127 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:10.419085 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:10.427796 21377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:10.431455 21378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:10.438441 21380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:10.445559 21026 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:10.447515 19603 server_base.cc:1061] running on GCE node
I20260709 12:21:10.449054 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:10.449999 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:10.450313 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783599670450294 us; error 0 us; skew 500 ppm
I20260709 12:21:10.451618 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:10.464744 21356 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:10.530967 19603 webserver.cc:533] Webserver started at http://127.19.36.196:45261/ using document root <none> and password file <none>
I20260709 12:21:10.532747 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:10.533231 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:10.539063 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:10.542119 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-3-root/instance:
uuid: "50c1ab13351d4427b836fbb449c6d7bb"
format_stamp: "Formatted at 2026-07-09 12:21:10 on dist-test-slave-hfcn"
I20260709 12:21:10.619865 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.076s	user 0.006s	sys 0.005s
I20260709 12:21:10.684398 21397 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:10.701850 19603 fs_manager.cc:730] Time spent opening block manager: real 0.077s	user 0.026s	sys 0.015s
I20260709 12:21:10.702275 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-3-root
uuid: "50c1ab13351d4427b836fbb449c6d7bb"
format_stamp: "Formatted at 2026-07-09 12:21:10 on dist-test-slave-hfcn"
I20260709 12:21:10.702725 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:10.752116 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:10.753782 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:10.756191 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:10.759816 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:10.760099 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:10.760586 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:10.760818 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:10.891857 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.196:44339
I20260709 12:21:10.891975 21459 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.196:44339 every 8 connection(s)
I20260709 12:21:10.898811 19603 auto_leader_rebalancer-test.cc:439] add a tserver: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:10.941110 21460 heartbeater.cc:344] Connected to a master server at 127.19.36.254:46039
I20260709 12:21:10.941684 21460 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:10.942574 21460 heartbeater.cc:507] Master 127.19.36.254:46039 requested a full tablet report, sending...
I20260709 12:21:10.944823 20631 ts_manager.cc:194] Registered new tserver with Master: 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:10.946139 20631 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33062
I20260709 12:21:11.950455 21460 heartbeater.cc:499] Master 127.19.36.254:46039 was elected leader, sending a full tablet report...
W20260709 12:21:12.319478 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48178us
I20260709 12:21:12.334933 20805 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:12.342693 20878 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:12.344884 21485 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:12.345402 20730 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:12.346753 21479 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:12.354442 21485 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:21:12.357300 20776 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 626209a224524080801f3a9ad7307fbe. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.356420 20878 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.367908 20730 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.379276 20631 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.395309 20878 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:12.404601 20805 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:12.406307 21470 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:12.412251 20730 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:12.421087 21470 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:12.425220 21465 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.431079 20802 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:21:12.450227 20850 consensus_peers.cc:597] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 54cd23b6f74a477a99461098dd4f236d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.450173 20633 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.463985 20729 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.472185 20730 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:12.487854 20878 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:12.489423 21466 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:12.492090 20810 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:12.494767 21466 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:12.500876 20701 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26ced453445642b09bda798a2963748b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.511351 21481 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.513024 20810 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.514194 20878 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.529690 20810 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:12.530745 20633 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.545897 20879 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:12.548753 20729 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:12.549695 20776 consensus_peers.cc:597] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba4e3bf73fae46119cf926d5af0782a6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.551079 21486 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:12.552204 21485 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:12.592190 21486 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.595144 20730 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.597895 20879 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.616392 20633 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:12.835804 21509 ts_tablet_manager.cc:933] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:12.837801 21509 tablet_copy_client.cc:323] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:12.850103 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 54cd23b6f74a477a99461098dd4f236d from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:12.850647 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 54cd23b6f74a477a99461098dd4f236d from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-54cd23b6f74a477a99461098dd4f236d
I20260709 12:21:12.860737 20894 tablet_copy_source_session.cc:215] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:12.866691 21509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cd23b6f74a477a99461098dd4f236d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.879042 21512 ts_tablet_manager.cc:933] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:12.888499 21512 tablet_copy_client.cc:323] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:12.890308 21509 tablet_copy_client.cc:806] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:12.891079 21509 tablet_copy_client.cc:670] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:12.909070 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 626209a224524080801f3a9ad7307fbe from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:12.909693 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 626209a224524080801f3a9ad7307fbe from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-626209a224524080801f3a9ad7307fbe
I20260709 12:21:12.910236 21509 tablet_copy_client.cc:538] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:12.911317 21515 ts_tablet_manager.cc:933] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
I20260709 12:21:12.920480 20820 tablet_copy_source_session.cc:215] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:12.920926 21509 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:12.925509 21512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626209a224524080801f3a9ad7307fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.959817 21515 tablet_copy_client.cc:323] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:21:12.963480 21509 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:12.964641 21509 tablet_bootstrap.cc:492] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:12.965353 21509 ts_tablet_manager.cc:1403] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.045s	user 0.037s	sys 0.001s
I20260709 12:21:12.967815 21512 tablet_copy_client.cc:806] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:12.968469 21512 tablet_copy_client.cc:670] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:12.970314 21509 raft_consensus.cc:359] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:12.971314 21509 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:12.973067 21509 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:12.989528 21512 tablet_copy_client.cc:538] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:12.990450 21509 ts_tablet_manager.cc:1434] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.025s	user 0.020s	sys 0.000s
I20260709 12:21:12.992697 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-54cd23b6f74a477a99461098dd4f236d received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:12.993083 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet 26ced453445642b09bda798a2963748b from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:21:12.993204 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-54cd23b6f74a477a99461098dd4f236d on tablet 54cd23b6f74a477a99461098dd4f236d with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:12.993666 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet 26ced453445642b09bda798a2963748b from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-26ced453445642b09bda798a2963748b
I20260709 12:21:12.998222 21512 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:13.011804 20745 tablet_copy_source_session.cc:215] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:13.015254 21515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ced453445642b09bda798a2963748b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.029685 21512 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:13.030531 21512 tablet_bootstrap.cc:492] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:13.032917 21512 ts_tablet_manager.cc:1403] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.006s
I20260709 12:21:13.035414 21512 raft_consensus.cc:359] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:13.036063 21512 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:13.036922 21515 tablet_copy_client.cc:806] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:13.036710 21512 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:13.037559 21515 tablet_copy_client.cc:670] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:13.038978 21512 ts_tablet_manager.cc:1434] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:13.043900 21515 tablet_copy_client.cc:538] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:13.044041 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-626209a224524080801f3a9ad7307fbe received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:13.044546 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-626209a224524080801f3a9ad7307fbe on tablet 626209a224524080801f3a9ad7307fbe with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:13.055444 21515 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:13.087812 21515 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:13.088826 21515 tablet_bootstrap.cc:492] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:13.089560 21515 ts_tablet_manager.cc:1403] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.017s
I20260709 12:21:13.092558 21515 raft_consensus.cc:359] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:13.093340 21512 ts_tablet_manager.cc:933] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:13.093443 21515 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:13.094214 21515 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:13.104569 21512 tablet_copy_client.cc:323] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:13.107299 21515 ts_tablet_manager.cc:1434] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:21:13.107658 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet ba4e3bf73fae46119cf926d5af0782a6 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:13.108628 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet ba4e3bf73fae46119cf926d5af0782a6 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-ba4e3bf73fae46119cf926d5af0782a6
I20260709 12:21:13.109093 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-26ced453445642b09bda798a2963748b received from {username='slave'} at 127.0.0.1:33906
I20260709 12:21:13.109537 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-26ced453445642b09bda798a2963748b on tablet 26ced453445642b09bda798a2963748b with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:13.118911 20820 tablet_copy_source_session.cc:215] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:13.123265 21512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4e3bf73fae46119cf926d5af0782a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.141391 21512 tablet_copy_client.cc:806] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:13.142068 21512 tablet_copy_client.cc:670] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
W20260709 12:21:13.148284 20831 debug-util.cc:398] Leaking SignalData structure 0x7b080016d1c0 after lost signal to thread 19606
W20260709 12:21:13.149355 20831 debug-util.cc:398] Leaking SignalData structure 0x7b080016d8a0 after lost signal to thread 20667
I20260709 12:21:13.187944 21512 tablet_copy_client.cc:538] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:13.198531 21512 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:13.245296 21512 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:13.246269 21512 tablet_bootstrap.cc:492] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:13.247258 21512 ts_tablet_manager.cc:1403] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.049s	user 0.031s	sys 0.009s
I20260709 12:21:13.250643 21512 raft_consensus.cc:359] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:13.251694 21512 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:13.252468 21512 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:13.258059 21512 ts_tablet_manager.cc:1434] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:13.260059 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ba4e3bf73fae46119cf926d5af0782a6 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:13.260533 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ba4e3bf73fae46119cf926d5af0782a6 on tablet ba4e3bf73fae46119cf926d5af0782a6 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:13.306460 21435 raft_consensus.cc:1217] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:13.334744 21435 raft_consensus.cc:1217] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:13.472231 21435 raft_consensus.cc:1217] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:13.483649 21435 raft_consensus.cc:1217] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:13.826752 21481 raft_consensus.cc:1064] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:13.832549 21481 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:13.848271 20882 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:13.850345 21435 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:13.851675 21481 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:13.859612 21475 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:13.876606 21485 raft_consensus.cc:1064] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:13.880105 21485 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:13.880699 21464 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:13.886457 20805 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:13.892287 21475 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:13.921012 20810 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:13.936435 20878 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:13.937988 21482 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:13.948983 21470 raft_consensus.cc:1064] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:13.951752 21470 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:13.967267 21434 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:13.969220 21522 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:13.969678 20629 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.976339 20882 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:13.978670 20730 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:13.980525 21482 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:13.894711 21435 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.032618 21522 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.041833 21433 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.050843 21432 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:14.053649 21468 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:14.066299 20882 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.078931 21485 raft_consensus.cc:1064] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:14.082089 21485 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:14.088524 20810 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:14.091318 20731 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:14.100914 21470 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:14.103888 21465 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:14.172920 21465 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.199611 21433 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:14.210487 20810 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.216709 20730 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.236711 20881 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:14.242074 21433 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.249716 21544 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:21:14.248620 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61281.7us
I20260709 12:21:14.253998 21545 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:14.262976 20730 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:14.264926 20729 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.291172 20631 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.335611 21548 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:14.306047 20631 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:14.400826 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76146.3us
I20260709 12:21:14.433948 20878 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:14.441943 21434 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.442420 20810 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.444240 21465 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:14.454531 21551 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.467438 20809 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:14.470856 20730 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:14.466106 21434 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.478439 20878 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.478849 21433 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.481142 21475 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:14.487886 21433 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.490240 21547 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:14.490561 21464 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:14.496399 21464 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.507459 21432 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.513227 21434 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.515276 20633 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.505307 20881 raft_consensus.cc:2955] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.510676 21485 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.525424 21548 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.536458 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 26ced453445642b09bda798a2963748b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:21:14.537307 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 81241.9us
I20260709 12:21:14.550160 21465 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:14.555701 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 54cd23b6f74a477a99461098dd4f236d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:14.563500 20731 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.563840 20633 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:14.565672 21554 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:21:14.569484 20633 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.578881 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 626209a224524080801f3a9ad7307fbe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:14.579654 20802 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.572827 21432 raft_consensus.cc:2955] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.612725 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet 626209a224524080801f3a9ad7307fbe with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:14.617295 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet 26ced453445642b09bda798a2963748b with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:14.625471 21544 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:14.636446 20631 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.640254 21557 tablet_replica.cc:333] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:14.640887 21556 tablet_replica.cc:333] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:14.641990 21557 raft_consensus.cc:2243] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.648907 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 54cd23b6f74a477a99461098dd4f236d with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:14.664060 20730 raft_consensus.cc:2955] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.665789 21557 raft_consensus.cc:2272] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.670367 21557 ts_tablet_manager.cc:1916] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:14.676127 20881 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.688889 20729 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.695312 21556 raft_consensus.cc:2243] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.696219 21556 raft_consensus.cc:2272] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.700222 21556 ts_tablet_manager.cc:1916] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:14.719880 21556 ts_tablet_manager.cc:1929] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:14.720489 21556 log.cc:1199] T 626209a224524080801f3a9ad7307fbe P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/626209a224524080801f3a9ad7307fbe
I20260709 12:21:14.723520 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet 626209a224524080801f3a9ad7307fbe (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:14.726450 21557 ts_tablet_manager.cc:1929] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:14.735477 21558 tablet_replica.cc:333] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:14.736622 21558 raft_consensus.cc:2243] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.737360 21558 raft_consensus.cc:2272] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.747908 21557 log.cc:1199] T 26ced453445642b09bda798a2963748b P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/26ced453445642b09bda798a2963748b
I20260709 12:21:14.749759 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet 26ced453445642b09bda798a2963748b (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:14.749739 21558 ts_tablet_manager.cc:1916] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:14.774139 20802 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:14.785071 21432 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.786614 21544 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:14.787375 20882 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:14.788796 21485 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:14.797864 21558 ts_tablet_manager.cc:1929] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:14.798467 21558 log.cc:1199] T 54cd23b6f74a477a99461098dd4f236d P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/54cd23b6f74a477a99461098dd4f236d
I20260709 12:21:14.801339 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 54cd23b6f74a477a99461098dd4f236d (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:14.804720 21548 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.806288 20882 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.814307 21432 raft_consensus.cc:2955] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:14.828105 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ba4e3bf73fae46119cf926d5af0782a6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:14.839745 20631 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.853616 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet ba4e3bf73fae46119cf926d5af0782a6 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:14.854527 21558 tablet_replica.cc:333] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:14.855520 21558 raft_consensus.cc:2243] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.856233 21558 raft_consensus.cc:2272] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.859288 21558 ts_tablet_manager.cc:1916] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:14.878619 21558 ts_tablet_manager.cc:1929] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:14.879031 21558 log.cc:1199] T ba4e3bf73fae46119cf926d5af0782a6 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/ba4e3bf73fae46119cf926d5af0782a6
I20260709 12:21:14.880596 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet ba4e3bf73fae46119cf926d5af0782a6 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:15.924769 20730 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:15.932057 20881 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:15.934695 21555 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:15.935603 20701 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3efcc9a1f14042de975fa0b662cad24d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:15.936738 20810 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:15.938161 21475 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:15.950531 21555 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:15.955807 20881 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:15.955807 20810 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:15.978159 20882 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:15.995147 20631 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:21:16.013813 20850 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be08f375c22d436fa6e99ac7bd7f6c93. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.013911 20810 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:16.014050 20730 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:16.016021 21470 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.020205 21535 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.040812 21470 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.061254 20802 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.087265 20731 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.116654 20802 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.123256 20633 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:16.189504 20730 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:16.190727 20776 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f61efc9c6d5f43ac84cfe1def41de8a0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.191021 20881 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:16.192991 21544 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.194132 21575 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:16.215791 21575 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.217537 20730 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.233126 20802 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.246613 20633 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:16.264850 20881 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:16.264271 20882 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.267663 21544 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 12:21:16.280333 20776 consensus_peers.cc:597] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f381d2ab1764a739c2d1b3d41e65181. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.286803 20730 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:16.286940 21547 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.288321 21575 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:16.293413 20878 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.315568 20633 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:16.317205 20730 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:16.473697 21582 ts_tablet_manager.cc:933] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
I20260709 12:21:16.476017 21582 tablet_copy_client.cc:323] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:21:16.477875 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet 3efcc9a1f14042de975fa0b662cad24d from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:21:16.478466 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet 3efcc9a1f14042de975fa0b662cad24d from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-3efcc9a1f14042de975fa0b662cad24d
I20260709 12:21:16.486290 20745 tablet_copy_source_session.cc:215] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:16.491057 21582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efcc9a1f14042de975fa0b662cad24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:16.547603 21582 tablet_copy_client.cc:806] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:16.558900 21584 ts_tablet_manager.cc:933] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:16.548223 21582 tablet_copy_client.cc:670] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:16.563413 21584 tablet_copy_client.cc:323] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:16.565649 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet be08f375c22d436fa6e99ac7bd7f6c93 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:16.566329 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet be08f375c22d436fa6e99ac7bd7f6c93 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-be08f375c22d436fa6e99ac7bd7f6c93
I20260709 12:21:16.576802 21582 tablet_copy_client.cc:538] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:16.579109 21586 ts_tablet_manager.cc:933] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:16.579741 20894 tablet_copy_source_session.cc:215] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:16.584488 21584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be08f375c22d436fa6e99ac7bd7f6c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:16.591763 21586 tablet_copy_client.cc:323] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:16.593755 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet f61efc9c6d5f43ac84cfe1def41de8a0 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:16.594324 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet f61efc9c6d5f43ac84cfe1def41de8a0 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-f61efc9c6d5f43ac84cfe1def41de8a0
I20260709 12:21:16.595588 21582 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:16.603483 20820 tablet_copy_source_session.cc:215] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:16.633536 21586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f61efc9c6d5f43ac84cfe1def41de8a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:16.658318 21584 tablet_copy_client.cc:806] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:16.659065 21584 tablet_copy_client.cc:670] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:16.663985 21584 tablet_copy_client.cc:538] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:16.680228 21584 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:16.685583 21586 tablet_copy_client.cc:806] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:16.686316 21586 tablet_copy_client.cc:670] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:16.691737 21582 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:16.693813 21586 tablet_copy_client.cc:538] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:16.694500 21582 tablet_bootstrap.cc:492] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:16.695329 21582 ts_tablet_manager.cc:1403] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.100s	user 0.018s	sys 0.011s
I20260709 12:21:16.703655 21586 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:16.728035 21582 raft_consensus.cc:359] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.729264 21582 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:16.735986 21582 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.752883 21582 ts_tablet_manager.cc:1434] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.002s
I20260709 12:21:16.758759 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-3efcc9a1f14042de975fa0b662cad24d received from {username='slave'} at 127.0.0.1:33906
I20260709 12:21:16.759289 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-3efcc9a1f14042de975fa0b662cad24d on tablet 3efcc9a1f14042de975fa0b662cad24d with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:16.760924 21586 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:16.763082 21586 tablet_bootstrap.cc:492] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:16.763839 21586 ts_tablet_manager.cc:1403] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.009s
I20260709 12:21:16.767096 21586 raft_consensus.cc:359] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.768025 21586 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:16.771570 21582 ts_tablet_manager.cc:933] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:16.768730 21586 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.777848 21586 ts_tablet_manager.cc:1434] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:21:16.780776 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-f61efc9c6d5f43ac84cfe1def41de8a0 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:16.781239 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-f61efc9c6d5f43ac84cfe1def41de8a0 on tablet f61efc9c6d5f43ac84cfe1def41de8a0 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:16.783010 21582 tablet_copy_client.cc:323] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:16.785326 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 8f381d2ab1764a739c2d1b3d41e65181 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:16.785903 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 8f381d2ab1764a739c2d1b3d41e65181 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-8f381d2ab1764a739c2d1b3d41e65181
I20260709 12:21:16.802307 20820 tablet_copy_source_session.cc:215] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:16.805969 21582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f381d2ab1764a739c2d1b3d41e65181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:16.830375 21584 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:16.831512 21584 tablet_bootstrap.cc:492] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:16.833348 21584 ts_tablet_manager.cc:1403] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.153s	user 0.020s	sys 0.009s
I20260709 12:21:16.836074 21582 tablet_copy_client.cc:806] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:16.839869 21582 tablet_copy_client.cc:670] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:16.841228 21584 raft_consensus.cc:359] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.844700 21584 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:16.845420 21584 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.846182 21582 tablet_copy_client.cc:538] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:16.848079 21584 ts_tablet_manager.cc:1434] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:21:16.852725 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-be08f375c22d436fa6e99ac7bd7f6c93 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:16.853189 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-be08f375c22d436fa6e99ac7bd7f6c93 on tablet be08f375c22d436fa6e99ac7bd7f6c93 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:16.869470 21582 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:16.888370 21432 raft_consensus.cc:1217] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:16.939240 21582 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:16.940251 21582 tablet_bootstrap.cc:492] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:16.941032 21582 ts_tablet_manager.cc:1403] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.072s	user 0.035s	sys 0.008s
I20260709 12:21:16.944013 21582 raft_consensus.cc:359] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.944943 21582 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:16.945659 21582 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:16.948455 21582 ts_tablet_manager.cc:1434] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:21:16.950392 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-8f381d2ab1764a739c2d1b3d41e65181 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:16.950819 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-8f381d2ab1764a739c2d1b3d41e65181 on tablet 8f381d2ab1764a739c2d1b3d41e65181 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:17.081434 21432 raft_consensus.cc:1217] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:17.135934 21432 raft_consensus.cc:1217] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:17.269255 21432 raft_consensus.cc:1217] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:21:17.383589 21595 raft_consensus.cc:1064] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:17.386540 21595 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:17.406813 21432 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.406920 20882 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.411329 21559 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:17.412009 20802 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.416307 21559 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.441568 21595 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:17.478324 21595 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.481148 20882 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.482384 20802 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.521880 20631 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.523164 21432 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.542214 21470 raft_consensus.cc:1064] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:17.544615 21470 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:17.549103 21544 raft_consensus.cc:1064] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:17.551893 21544 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:17.573649 20882 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.577983 21434 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.582454 21573 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.584383 21573 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.600602 20731 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.602095 21434 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.602210 21544 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.608345 21568 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:21:17.652822 20804 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.652333 21544 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.655156 20731 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.658902 20730 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:17.676707 21612 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:17.678035 21596 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:17.698568 21433 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.717834 20879 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.759418 20633 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.764843 21613 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.774331 20804 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.811617 20804 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:17.832597 20731 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:17.852659 21433 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:17.856141 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58476.4us
W20260709 12:21:17.857844 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52509.2us
I20260709 12:21:17.860896 21432 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:17.867983 21432 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:17.878186 21544 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:21:17.880379 20730 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.887890 20727 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:17.862438 20630 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:17.889667 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50688.8us
I20260709 12:21:17.892750 21625 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:17.899955 21597 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:17.891992 21547 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.894997 21434 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.910357 20802 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:17.915946 20729 raft_consensus.cc:2955] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.917447 21595 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:17.922062 21621 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.925572 21431 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.939603 21624 raft_consensus.cc:1064] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:17.939975 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f61efc9c6d5f43ac84cfe1def41de8a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:17.942539 21624 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:17.953881 20802 raft_consensus.cc:2955] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:17.958325 20631 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:17.965698 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3efcc9a1f14042de975fa0b662cad24d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:17.969750 20882 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:17.974745 20633 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:17.980963 20730 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:17.982164 21568 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:17.982877 21433 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:17.984418 21568 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:18.002223 20882 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:18.004175 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet f61efc9c6d5f43ac84cfe1def41de8a0 with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:18.010228 21432 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:18.001926 21623 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.007077 21547 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:21:18.017666 20873 tablet_service.cc:1558] Processing DeleteTablet for tablet 3efcc9a1f14042de975fa0b662cad24d with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:18.027549 21628 tablet_replica.cc:333] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:18.029388 21626 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:21:18.035996 21628 raft_consensus.cc:2243] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:18.036840 21628 raft_consensus.cc:2272] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:18.036757 20730 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:18.039851 21544 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:18.043869 20730 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.030633 21432 raft_consensus.cc:2955] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.052489 21628 ts_tablet_manager.cc:1916] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:18.080709 20633 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.089699 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet be08f375c22d436fa6e99ac7bd7f6c93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:18.104445 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet be08f375c22d436fa6e99ac7bd7f6c93 with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:18.127991 21628 ts_tablet_manager.cc:1929] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:18.128489 21628 log.cc:1199] T f61efc9c6d5f43ac84cfe1def41de8a0 P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/f61efc9c6d5f43ac84cfe1def41de8a0
I20260709 12:21:18.130090 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet f61efc9c6d5f43ac84cfe1def41de8a0 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:18.132601 21628 tablet_replica.cc:333] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:18.135010 21628 raft_consensus.cc:2243] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:18.144475 21628 raft_consensus.cc:2272] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:18.148928 21632 tablet_replica.cc:333] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:18.173461 21632 raft_consensus.cc:2243] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:18.174891 21632 raft_consensus.cc:2272] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:18.188685 21628 ts_tablet_manager.cc:1916] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:18.248103 21573 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.268981 21431 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.284654 20633 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:18.286873 20882 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.304309 21632 ts_tablet_manager.cc:1916] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:21:18.311107 21629 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.129s	user 0.004s	sys 0.009s
I20260709 12:21:18.323943 21628 ts_tablet_manager.cc:1929] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:18.351063 21628 log.cc:1199] T 3efcc9a1f14042de975fa0b662cad24d P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/3efcc9a1f14042de975fa0b662cad24d
I20260709 12:21:18.381003 20804 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:18.383843 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet 3efcc9a1f14042de975fa0b662cad24d (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:18.399016 21632 ts_tablet_manager.cc:1929] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:18.399627 21632 log.cc:1199] T be08f375c22d436fa6e99ac7bd7f6c93 P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/be08f375c22d436fa6e99ac7bd7f6c93
I20260709 12:21:18.402341 20729 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.404491 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet be08f375c22d436fa6e99ac7bd7f6c93 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:18.416846 21432 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:18.419911 21544 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:18.432626 20881 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:18.434652 21544 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:21:18.454799 21624 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.455724 21432 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.478404 20633 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.500769 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f381d2ab1764a739c2d1b3d41e65181 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:18.502336 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f381d2ab1764a739c2d1b3d41e65181 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:18.502738 21633 tablet_replica.cc:333] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:18.503854 21633 raft_consensus.cc:2243] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:18.504706 21633 raft_consensus.cc:2272] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:18.517822 20882 raft_consensus.cc:2955] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:18.522774 21633 ts_tablet_manager.cc:1916] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:18.563644 21633 ts_tablet_manager.cc:1929] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:21:18.564169 21633 log.cc:1199] T 8f381d2ab1764a739c2d1b3d41e65181 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/8f381d2ab1764a739c2d1b3d41e65181
I20260709 12:21:18.565960 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 8f381d2ab1764a739c2d1b3d41e65181 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:19.561856 20881 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:19.568161 20802 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.568436 20730 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.570274 21612 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:19.571672 21623 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:19.595019 21612 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.596098 20802 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.612015 20630 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:21:19.617843 20850 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5374e2302374917828294b0a4e3313a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.624024 20881 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:19.626525 20730 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.638408 20802 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.639899 20727 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.642313 21623 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:19.643431 21635 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:19.646001 20850 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d82740a2c094456a91cf0e411dd72159. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.659919 21635 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.665722 20727 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.668453 20802 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.690354 20633 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.691838 20882 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:19.703447 20810 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.703466 20727 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.705978 21637 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:19.707082 20850 consensus_peers.cc:597] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3046bc775e64931b4a1ff963cccf0e2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.707867 21623 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:19.721518 21637 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.725320 20810 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.727746 20727 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.742976 20802 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:19.749778 20631 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.757543 20730 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.759183 21547 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:19.762029 20776 consensus_peers.cc:597] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75e78bb14a7646c58f2f948b2d33b2cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.765108 20882 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:19.767834 21547 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:19.784287 21615 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.790553 20729 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.793570 20881 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:19.809551 20631 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.097582 21654 ts_tablet_manager.cc:933] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:20.101238 21654 tablet_copy_client.cc:323] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:20.103070 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet d82740a2c094456a91cf0e411dd72159 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:20.103696 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet d82740a2c094456a91cf0e411dd72159 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-d82740a2c094456a91cf0e411dd72159
I20260709 12:21:20.111726 21656 ts_tablet_manager.cc:933] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:20.116273 20894 tablet_copy_source_session.cc:215] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:20.120998 21654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82740a2c094456a91cf0e411dd72159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:20.140456 21656 tablet_copy_client.cc:323] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:20.142403 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet d5374e2302374917828294b0a4e3313a from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:20.143023 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet d5374e2302374917828294b0a4e3313a from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-d5374e2302374917828294b0a4e3313a
I20260709 12:21:20.151889 20894 tablet_copy_source_session.cc:215] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:20.160419 21656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5374e2302374917828294b0a4e3313a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:20.165037 21654 tablet_copy_client.cc:806] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:20.165845 21654 tablet_copy_client.cc:670] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:20.167482 21658 ts_tablet_manager.cc:933] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:20.176599 21654 tablet_copy_client.cc:538] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:20.188930 21654 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:20.192760 21658 tablet_copy_client.cc:323] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:20.194658 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 75e78bb14a7646c58f2f948b2d33b2cf from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:20.195356 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 75e78bb14a7646c58f2f948b2d33b2cf from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-75e78bb14a7646c58f2f948b2d33b2cf
I20260709 12:21:20.198586 21656 tablet_copy_client.cc:806] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:20.199199 21656 tablet_copy_client.cc:670] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:20.204048 20820 tablet_copy_source_session.cc:215] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:20.207583 21656 tablet_copy_client.cc:538] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:20.208587 21658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e78bb14a7646c58f2f948b2d33b2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:20.218701 21656 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:20.221863 21654 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.222982 21654 tablet_bootstrap.cc:492] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:20.223834 21654 ts_tablet_manager.cc:1403] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.002s
I20260709 12:21:20.227744 21654 raft_consensus.cc:359] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.228753 21654 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:20.229544 21654 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.230535 21658 tablet_copy_client.cc:806] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:20.231508 21658 tablet_copy_client.cc:670] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:20.235073 21658 tablet_copy_client.cc:538] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:20.243248 21654 ts_tablet_manager.cc:1434] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.006s
I20260709 12:21:20.245326 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d82740a2c094456a91cf0e411dd72159 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:20.245810 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d82740a2c094456a91cf0e411dd72159 on tablet d82740a2c094456a91cf0e411dd72159 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:20.265079 21658 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:20.274701 21656 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.275990 21656 tablet_bootstrap.cc:492] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:20.276731 21656 ts_tablet_manager.cc:1403] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.058s	user 0.023s	sys 0.015s
I20260709 12:21:20.280696 21656 raft_consensus.cc:359] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.281622 21656 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:20.282286 21656 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.287717 21656 ts_tablet_manager.cc:1434] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:21:20.295166 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d5374e2302374917828294b0a4e3313a received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:20.295686 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d5374e2302374917828294b0a4e3313a on tablet d5374e2302374917828294b0a4e3313a with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:20.309101 21658 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.310351 21658 tablet_bootstrap.cc:492] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:20.311385 21658 ts_tablet_manager.cc:1403] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.010s
I20260709 12:21:20.312422 21656 ts_tablet_manager.cc:933] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:20.313910 21658 raft_consensus.cc:359] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.314718 21658 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:20.315378 21656 tablet_copy_client.cc:323] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:20.315475 21658 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.317655 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet c3046bc775e64931b4a1ff963cccf0e2 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:20.317997 21658 ts_tablet_manager.cc:1434] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:20.318204 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet c3046bc775e64931b4a1ff963cccf0e2 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-c3046bc775e64931b4a1ff963cccf0e2
I20260709 12:21:20.323096 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-75e78bb14a7646c58f2f948b2d33b2cf received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:20.323715 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-75e78bb14a7646c58f2f948b2d33b2cf on tablet 75e78bb14a7646c58f2f948b2d33b2cf with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:20.333112 20894 tablet_copy_source_session.cc:215] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:20.336156 21656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3046bc775e64931b4a1ff963cccf0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:20.354168 21656 tablet_copy_client.cc:806] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:20.354746 21656 tablet_copy_client.cc:670] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:20.358368 21656 tablet_copy_client.cc:538] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:20.366887 21656 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:20.385918 21656 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.387001 21656 tablet_bootstrap.cc:492] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:20.387697 21656 ts_tablet_manager.cc:1403] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.003s
I20260709 12:21:20.390635 21656 raft_consensus.cc:359] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.391525 21656 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:20.392181 21656 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:20.394506 21656 ts_tablet_manager.cc:1434] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:20.396256 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-c3046bc775e64931b4a1ff963cccf0e2 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:20.396692 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-c3046bc775e64931b4a1ff963cccf0e2 on tablet c3046bc775e64931b4a1ff963cccf0e2 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:20.612959 21432 raft_consensus.cc:1217] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:20.755959 21432 raft_consensus.cc:1217] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:20.899446 21432 raft_consensus.cc:1217] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:20.903738 21434 raft_consensus.cc:1217] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:21.188820 21668 raft_consensus.cc:1064] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:21.191956 21668 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.199604 20882 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.199676 21431 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.201843 21672 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.202935 21678 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.207754 20729 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.216141 21674 raft_consensus.cc:1064] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:21.216421 21668 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:21.219950 21674 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.253091 21678 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.259575 21431 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.276657 21676 raft_consensus.cc:1064] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:21.286720 20630 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.281052 21676 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.314990 21431 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.317605 20730 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.321707 21674 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:21.325225 20810 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.328379 20729 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.350891 21612 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.353205 21612 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:21.356369 21434 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.361297 21637 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.367147 20879 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.380270 20727 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.388854 20805 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.402459 21679 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.419986 21664 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.446867 21676 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.454528 20810 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.460462 21612 raft_consensus.cc:1064] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:21.462652 21612 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.488063 20733 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:21.545876 21691 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:21:21.566170 20810 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.572801 21434 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.574210 21678 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:21:21.575496 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 97046us
I20260709 12:21:21.588231 21431 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.589658 21432 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.590420 21699 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.596196 20729 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.597429 21668 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.608417 21434 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.609484 21615 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:21.613462 21612 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.625041 20629 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.627664 20733 raft_consensus.cc:2955] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.644023 21431 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.645996 20878 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.648093 20727 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.657275 20729 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.671442 20629 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:21.672777 20727 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.686024 20878 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.689183 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 75e78bb14a7646c58f2f948b2d33b2cf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:21.692442 21433 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.695375 21698 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:21.703433 20809 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:21.706110 20802 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.708438 21637 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:21.710229 21637 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:21.715780 20629 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.729290 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet 75e78bb14a7646c58f2f948b2d33b2cf with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:21.735829 21433 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.739099 21637 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:21.743220 21612 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.748034 20727 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.754377 21699 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:21.758064 21637 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:21.767347 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d5374e2302374917828294b0a4e3313a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:21.768405 21701 tablet_replica.cc:333] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:21.770164 21431 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.773274 21701 raft_consensus.cc:2243] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:21.774057 21701 raft_consensus.cc:2272] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:21.778246 21673 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.783831 21701 ts_tablet_manager.cc:1916] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:21.793475 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d82740a2c094456a91cf0e411dd72159 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:21.788641 21434 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.797250 20633 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.800850 21699 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.818383 21431 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.819878 21701 ts_tablet_manager.cc:1929] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:21.820535 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet d5374e2302374917828294b0a4e3313a with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:21.829126 20729 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.829733 20728 raft_consensus.cc:2955] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.839502 20733 raft_consensus.cc:2955] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.845119 20633 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.820544 21701 log.cc:1199] T 75e78bb14a7646c58f2f948b2d33b2cf P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/75e78bb14a7646c58f2f948b2d33b2cf
I20260709 12:21:21.847615 21703 tablet_replica.cc:333] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:21.866641 21703 raft_consensus.cc:2243] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:21.870229 20629 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:21.873742 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet 75e78bb14a7646c58f2f948b2d33b2cf (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:21.879927 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet d82740a2c094456a91cf0e411dd72159 with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:21.879688 21703 raft_consensus.cc:2272] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:21.885201 20805 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.886811 21703 ts_tablet_manager.cc:1916] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:21.906172 20879 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:21.911342 21431 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.912953 21673 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:21.913267 20805 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:21.917722 21703 ts_tablet_manager.cc:1929] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:21.918347 21703 log.cc:1199] T d5374e2302374917828294b0a4e3313a P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/d5374e2302374917828294b0a4e3313a
I20260709 12:21:21.919360 21637 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:21.920876 21703 tablet_replica.cc:333] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:21.921836 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet d5374e2302374917828294b0a4e3313a (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:21.925099 21703 raft_consensus.cc:2243] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:21.925894 21703 raft_consensus.cc:2272] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:21.927953 21673 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.929970 20805 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.937085 21434 raft_consensus.cc:2955] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.940198 21703 ts_tablet_manager.cc:1916] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:21.948818 20633 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:21.959511 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c3046bc775e64931b4a1ff963cccf0e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:21.959511 21703 ts_tablet_manager.cc:1929] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:21.961100 21703 log.cc:1199] T d82740a2c094456a91cf0e411dd72159 P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/d82740a2c094456a91cf0e411dd72159
I20260709 12:21:21.965153 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet c3046bc775e64931b4a1ff963cccf0e2 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:21.966533 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet d82740a2c094456a91cf0e411dd72159 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:21.968200 21705 tablet_replica.cc:333] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:21.969552 21705 raft_consensus.cc:2243] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:21.970235 21705 raft_consensus.cc:2272] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:21.974022 21705 ts_tablet_manager.cc:1916] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:21.990680 21705 ts_tablet_manager.cc:1929] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:21.991089 21705 log.cc:1199] T c3046bc775e64931b4a1ff963cccf0e2 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/c3046bc775e64931b4a1ff963cccf0e2
I20260709 12:21:21.992942 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet c3046bc775e64931b4a1ff963cccf0e2 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:23.043994 20802 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.049656 20878 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:23.050274 20776 consensus_peers.cc:597] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b098f0a68194f18be24dc4f3742a4ab. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.051267 21672 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.051673 20733 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:23.052866 21678 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.063830 20878 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.061553 21710 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.080857 20630 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.087791 20733 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.095513 20802 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.108443 20878 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:23.109673 20733 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:23.110877 20776 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a4e4474d2fd45438665993211ae862a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.111588 21709 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.113081 21678 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.129995 21709 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.131680 20878 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.132493 20733 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.154866 20629 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.147040 20729 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.172770 20878 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:23.173605 20802 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:23.174187 20701 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a8b742e1fe24e6d82f1a1498f17a5fb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.174027 21677 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.177105 21669 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.184437 21706 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.185983 20879 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.203802 20805 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.221171 20630 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.228516 20879 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:23.230811 20776 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3199c6fc2e3441849ac7dd78244a3a18. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.233347 21709 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.235661 20733 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:23.237356 21709 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.252408 21678 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.261902 20879 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.270712 20802 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.266481 20729 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.294876 20633 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:23.571153 21727 ts_tablet_manager.cc:933] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
I20260709 12:21:23.576570 21727 tablet_copy_client.cc:323] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:21:23.578727 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:21:23.579388 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet 6a8b742e1fe24e6d82f1a1498f17a5fb from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-6a8b742e1fe24e6d82f1a1498f17a5fb
I20260709 12:21:23.600970 20745 tablet_copy_source_session.cc:215] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:23.604367 21727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:23.621351 21727 tablet_copy_client.cc:806] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:23.622224 21727 tablet_copy_client.cc:670] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:23.636332 21727 tablet_copy_client.cc:538] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:23.650545 21727 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:23.667917 21730 ts_tablet_manager.cc:933] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:23.677500 21730 tablet_copy_client.cc:323] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:23.683542 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 3199c6fc2e3441849ac7dd78244a3a18 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:23.684151 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 3199c6fc2e3441849ac7dd78244a3a18 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-3199c6fc2e3441849ac7dd78244a3a18
I20260709 12:21:23.686844 21731 ts_tablet_manager.cc:933] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:23.696861 20820 tablet_copy_source_session.cc:215] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:23.699958 21731 tablet_copy_client.cc:323] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:23.711927 21727 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:23.713086 21727 tablet_bootstrap.cc:492] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:23.713253 21730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3199c6fc2e3441849ac7dd78244a3a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:23.714115 21727 ts_tablet_manager.cc:1403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.014s
I20260709 12:21:23.714579 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 2b098f0a68194f18be24dc4f3742a4ab from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:23.717365 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 2b098f0a68194f18be24dc4f3742a4ab from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-2b098f0a68194f18be24dc4f3742a4ab
I20260709 12:21:23.704099 21734 ts_tablet_manager.cc:933] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:23.730147 21727 raft_consensus.cc:359] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.731156 21727 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:23.731870 21727 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.733827 20820 tablet_copy_source_session.cc:215] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:23.737550 21731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b098f0a68194f18be24dc4f3742a4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:23.752946 21727 ts_tablet_manager.cc:1434] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.004s
I20260709 12:21:23.756841 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-6a8b742e1fe24e6d82f1a1498f17a5fb received from {username='slave'} at 127.0.0.1:33906
I20260709 12:21:23.757479 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-6a8b742e1fe24e6d82f1a1498f17a5fb on tablet 6a8b742e1fe24e6d82f1a1498f17a5fb with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:23.757580 21731 tablet_copy_client.cc:806] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:23.758265 21731 tablet_copy_client.cc:670] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:23.762315 21731 tablet_copy_client.cc:538] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:23.791821 21731 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:23.804633 21734 tablet_copy_client.cc:323] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:23.806706 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 0a4e4474d2fd45438665993211ae862a from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:23.807314 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 0a4e4474d2fd45438665993211ae862a from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-0a4e4474d2fd45438665993211ae862a
I20260709 12:21:23.811993 21730 tablet_copy_client.cc:806] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:23.813359 21730 tablet_copy_client.cc:670] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:23.816184 20820 tablet_copy_source_session.cc:215] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:23.817301 21730 tablet_copy_client.cc:538] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:23.819522 21734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a4e4474d2fd45438665993211ae862a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:23.827347 21731 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:23.828372 21731 tablet_bootstrap.cc:492] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:23.829118 21731 ts_tablet_manager.cc:1403] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.038s	user 0.022s	sys 0.002s
I20260709 12:21:23.829447 21730 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:23.832228 21731 raft_consensus.cc:359] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.833098 21731 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:23.834687 21731 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.837916 21731 ts_tablet_manager.cc:1434] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:23.838929 21734 tablet_copy_client.cc:806] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:23.839797 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-2b098f0a68194f18be24dc4f3742a4ab received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:23.840269 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-2b098f0a68194f18be24dc4f3742a4ab on tablet 2b098f0a68194f18be24dc4f3742a4ab with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:23.839567 21734 tablet_copy_client.cc:670] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:23.844686 21734 tablet_copy_client.cc:538] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:23.857355 21734 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:23.907087 21734 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:23.908083 21734 tablet_bootstrap.cc:492] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:23.908761 21734 ts_tablet_manager.cc:1403] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.052s	user 0.024s	sys 0.027s
I20260709 12:21:23.911787 21734 raft_consensus.cc:359] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.912617 21734 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:23.913282 21734 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.915043 21730 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:23.916067 21730 tablet_bootstrap.cc:492] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:23.916654 21734 ts_tablet_manager.cc:1434] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:21:23.916783 21730 ts_tablet_manager.cc:1403] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.021s
I20260709 12:21:23.922705 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-0a4e4474d2fd45438665993211ae862a received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:23.923219 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-0a4e4474d2fd45438665993211ae862a on tablet 0a4e4474d2fd45438665993211ae862a with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:23.924541 21730 raft_consensus.cc:359] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.925428 21730 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:23.926081 21730 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:23.928503 21730 ts_tablet_manager.cc:1434] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:23.930982 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-3199c6fc2e3441849ac7dd78244a3a18 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:23.931452 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-3199c6fc2e3441849ac7dd78244a3a18 on tablet 3199c6fc2e3441849ac7dd78244a3a18 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:24.030622 21433 raft_consensus.cc:1217] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:24.047643 21433 raft_consensus.cc:1217] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:24.069780 21433 raft_consensus.cc:1217] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:24.077657 21433 raft_consensus.cc:1217] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:24.436544 21672 raft_consensus.cc:1064] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:24.438910 21672 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:24.447254 20876 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.448589 20729 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.449143 21433 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.449852 21709 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.450672 21672 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:24.451692 21678 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.474506 21678 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.476230 20876 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.481047 21433 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.489696 20729 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.500794 20630 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.529698 20802 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:24.534444 21433 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:24.535477 20733 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:24.535883 21709 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:24.536935 21678 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:24.544335 21672 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.551756 21433 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.554538 20733 raft_consensus.cc:2955] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.565025 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3199c6fc2e3441849ac7dd78244a3a18 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:24.572189 20629 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.586971 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet 3199c6fc2e3441849ac7dd78244a3a18 with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:24.588702 21749 tablet_replica.cc:333] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:24.589883 21749 raft_consensus.cc:2243] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:24.590773 21749 raft_consensus.cc:2272] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:24.594585 21749 ts_tablet_manager.cc:1916] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:24.605803 21709 raft_consensus.cc:1064] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:24.613436 21749 ts_tablet_manager.cc:1929] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:24.615211 21749 log.cc:1199] T 3199c6fc2e3441849ac7dd78244a3a18 P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/3199c6fc2e3441849ac7dd78244a3a18
I20260709 12:21:24.617910 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet 3199c6fc2e3441849ac7dd78244a3a18 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:24.621766 21751 raft_consensus.cc:1064] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:24.622079 21677 raft_consensus.cc:1064] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:24.624517 21751 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:24.624593 21677 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:24.630868 21709 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:24.665102 20876 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.666725 21706 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.672108 20802 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.674274 21433 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.674968 21750 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.677413 21750 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.694970 20878 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.697007 21678 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.697257 21434 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.701391 21672 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.701093 20733 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.716817 21672 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.761482 20733 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.776737 21762 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.782579 21706 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.785589 20876 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.809047 20876 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.827978 20876 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.830277 21434 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.830257 21709 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:24.834178 21709 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.867699 21434 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.900609 20802 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.901175 20629 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.915642 20633 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.918916 21431 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.921736 21709 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.927515 20727 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.933529 21433 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.948277 20878 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.954849 20631 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.960320 21672 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.985952 20728 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:24.990715 20731 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:24.994560 20802 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:24.997884 20809 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:25.003113 21433 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:25.003324 20878 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:25.004535 21765 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:25.006541 21770 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:25.009147 21433 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:25.011852 21709 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:25.019402 20878 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:25.022677 21752 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:25.027792 21770 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.028218 21672 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.031894 20878 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.033346 21762 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.036500 21431 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.036640 20876 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.041656 21434 raft_consensus.cc:2955] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.051316 20728 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:25.052834 21752 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:25.060055 20731 raft_consensus.cc:2955] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.062353 21433 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:25.064159 21709 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:25.064738 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:25.065137 20630 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.069588 21432 raft_consensus.cc:2955] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.076149 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a4e4474d2fd45438665993211ae862a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:25.079113 20631 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:25.084499 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 2b098f0a68194f18be24dc4f3742a4ab with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:25.089468 21773 tablet_replica.cc:333] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:25.090669 21773 raft_consensus.cc:2243] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.091780 21773 raft_consensus.cc:2272] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.094817 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a8b742e1fe24e6d82f1a1498f17a5fb with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:25.099452 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2b098f0a68194f18be24dc4f3742a4ab with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:25.108935 21773 ts_tablet_manager.cc:1916] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:25.124279 20631 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:25.135660 21774 tablet_replica.cc:333] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:25.139681 21773 ts_tablet_manager.cc:1929] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:25.140219 21773 log.cc:1199] T 2b098f0a68194f18be24dc4f3742a4ab P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/2b098f0a68194f18be24dc4f3742a4ab
I20260709 12:21:25.142288 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 2b098f0a68194f18be24dc4f3742a4ab (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:25.147480 21774 raft_consensus.cc:2243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.148530 21774 raft_consensus.cc:2272] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.158159 21774 ts_tablet_manager.cc:1916] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:25.161988 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a4e4474d2fd45438665993211ae862a with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:25.171418 21775 tablet_replica.cc:333] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:25.172629 21775 raft_consensus.cc:2243] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.173329 21775 raft_consensus.cc:2272] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.177237 21775 ts_tablet_manager.cc:1916] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:25.180907 21774 ts_tablet_manager.cc:1929] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:25.181499 21774 log.cc:1199] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/6a8b742e1fe24e6d82f1a1498f17a5fb
I20260709 12:21:25.187937 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet 6a8b742e1fe24e6d82f1a1498f17a5fb (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:25.194810 21775 ts_tablet_manager.cc:1929] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:25.195367 21775 log.cc:1199] T 0a4e4474d2fd45438665993211ae862a P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/0a4e4474d2fd45438665993211ae862a
I20260709 12:21:25.196894 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet 0a4e4474d2fd45438665993211ae862a (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:26.242442 20882 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:21:26.250321 20850 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f3454624b7c40c6934257c77f2174de. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.251714 20728 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:26.252600 20805 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:26.254251 21782 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:26.255043 21675 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:26.265811 20731 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.264789 21776 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.285513 20882 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.290092 20631 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:21:26.299671 20850 consensus_peers.cc:597] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad57e68a6a114847ac252579307658f7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.298524 20805 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.314393 20809 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:26.314728 20731 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:26.316268 21776 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:26.318526 21782 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:26.372987 21785 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.375712 20731 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.381990 20802 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.394520 20631 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.400880 20876 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:21:26.420904 20850 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c00fffbb02ff4bc2a974d8a5b179a97d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.424695 20805 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:26.427281 20731 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:26.427490 21675 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:26.428579 21785 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:26.437156 21782 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.442953 20802 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.450350 20731 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.465969 20631 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.474457 20876 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.489737 20802 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:26.490211 20850 consensus_peers.cc:597] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b31661191c14b44ac964555853900f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.491030 21785 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:26.491412 20731 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:26.492935 21785 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:26.501097 21782 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.504539 20802 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.515808 20731 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.520258 20630 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:26.722183 21800 ts_tablet_manager.cc:933] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:26.728789 21800 tablet_copy_client.cc:323] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:26.731660 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet ad57e68a6a114847ac252579307658f7 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:26.732215 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet ad57e68a6a114847ac252579307658f7 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-ad57e68a6a114847ac252579307658f7
I20260709 12:21:26.740136 20894 tablet_copy_source_session.cc:215] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:26.742843 21800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad57e68a6a114847ac252579307658f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:26.769729 21800 tablet_copy_client.cc:806] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:26.770346 21800 tablet_copy_client.cc:670] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:26.773908 21800 tablet_copy_client.cc:538] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:26.782087 21800 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:26.803478 21800 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:26.804419 21800 tablet_bootstrap.cc:492] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:26.805101 21800 ts_tablet_manager.cc:1403] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.006s
I20260709 12:21:26.808142 21800 raft_consensus.cc:359] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.808980 21800 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:26.809621 21800 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.812829 21800 ts_tablet_manager.cc:1434] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:21:26.815160 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ad57e68a6a114847ac252579307658f7 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:26.815621 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ad57e68a6a114847ac252579307658f7 on tablet ad57e68a6a114847ac252579307658f7 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:26.857223 21800 ts_tablet_manager.cc:933] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:26.858937 21800 tablet_copy_client.cc:323] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:26.860580 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 8f3454624b7c40c6934257c77f2174de from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:26.861075 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 8f3454624b7c40c6934257c77f2174de from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-8f3454624b7c40c6934257c77f2174de
I20260709 12:21:26.867880 20894 tablet_copy_source_session.cc:215] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:26.872958 21800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f3454624b7c40c6934257c77f2174de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:26.880064 21805 ts_tablet_manager.cc:933] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:26.889197 21800 tablet_copy_client.cc:806] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:26.890645 21800 tablet_copy_client.cc:670] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:26.893023 21805 tablet_copy_client.cc:323] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:26.896201 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet c00fffbb02ff4bc2a974d8a5b179a97d from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:26.896853 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet c00fffbb02ff4bc2a974d8a5b179a97d from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-c00fffbb02ff4bc2a974d8a5b179a97d
I20260709 12:21:26.898384 21800 tablet_copy_client.cc:538] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:26.905016 20894 tablet_copy_source_session.cc:215] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:26.908540 21805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c00fffbb02ff4bc2a974d8a5b179a97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:26.909596 21800 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:26.926272 21805 tablet_copy_client.cc:806] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:26.926988 21805 tablet_copy_client.cc:670] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:26.930688 21805 tablet_copy_client.cc:538] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:26.933332 21800 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:26.934314 21800 tablet_bootstrap.cc:492] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:26.935026 21800 ts_tablet_manager.cc:1403] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.002s
I20260709 12:21:26.938122 21800 raft_consensus.cc:359] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.939006 21800 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:26.940371 21805 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:26.939704 21800 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.942655 21800 ts_tablet_manager.cc:1434] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:26.945047 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-8f3454624b7c40c6934257c77f2174de received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:26.945461 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-8f3454624b7c40c6934257c77f2174de on tablet 8f3454624b7c40c6934257c77f2174de with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:26.968669 21805 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:26.969594 21805 tablet_bootstrap.cc:492] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:26.970307 21805 ts_tablet_manager.cc:1403] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.005s
I20260709 12:21:26.972965 21805 raft_consensus.cc:359] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.973582 21805 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:26.974123 21805 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:26.976094 21805 ts_tablet_manager.cc:1434] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:21:26.977918 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-c00fffbb02ff4bc2a974d8a5b179a97d received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:26.978329 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-c00fffbb02ff4bc2a974d8a5b179a97d on tablet c00fffbb02ff4bc2a974d8a5b179a97d with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:27.054389 21805 ts_tablet_manager.cc:933] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:27.056412 21805 tablet_copy_client.cc:323] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:27.057921 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 0b31661191c14b44ac964555853900f1 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:27.058490 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 0b31661191c14b44ac964555853900f1 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-0b31661191c14b44ac964555853900f1
I20260709 12:21:27.066237 20894 tablet_copy_source_session.cc:215] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:27.070407 21805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b31661191c14b44ac964555853900f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:27.088687 21805 tablet_copy_client.cc:806] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:27.089319 21805 tablet_copy_client.cc:670] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:27.093230 21805 tablet_copy_client.cc:538] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:27.102628 21805 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:27.123730 21805 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:27.124665 21805 tablet_bootstrap.cc:492] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:27.125347 21805 ts_tablet_manager.cc:1403] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.023s	user 0.023s	sys 0.000s
I20260709 12:21:27.128334 21805 raft_consensus.cc:359] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:27.129185 21805 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:27.129835 21805 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:27.132277 21805 ts_tablet_manager.cc:1434] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:21:27.134298 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-0b31661191c14b44ac964555853900f1 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:27.134721 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-0b31661191c14b44ac964555853900f1 on tablet 0b31661191c14b44ac964555853900f1 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:27.150686 21431 raft_consensus.cc:1217] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:27.309058 21431 raft_consensus.cc:1217] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:27.455428 21431 raft_consensus.cc:1217] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:27.613057 21431 raft_consensus.cc:1217] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:27.706971 21812 raft_consensus.cc:1064] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:27.709594 21812 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:27.716830 21431 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.716890 20802 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.718449 20728 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.718568 21782 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:27.720131 21776 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:27.721016 21812 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 12:21:27.795773 21814 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:21:27.797043 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54371.6us
I20260709 12:21:27.812023 21812 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.813191 21431 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:27.825538 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73326.8us
I20260709 12:21:27.846421 20630 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.847967 20728 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.849107 20805 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.870337 21822 raft_consensus.cc:1064] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:27.874389 21822 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:27.881668 21431 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.882858 20805 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.883975 21675 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:27.885308 21812 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:27.890362 20876 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:27.893267 20728 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.897578 21812 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:27.903566 21434 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:27.905069 21675 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:27.911057 20802 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:27.913067 21782 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.914719 21824 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:27.917075 21432 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.928687 21824 raft_consensus.cc:1064] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:27.925889 20809 raft_consensus.cc:2955] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.931754 21824 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:27.935019 20629 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:27.945320 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ad57e68a6a114847ac252579307658f7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:27.951462 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet ad57e68a6a114847ac252579307658f7 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:27.992492 21432 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.994040 20730 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.994549 21822 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:27.995656 21782 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:27.996811 21834 tablet_replica.cc:333] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:27.997835 21834 raft_consensus.cc:2243] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.999560 20809 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:27.999156 21812 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.000936 21675 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:28.006486 20805 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.008405 21434 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.023367 21834 raft_consensus.cc:2272] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:28.028127 21834 ts_tablet_manager.cc:1916] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:28.034938 20629 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.056550 21834 ts_tablet_manager.cc:1929] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:28.079792 21834 log.cc:1199] T ad57e68a6a114847ac252579307658f7 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/ad57e68a6a114847ac252579307658f7
I20260709 12:21:28.081456 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet ad57e68a6a114847ac252579307658f7 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:28.080536 21782 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.088094 20728 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.091540 20805 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.099566 21434 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.115592 20633 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.126369 21812 raft_consensus.cc:1064] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:28.130266 21812 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:28.130663 20728 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.140491 20882 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:28.148870 21434 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:28.150424 21782 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:28.152115 21434 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:28.153493 21782 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.155506 20728 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:28.157497 20805 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:28.159901 20728 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:28.161728 21782 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:28.163096 21822 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.165802 21782 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.167009 21432 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.169242 21839 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:28.175091 20882 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:28.184881 21431 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:28.186062 20730 raft_consensus.cc:2955] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.188833 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c00fffbb02ff4bc2a974d8a5b179a97d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:28.186271 20633 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.189181 21782 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.200007 20730 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:28.200182 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet c00fffbb02ff4bc2a974d8a5b179a97d with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:28.212903 21782 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.215214 21822 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.215612 21431 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.225452 21812 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.229908 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f3454624b7c40c6934257c77f2174de with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:28.238029 21432 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.247821 20728 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.256976 20633 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.258008 20730 raft_consensus.cc:2955] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.263545 21844 tablet_replica.cc:333] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:28.265142 21844 raft_consensus.cc:2243] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:28.266232 21844 raft_consensus.cc:2272] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:28.269991 21844 ts_tablet_manager.cc:1916] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:28.278658 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f3454624b7c40c6934257c77f2174de with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:28.282389 20805 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.284085 20633 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.316988 21844 ts_tablet_manager.cc:1929] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:28.317530 21844 log.cc:1199] T c00fffbb02ff4bc2a974d8a5b179a97d P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/c00fffbb02ff4bc2a974d8a5b179a97d
I20260709 12:21:28.319566 21844 tablet_replica.cc:333] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:28.320657 20882 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:28.321702 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet c00fffbb02ff4bc2a974d8a5b179a97d (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:28.320815 21844 raft_consensus.cc:2243] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:28.325103 21844 raft_consensus.cc:2272] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:28.329892 21432 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:28.331067 21844 ts_tablet_manager.cc:1916] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:28.331712 21782 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.335939 20805 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:28.337208 21782 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.339000 21839 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.342168 21431 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.349889 20802 raft_consensus.cc:2955] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.362218 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0b31661191c14b44ac964555853900f1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:28.369910 20633 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:28.393853 21844 ts_tablet_manager.cc:1929] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:28.394517 21844 log.cc:1199] T 8f3454624b7c40c6934257c77f2174de P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/8f3454624b7c40c6934257c77f2174de
I20260709 12:21:28.396013 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet 8f3454624b7c40c6934257c77f2174de (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:28.403514 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b31661191c14b44ac964555853900f1 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:28.404623 21834 tablet_replica.cc:333] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:28.405748 21834 raft_consensus.cc:2243] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:28.406581 21834 raft_consensus.cc:2272] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:28.410320 21834 ts_tablet_manager.cc:1916] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:28.423050 21834 ts_tablet_manager.cc:1929] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:28.423501 21834 log.cc:1199] T 0b31661191c14b44ac964555853900f1 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/0b31661191c14b44ac964555853900f1
I20260709 12:21:28.425034 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 0b31661191c14b44ac964555853900f1 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:29.475121 20730 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:29.481024 20882 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:29.482259 20701 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4415902e5b30489691f67f4a635305b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:29.482378 20805 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:29.482488 21777 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:29.483887 21821 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:29.495565 21777 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.497269 20805 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.500000 20882 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.514312 20805 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:29.514983 20631 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:29.522384 20876 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:29.524243 20776 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec190d02960a4e88af0807817b789e6c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:29.524950 20728 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:29.524578 21818 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:29.526461 21780 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:29.623617 21818 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.635531 20882 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.640182 20730 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.662279 20882 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:29.664803 20629 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.671216 20805 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:29.673424 20850 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 210839e60f364f8cab9f53ff440c461d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:29.673834 20730 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:29.675745 21822 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:29.678511 21854 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:29.692873 21854 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.694226 20805 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.729933 20882 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:29.728683 20731 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.733628 20631 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:29.764106 20850 consensus_peers.cc:597] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5e3c520e78241879a779a912c66a4ba. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:29.765125 20731 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:29.765314 20805 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:29.766546 21856 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:29.767477 21854 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:29.779796 21854 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.802081 20731 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.814996 20805 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:29.819751 20633 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:29.926224 21870 ts_tablet_manager.cc:933] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:29.928323 21870 tablet_copy_client.cc:323] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:29.929886 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet ec190d02960a4e88af0807817b789e6c from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:29.930361 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet ec190d02960a4e88af0807817b789e6c from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-ec190d02960a4e88af0807817b789e6c
I20260709 12:21:29.935772 20820 tablet_copy_source_session.cc:215] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:29.938474 21870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec190d02960a4e88af0807817b789e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.953506 21870 tablet_copy_client.cc:806] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:29.954075 21870 tablet_copy_client.cc:670] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:29.957787 21870 tablet_copy_client.cc:538] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:29.967734 21870 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:30.007652 21870 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:30.008545 21870 tablet_bootstrap.cc:492] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:30.009279 21870 ts_tablet_manager.cc:1403] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.042s	user 0.025s	sys 0.011s
I20260709 12:21:30.012140 21870 raft_consensus.cc:359] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.012935 21870 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:30.013456 21870 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.016266 21870 ts_tablet_manager.cc:1434] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:21:30.018829 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ec190d02960a4e88af0807817b789e6c received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:30.019582 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ec190d02960a4e88af0807817b789e6c on tablet ec190d02960a4e88af0807817b789e6c with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:30.040557 21870 ts_tablet_manager.cc:933] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
I20260709 12:21:30.042469 21870 tablet_copy_client.cc:323] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:21:30.043866 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet 4415902e5b30489691f67f4a635305b4 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:21:30.044303 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet 4415902e5b30489691f67f4a635305b4 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-4415902e5b30489691f67f4a635305b4
I20260709 12:21:30.050984 20745 tablet_copy_source_session.cc:215] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:30.053891 21870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4415902e5b30489691f67f4a635305b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.065714 21870 tablet_copy_client.cc:806] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:30.066308 21870 tablet_copy_client.cc:670] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:30.069463 21870 tablet_copy_client.cc:538] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:30.077986 21870 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:30.078011 21875 ts_tablet_manager.cc:933] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:30.084142 21875 tablet_copy_client.cc:323] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:30.085861 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 210839e60f364f8cab9f53ff440c461d from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:30.086436 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 210839e60f364f8cab9f53ff440c461d from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-210839e60f364f8cab9f53ff440c461d
I20260709 12:21:30.097415 20894 tablet_copy_source_session.cc:215] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:30.101611 21875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210839e60f364f8cab9f53ff440c461d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.104266 21870 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:30.105120 21870 tablet_bootstrap.cc:492] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:30.105798 21870 ts_tablet_manager.cc:1403] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.004s
I20260709 12:21:30.108731 21870 raft_consensus.cc:359] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.109602 21870 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:30.110282 21870 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.112123 21870 ts_tablet_manager.cc:1434] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:30.118439 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-4415902e5b30489691f67f4a635305b4 received from {username='slave'} at 127.0.0.1:33906
I20260709 12:21:30.118882 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-4415902e5b30489691f67f4a635305b4 on tablet 4415902e5b30489691f67f4a635305b4 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:30.153613 21875 tablet_copy_client.cc:806] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:30.154157 21875 tablet_copy_client.cc:670] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:30.157291 21875 tablet_copy_client.cc:538] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:30.164680 21875 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:30.173751 21870 ts_tablet_manager.cc:933] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:30.183633 21870 tablet_copy_client.cc:323] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:30.185336 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet d5e3c520e78241879a779a912c66a4ba from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:30.185875 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet d5e3c520e78241879a779a912c66a4ba from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-d5e3c520e78241879a779a912c66a4ba
I20260709 12:21:30.192391 21875 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:30.193360 21875 tablet_bootstrap.cc:492] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:30.194056 21875 ts_tablet_manager.cc:1403] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.030s	user 0.023s	sys 0.001s
I20260709 12:21:30.197665 20894 tablet_copy_source_session.cc:215] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:30.197293 21875 raft_consensus.cc:359] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.198307 21875 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:30.199121 21875 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.201618 21870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5e3c520e78241879a779a912c66a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.204147 21875 ts_tablet_manager.cc:1434] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:21:30.207996 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-210839e60f364f8cab9f53ff440c461d received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:30.208472 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-210839e60f364f8cab9f53ff440c461d on tablet 210839e60f364f8cab9f53ff440c461d with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:30.220728 21870 tablet_copy_client.cc:806] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:30.221344 21870 tablet_copy_client.cc:670] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:30.224435 21870 tablet_copy_client.cc:538] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:30.230227 21870 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:30.243563 21870 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:30.244194 21870 tablet_bootstrap.cc:492] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:30.244686 21870 ts_tablet_manager.cc:1403] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.015s	user 0.015s	sys 0.001s
I20260709 12:21:30.246970 21870 raft_consensus.cc:359] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.247915 21870 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:30.248509 21870 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:30.250525 21870 ts_tablet_manager.cc:1434] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:30.252115 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d5e3c520e78241879a779a912c66a4ba received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:30.252521 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d5e3c520e78241879a779a912c66a4ba on tablet d5e3c520e78241879a779a912c66a4ba with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:30.444597 21432 raft_consensus.cc:1217] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:30.518153 21432 raft_consensus.cc:1217] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:21:30.635707 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 79672.4us
I20260709 12:21:30.664654 21432 raft_consensus.cc:1217] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:30.685353 21432 raft_consensus.cc:1217] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:21:30.707294 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55601.9us
W20260709 12:21:30.715523 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59027.3us
I20260709 12:21:30.958891 21818 raft_consensus.cc:1064] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:30.963542 21818 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:30.976063 20876 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:30.978874 21432 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:30.979236 21860 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:30.980499 20731 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:30.981735 21860 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:30.982514 21818 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:21:31.019271 21886 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.025745 20730 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.036293 21853 raft_consensus.cc:1064] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:31.028971 21432 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.044149 21853 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.044598 20876 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.060757 20629 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.087222 20876 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.088737 21432 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.092202 21845 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:31.093640 21896 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:31.111606 20805 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.113751 20876 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.116106 21891 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.120661 20730 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.123842 21434 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.125617 21885 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:31.129155 20805 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.129410 21860 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:31.134443 21433 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.134115 21777 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:31.140606 21839 raft_consensus.cc:1064] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:31.145803 20630 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:31.149333 21839 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.156054 21884 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.161885 20730 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.175329 21432 raft_consensus.cc:2955] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.177146 20805 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.193729 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec190d02960a4e88af0807817b789e6c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:31.203152 20630 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:31.217516 21856 raft_consensus.cc:1064] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:31.239018 21856 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.248872 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet ec190d02960a4e88af0807817b789e6c with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:31.267817 21432 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.270205 21822 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:31.280419 20805 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.283263 21902 tablet_replica.cc:333] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:31.281744 20730 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.284560 21822 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:31.289392 21839 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:31.295754 21433 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.297338 21856 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:31.301676 21902 raft_consensus.cc:2243] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.303404 20731 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.304240 20809 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:31.313586 21902 raft_consensus.cc:2272] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.313519 21904 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:31.317162 21822 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:31.322068 21902 ts_tablet_manager.cc:1916] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:31.343215 21904 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.352444 20730 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.371883 21856 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.363068 20805 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.382268 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:31.387413 21432 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.395648 20630 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:31.389257 20731 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.410802 21433 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.414085 20882 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.423414 21902 ts_tablet_manager.cc:1929] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:31.424000 21902 log.cc:1199] T ec190d02960a4e88af0807817b789e6c P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/ec190d02960a4e88af0807817b789e6c
I20260709 12:21:31.426059 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet ec190d02960a4e88af0807817b789e6c (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:31.431624 21433 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.434427 21904 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:31.436643 20630 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.444015 21839 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.446738 21435 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.449771 20728 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.470067 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 210839e60f364f8cab9f53ff440c461d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:31.471632 21882 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:31.471452 20630 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.475152 20805 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.490705 20730 raft_consensus.cc:2955] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.494086 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet 210839e60f364f8cab9f53ff440c461d with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:31.507045 20876 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.514907 21911 tablet_replica.cc:333] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:31.541333 21435 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.542932 21882 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:31.549641 20805 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.551160 21856 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:31.557485 21856 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.558974 21435 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.562471 20809 raft_consensus.cc:2955] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.576741 21911 raft_consensus.cc:2243] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.577605 21911 raft_consensus.cc:2272] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.586903 20632 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.592578 21911 ts_tablet_manager.cc:1916] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:31.598563 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet d5e3c520e78241879a779a912c66a4ba with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:31.601223 21912 tablet_replica.cc:333] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:31.606314 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d5e3c520e78241879a779a912c66a4ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:31.620417 21911 ts_tablet_manager.cc:1929] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:31.621021 21911 log.cc:1199] T 210839e60f364f8cab9f53ff440c461d P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/210839e60f364f8cab9f53ff440c461d
I20260709 12:21:31.623384 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet 210839e60f364f8cab9f53ff440c461d (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:31.626955 21912 raft_consensus.cc:2243] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.627807 21912 raft_consensus.cc:2272] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.631690 21912 ts_tablet_manager.cc:1916] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:31.651114 20882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:31.652547 20882 raft_consensus.cc:606] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:31.656529 20730 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.664717 21435 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.666407 21891 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:31.669611 21912 ts_tablet_manager.cc:1929] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:31.670259 21912 log.cc:1199] T d5e3c520e78241879a779a912c66a4ba P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/d5e3c520e78241879a779a912c66a4ba
I20260709 12:21:31.672453 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet d5e3c520e78241879a779a912c66a4ba (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:31.675160 21845 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.677050 20809 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:31.676919 21433 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.685361 21777 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:31.692807 20809 raft_consensus.cc:2955] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.695003 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4415902e5b30489691f67f4a635305b4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:31.698407 20631 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:31.697847 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4415902e5b30489691f67f4a635305b4"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:31.701833 20730 raft_consensus.cc:606] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:31.708192 20882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "761ea542665e419c8a63392e3014efa7"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:31.708801 20882 raft_consensus.cc:606] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:31.710883 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet 4415902e5b30489691f67f4a635305b4 with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:31.712538 21902 tablet_replica.cc:333] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:31.713925 21902 raft_consensus.cc:2243] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.714835 21902 raft_consensus.cc:2272] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.714956 20876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:31.715574 20876 raft_consensus.cc:606] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:31.718914 21902 ts_tablet_manager.cc:1916] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:31.721422 20882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:31.721982 20882 raft_consensus.cc:606] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:31.737987 21902 ts_tablet_manager.cc:1929] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:31.738606 21902 log.cc:1199] T 4415902e5b30489691f67f4a635305b4 P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/4415902e5b30489691f67f4a635305b4
I20260709 12:21:31.740582 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet 4415902e5b30489691f67f4a635305b4 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:31.741336 20809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:31.741933 20809 raft_consensus.cc:606] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:31.744612 20882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:31.745119 20882 raft_consensus.cc:606] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:31.746341 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:21:31.748145 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:31.749078 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:31.871646 21856 raft_consensus.cc:993] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:31.872212 21904 raft_consensus.cc:1081] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:31.873633 20802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:21:31.874203 20802 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:31.874531 20802 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.880920 20802 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:31.882782 20802 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:31.883245 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:31.883301 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:31.884029 20732 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.884081 20882 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:31.884567 20882 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:31.885383 20882 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:31.886238 20882 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.889724 20732 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:31.890928 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:31.891664 21860 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:31.891954 20882 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:31.892060 21860 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:31.893045 21860 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:31.903615 20629 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:31.911420 21904 raft_consensus.cc:993] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:31.911932 21856 raft_consensus.cc:1081] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:31.913214 21433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:21:31.913650 21433 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:31.913970 21433 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.919495 21433 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.921244 21433 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:31.921983 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:31.922151 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:31.922565 20732 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.922608 20882 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:31.923329 20882 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:31.924069 20882 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.925248 20882 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.929033 20732 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:31.930037 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:31.930796 21873 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:31.931485 20882 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:31.931991 21873 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:31.932914 21873 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:31.943365 20629 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:32.139714 21904 raft_consensus.cc:993] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:32.140168 21856 raft_consensus.cc:1081] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:32.141459 20732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "761ea542665e419c8a63392e3014efa7"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:21:32.141911 20732 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:32.142195 20732 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.146521 20732 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:32.148278 20732 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:32.149241 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:32.149298 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:32.149962 20802 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.150051 20882 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.150554 20882 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:32.151732 20882 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:32.152686 20882 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.155416 20802 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:32.156818 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:32.157613 21891 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:32.158088 21891 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:32.158900 21891 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:32.158082 20882 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:32.165846 21856 raft_consensus.cc:993] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:32.166325 21882 raft_consensus.cc:1081] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:32.168052 21435 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:21:32.168701 21435 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:32.169111 21435 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.170792 21885 raft_consensus.cc:993] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:32.171331 21884 raft_consensus.cc:1081] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:32.171119 20629 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.173204 21433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44460
I20260709 12:21:32.173728 21433 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:32.174068 21433 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.176750 21435 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.179279 21435 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:32.182636 21845 raft_consensus.cc:993] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:32.183276 21845 raft_consensus.cc:1081] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:32.184872 21435 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4415902e5b30489691f67f4a635305b4"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44466
I20260709 12:21:32.185367 21904 raft_consensus.cc:993] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:32.185500 21435 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:32.185913 21904 raft_consensus.cc:1081] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:32.186093 21435 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.187706 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:32.187885 20730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:21:32.188412 20730 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:32.188524 20732 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.188817 20730 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.189330 21433 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.193491 21435 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.195513 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:32.196368 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:32.196894 20728 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.197165 20728 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:32.197371 21435 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:32.197923 20728 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.199021 20728 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.200197 20732 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:32.202127 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:32.202105 20730 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:32.203416 21873 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:32.203508 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:32.203934 21873 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:32.204885 21873 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.205196 20728 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:32.206300 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:32.196053 20882 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.207355 21918 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:32.207517 20882 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:32.208456 21433 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:32.204027 20802 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.208750 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:32.209265 20809 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.209578 20809 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:32.209353 20882 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.210047 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:32.210610 20882 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.210274 20809 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.211531 20809 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.210628 20876 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.215513 21918 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:32.216477 21918 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.217646 20882 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:32.219849 20802 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:32.224560 20876 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:32.224893 20809 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:32.225520 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:32.227447 20730 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:32.227447 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:32.228142 20882 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.228399 20882 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:32.229038 20882 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:32.229846 20882 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.230295 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:32.231009 20805 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.233552 21918 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:32.234057 21918 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:32.234906 21918 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:32.256175 20805 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:32.257239 20882 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:32.258188 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:32.263015 21891 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:32.268009 21891 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:32.271785 21891 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:32.299485 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57666.7us
I20260709 12:21:32.304044 20629 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:32.322081 20629 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:32.324384 20629 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:32.325318 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67069.6us
I20260709 12:21:32.341442 20731 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.344811 21928 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:32.357985 20629 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.371352 20882 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.372614 21885 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:32.420428 20882 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.421535 21873 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.432471 20731 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.433661 21921 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.650095 20731 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.651283 21921 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.665405 20882 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.668009 21873 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.689702 20732 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.690973 21921 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.696966 20809 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.698596 21891 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:32.700224 20809 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.702448 21918 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.712733 20876 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.714344 21891 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:32.792702 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:32.794425 20876 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:32.796001 20809 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.797894 21845 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:32.801965 20805 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:32.802997 20850 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f517f5d84b3437eb2fd2ae144fc6380. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:32.804066 21856 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:32.815487 20732 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:32.817340 21924 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:32.829483 20876 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.829999 20809 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.831621 21918 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.836316 21845 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:32.838824 20879 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.840886 21873 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:32.882071 21856 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:32.895277 20731 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:32.918391 20802 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:32.944888 20882 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:32.946115 21860 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:32.947476 20776 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff0a54292b924069aae5a0cd6c1366d1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:32.954835 20732 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:32.957409 21860 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:32.957691 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:32.958285 20802 raft_consensus.cc:606] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:32.961199 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b31661191c14b44ac964555853900f1"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:32.961721 20879 raft_consensus.cc:606] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:32.968309 20728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:32.969069 20728 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:32.980528 21885 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:32.990063 20731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "761ea542665e419c8a63392e3014efa7"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:32.990679 20731 raft_consensus.cc:606] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:32.990236 20879 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:32.998206 20805 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:32.999087 20876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:32.999675 20876 raft_consensus.cc:606] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:33.006039 20876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46401ecb9b443118647ee361081a01e"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:33.006510 20876 raft_consensus.cc:606] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:33.007577 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:21:33.012182 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:33.013154 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:33.014500 20776 consensus_peers.cc:597] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 981a4d0731e940b79eccfb33d1928fb4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:33.015154 20876 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:33.016637 20728 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:33.018316 21928 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:33.017951 20632 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.021011 21928 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:33.066301 20802 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.081288 20732 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.099659 21928 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.102792 20728 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.126238 21904 raft_consensus.cc:993] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:33.126766 21856 raft_consensus.cc:1081] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:33.127734 20629 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.128724 21432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:21:33.129455 21432 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:33.129766 21432 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.139860 20882 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.142691 20732 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:33.158264 20876 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:33.159685 21925 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:33.160482 20802 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:33.160692 20701 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7040eacc899473abc05a913812e82a4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:33.162277 21432 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.163587 21925 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:33.164453 21432 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:33.165565 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:33.165912 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:33.166116 20876 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.166513 20876 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:33.166590 20732 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.174192 20876 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.175432 20876 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.175681 21845 raft_consensus.cc:993] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:33.176126 21845 raft_consensus.cc:1081] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:33.178013 20732 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:33.179582 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:33.180662 21918 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:33.181164 21918 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:33.181996 21918 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.187922 21891 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.191431 20876 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:33.172242 20630 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.198769 20882 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.199875 20632 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.208521 20876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "761ea542665e419c8a63392e3014efa7"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:33.209120 20876 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:33.209470 20876 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:33.244455 20876 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.245934 20876 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:33.251370 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:33.251933 20731 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:33.252259 20731 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:33.252952 20731 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.254096 20731 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:33.255779 21966 ts_tablet_manager.cc:933] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:33.254839 20805 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.260540 21924 raft_consensus.cc:993] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:33.261039 21924 raft_consensus.cc:1081] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:33.261619 21966 tablet_copy_client.cc:323] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:33.262491 21432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b31661191c14b44ac964555853900f1"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:21:33.262928 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:33.263288 21432 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:33.263631 20802 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:33.263695 21432 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.270709 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 0f517f5d84b3437eb2fd2ae144fc6380 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:33.271298 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 0f517f5d84b3437eb2fd2ae144fc6380 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-0f517f5d84b3437eb2fd2ae144fc6380
I20260709 12:21:33.272049 20731 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:33.272917 21432 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.278635 21432 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:33.284709 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:33.285287 20809 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.291826 20632 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:33.293622 20802 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:33.296947 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:33.297519 20882 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.297824 20882 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:33.298532 20882 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.299688 20882 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.300588 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:33.302098 21882 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:33.302516 21904 raft_consensus.cc:993] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:33.302556 21882 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:33.304291 21882 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.307473 20882 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:33.303480 21904 raft_consensus.cc:1081] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:33.308719 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:33.309502 21873 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:33.310001 21873 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:33.310871 21873 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.316602 20808 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d46401ecb9b443118647ee361081a01e"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:21:33.317196 20808 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:33.317566 20808 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.319862 20809 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:33.322566 20894 tablet_copy_source_session.cc:215] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:33.324859 20808 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.330844 20808 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:33.332643 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:33.333413 21966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f517f5d84b3437eb2fd2ae144fc6380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.337423 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:33.341156 21891 raft_consensus.cc:993] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:33.341534 20878 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.341732 21891 raft_consensus.cc:1081] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:33.341868 20878 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:33.341183 20731 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.342767 20878 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.349236 20629 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 3, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.351485 20876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:33.355165 20876 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:33.355558 20876 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.348387 20632 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.386765 20876 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.425362 20876 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:33.427834 20731 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:33.431988 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:33.432801 20802 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.433329 20775 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:33.435096 20878 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.440083 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:33.440660 20731 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.440991 20731 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:33.441984 20731 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.438835 21959 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:33.444900 21959 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:33.445715 21959 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.448975 20802 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:33.450309 20731 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.479508 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:33.492969 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 121766us
I20260709 12:21:33.496110 21856 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:33.496579 21856 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:33.499076 20731 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:33.499713 21856 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:33.535915 20878 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:33.567595 21966 tablet_copy_client.cc:806] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:33.568295 21966 tablet_copy_client.cc:670] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:33.583861 21966 tablet_copy_client.cc:538] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:33.584193 21979 ts_tablet_manager.cc:933] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:33.584697 21860 raft_consensus.cc:993] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:33.585173 21860 raft_consensus.cc:1081] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:33.593446 21433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44460
I20260709 12:21:33.594090 21433 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:33.599138 20876 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:33.601742 21873 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:33.594506 21433 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.640537 21433 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.644675 20629 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.657322 21981 ts_tablet_manager.cc:933] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:33.660997 20632 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.675699 21981 tablet_copy_client.cc:323] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:33.677721 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 981a4d0731e940b79eccfb33d1928fb4 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:33.678275 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 981a4d0731e940b79eccfb33d1928fb4 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-981a4d0731e940b79eccfb33d1928fb4
I20260709 12:21:33.679881 21966 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:33.680280 21984 ts_tablet_manager.cc:933] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
I20260709 12:21:33.682354 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:33.683220 20732 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.684913 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:33.685483 20802 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.685760 20802 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:33.686484 20802 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.687871 20802 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.682569 21433 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:33.694581 20731 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:33.695928 21921 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:33.698341 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:33.711944 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:33.712947 21921 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:33.713462 21921 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:33.713809 20820 tablet_copy_source_session.cc:215] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:33.714265 21921 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:33.722733 21984 tablet_copy_client.cc:323] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:21:33.723954 21979 tablet_copy_client.cc:323] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:33.724676 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet b7040eacc899473abc05a913812e82a4 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:21:33.725416 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet b7040eacc899473abc05a913812e82a4 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-b7040eacc899473abc05a913812e82a4
I20260709 12:21:33.728310 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet ff0a54292b924069aae5a0cd6c1366d1 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:33.729410 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet ff0a54292b924069aae5a0cd6c1366d1 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-ff0a54292b924069aae5a0cd6c1366d1
I20260709 12:21:33.715742 20802 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:33.729707 21981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981a4d0731e940b79eccfb33d1928fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.754384 20629 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.754926 20820 tablet_copy_source_session.cc:215] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:33.772795 21966 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:33.773777 21966 tablet_bootstrap.cc:492] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:33.774466 21966 ts_tablet_manager.cc:1403] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.095s	user 0.014s	sys 0.008s
I20260709 12:21:33.776081 21979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff0a54292b924069aae5a0cd6c1366d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.777609 21966 raft_consensus.cc:359] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:33.783270 21966 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:33.783497 20745 tablet_copy_source_session.cc:215] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:33.782545 21981 tablet_copy_client.cc:806] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:33.787048 21981 tablet_copy_client.cc:670] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:33.784030 21966 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:33.797533 20876 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:33.801316 21921 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:33.801986 21984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7040eacc899473abc05a913812e82a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.807544 20802 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:33.824011 20802 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:33.830484 21966 ts_tablet_manager.cc:1434] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.056s	user 0.010s	sys 0.000s
I20260709 12:21:33.831981 21924 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:33.835151 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-0f517f5d84b3437eb2fd2ae144fc6380 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:33.835868 21918 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:33.835891 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-0f517f5d84b3437eb2fd2ae144fc6380 on tablet 0f517f5d84b3437eb2fd2ae144fc6380 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:33.843204 21981 tablet_copy_client.cc:538] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:33.850337 21984 tablet_copy_client.cc:806] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:33.850976 21984 tablet_copy_client.cc:670] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:33.885893 21984 tablet_copy_client.cc:538] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:33.892307 20732 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:33.894389 21860 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:33.930378 21979 tablet_copy_client.cc:806] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:33.935930 21979 tablet_copy_client.cc:670] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:33.935443 20733 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:33.944875 20730 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:33.946802 21856 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:33.954277 21856 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:33.970157 21979 tablet_copy_client.cc:538] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:33.986193 21981 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
W20260709 12:21:34.045053 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51733.7us
I20260709 12:21:34.045310 20805 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.057792 21856 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:34.063876 21984 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:34.079938 20882 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.168296 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:34.108678 21979 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:34.197165 21928 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:21:34.294868 20808 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:34.312994 21918 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:21:34.331450 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69974us
I20260709 12:21:34.333868 20732 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:34.336954 21873 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:34.371623 21435 raft_consensus.cc:1217] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:34.386664 21981 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:34.388550 21981 tablet_bootstrap.cc:492] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:34.389288 21981 ts_tablet_manager.cc:1403] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.403s	user 0.024s	sys 0.011s
I20260709 12:21:34.389971 20876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:34.390681 20876 raft_consensus.cc:606] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:34.392263 21981 raft_consensus.cc:359] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:34.393131 21981 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:34.393863 21981 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:34.394798 20805 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:34.395462 20805 raft_consensus.cc:606] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
W20260709 12:21:34.404459 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62646.4us
I20260709 12:21:34.419951 20733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:34.424229 20733 raft_consensus.cc:606] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:34.455878 21984 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:34.456796 21984 tablet_bootstrap.cc:492] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:34.457192 21981 ts_tablet_manager.cc:1434] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.002s
I20260709 12:21:34.457455 21984 ts_tablet_manager.cc:1403] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.394s	user 0.014s	sys 0.019s
I20260709 12:21:34.457472 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:34.458397 20730 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:34.468943 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-981a4d0731e940b79eccfb33d1928fb4 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:34.469386 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-981a4d0731e940b79eccfb33d1928fb4 on tablet 981a4d0731e940b79eccfb33d1928fb4 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:34.475236 21979 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:34.476184 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6fe576036c04c149240cd6107285eca"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:34.476780 20878 raft_consensus.cc:606] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:34.476871 21979 tablet_bootstrap.cc:492] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:34.477591 21979 ts_tablet_manager.cc:1403] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.386s	user 0.021s	sys 0.017s
I20260709 12:21:34.483011 21984 raft_consensus.cc:359] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:34.483978 21984 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:34.484400 21979 raft_consensus.cc:359] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:34.485065 21979 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:34.484589 21984 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:34.485684 21979 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:34.487588 21984 ts_tablet_manager.cc:1434] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.005s
I20260709 12:21:34.487984 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:34.489771 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:34.490521 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:34.492516 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-b7040eacc899473abc05a913812e82a4 received from {username='slave'} at 127.0.0.1:33906
I20260709 12:21:34.492937 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-b7040eacc899473abc05a913812e82a4 on tablet b7040eacc899473abc05a913812e82a4 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:34.488045 21979 ts_tablet_manager.cc:1434] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:21:34.511248 20820 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ff0a54292b924069aae5a0cd6c1366d1 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:34.511703 20820 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ff0a54292b924069aae5a0cd6c1366d1 on tablet ff0a54292b924069aae5a0cd6c1366d1 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:34.518553 21882 raft_consensus.cc:993] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:34.519021 22002 raft_consensus.cc:1081] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:34.521967 20730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6fe576036c04c149240cd6107285eca"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:21:34.522502 20730 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:34.522837 20730 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.529587 21435 raft_consensus.cc:1217] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:34.529735 20730 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:34.532269 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:34.532825 20810 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.533056 20730 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:34.534202 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:34.534741 20876 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:34.534991 20876 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:34.535748 20876 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:34.539137 20810 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:34.539387 20876 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.540210 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:34.541172 22008 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:34.541623 22008 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:34.542464 22008 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:34.551630 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:34.567800 20629 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.570825 21435 raft_consensus.cc:1217] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:34.668722 21974 raft_consensus.cc:993] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:34.669333 21960 raft_consensus.cc:1081] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:34.670857 20733 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37282
I20260709 12:21:34.671387 20733 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:34.671679 20733 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.676780 20733 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:34.679167 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:34.679739 20810 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:34.680033 20810 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:34.680236 20733 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:34.680720 20810 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:34.681968 20810 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.683498 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:34.684029 20876 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.688230 20810 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:34.689998 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:34.690515 20876 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:34.690793 22008 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:34.691366 22008 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:34.692191 22008 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:34.703951 20629 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.768155 21904 raft_consensus.cc:1064] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:34.770912 21904 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:34.789100 21435 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:34.791842 21924 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:34.792846 20802 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:34.796780 20730 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:34.796895 21924 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:34.804155 21882 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:34.816651 21976 raft_consensus.cc:993] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:34.817097 21976 raft_consensus.cc:1081] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:34.818710 21433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:21:34.819254 21433 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:34.819594 21433 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.836047 21433 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:34.843712 21856 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.849038 21432 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.856103 20733 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.861972 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:34.862586 20810 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.870389 20810 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:34.870433 20629 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.871459 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:34.872242 21918 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:34.872715 21918 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:34.873526 21918 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:34.874819 21433 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:34.878822 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:34.883483 20876 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:34.883798 20876 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:34.884506 20876 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:34.887359 20808 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.894517 20876 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.902510 21432 raft_consensus.cc:1217] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:34.912232 20631 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:34.918865 20878 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:34.925366 20876 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:34.926252 20730 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:34.928001 21433 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:34.931237 21924 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:34.932317 21904 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:34.945437 21924 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.946889 21433 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.951335 20730 raft_consensus.cc:2955] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.967370 20618 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0f517f5d84b3437eb2fd2ae144fc6380 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:34.974632 20630 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:34.976821 20808 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.979061 22004 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:34.987855 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f517f5d84b3437eb2fd2ae144fc6380 with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:34.989760 22026 tablet_replica.cc:333] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:34.991863 22026 raft_consensus.cc:2243] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.995950 22026 raft_consensus.cc:2272] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.999720 22026 ts_tablet_manager.cc:1916] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:35.007881 20878 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:35.012998 21928 raft_consensus.cc:1064] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:35.015516 21928 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:35.019929 21845 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:35.021502 21845 raft_consensus.cc:1064] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:35.040702 22026 ts_tablet_manager.cc:1929] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:35.041285 22026 log.cc:1199] T 0f517f5d84b3437eb2fd2ae144fc6380 P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/0f517f5d84b3437eb2fd2ae144fc6380
I20260709 12:21:35.043462 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet 0f517f5d84b3437eb2fd2ae144fc6380 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:35.045674 22008 raft_consensus.cc:993] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:35.046140 22008 raft_consensus.cc:1081] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:35.057842 21845 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:35.067021 20876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:35.067669 20876 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:35.068168 20876 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.096256 20876 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:35.101346 21972 raft_consensus.cc:993] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:35.101852 21972 raft_consensus.cc:1081] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:35.108884 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:35.109500 20881 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:35.109823 20881 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.111398 20876 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:35.111454 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:35.112365 20730 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:35.112634 20730 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:35.114809 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:35.115410 20805 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.116264 20876 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:35.113329 20730 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:35.123932 20730 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.131525 21960 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:35.137120 20805 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:35.138371 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:35.139168 21904 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:35.139730 21904 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:35.140805 20881 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:35.140949 21904 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:35.149993 20882 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:35.152256 20805 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:35.154874 21433 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:35.161882 21845 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:35.164285 22004 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:35.168845 21433 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:35.176407 21971 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:35.156837 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:35.173718 22004 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:21:35.262432 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74731.1us
I20260709 12:21:35.266925 20881 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:35.269482 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:35.271435 20730 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:35.271770 20730 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:35.275518 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:35.276132 20802 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.278831 20810 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:35.280295 20730 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:35.281149 21845 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:35.281555 20730 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.288332 20730 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:35.289779 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:35.319401 20881 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:35.320976 21845 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:35.323239 22038 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:35.331885 22038 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:35.332837 22038 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:35.335027 20878 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:35.336562 22001 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:35.352411 20631 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.374137 20805 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:35.377102 21918 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:35.389321 20802 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
W20260709 12:21:35.420733 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 139721us
I20260709 12:21:35.425297 21845 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:35.428839 20730 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:35.476765 21433 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:35.475803 20881 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:35.471221 20808 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:35.528730 21928 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:21:35.536466 21885 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:35.547247 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62264.4us
I20260709 12:21:35.556279 21433 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:35.660092 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:35.771340 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 255915us
I20260709 12:21:35.820397 20631 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.837325 20629 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.862016 20732 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:35.866168 22038 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:35.884160 20731 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:35.888777 21904 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:35.938069 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 356206us
W20260709 12:21:35.951753 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 369736us
I20260709 12:21:35.969899 22046 raft_consensus.cc:1064] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:35.973997 22046 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:35.985574 20878 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:35.997575 20732 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.020447 20805 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:36.022193 20805 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:36.026335 21435 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:36.028087 22046 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:36.042969 20732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:36.043643 20732 raft_consensus.cc:606] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:36.050802 20878 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:36.054164 20809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:36.054826 20809 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:36.059329 22068 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:36.061210 20808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:36.061928 20808 raft_consensus.cc:606] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:36.064687 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:36.065358 20730 raft_consensus.cc:606] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:36.066941 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:21:36.068720 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:36.075950 21856 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:36.090709 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:36.099077 21904 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:36.100684 20733 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:36.108727 22041 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.118829 20878 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.122329 20730 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:36.127135 22049 raft_consensus.cc:993] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:36.127626 22061 raft_consensus.cc:993] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:36.127868 22049 raft_consensus.cc:1081] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:36.129562 22059 raft_consensus.cc:1081] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:36.130290 20882 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:36.131026 20882 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:36.131572 20882 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.138324 20632 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.145291 20877 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:36.146059 20877 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:36.146493 20877 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.157976 21435 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:36.164008 22054 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:36.165722 20877 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.180678 20877 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:36.181727 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:36.183157 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:36.183341 20802 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.182968 22049 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.185230 21435 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.183772 20730 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:36.196228 20730 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:36.196509 21430 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.198239 20882 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.200488 20882 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:36.201148 20802 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:36.201727 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:36.202481 20734 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:36.202855 20734 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:36.203847 20734 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.205192 20734 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.206827 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:36.208258 21904 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:36.208868 21904 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:36.197332 20730 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.212718 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:36.213294 20802 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.209801 21904 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.223313 20802 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:36.224381 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:36.225198 21924 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:36.225741 21924 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:36.231805 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7040eacc899473abc05a913812e82a4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:36.234328 20730 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.226605 21924 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.236433 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:36.237262 21928 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:36.240005 22071 raft_consensus.cc:993] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:36.240459 22071 raft_consensus.cc:1081] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:36.250571 20730 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:36.264418 20730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37282
I20260709 12:21:36.265172 20730 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:36.265540 20730 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.268764 20808 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:36.269395 20802 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:36.275780 21430 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:36.276408 20882 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:36.277546 22068 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:36.278306 21928 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:36.279632 22049 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:36.286165 20730 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.294581 20730 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:36.296952 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:36.297559 20878 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.304414 21928 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.309512 20877 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.312041 20734 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.314806 20802 raft_consensus.cc:2955] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.314792 20631 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.316601 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:36.321492 20808 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:36.321821 20808 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:36.322526 20808 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.324368 20878 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:36.330240 20808 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:36.340104 20808 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:36.343381 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:36.349282 22054 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:36.349805 22054 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:36.350649 22054 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:36.352175 21435 raft_consensus.cc:2955] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.359369 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 981a4d0731e940b79eccfb33d1928fb4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:36.360631 20630 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.374270 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet b7040eacc899473abc05a913812e82a4 with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:36.376513 22087 tablet_replica.cc:333] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:36.377543 22087 raft_consensus.cc:2243] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:36.384816 22087 raft_consensus.cc:2272] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:36.388674 22087 ts_tablet_manager.cc:1916] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:36.392266 20632 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.416769 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 981a4d0731e940b79eccfb33d1928fb4 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:36.432293 22087 ts_tablet_manager.cc:1929] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:36.438505 22087 log.cc:1199] T b7040eacc899473abc05a913812e82a4 P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/b7040eacc899473abc05a913812e82a4
I20260709 12:21:36.440647 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet b7040eacc899473abc05a913812e82a4 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:36.458683 20633 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.460201 22088 tablet_replica.cc:333] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:36.466773 22088 raft_consensus.cc:2243] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:36.467689 22088 raft_consensus.cc:2272] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:36.477555 20628 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.483475 20628 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.496681 22088 ts_tablet_manager.cc:1916] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:36.530323 22088 ts_tablet_manager.cc:1929] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:36.530838 22088 log.cc:1199] T 981a4d0731e940b79eccfb33d1928fb4 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/981a4d0731e940b79eccfb33d1928fb4
I20260709 12:21:36.532647 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 981a4d0731e940b79eccfb33d1928fb4 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:36.598105 22046 raft_consensus.cc:993] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:36.598650 22046 raft_consensus.cc:1081] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:36.600248 21430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44460
I20260709 12:21:36.600803 21430 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:36.601155 21430 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:36.608201 21430 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:36.610597 21430 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:36.611135 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:36.611123 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:36.612001 20802 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:36.612035 20878 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:36.612511 20802 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:36.613336 20802 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:36.614639 20802 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:36.620224 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:21:36.629722 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:36.630652 22050 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:36.631325 22050 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:36.632300 22050 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:36.633716 20802 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:21:36.647315 20808 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:36.649394 20805 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:36.654000 21904 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:36.656075 21435 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:36.658535 20730 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:36.662075 22046 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:36.664690 21928 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:36.672973 21928 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.678191 20633 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 4, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 4 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.684211 20733 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.686291 21435 raft_consensus.cc:2955] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.704061 20805 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:36.706250 21924 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:36.709901 20633 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:36.721042 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ff0a54292b924069aae5a0cd6c1366d1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:36.736430 20730 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:36.738989 21904 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:36.745782 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet ff0a54292b924069aae5a0cd6c1366d1 with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
I20260709 12:21:36.747036 22087 tablet_replica.cc:333] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:36.748169 22087 raft_consensus.cc:2243] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:36.748860 22087 raft_consensus.cc:2272] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:36.751180 20730 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:36.757228 21904 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:36.757676 22087 ts_tablet_manager.cc:1916] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:36.778831 22087 ts_tablet_manager.cc:1929] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:36.779268 22087 log.cc:1199] T ff0a54292b924069aae5a0cd6c1366d1 P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/ff0a54292b924069aae5a0cd6c1366d1
I20260709 12:21:36.780876 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet ff0a54292b924069aae5a0cd6c1366d1 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:36.787922 20878 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:36.789641 22049 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:36.794492 20808 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:36.797518 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08000bf560 after lost signal to thread 19606
I20260709 12:21:36.798180 22049 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:36.798664 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08000bfbc0 after lost signal to thread 20667
W20260709 12:21:36.799969 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08002861a0 after lost signal to thread 20759
W20260709 12:21:36.804102 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800368a20 after lost signal to thread 20834
W20260709 12:21:36.805210 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800315da0 after lost signal to thread 20908
W20260709 12:21:36.806805 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4da0 after lost signal to thread 21459
I20260709 12:21:37.190124 20878 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:37.191478 22050 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:37.214587 20808 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:37.216378 22102 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:37.466621 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:37.509608 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:37.510223 20877 raft_consensus.cc:606] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:37.512706 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:37.513183 20877 raft_consensus.cc:606] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:37.515311 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad57e68a6a114847ac252579307658f7"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:37.515723 20877 raft_consensus.cc:606] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:37.517828 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:37.518230 20877 raft_consensus.cc:606] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:37.521234 20733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:37.522192 20733 raft_consensus.cc:606] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:37.523516 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:37.525096 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:37.525820 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:37.538519 21904 raft_consensus.cc:993] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:37.539052 22094 raft_consensus.cc:1081] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:37.541127 21430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:21:37.541733 21430 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.542148 21430 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.554752 21430 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:37.567806 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:37.568416 20805 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.576378 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:37.575891 21430 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:37.576877 20805 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:37.576903 20877 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.578058 20877 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:37.578717 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:37.578816 20877 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:37.579617 22050 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.580157 22050 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:37.580386 20877 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.581033 22050 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:37.586841 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:37.591866 20633 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.652486 22094 raft_consensus.cc:993] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:37.652999 22094 raft_consensus.cc:1081] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:37.656208 20733 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:21:37.656783 20733 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.657174 20733 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.660509 22038 raft_consensus.cc:993] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:37.661083 21904 raft_consensus.cc:1081] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:37.663072 20808 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad57e68a6a114847ac252579307658f7"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:21:37.663754 20808 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.664184 20808 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.665261 20733 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.668170 20733 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:37.668098 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:37.668380 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:37.668768 20805 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.669206 20877 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.669589 20877 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:37.670516 20877 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.671689 20877 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.672300 20808 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:37.677875 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:37.678457 20878 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.678794 20877 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:37.678807 20878 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:37.684883 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:37.685004 20878 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:37.684942 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:37.685664 21430 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.686311 20878 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.687135 21972 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.687683 21972 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:37.688588 21972 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.681438 20808 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:37.681231 20805 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:37.696460 21430 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:37.702582 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:37.704329 22068 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.704838 22068 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:37.705746 22068 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:37.707468 20878 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:37.717464 20633 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.730373 20628 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.829510 20734 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:37.836269 20882 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:21:37.837442 20701 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7cd857e30c3240c5a4fdd11bc0602c00. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:37.838662 20805 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:37.838444 21972 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:37.840973 22054 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:37.847486 22038 raft_consensus.cc:993] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:37.848030 22038 raft_consensus.cc:1081] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:37.849900 20808 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:21:37.850437 20808 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.850816 20808 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.851476 22054 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:37.859460 20805 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:37.859783 20882 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:37.861619 20808 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.870761 20802 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:37.871912 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:37.872514 20734 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.884092 20628 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.888190 20734 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:37.901115 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:37.901736 20878 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:37.902042 20878 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:37.902730 20878 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.903731 20878 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.914548 21972 raft_consensus.cc:993] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:37.915100 21972 raft_consensus.cc:1081] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:37.917176 20877 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:37.917697 20877 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.918035 20877 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.925994 20877 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.937219 20882 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:21:37.937952 20776 consensus_peers.cc:597] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d46401ecb9b443118647ee361081a01e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:37.940382 22068 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:37.944077 20730 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:37.944530 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:37.951109 22068 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:37.952039 21928 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:37.955319 20808 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:37.961253 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:37.961932 20734 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.962282 20734 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:37.963094 20734 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.964874 20734 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.974032 22068 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:37.975910 22068 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:37.987876 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:37.988298 20878 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
W20260709 12:21:37.997684 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46188.1us
I20260709 12:21:37.988559 20808 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.989495 20877 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:38.020236 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:38.021549 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:38.022532 22094 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.023119 22094 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:38.024144 22094 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:38.029891 20877 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:38.031337 20808 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:38.033603 22118 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:38.033314 22041 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.026661 20628 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 3, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.055445 20808 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:38.055876 20878 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.082720 20881 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:38.083976 20632 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:38.085330 21928 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:38.090344 20776 consensus_peers.cc:597] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1668b10d46ef4bfeaa2dc7ccbad48f81. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:38.074121 20730 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.094339 20733 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:38.096421 20808 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:38.097149 21928 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.099041 22118 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:38.099962 20628 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 2 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:38.120278 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 111393us
I20260709 12:21:38.141276 22068 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.151439 21430 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:38.154073 22116 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:38.171296 20881 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.206064 20805 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:38.213725 20881 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:38.215138 22041 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:21:38.225456 20776 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5c70f45aef146f4bcb47f1498221b99. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:38.223048 20878 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:38.228637 20730 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.232517 22068 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:38.237389 20878 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.240307 21845 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:38.244210 20734 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:38.246161 22068 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:38.262979 22041 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.269335 20734 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.276124 20805 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.278028 22054 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:38.302366 20882 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.313638 20632 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.416713 20628 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:38.439572 22143 ts_tablet_manager.cc:933] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
I20260709 12:21:38.500175 20734 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:38.533706 21928 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:38.514099 20878 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:38.518391 20808 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.537950 21928 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:38.555609 22141 ts_tablet_manager.cc:933] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:38.571964 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:38.643033 22143 tablet_copy_client.cc:323] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:21:38.643765 22149 ts_tablet_manager.cc:933] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:38.647814 22141 tablet_copy_client.cc:323] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:38.648149 22149 tablet_copy_client.cc:323] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:38.650460 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet 7cd857e30c3240c5a4fdd11bc0602c00 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:21:38.651002 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet 7cd857e30c3240c5a4fdd11bc0602c00 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-7cd857e30c3240c5a4fdd11bc0602c00
I20260709 12:21:38.655453 20819 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:38.656075 20819 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-1668b10d46ef4bfeaa2dc7ccbad48f81
I20260709 12:21:38.650475 20820 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet d46401ecb9b443118647ee361081a01e from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:38.658209 20820 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet d46401ecb9b443118647ee361081a01e from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-d46401ecb9b443118647ee361081a01e
I20260709 12:21:38.666354 20820 tablet_copy_source_session.cc:215] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:38.673818 22141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46401ecb9b443118647ee361081a01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:38.688133 20745 tablet_copy_source_session.cc:215] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:38.693243 20819 tablet_copy_source_session.cc:215] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:38.700868 22149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:38.701922 22143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd857e30c3240c5a4fdd11bc0602c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:38.709165 22141 tablet_copy_client.cc:806] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:38.717835 20734 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.723276 22143 tablet_copy_client.cc:806] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:38.764350 22143 tablet_copy_client.cc:670] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:38.726472 22038 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.728632 22152 ts_tablet_manager.cc:933] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:38.738032 22149 tablet_copy_client.cc:806] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:38.767410 22149 tablet_copy_client.cc:670] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:38.773551 22143 tablet_copy_client.cc:538] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:38.783245 22152 tablet_copy_client.cc:323] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
I20260709 12:21:38.788447 21904 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.792431 22149 tablet_copy_client.cc:538] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:38.793152 22141 tablet_copy_client.cc:670] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:38.795082 20819 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet d5c70f45aef146f4bcb47f1498221b99 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:21:38.795737 20819 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet d5c70f45aef146f4bcb47f1498221b99 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-d5c70f45aef146f4bcb47f1498221b99
I20260709 12:21:38.806527 20819 tablet_copy_source_session.cc:215] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:38.807461 22141 tablet_copy_client.cc:538] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:38.819296 22152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5c70f45aef146f4bcb47f1498221b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:38.822068 22149 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:38.842288 22143 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:38.865969 20882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:38.866657 20882 raft_consensus.cc:606] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:38.869457 20808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:38.870076 20808 raft_consensus.cc:606] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:38.874171 20734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6fe576036c04c149240cd6107285eca"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:38.889290 20734 raft_consensus.cc:606] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:38.894698 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:38.896386 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:38.897128 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:38.908998 22141 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:38.928287 22152 tablet_copy_client.cc:806] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:38.928896 22152 tablet_copy_client.cc:670] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:38.933097 22152 tablet_copy_client.cc:538] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:38.944667 22152 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:38.965409 22141 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:38.966346 22141 tablet_bootstrap.cc:492] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:38.967029 22141 ts_tablet_manager.cc:1403] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.007s
I20260709 12:21:38.975272 22141 raft_consensus.cc:359] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:38.976122 22141 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:38.976770 22141 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:38.984378 22141 ts_tablet_manager.cc:1434] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:21:38.991321 22149 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:38.992473 22149 tablet_bootstrap.cc:492] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:38.992745 22152 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:38.993276 22149 ts_tablet_manager.cc:1403] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.171s	user 0.014s	sys 0.011s
I20260709 12:21:38.993702 22152 tablet_bootstrap.cc:492] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:38.994377 22152 ts_tablet_manager.cc:1403] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.009s
I20260709 12:21:38.992693 21430 raft_consensus.cc:1217] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:21:38.998622 22149 raft_consensus.cc:359] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:39.002524 20819 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d46401ecb9b443118647ee361081a01e received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:39.006217 20819 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d46401ecb9b443118647ee361081a01e on tablet d46401ecb9b443118647ee361081a01e with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:39.005824 22149 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:39.007078 22149 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:39.002195 22152 raft_consensus.cc:359] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:39.024828 22152 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:39.025552 22152 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:39.031406 22143 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:39.032356 22143 tablet_bootstrap.cc:492] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:39.034680 22152 ts_tablet_manager.cc:1434] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.001s
I20260709 12:21:39.033082 22143 ts_tablet_manager.cc:1403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.191s	user 0.022s	sys 0.003s
I20260709 12:21:39.037542 22149 ts_tablet_manager.cc:1434] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:21:39.037648 20819 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d5c70f45aef146f4bcb47f1498221b99 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:39.038856 20819 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-d5c70f45aef146f4bcb47f1498221b99 on tablet d5c70f45aef146f4bcb47f1498221b99 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:39.040362 22143 raft_consensus.cc:359] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:39.041267 22143 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:39.042833 22143 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:39.049949 20819 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-1668b10d46ef4bfeaa2dc7ccbad48f81 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:21:39.050437 20819 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-1668b10d46ef4bfeaa2dc7ccbad48f81 on tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:39.045337 22143 ts_tablet_manager.cc:1434] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:39.067219 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-7cd857e30c3240c5a4fdd11bc0602c00 received from {username='slave'} at 127.0.0.1:33906
I20260709 12:21:39.067726 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-7cd857e30c3240c5a4fdd11bc0602c00 on tablet 7cd857e30c3240c5a4fdd11bc0602c00 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:39.164925 22142 raft_consensus.cc:993] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:39.165450 22142 raft_consensus.cc:1081] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:39.168251 20734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:21:39.168831 20734 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:39.169219 20734 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.182271 20734 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.209649 21928 raft_consensus.cc:993] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:39.210130 21928 raft_consensus.cc:1081] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:39.215230 20729 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37282
I20260709 12:21:39.215793 20729 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:39.216112 20729 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.224227 20729 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.228591 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:39.229230 20808 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:39.229538 20808 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:39.229653 20729 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:39.230321 20808 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.231727 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:39.232322 20878 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.244913 20808 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.290438 21845 raft_consensus.cc:993] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:39.290922 21845 raft_consensus.cc:1081] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:39.292282 20878 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:39.293663 21430 raft_consensus.cc:1217] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:21:39.300406 20882 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6fe576036c04c149240cd6107285eca"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:39.310293 20882 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:39.310688 20882 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.301188 20734 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:39.301759 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:39.306154 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:39.312582 20802 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.322551 20808 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:39.323383 20802 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:39.324304 22054 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:39.324818 22054 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:39.325182 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:39.325908 21972 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:39.325667 22054 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.326411 21972 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:39.328509 20882 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.332623 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:39.333190 20877 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:39.333468 20877 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:39.334160 20877 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.341756 20877 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.347163 20882 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:39.349622 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:39.350205 20734 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:39.350504 20734 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:39.351215 20734 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.352314 20734 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.359089 20877 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:39.330179 21972 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:39.371160 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:39.373838 20805 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.376765 20734 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:39.378759 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:39.379575 21430 raft_consensus.cc:1217] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:39.382097 21904 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:39.403548 20805 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:39.407660 21904 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:39.408499 21904 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:39.429338 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61714.2us
W20260709 12:21:39.447811 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64282us
I20260709 12:21:39.493995 20628 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.496505 20628 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.496028 20632 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 3, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.505654 22068 raft_consensus.cc:1064] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:39.508602 22068 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:39.531730 21430 raft_consensus.cc:1217] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:39.538551 21433 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:39.541332 22173 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:39.542480 20734 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:39.543880 22117 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:39.617532 20877 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:39.620992 22117 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:39.659909 21928 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:39.692440 20882 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:39.705433 22163 raft_consensus.cc:1064] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:39.706116 20733 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:39.708967 22163 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:39.723546 21433 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:39.732021 20882 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:39.734306 20802 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:39.732057 21430 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:39.735467 22163 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:21:39.743331 21845 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:39.744621 21972 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:39.754256 20877 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:39.760530 22054 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:39.750273 20633 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.805264 21928 raft_consensus.cc:1064] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:39.808349 21928 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:39.819473 20805 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:39.821318 21972 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:39.841095 20808 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:39.842660 21972 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:39.855204 21430 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:39.857919 20733 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:39.861981 22173 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:39.868873 21928 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:39.869289 20882 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:39.878702 21845 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:39.892853 21435 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:39.919257 20808 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:39.996608 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72475.6us
I20260709 12:21:40.005498 20633 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.006691 20802 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:40.010010 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 86200.7us
I20260709 12:21:40.011165 22173 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:40.016930 21433 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:40.033277 20804 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:21:40.035578 22191 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:21:40.048847 20876 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.058264 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:40.063742 22174 raft_consensus.cc:1064] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:40.064438 22179 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:40.066428 22174 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:40.076494 22173 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.079692 20877 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:40.089263 21972 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:40.096472 22145 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:40.098721 20877 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:40.134426 22178 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:40.140825 20734 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:40.150074 21430 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.158481 20877 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.163069 20881 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:40.204821 22174 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.210640 22179 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:40.215410 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d46401ecb9b443118647ee361081a01e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:40.217478 20878 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.220623 21435 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:40.230513 22145 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:40.246824 22174 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:40.249538 21430 raft_consensus.cc:2955] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.206971 20633 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:40.269618 20730 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:40.295684 21435 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.294179 22174 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.326982 21928 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:40.337252 20729 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:40.354600 20879 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:40.355448 20633 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 4 to 5, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 2 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.356168 21435 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:40.356271 22199 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:40.358777 22198 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:40.361358 20878 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.367470 21845 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.373323 20879 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.385748 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.388581 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7cd857e30c3240c5a4fdd11bc0602c00 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:21:40.385525 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53595.3us
I20260709 12:21:40.399642 20730 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.402012 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:40.402896 20878 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:40.430809 20804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:40.431494 20804 raft_consensus.cc:606] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:40.439291 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:40.439932 20730 raft_consensus.cc:606] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:40.441320 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:40.442983 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:40.443768 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:40.454409 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet d46401ecb9b443118647ee361081a01e with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:40.472326 20729 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.480070 20633 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 4 to 5, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.491086 22210 tablet_replica.cc:333] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:40.492612 22210 raft_consensus.cc:2243] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:40.493392 22210 raft_consensus.cc:2272] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:40.494897 21430 raft_consensus.cc:2955] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.497283 20804 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:40.502842 22210 ts_tablet_manager.cc:1916] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:40.506863 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet 7cd857e30c3240c5a4fdd11bc0602c00 with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:40.512190 21435 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:40.522356 22117 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:40.527984 20878 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:40.531844 22199 raft_consensus.cc:993] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:40.532269 22199 raft_consensus.cc:1081] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:40.534008 22174 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:40.533996 21928 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.539625 20878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:40.540288 20878 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:40.540686 20878 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.538802 20877 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.548105 21435 raft_consensus.cc:2955] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.570894 22211 tablet_replica.cc:333] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:21:40.573935 22210 ts_tablet_manager.cc:1929] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:21:40.587318 22210 log.cc:1199] T d46401ecb9b443118647ee361081a01e P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/d46401ecb9b443118647ee361081a01e
I20260709 12:21:40.580828 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:40.586256 22174 raft_consensus.cc:993] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:40.592458 22174 raft_consensus.cc:1081] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:40.594121 20879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54488
I20260709 12:21:40.594620 20879 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:40.594934 20879 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.562732 20878 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:40.587482 22211 raft_consensus.cc:2243] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:40.598102 22211 raft_consensus.cc:2272] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:40.600514 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:40.601078 20802 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.601377 20879 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:40.603933 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:40.604475 20804 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:40.604770 20804 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:40.605492 20804 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:40.606539 20804 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.608315 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:40.608901 20729 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:40.609248 20729 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:40.609848 20802 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:40.609998 20729 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:40.611246 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:40.612231 22145 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:40.612560 20878 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:40.612787 22145 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:40.613842 22145 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:40.616128 20879 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:40.617022 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:40.617575 21435 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.620757 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet d46401ecb9b443118647ee361081a01e (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:40.623670 20729 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.624398 21435 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:40.625521 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:40.626294 22170 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:40.613842 22211 ts_tablet_manager.cc:1916] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:40.626784 22170 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:40.628151 20804 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:40.628213 22170 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:40.639576 20729 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:40.626742 20632 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 3 to 4, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.644850 20802 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:40.670585 22211 ts_tablet_manager.cc:1929] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:21:40.671173 22211 log.cc:1199] T 7cd857e30c3240c5a4fdd11bc0602c00 P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/7cd857e30c3240c5a4fdd11bc0602c00
I20260709 12:21:40.673182 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet 7cd857e30c3240c5a4fdd11bc0602c00 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:40.681437 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37260
W20260709 12:21:40.682717 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67560.7us
I20260709 12:21:40.683604 22210 tablet_replica.cc:333] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:21:40.684702 22210 raft_consensus.cc:2243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:40.685403 22210 raft_consensus.cc:2272] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:40.688691 21435 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:40.690635 21928 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:40.691267 22210 ts_tablet_manager.cc:1916] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:40.697297 20730 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:40.698849 22170 raft_consensus.cc:993] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:40.699394 22200 raft_consensus.cc:1081] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:40.701299 20805 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:21:40.701835 20805 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:40.702211 20805 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.703435 21928 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.704814 22173 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:40.712910 20805 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:40.715422 20730 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.717483 21435 raft_consensus.cc:2955] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.742415 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d5c70f45aef146f4bcb47f1498221b99 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:40.768208 22210 ts_tablet_manager.cc:1929] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:40.768752 22210 log.cc:1199] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/1668b10d46ef4bfeaa2dc7ccbad48f81
I20260709 12:21:40.782064 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 1668b10d46ef4bfeaa2dc7ccbad48f81 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:40.784825 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:40.785617 20731 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.811170 20630 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 3 to 4, VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) evicted. New cstate: current_term: 1 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:40.811866 20633 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:40.812852 20731 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:40.813637 20633 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:40.815908 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:40.817749 22117 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:40.818249 22117 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:40.819089 22117 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:40.833766 20805 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:40.834782 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:40.835407 20879 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:40.835681 20879 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:40.836360 20879 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:40.837545 20879 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.847805 20879 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:40.904822 20869 tablet_service.cc:1558] Processing DeleteTablet for tablet d5c70f45aef146f4bcb47f1498221b99 with delete_type TABLET_DATA_TOMBSTONED (TS 116023b4c0c94bfea737fc238bda3838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54462
W20260709 12:21:40.969199 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45264.7us
I20260709 12:21:40.959756 22216 tablet_replica.cc:333] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:21:40.990646 20632 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 3, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.051026 22216 raft_consensus.cc:2243] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:41.052474 22216 raft_consensus.cc:2272] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:41.057752 22216 ts_tablet_manager.cc:1916] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:41.058002 20731 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:41.059944 22200 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:41.071506 21435 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:41.074357 22200 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:41.089722 22216 ts_tablet_manager.cc:1929] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:41.090379 22216 log.cc:1199] T d5c70f45aef146f4bcb47f1498221b99 P 116023b4c0c94bfea737fc238bda3838: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/wals/d5c70f45aef146f4bcb47f1498221b99
I20260709 12:21:41.092609 20618 catalog_manager.cc:5028] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): tablet d5c70f45aef146f4bcb47f1498221b99 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:21:41.116863 20802 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:41.119611 22200 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.133616 20805 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:41.135957 22200 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:41.295137 20731 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:41.296418 22117 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:41.303128 20879 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:41.304901 22178 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:41.766825 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:41.812268 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:41.812994 20802 raft_consensus.cc:606] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:41.816087 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:41.816691 20879 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:41.820894 22224 raft_consensus.cc:993] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:41.821318 20731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:41.821521 22170 raft_consensus.cc:1081] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:41.821890 20731 raft_consensus.cc:606] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:41.823334 20802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:21:41.823976 20802 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:41.824352 20802 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:41.824661 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:41.825184 20879 raft_consensus.cc:606] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:41.827119 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:21:41.828773 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:41.830010 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:41.839527 20802 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:41.842772 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:41.843427 20729 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:41.849499 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:41.850169 20879 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:41.850503 20879 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:41.851152 20729 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:41.851322 20879 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:41.854589 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:41.855518 21928 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:41.855996 21928 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:41.856889 21928 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:41.854592 20879 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:41.925964 22224 raft_consensus.cc:993] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:41.926484 22224 raft_consensus.cc:1081] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:21:41.928504 21430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:21:41.929129 21430 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:41.929600 21430 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.947638 21430 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:41.959569 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:41.960175 20731 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.960816 20879 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:41.948848 20802 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:41.980046 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:41.985019 21430 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:41.985010 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:41.992779 20879 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.993139 20879 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:41.997701 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:42.010752 22217 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:42.011345 22217 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:42.012332 22217 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:42.014349 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72640.2us
I20260709 12:21:42.018332 20879 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:42.019738 20879 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.035328 20879 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:42.044098 20632 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 3, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.051884 21928 raft_consensus.cc:993] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:42.057933 22241 raft_consensus.cc:1081] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:42.059978 20876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54488
I20260709 12:21:42.060493 20876 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:42.060829 20876 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.063005 20628 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:42.080264 20876 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:42.082525 20876 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:42.083171 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:42.083810 20802 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:42.084156 20802 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:42.084270 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:42.084894 21433 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.084890 20802 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:42.086443 20802 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.092446 21433 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:42.093967 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:42.094940 22200 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:42.095479 22200 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:42.096392 22200 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:42.109243 20802 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:42.136432 20630 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.204627 20879 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:42.211889 20802 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:21:42.212105 20850 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9c17d2fc98f481597ce7b97e8f98c43. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:42.213349 22200 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:42.213707 20731 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:42.216449 22200 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:42.232707 22170 raft_consensus.cc:2955] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.239897 20730 raft_consensus.cc:2955] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.261302 20802 raft_consensus.cc:2955] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.264158 20876 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:42.278558 22198 raft_consensus.cc:993] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:42.279202 22239 raft_consensus.cc:1081] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:42.281440 20879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:42.281966 20879 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:42.282322 20879 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.264479 20632 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.293380 20879 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:42.296788 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:42.297353 20730 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:42.297648 20730 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:42.298370 20730 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:42.299435 20730 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.306838 20730 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:42.309187 20804 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:42.313325 22145 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:42.313463 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:42.318948 22224 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:42.319510 22224 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:42.320500 22224 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:42.315709 20730 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:42.317883 20850 consensus_peers.cc:597] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ccae903e8484f8d8c3af847e2c4c3d4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:42.332407 22145 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:42.344451 20632 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 3, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:42.346134 20879 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:42.353308 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:42.354027 20802 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.368460 20802 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:42.370642 22224 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.381758 20734 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.382345 20804 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.400511 20632 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.402570 20879 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:42.415012 20734 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:42.416908 22241 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:42.428656 20878 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:21:42.430089 20850 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f15b5f1ac1444b63bad60a89fa9f8a29. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:42.430157 22241 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:42.436345 20804 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:42.440390 22224 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:42.442904 20734 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:42.450115 22145 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:42.477552 20804 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.475636 22200 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.507537 20734 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:42.520614 22242 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:42.521891 20729 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:42.524178 20731 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.532047 20628 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 3, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.542371 20878 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:42.544344 20879 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:42.547685 22218 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:42.549624 22239 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:42.579669 20701 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37c593b286274096a8c431979866be8a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:42.583220 20804 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:42.586367 22239 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:42.645754 22239 raft_consensus.cc:2955] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.652212 22274 ts_tablet_manager.cc:933] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:42.662279 20879 raft_consensus.cc:2955] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.682286 21430 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:42.659401 20804 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:42.686584 22200 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:42.688225 22267 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:42.697994 20628 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.768632 22274 tablet_copy_client.cc:323] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:42.770841 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet a9c17d2fc98f481597ce7b97e8f98c43 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:42.771402 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet a9c17d2fc98f481597ce7b97e8f98c43 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-a9c17d2fc98f481597ce7b97e8f98c43
I20260709 12:21:42.775555 20802 raft_consensus.cc:2955] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:42.782114 20894 tablet_copy_source_session.cc:215] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:42.792086 22274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c17d2fc98f481597ce7b97e8f98c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:42.794756 20805 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:42.798281 20730 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:42.799619 22243 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:42.802560 22224 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:42.828529 22278 ts_tablet_manager.cc:933] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:42.836709 22274 tablet_copy_client.cc:806] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:42.837410 22274 tablet_copy_client.cc:670] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:42.843084 22274 tablet_copy_client.cc:538] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:42.863140 22278 tablet_copy_client.cc:323] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:42.864985 20894 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 7ccae903e8484f8d8c3af847e2c4c3d4 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:42.865509 20894 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 7ccae903e8484f8d8c3af847e2c4c3d4 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-7ccae903e8484f8d8c3af847e2c4c3d4
I20260709 12:21:42.874843 20894 tablet_copy_source_session.cc:215] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:42.863161 22274 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:42.880805 22278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ccae903e8484f8d8c3af847e2c4c3d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:43.036209 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:43.226253 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "626209a224524080801f3a9ad7307fbe"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:43.227824 20802 raft_consensus.cc:606] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:43.332132 22284 ts_tablet_manager.cc:933] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:43.336498 22283 ts_tablet_manager.cc:933] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
W20260709 12:21:43.457859 21430 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.076s	sys 0.077s
W20260709 12:21:43.464484 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 117997us
W20260709 12:21:43.559755 21430 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.076s	sys 0.077s
I20260709 12:21:43.469728 22278 tablet_copy_client.cc:806] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:43.564100 22278 tablet_copy_client.cc:670] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
W20260709 12:21:43.523308 22274 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.121s	sys 0.310s
W20260709 12:21:43.564827 22274 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.121s	sys 0.311s
I20260709 12:21:43.588966 22274 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:43.590283 22274 tablet_bootstrap.cc:492] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:43.591202 22274 ts_tablet_manager.cc:1403] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.728s	user 0.141s	sys 0.316s
I20260709 12:21:43.609673 22274 raft_consensus.cc:359] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:43.610798 22274 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:44.230975 22298 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.231786 22298 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:44.321334 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.066s	sys 0.069s
W20260709 12:21:44.324894 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.067s	sys 0.069s
I20260709 12:21:44.326010 22298 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:44.326886 22283 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.008s	sys 0.015s
I20260709 12:21:44.327803 22298 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.351227 22298 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:44.353693 22298 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:44.327872 22310 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:44.355297 22310 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:44.358304 22310 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:44.335937 22311 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:44.348197 22290 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.359400 22311 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:44.359508 22290 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:44.361398 22283 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.008s	sys 0.016s
I20260709 12:21:44.361714 22290 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.361977 22283 tablet_copy_client.cc:323] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:21:44.358870 22293 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:44.363752 22293 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.365893 22293 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.367547 22311 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:44.371307 22242 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.372257 22242 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.374624 22242 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:44.375278 22242 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:21:44.376888 22284 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.011s	sys 0.005s
W20260709 12:21:44.380131 22284 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.011s	sys 0.005s
I20260709 12:21:44.380714 22284 tablet_copy_client.cc:323] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:21:44.381916 22242 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:43.613883 22274 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:44.397266 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:44.398456 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet 37c593b286274096a8c431979866be8a from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:21:44.398984 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet 37c593b286274096a8c431979866be8a from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-37c593b286274096a8c431979866be8a
I20260709 12:21:44.402872 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:44.397183 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:44.402699 22239 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.530282 22240 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.531037 22240 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:44.533730 22240 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:44.404073 22239 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.524088 22242 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:44.565553 22239 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.566924 22233 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:44.567605 22233 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.571025 22233 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.589355 22274 ts_tablet_manager.cc:1434] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.998s	user 0.024s	sys 0.014s
I20260709 12:21:44.602034 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:44.602288 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:44.604069 22239 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:44.622829 22239 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:44.624167 22274 ts_tablet_manager.cc:1467] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Tablet startup took 1032ms
I20260709 12:21:44.625278 22239 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.641356 22239 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:21:44.641741 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 239863us
I20260709 12:21:44.642196 22239 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.644934 22239 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.650745 20745 tablet_copy_source_session.cc:215] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:44.670022 22239 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:44.671006 22239 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.682183 22321 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:44.683064 22321 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:44.624732 22274 ts_tablet_manager.cc:1469] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Trace:
0709 12:21:42.605689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.606138 (+   449us) service_pool.cc:224] Handling call
0709 12:21:42.652702 (+ 46564us) ts_tablet_manager.cc:934] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
0709 12:21:42.790730 (+138028us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:21:42.801750 (+ 11020us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:42.801865 (+   115us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:21:42.822818 (+ 20953us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:21:42.823154 (+   336us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:42.823507 (+   353us) ts_tablet_manager.cc:989] Replica registered.
0709 12:21:42.823559 (+    52us) inbound_call.cc:177] Queueing success response
0709 12:21:42.862769 (+ 39210us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:21:43.591520 (+728751us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:21:43.603615 (+ 12095us) tablet_replica.cc:261] Starting instrumentation
0709 12:21:43.607602 (+  3987us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":254,"raft.queue_time_us":7740,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":91,"tablet-copy.queue_time_us":7317,"tablet-download-pool.queue_time_us":1112,"tablet-download-pool.run_cpu_time_us":1223,"tablet-download-pool.run_wall_time_us":3984,"thread_start_us":136101,"threads_started":3}
I20260709 12:21:44.816748 22233 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.817651 22290 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:44.817411 22233 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.813694 22293 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:44.818216 22290 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:44.818601 22293 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:44.820307 22290 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:44.820966 22293 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:44.811489 22239 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:44.822053 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 179470us
I20260709 12:21:44.825997 22319 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:44.826546 22319 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.821009 22233 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:44.832152 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 398859us
I20260709 12:21:44.835333 22319 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.838598 22283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37c593b286274096a8c431979866be8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:44.865164 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:44.866719 22323 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:44.866818 22321 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.872119 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:44.867416 22323 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:44.875675 22242 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.885041 22242 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.878181 22317 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:44.886488 22317 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.886790 22224 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.887545 22224 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.888659 22317 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:44.890254 22224 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:44.891403 22325 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:44.890690 22317 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:44.884495 22315 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:44.894243 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:44.905001 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:44.891940 22325 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:44.887063 22242 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:44.891582 22224 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:44.892112 22317 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.876511 22321 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:44.983621 22321 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:44.947458 22319 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:44.986824 22329 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:44.909291 22224 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.011972 22323 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.013466 22317 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.003662 22319 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.016111 22319 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:44.992800 22329 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.039891 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.049919 22328 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.050710 22328 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.053922 22328 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.055578 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.058066 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.043891 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.061318 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.063202 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.065037 22224 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.065565 22224 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.065944 22283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44466 (request call id 225) took 1930 ms. Trace:
I20260709 12:21:45.066013 22224 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.067732 22321 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.076810 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.077740 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.090912 22313 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.066473 22283 rpcz_store.cc:276] 0709 12:21:43.135688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.137482 (+  1794us) service_pool.cc:224] Handling call
0709 12:21:43.558928 (+421446us) ts_tablet_manager.cc:934] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
0709 12:21:44.836885 (+1277957us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:21:44.904980 (+ 68095us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:44.907566 (+  2586us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:21:45.065229 (+157663us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:21:45.065512 (+   283us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:45.065807 (+   295us) ts_tablet_manager.cc:989] Replica registered.
0709 12:21:45.065869 (+    62us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":230,"tablet-copy.queue_time_us":198313,"thread_start_us":999156,"threads_started":2}
I20260709 12:21:45.094583 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.097509 22335 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.095203 22224 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.108696 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.121407 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.095409 22315 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.106048 22283 tablet_copy_client.cc:806] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:45.126971 22314 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.098107 22335 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.118688 22328 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.141922 22288 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.091566 22313 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.130232 22224 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.140106 22283 tablet_copy_client.cc:670] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:45.153117 22330 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.153896 22330 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.156971 22330 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.132951 22325 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.144520 22233 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.142508 22288 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:45.164439 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.158560 22239 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.158821 22293 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.239363 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.239523 22309 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.144171 22328 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.163448 22290 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.208925 22330 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.275249 22335 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.276046 22335 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.276741 22335 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.159119 22224 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.231369 22325 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.130491 22315 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.281800 22200 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.284019 22312 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.284053 22200 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.285387 22315 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.286170 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.284965 22312 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.232970 22329 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.290735 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.243016 22317 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.289983 22200 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.213464 22233 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.144871 22314 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.279592 22240 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.297983 22233 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.299598 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.300807 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.302680 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.246146 22330 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.293051 22315 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.246459 22293 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.244007 22239 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:45.282473 22290 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.293782 22325 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.329428 22314 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.241048 22309 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.337659 22340 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.343586 22239 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.340029 22313 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.312837 22240 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.340870 22333 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.318166 22200 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.295334 22317 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.349852 22338 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.340437 22293 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.352716 22341 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.353547 22224 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.358608 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.383770 22337 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.330672 22315 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.403560 20808 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:45.359364 22340 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:45.410074 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc7c0 after lost signal to thread 19606
W20260709 12:21:45.411125 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4d20 after lost signal to thread 20667
W20260709 12:21:45.412492 20831 debug-util.cc:398] Leaking SignalData structure 0x7b080016dbc0 after lost signal to thread 20759
W20260709 12:21:45.413621 20831 debug-util.cc:398] Leaking SignalData structure 0x7b080016d360 after lost signal to thread 20834
I20260709 12:21:45.383796 22312 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.401484 22200 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.437695 22340 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.406404 22333 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.442533 22290 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.424010 22335 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.433467 22330 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.419724 22341 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.418205 22337 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.443418 22332 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.402825 22338 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.458035 22309 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.467291 22240 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.472539 22333 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.462342 22288 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.470229 22325 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.476615 22332 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.523897 22302 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.524590 22302 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:45.530704 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800194ba0 after lost signal to thread 20908
W20260709 12:21:45.531026 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800196080 after lost signal to thread 20909
W20260709 12:21:45.532142 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800325680 after lost signal to thread 21459
I20260709 12:21:45.516394 22317 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.531148 22315 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.541215 22337 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.542340 22200 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.523278 22332 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.530082 22341 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.549108 22328 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:45.554569 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800361320 after lost signal to thread 22281
I20260709 12:21:45.545238 22338 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.577346 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.578222 20733 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:45.580819 22302 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:45.586695 22306 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.587661 22306 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.600195 22306 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:45.618433 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 494338us
I20260709 12:21:45.619369 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:45.631409 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:45.632926 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:45.633461 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.639827 22336 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.640710 22336 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.617875 22242 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.636839 22317 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:45.651568 22242 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.653069 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.653959 22242 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.634178 22337 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:45.630648 22341 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.661981 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.630716 22328 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.637275 22315 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:45.631415 22327 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.667603 22336 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.655331 22295 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.670281 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:45.655380 22340 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.667512 22327 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:45.702149 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:45.697796 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.013s	sys 0.042s
I20260709 12:21:45.698113 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.698347 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.665488 22335 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.672237 22224 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:21:45.708987 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.014s	sys 0.042s
I20260709 12:21:45.712218 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.693161 22329 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.699043 22336 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.671941 22341 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.717696 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.714332 21431 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.718135 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.695698 20879 raft_consensus.cc:606] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:45.718987 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.719043 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.719839 20729 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:45.723498 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.729180 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.701489 22349 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.672782 22328 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.731653 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.715473 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.715065 22295 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.739774 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.741145 22326 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.741626 22326 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.743075 22300 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.743692 22300 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:45.746127 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.746418 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:45.748551 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 327103us
I20260709 12:21:45.749888 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.720810 22332 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.755479 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.718278 22306 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.723181 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.730996 22283 tablet_copy_client.cc:538] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:45.733876 22333 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.708909 22340 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.754480 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.756296 21433 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:45.712755 22335 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.748813 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:45.733878 22311 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:45.714543 22224 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.766346 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.780241 21431 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.762569 22349 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.784137 22349 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.766506 22306 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.719606 22329 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.790009 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.795099 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.799402 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.727413 22336 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.802024 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.802646 21431 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.768028 20727 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:45.778625 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.806617 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.804306 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.806533 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.807446 20808 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:45.800034 20804 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.805364 22200 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.809296 22200 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.791940 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:45.801616 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.810230 20732 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:45.761210 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.812278 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.810468 21432 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.807544 20802 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:45.813145 21430 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.768819 22342 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.828737 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.762796 22332 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.803020 22306 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.843626 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.765141 21427 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.794250 22311 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.844238 20893 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet f15b5f1ac1444b63bad60a89fa9f8a29 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:21:45.836673 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.846243 22200 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.806589 22338 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.844496 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.854444 20809 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:21:45.859618 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 115657us
I20260709 12:21:45.797816 22333 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.844471 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.863744 22309 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.864444 22309 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.844749 20894 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-a9c17d2fc98f481597ce7b97e8f98c43 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:45.860865 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.845213 22267 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.861969 20810 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:45.871316 21435 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:45.844870 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.867619 20894 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-a9c17d2fc98f481597ce7b97e8f98c43 on tablet a9c17d2fc98f481597ce7b97e8f98c43 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:45.888945 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.861416 22287 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:45.886570 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.867779 22344 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.890501 22352 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.878674 22353 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:45.890908 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.867669 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.861832 22300 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.859922 22338 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.907837 22344 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.867018 21426 raft_consensus.cc:1217] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:21:45.912052 22353 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:45.921471 22348 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:21:45.925261 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62156.8us
I20260709 12:21:45.893658 22341 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.901036 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:45.867430 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.927011 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.927408 20879 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:45.928251 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.851922 22342 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.931981 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:45.919487 21429 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:45.933058 22342 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.933331 22309 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:45.933974 20699 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.576s (SENT) [suppressed 3 similar messages]
I20260709 12:21:45.912861 22287 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.935473 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:45.932137 22336 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.939975 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:45.921950 20893 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet f15b5f1ac1444b63bad60a89fa9f8a29 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-f15b5f1ac1444b63bad60a89fa9f8a29
I20260709 12:21:45.944062 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.946791 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:45.878568 22267 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.912581 22343 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:21:45.951108 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60058.4us
I20260709 12:21:45.939088 22287 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:45.936683 22309 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:45.952791 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:45.924152 22327 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:45.942544 22328 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.943112 22337 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:45.938645 22348 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:45.926672 22352 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:45.955317 22333 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.110697 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.196638 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.046195 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.207963 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.086565 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:46.208766 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.025163 22332 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.102589 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.070917 22278 tablet_copy_client.cc:538] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:46.209424 22350 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.209972 22350 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:46.210052 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.210438 22350 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.210702 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:46.210901 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.211630 20875 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:46.212518 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:45.996018 22240 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:21:46.103327 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 107975us
I20260709 12:21:46.195207 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:46.040576 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.063592 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.057639 22300 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.104300 22349 raft_consensus.cc:993] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:46.050278 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.222345 22341 raft_consensus.cc:1081] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:21:45.955669 22288 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:46.194511 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.059427 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.223726 20876 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:46.026885 22353 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:46.067296 22295 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.217834 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:46.063153 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:46.031692 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:46.226006 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 162917us
I20260709 12:21:46.214605 22309 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.226816 22309 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:46.023905 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.074841 22267 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:46.226944 22332 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.227708 22332 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.228128 22332 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.227842 22337 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.230491 22317 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:46.108081 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:46.045012 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:45.967713 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:46.137099 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.107439 22326 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.105782 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.104348 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.126919 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.236944 20810 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:46.219811 22352 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.196462 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.013321 22329 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:46.129638 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.101940 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.123322 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.024308 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.071111 22290 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.071415 22293 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:45.968439 22344 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.219679 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.248078 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.249395 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.024569 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.220570 22240 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.049871 22311 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.221634 22327 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.221540 22300 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:46.223210 22288 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.224596 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.262482 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.029233 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.230974 22317 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.093041 22283 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:46.277629 22317 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:46.277961 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.278630 20876 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:46.217664 22348 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.281034 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.241271 22337 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.285645 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.234750 22224 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.180421 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.232075 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:21:46.234177 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 126191us
W20260709 12:21:46.239831 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 272201us
I20260709 12:21:46.294440 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.283506 22288 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:46.045688 22338 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:45.955540 22335 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.299044 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.230222 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.237615 22267 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.025027 22340 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.245146 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.246428 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.012179 22343 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.258033 22327 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.260401 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.157440 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:46.255998 22293 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.271456 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.272158 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.273324 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.273937 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.241808 20880 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:46.260838 22233 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:46.276932 22300 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.182818 20893 tablet_copy_source_session.cc:215] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:21:46.234181 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 153132us
W20260709 12:21:46.288375 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 354515us
I20260709 12:21:46.367971 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.289234 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.290144 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.291116 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.256850 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:46.217725 22350 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.293706 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.295926 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.296096 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:46.296119 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64124.7us
I20260709 12:21:46.295303 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:46.300468 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.300396 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.301226 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.301425 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.192479 22287 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:46.304504 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.303805 22267 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:46.377420 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.257803 22240 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.305187 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.385324 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:46.387836 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 131090us
I20260709 12:21:46.389406 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:46.359786 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:46.310776 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.361728 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:46.361603 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.360080 20883 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:46.358279 22327 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.300190 22337 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.363289 22233 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.257884 22290 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:46.370333 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.370514 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.373564 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.423434 20802 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:46.374737 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.280647 22309 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:46.392792 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.365167 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:46.407719 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48024.6us
I20260709 12:21:46.428076 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.410519 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.411216 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:46.416779 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 131876us
W20260709 12:21:46.420821 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49553us
I20260709 12:21:46.374614 22347 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.415488 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.394254 22305 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.428838 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:46.434777 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.442479 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.386152 22267 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.435691 22200 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.446005 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.447379 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.421564 22233 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:46.450874 20729 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:21:46.458846 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.418895 22349 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.460516 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.459448 22200 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.458868 20729 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:46.465452 20729 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.448586 22353 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:46.460511 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.461313 22293 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.461424 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:46.470088 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 80765.3us
I20260709 12:21:46.471822 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:46.473209 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 177991us
I20260709 12:21:46.474524 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.449268 22342 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.464411 22351 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.462883 22323 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.467736 22340 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.493673 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.511957 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:46.520511 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73237.3us
I20260709 12:21:46.521631 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.516359 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:46.476337 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103897us
I20260709 12:21:46.590447 22351 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:46.591755 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70211.6us
I20260709 12:21:46.592656 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:46.591475 20729 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.596438 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.495810 22358 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:46.612138 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:21:46.612957 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 185829us
I20260709 12:21:46.614218 20729 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:46.615154 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 169252us
W20260709 12:21:46.617997 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 184442us
I20260709 12:21:46.622063 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:46.624817 20731 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:46.620105 22309 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.624485 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.649541 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.650538 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.652729 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.652843 22346 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.652912 22364 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:46.654435 22364 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.613283 22355 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.649250 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.660527 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.616117 22342 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:46.615821 22358 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.666415 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:46.671233 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:46.627455 22347 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:46.675050 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50433.8us
I20260709 12:21:46.653738 22346 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:46.689306 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.690881 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:46.691027 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:46.692652 20877 raft_consensus.cc:606] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:46.657538 22340 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.702666 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.705291 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:46.706367 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:46.678226 22335 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.708003 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:46.669914 22342 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:46.684558 22350 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.666188 22324 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:46.713035 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.675465 22293 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:46.714085 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:46.684851 22358 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:46.720779 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.721928 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.714849 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:46.716848 22361 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.717108 22278 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:46.695137 22328 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:46.729008 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.718560 22342 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:46.691303 22362 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.727866 22293 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.725231 22290 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.724623 22348 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.730031 22309 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:46.727732 22328 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.745693 22364 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.746567 22293 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.719552 22359 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.733945 22361 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:46.740598 22350 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.741067 22337 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.726549 22324 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.739804 22305 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:46.798421 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:46.799233 20881 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:46.747958 22348 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.762540 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:46.799598 20881 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:46.764917 22288 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:46.752703 22328 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:46.766729 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.767469 22375 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.776296 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.767675 22373 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.727236 22332 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.865631 22365 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:46.866220 22365 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:46.867578 22332 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.868431 22365 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:46.869858 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 192279us
W20260709 12:21:46.871687 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 165358us
I20260709 12:21:46.874210 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:21:46.794839 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80818us
I20260709 12:21:46.748102 22323 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:46.802443 22305 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:46.764788 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.748106 22309 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:46.748387 22326 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:46.809294 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:46.825050 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:46.892341 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:46.787446 22361 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.881012 20881 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:46.894060 20881 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.823374 22364 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:46.876055 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:46.812398 22348 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.880510 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:46.880384 22323 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:46.883373 22373 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:46.873255 22374 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:46.875385 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.882969 22290 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:21:46.910341 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 169943us
I20260709 12:21:46.914372 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:46.943934 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:46.944713 20804 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:46.812052 22287 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:46.915499 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:46.919848 22373 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.930537 22327 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:46.910002 20806 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:46.932377 22324 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:46.813104 22240 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:46.917567 22323 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:46.893338 22309 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:46.918828 22375 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.945437 22290 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:21:46.973358 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98041.9us
W20260709 12:21:46.976002 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60578.1us
I20260709 12:21:46.917160 22311 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:46.982592 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 260707us
I20260709 12:21:47.009562 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:46.973713 22337 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:46.977280 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.911504 22326 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:47.015560 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:46.985378 22368 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.003506 20734 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.003325 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:46.998174 22375 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:46.918542 22364 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:46.976269 22317 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.668s	user 0.006s	sys 0.005s
I20260709 12:21:46.978420 22300 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:46.973655 22351 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:21:47.040306 22366 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:46.980060 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.041023 22288 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:47.041327 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61373.5us
I20260709 12:21:47.041654 22335 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.040119 22317 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.042105 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.043210 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.011488 22347 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:47.013388 20810 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:46.938882 22358 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:47.014761 22337 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:47.001121 22290 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.001636 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:47.022338 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.050693 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:47.031150 22305 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.032312 22284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f15b5f1ac1444b63bad60a89fa9f8a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.033007 22327 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.033764 22326 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.019385 22340 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.017128 22370 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.039247 22364 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.040468 22351 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:47.044909 20881 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:47.045466 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:47.008409 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.016687 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.026415 22375 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:47.054759 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 53228.5us
I20260709 12:21:47.057387 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:47.059391 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.047566 22344 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:47.048556 22335 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.048348 22347 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:47.050556 22337 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:47.088030 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79717.8us
W20260709 12:21:47.088691 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72061us
I20260709 12:21:47.060318 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:47.094349 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.058204 22300 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.059068 22368 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:47.059894 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:47.105141 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.089107 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:47.106325 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.084432 22351 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:47.092471 22329 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.092379 22288 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:47.096674 22347 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.114121 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.096000 22364 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.116955 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.115027 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.117657 22344 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.128646 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.129096 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.130505 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:47.107424 22240 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.133088 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.133714 20734 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:47.118742 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.113107 22305 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:47.120276 22379 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.119822 22351 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:47.108378 22368 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:47.137312 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.138501 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.138585 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.139672 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.150523 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.140785 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.113057 22335 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.115391 22374 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:47.143170 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.131832 22332 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.171653 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:21:47.142624 22267 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.678s	user 0.006s	sys 0.006s
I20260709 12:21:47.145452 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.141950 22293 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.139606 22352 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:47.144450 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.141557 22305 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:47.177903 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:47.178341 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.145720 22329 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:21:47.179925 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.146234 22368 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.172539 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.169409 22330 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:47.199690 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.175696 22344 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.196265 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.174957 22374 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:47.197665 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.198137 20734 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:47.179165 22370 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.200481 20731 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:47.196455 22305 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:47.201594 22233 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.003s
I20260709 12:21:47.184160 22267 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.199533 22358 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.201293 22379 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:47.203537 22374 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.211472 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.212327 22382 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:47.203469 22330 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.210918 22346 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:21:47.221372 22360 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:47.223333 22357 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.205024 22293 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:47.225773 22379 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:47.203351 22340 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.211076 22233 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:47.240043 20875 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.239688 22290 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.239749 22330 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:47.241845 22293 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:47.232084 22382 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.248121 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.250543 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:47.251682 20728 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:47.254992 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.256718 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.258606 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.259828 21431 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.261430 20729 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:47.266597 20730 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.272089 22346 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:47.272625 22343 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:47.273028 22346 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:47.273375 22346 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:47.273846 22233 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.275094 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:47.276962 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:47.280450 22373 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.282578 22373 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:47.260640 22350 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.260095 22330 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.288105 20731 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.242695 22360 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.274196 22346 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:47.268414 22381 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.296550 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.259737 22348 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.291823 22361 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.316828 22233 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:21:47.305531 22378 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:47.322465 22290 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:47.304368 22309 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:47.381573 22233 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:47.381964 22233 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:47.382387 20733 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:47.382772 22233 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.384122 22233 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.384538 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.385668 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:47.386636 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.301443 22382 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.299748 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:47.388379 20727 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.304554 22379 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.372545 20728 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:47.391034 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.357393 22337 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.329934 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.305974 22341 raft_consensus.cc:993] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:47.365278 22348 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:47.379786 22342 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.000s
I20260709 12:21:47.409835 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.388667 22361 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:47.368896 20801 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.308041 22283 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:47.412287 21432 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.402762 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:47.402412 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134736us
I20260709 12:21:47.307152 22347 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.407244 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:47.400385 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:47.415657 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:47.371701 22267 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:47.321302 22293 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.380404 22378 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.412985 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.459524 20733 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:47.579885 22372 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:47.459474 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:47.587301 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159384us
I20260709 12:21:47.590626 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.605679 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:47.410737 22369 raft_consensus.cc:1081] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:21:47.624524 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.633857 22295 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:47.645184 20775 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.377131 22323 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:47.654834 22332 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.434911 22287 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:47.586485 22346 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.663806 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.655426 22341 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:47.652110 22300 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:47.693259 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.412483 22342 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.695261 22371 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.695616 22366 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.696113 20803 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:47.694890 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.698060 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.678264 22284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44462 (request call id 427) took 4774 ms. Trace:
I20260709 12:21:47.680794 22337 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.583115 22348 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:47.649818 22328 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.000s
I20260709 12:21:47.697009 22324 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.682184 22287 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:47.656452 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:47.705364 22371 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.733353 22366 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:21:47.739871 22328 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:47.746397 22375 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.642s	user 0.002s	sys 0.007s
I20260709 12:21:47.746572 22328 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.741163 20883 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:47.733806 22283 tablet_bootstrap.cc:492] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:47.746973 22327 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:47.747382 21434 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:47.694380 22369 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:47.762231 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:47.762547 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.768785 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.744233 22287 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:47.755868 22368 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.551s	user 0.006s	sys 0.003s
I20260709 12:21:47.746816 22375 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.774556 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:47.779469 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:47.780318 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.746425 22323 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.783313 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:47.784317 20733 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:47.786499 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:47.754082 22324 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:47.760314 22369 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:47.796027 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 232) took 1108 ms. Trace:
I20260709 12:21:47.800347 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.804204 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:47.805758 22351 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:47.806273 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.808372 22342 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:47.808949 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:21:47.813053 22374 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.001s
I20260709 12:21:47.815881 22317 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.795413 22347 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.823829 20883 rpcz_store.cc:276] 0709 12:21:46.687279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.691632 (+  4353us) service_pool.cc:224] Handling call
0709 12:21:46.699907 (+  8275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.753896 (+1053989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.795792 (+ 41896us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:47.795949 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2394181376}
W20260709 12:21:47.851486 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51215.3us
I20260709 12:21:47.855674 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.862056 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.864660 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.867915 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.877022 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.815080 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1391) took 1041 ms. Trace:
I20260709 12:21:47.877810 20803 rpcz_store.cc:276] 0709 12:21:46.773407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.787814 (+ 14407us) service_pool.cc:224] Handling call
0709 12:21:46.937455 (+149641us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:21:46.937665 (+   210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.696569 (+758904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.814849 (+118280us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b54002b6bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:47.815017 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1740541952}
I20260709 12:21:47.876158 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.734946 22284 rpcz_store.cc:276] 0709 12:21:42.903865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.904004 (+   139us) service_pool.cc:224] Handling call
0709 12:21:43.551015 (+647011us) ts_tablet_manager.cc:934] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
0709 12:21:47.015916 (+3464901us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:21:47.262345 (+246429us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:47.270263 (+  7918us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:21:47.677480 (+407217us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:21:47.677757 (+   277us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:47.678068 (+   311us) ts_tablet_manager.cc:989] Replica registered.
0709 12:21:47.678145 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":244,"tablet-copy.queue_time_us":427503,"thread_start_us":1484633,"threads_started":2}
I20260709 12:21:47.880147 22377 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.884763 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.885780 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.887944 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.888314 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.888666 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.888108 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:47.889714 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.889815 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:47.869411 22305 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.642s	user 0.005s	sys 0.005s
I20260709 12:21:47.771715 22327 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.891613 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:47.898700 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.848596 22309 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:47.862051 22330 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.894395 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.789007 22283 ts_tablet_manager.cc:1403] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 1.696s	user 0.039s	sys 0.030s
I20260709 12:21:47.871477 22323 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:47.907011 22278 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:21:47.899799 22284 tablet_copy_client.cc:806] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:47.907294 22366 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.908593 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:47.951869 22284 tablet_copy_client.cc:670] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:47.954342 21427 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.909403 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:47.959764 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60907.5us
I20260709 12:21:47.849102 22324 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.916602 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:47.906072 22350 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:47.930599 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:47.850841 22374 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:47.945482 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.891867 22377 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.947993 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:47.894176 22380 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.892154 22365 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.788836 22368 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:47.970628 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54100us
I20260709 12:21:47.977664 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.990160 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:47.980618 22373 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.902361 22305 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:47.995399 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:47.997136 20732 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:47.976323 22233 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.014796 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:48.022274 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44215.8us
I20260709 12:21:48.026753 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:48.037819 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76402.2us
I20260709 12:21:48.039610 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:48.040694 20731 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:48.043874 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.044049 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.044541 20882 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:47.997855 22379 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.049258 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.054207 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.058533 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:48.057729 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47514.3us
W20260709 12:21:48.053032 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 352239us
I20260709 12:21:48.000432 22387 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:48.061069 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.048393 22373 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:48.066604 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.069419 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.070951 22295 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:48.071871 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:48.072400 22295 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.072788 22295 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.073632 22295 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.074158 22332 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.074837 22295 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.073827 22283 raft_consensus.cc:359] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.045025 22336 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:21:48.085156 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.117969 20810 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:48.117734 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57954.7us
I20260709 12:21:48.088332 22375 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.092164 22351 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.092298 22374 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:48.093806 22324 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:48.094362 22344 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.094463 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:48.123591 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.097046 22360 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:48.118012 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.097405 22278 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:48.064213 22387 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:48.133675 22387 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:48.134002 22368 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:48.082357 22394 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:21:48.080673 22396 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.106402 20882 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:48.108536 22305 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.109112 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 459) took 1057 ms. Trace:
I20260709 12:21:48.066792 22366 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.004467 22369 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.112310 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.072444 21429 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.085485 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.119557 22287 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.058547 22349 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:21:48.120165 22336 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:48.122256 22371 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:48.117419 22283 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:48.119593 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.132723 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:48.130905 22344 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.142373 21434 rpcz_store.cc:276] 0709 12:21:47.051268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.052500 (+  1232us) service_pool.cc:224] Handling call
0709 12:21:47.052766 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.756001 (+703235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:48.108882 (+352881us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:48.109033 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597028096}
I20260709 12:21:48.139140 20810 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:21:48.140432 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 460) took 1081 ms. Trace:
I20260709 12:21:48.148119 20804 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:48.140518 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.139833 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:48.156905 21432 rpcz_store.cc:276] 0709 12:21:47.058613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.058732 (+   119us) service_pool.cc:224] Handling call
0709 12:21:47.084114 (+ 25382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.412691 (+328577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:48.140217 (+727526us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54007201d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:48.140379 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754276096}
I20260709 12:21:48.146972 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.146997 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:48.139482 21435 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.155759 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.133220 22401 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.146972 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:48.150195 22336 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.159837 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.123241 22300 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.167945 20882 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:48.161697 21433 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:21:48.179383 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 125241us
I20260709 12:21:48.143051 22369 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.183126 22278 tablet_bootstrap.cc:492] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:48.183899 22278 ts_tablet_manager.cc:1403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 1.492s	user 0.035s	sys 0.014s
I20260709 12:21:48.145228 22330 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:48.185729 22330 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:48.175882 22387 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.145188 22287 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.152796 22371 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.190135 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.168480 22394 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.191406 22394 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.183146 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.192864 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.193392 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1660) took 1459 ms. Trace:
I20260709 12:21:48.176960 22397 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:21:48.193771 20729 rpcz_store.cc:276] 0709 12:21:46.728619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.728755 (+   136us) service_pool.cc:224] Handling call
0709 12:21:46.729058 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.263156 (+534098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:48.187866 (+924710us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54002279d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:48.188020 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223914368}
I20260709 12:21:48.200119 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:48.202136 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:48.204159 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.205654 22361 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:48.205952 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:48.145758 22342 consensus_queue.cc:1175] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 12:21:48.209175 21433 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:48.209982 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.212905 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.184600 22309 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:21:48.213207 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.195809 22351 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.221168 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.221168 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.186645 22330 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.179893 22373 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:21:48.177129 22283 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.223816 22384 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:21:48.223868 20883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70cdcdfe95804895979397c927fae46f"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:48.231730 22389 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.232002 22363 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:48.234176 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:48.236102 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.205672 22400 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:21:48.225998 20877 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:48.212035 22396 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:21:48.238782 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.212155 22361 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.192272 22394 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.215626 22371 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.205698 22336 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.275126 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:48.278642 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 200380us
W20260709 12:21:48.292732 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 229325us
I20260709 12:21:48.248785 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.245551 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:21:48.294282 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45604.4us
I20260709 12:21:48.295876 22283 ts_tablet_manager.cc:1434] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.366s	user 0.006s	sys 0.002s
I20260709 12:21:48.315725 22388 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:48.242487 22305 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:21:48.243031 22284 tablet_copy_client.cc:538] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:21:48.294697 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49233.3us
I20260709 12:21:48.297410 22309 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:48.227726 22293 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.252933 22330 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.300596 22393 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:48.300697 22407 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:48.301074 22403 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:48.301367 22336 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.243402 22387 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.301839 22278 raft_consensus.cc:359] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.302829 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:48.299825 22360 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.228039 22335 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:21:48.329173 22278 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:48.332170 22341 raft_consensus.cc:1064] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:48.332736 22278 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.244000 22373 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.334575 22378 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:48.334676 22373 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:48.335853 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:48.337124 22327 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:48.274251 22402 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.308799 22323 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.335621 22373 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.244055 22384 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.309798 22371 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:48.348496 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1281) took 1208 ms (client timeout 1542 ms). Trace:
I20260709 12:21:48.251992 22363 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:48.237579 22344 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.310820 22394 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.311611 21429 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:48.272782 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.316318 22388 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.321729 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.323107 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-37c593b286274096a8c431979866be8a received from {username='slave'} at 127.0.0.1:33906
I20260709 12:21:48.328691 22403 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:21:48.357090 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 84396.2us
I20260709 12:21:48.357425 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.356410 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:48.330542 20729 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.323766 22407 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.328831 22393 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:48.325023 22309 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:48.338050 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:21:48.308494 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69824.7us
I20260709 12:21:48.367585 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.366370 22401 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:48.368752 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.369105 21433 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:48.346664 22335 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:48.356449 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-37c593b286274096a8c431979866be8a on tablet 37c593b286274096a8c431979866be8a with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.233072 22350 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:48.351326 20876 rpcz_store.cc:269] 0709 12:21:47.136947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.137082 (+   135us) service_pool.cc:224] Handling call
0709 12:21:48.340327 (+1203245us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b540019cdd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:48.345234 (+  4907us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147256192}
I20260709 12:21:48.348201 22278 ts_tablet_manager.cc:1434] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.164s	user 0.005s	sys 0.003s
W20260709 12:21:48.330209 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.373318 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:21:48.373639 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.350364 22384 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.359153 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:48.364078 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.380822 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.372351 22344 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.372254 22233 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:21:48.362700 22402 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.378983 20734 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.371878 22393 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.372566 22348 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.379571 22329 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:21:48.385782 22351 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:48.372551 22335 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.365536 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 246) took 1108 ms. Trace:
I20260709 12:21:48.392338 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.361420 22293 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.373060 22365 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:48.393832 22407 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:48.394042 20803 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:48.376752 22327 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.396986 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1395) took 1195 ms. Trace:
I20260709 12:21:48.390990 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.376194 22305 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.386857 22387 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:21:48.404664 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.390285 22233 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.406411 20732 rpcz_store.cc:276] 0709 12:21:47.201980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.202132 (+   152us) service_pool.cc:224] Handling call
0709 12:21:47.207929 (+  5797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:48.001989 (+794060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:48.396772 (+394783us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54003acfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:48.396933 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814451968}
W20260709 12:21:48.463235 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.392697 22351 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.409548 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:48.397815 21429 rpcz_store.cc:276] 0709 12:21:47.254088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.259904 (+  5816us) service_pool.cc:224] Handling call
0709 12:21:48.071703 (+811799us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:21:48.359275 (+287572us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21596928}
I20260709 12:21:48.467830 22406 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:48.472002 20808 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.495621 22388 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.496429 20806 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:48.481856 20893 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-7ccae903e8484f8d8c3af847e2c4c3d4 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:48.503501 20893 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-7ccae903e8484f8d8c3af847e2c4c3d4 on tablet 7ccae903e8484f8d8c3af847e2c4c3d4 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.407918 22384 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.392509 22309 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.412919 22363 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.512005 22363 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:48.517017 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.518007 22373 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.519589 22380 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:48.520200 22380 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.393225 22341 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.462285 22403 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.545715 22351 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.546988 22351 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.548203 22384 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.553359 20727 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:48.557019 22317 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:48.557583 22317 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.557029 22388 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:48.558800 22388 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.560204 22317 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:48.560834 22388 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.487746 22335 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.522464 22380 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:48.577991 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:48.580907 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:48.582093 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:48.587929 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:48.588819 20805 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:48.589828 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.591756 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.567148 22335 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.599290 22317 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.600915 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.602576 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:48.604216 20810 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.600446 22382 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.608898 20805 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:48.617503 22349 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.618093 22349 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.618317 22284 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:21:48.622816 20804 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:21:48.623410 20804 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:48.623692 20802 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:48.623759 20804 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.631073 20804 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.633106 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.637734 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.639007 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.639740 20804 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.641079 22384 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.639809 22388 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.645895 20883 raft_consensus.cc:606] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:21:48.646296 22287 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.646747 22287 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.646436 22349 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.647806 22349 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.647606 22287 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.648320 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.649036 22287 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.650563 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:48.651434 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:48.652045 20879 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:48.653533 22233 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:48.641451 22340 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.646342 20883 raft_consensus.cc:614] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:48.657992 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.663998 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:48.667768 22346 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:48.668174 22346 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.669034 22346 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.670018 22346 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:48.670469 22392 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.672420 20807 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.672946 22233 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:48.673952 22233 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.661506 20632 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:48.674278 22233 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:48.679541 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:48.688844 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.689465 20882 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:48.689759 20882 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.705765 20729 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:48.705739 22326 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:48.707211 22371 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.707569 22371 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.708426 22371 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.709633 22371 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.713279 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:48.678994 22387 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:48.714869 22387 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.716572 21432 raft_consensus.cc:1217] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:48.717657 22387 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:48.719298 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.719722 22295 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:48.720319 22295 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.690574 20882 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.727026 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:48.727602 20882 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.732975 20804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "986267de598f431fb25840d9d741436d"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:48.733628 20804 raft_consensus.cc:606] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:48.739032 20808 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:48.739148 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:48.702556 22388 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:48.741245 22388 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:48.741757 22388 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:48.747422 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82832.5us
I20260709 12:21:48.747671 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:48.753717 20876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:48970
I20260709 12:21:48.754354 20876 raft_consensus.cc:606] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.754684 20876 raft_consensus.cc:614] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:48.758409 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.758643 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:48.759280 20734 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.760345 22340 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:48.761014 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:48.762768 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:48.763540 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:48.772403 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.772692 21429 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.771548 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:48.721455 22295 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.791437 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:48.781144 20876 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.789047 22293 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:48.793224 20882 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:48.803367 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:48.807372 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:48.675026 22233 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.810191 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:21:48.812703 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 112259us
I20260709 12:21:48.789664 22311 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.795049 22393 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.821676 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:48.854663 20804 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:48.816521 22340 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:48.827178 20727 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:48.815450 22335 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:21:48.830992 22366 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:48.855942 22378 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.862363 22342 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 12:21:48.887243 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 93625.8us
I20260709 12:21:48.880028 22293 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:48.880374 22366 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.892874 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.893615 20877 raft_consensus.cc:2376] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4. Re-sending same reply.
I20260709 12:21:48.898090 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:48.892167 22287 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:21:48.902952 22421 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:48.914418 22293 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.920480 22346 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:48.920949 22346 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:48.921847 22346 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.930331 22366 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:48.948807 21432 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:48.944842 22346 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:48.943615 22421 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:48.957906 22421 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:48.955937 22342 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:21:48.972429 22388 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:48.985213 22295 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.987676 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:48.990304 22388 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:48.997917 20729 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:49.001559 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:49.002549 20808 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:21:49.002810 22423 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:21:49.007666 22233 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.020648 22311 raft_consensus.cc:993] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:49.021176 22311 raft_consensus.cc:1081] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:49.026921 20729 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:49.033577 20877 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "986267de598f431fb25840d9d741436d"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54488
I20260709 12:21:49.034312 20877 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:49.034720 20877 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.043503 22336 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:49.044775 22336 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:49.045173 22336 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:49.047784 22375 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:49.046105 22336 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:49.054803 22422 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.159s	user 0.000s	sys 0.004s
I20260709 12:21:49.055420 22361 raft_consensus.cc:1064] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:49.055735 20883 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:49.056624 20804 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:49.058182 22348 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:49.059239 20882 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:49.060921 21433 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Advancing to term 2
I20260709 12:21:49.058051 22361 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.062451 22373 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:49.067374 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:49.078320 21433 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:49.067785 20877 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.069185 20633 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 3, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.080680 22336 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.081341 22293 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.081873 22293 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.084116 22293 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:49.087363 22363 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:49.088474 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:49.089079 21433 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:49.093578 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:49.094108 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:49.094692 20882 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:49.105337 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:49.106004 20732 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.114331 20732 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:49.116119 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:49.117249 22346 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.117930 22346 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:49.119117 22346 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.103761 22293 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:49.128014 22293 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.128391 22293 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.129076 20877 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:49.103366 22392 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:21:49.099537 22375 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.152367 20808 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:49.159610 22233 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:49.156420 22375 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:49.166972 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:49.183861 20807 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:49.184258 20807 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:49.185017 20807 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.186499 20807 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.177085 20884 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:49.202353 22375 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.203589 22361 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:49.213704 20807 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:49.214190 22293 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.228020 20884 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:49.252648 21433 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:49.271836 20806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:49.277038 20806 raft_consensus.cc:606] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:21:49.314591 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:49.318298 22293 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:49.267310 22233 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:21:49.325459 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:49.326795 21433 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.343760 21433 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:49.345461 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:49.351064 22387 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:49.355597 20807 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:49.343138 22366 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:49.357455 22366 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.380514 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:49.384713 20802 raft_consensus.cc:606] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:21:49.385892 20628 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:49.386055 22309 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:21:49.386324 22401 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:49.400825 22346 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:49.396536 22375 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:49.405483 22366 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:49.407027 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:49.416421 21432 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:49.417834 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:49.415895 22384 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:49.424790 22366 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:49.442062 21432 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.421900 22401 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:49.453709 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:49.437028 22349 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.463354 20883 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:49.440845 22366 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:49.420966 22346 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.423951 22375 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:49.504966 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 128393us
I20260709 12:21:49.513756 20875 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:49.446357 22384 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.488358 22366 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.532949 22375 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:49.497109 22361 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:49.483789 22336 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:49.491302 22349 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.579754 20727 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:49.606101 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:49.574779 22336 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.581105 22361 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.571022 22401 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:49.676651 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 218122us
I20260709 12:21:49.682156 21432 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:49.610385 22397 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:21:49.723325 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:49.724263 20808 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:49.725616 22330 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.726292 22330 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.735776 22406 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.736495 22406 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.739454 22406 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:49.741920 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:49.749615 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:49.750602 21432 raft_consensus.cc:2368] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
W20260709 12:21:49.754912 22441 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 12:21:49.703150 22421 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.712249 22346 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:49.781842 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68749.6us
W20260709 12:21:49.786540 20776 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.758440 22393 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.802881 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:49.803772 20883 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:49.805656 20883 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:49.791340 22384 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:49.810108 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:49.812039 20802 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:49.792162 22393 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:49.784077 22421 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.815248 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:49.816311 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53950us
I20260709 12:21:49.816556 20729 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:49.819574 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:49.819690 22361 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:49.820432 20727 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:49.807844 22309 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.828126 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:49.832587 20802 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:49.813246 22349 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:49.822765 22393 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.838474 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:49.839152 20802 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:49.835793 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:49.814354 22452 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.842496 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:49.845152 20727 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
W20260709 12:21:49.835433 22445 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:21:49.849786 20727 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.847994 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:49.852715 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:49.848272 20880 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:49.853775 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:49.847411 22336 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:49.848393 22452 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.880698 22421 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:49.882169 22452 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:49.883757 22366 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.885938 22366 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:49.886586 22467 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.887094 22467 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.835064 22309 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.889204 22467 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:49.889298 22309 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:49.901808 22393 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:49.905956 20850 leader_election.cc:400] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:21:49.906256 20850 leader_election.cc:403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:21:49.889484 22461 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:49.909757 22461 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.910132 22461 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.856354 22457 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:49.889791 22467 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:49.919762 22453 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.923535 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:49.928169 20805 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:49.929261 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:49.933104 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:49.933571 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:49.933847 20876 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:49.914902 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:49.907164 22452 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:49.903441 22336 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:21:49.940801 22311 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:49.946918 22311 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.947386 22311 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:49.931834 22330 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:49.917567 22467 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:49.938164 22464 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.941093 22452 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:49.948879 20807 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:21:49.980126 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 126429us
I20260709 12:21:49.984416 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:49.985205 22475 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.985253 20806 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:49.985936 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:49.986518 20729 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:49.986636 20809 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.985625 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:49.987798 20727 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:49.988871 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:49.989874 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:49.993989 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:49.995260 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:49.953889 22467 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.997757 20809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33466
I20260709 12:21:50.001132 20809 raft_consensus.cc:606] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:21:49.981966 22452 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:49.942657 22449 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
W20260709 12:21:50.007350 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62018.8us
I20260709 12:21:49.985730 22475 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:49.999238 22447 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.999356 22346 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:50.003508 22421 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.931756 22457 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:50.040030 22473 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:21:50.009527 22295 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.003661 22462 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:49.975953 22464 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.104874 22457 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:50.105592 20884 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:21:50.106844 22363 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.024129 22447 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:50.107482 22447 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:50.107389 22363 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.026362 22378 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.024564 22346 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.016217 22393 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:50.085870 22441 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:21:50.084134 22327 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.093680 22462 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.148267 22449 raft_consensus.cc:993] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:50.148862 22377 raft_consensus.cc:1081] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:50.152842 20883 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54488
I20260709 12:21:50.153564 20883 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:50.153978 20883 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:50.106590 22461 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.113612 22451 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.133141 22374 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.130131 22346 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:50.100356 22349 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:21:50.086310 22421 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.100919 22295 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.151755 22462 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:50.154765 21429 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:50.135025 22378 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.229269 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:50.154557 22327 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.236528 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:50.239077 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:50.239899 20876 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:50.252064 22457 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.252534 22360 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:50.252036 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:50.253171 22360 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.191535 22453 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:21:50.257485 20727 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:50.225185 22384 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.179075 22374 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.258450 20875 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:50.280639 22421 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.282781 22327 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:50.283694 22349 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.284350 22349 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.288337 22295 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:50.094106 22233 raft_consensus.cc:2955] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:50.258131 22464 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:50.290517 22464 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:50.290958 22464 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.293341 22464 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:50.209545 22348 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:21:50.171201 22451 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.257110 22476 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:50.260568 22406 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:50.252652 22457 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.305564 22450 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.275983 22377 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:21:50.334276 22450 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.336236 22475 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:50.336539 22450 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.340070 22378 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.340819 22451 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.343569 22363 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:50.345961 22284 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:50.346961 22284 tablet_bootstrap.cc:492] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:21:50.349887 22397 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.350383 22397 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.351600 22467 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.352607 22397 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.318159 22476 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.353631 22467 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:50.324579 22349 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.325220 22454 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:50.294251 22384 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.327169 22374 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.319878 22406 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.377421 20881 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:50.329010 22457 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:50.330272 22360 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:50.356274 20883 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:50.383533 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:50.384395 20877 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:50.360460 22454 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.384997 22454 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:50.369472 22476 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:50.369549 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 367291us
I20260709 12:21:50.416214 22346 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.386255 22284 ts_tablet_manager.cc:1403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 1.768s	user 0.057s	sys 0.088s
I20260709 12:21:50.371114 22447 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.425688 22462 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.589231 21432 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:50.766109 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:50.766752 20882 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:50.742677 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:50.768117 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:50.768755 20882 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:50.627825 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:50.768052 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:50.768858 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:50.752462 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 500474us
I20260709 12:21:50.770850 20808 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:50.943936 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:50.957147 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:51.234825 20884 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:51.234946 20732 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
W20260709 12:21:51.024796 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.027s	sys 0.014s
I20260709 12:21:50.840440 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:50.772413 20878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.015s	sys 0.023s
W20260709 12:21:51.237339 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 218091us
I20260709 12:21:50.771642 21433 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
W20260709 12:21:51.252897 20878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.255s	user 0.019s	sys 0.049s
I20260709 12:21:51.252818 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:51.253785 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.030s	sys 0.019s
W20260709 12:21:50.762130 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.015s
W20260709 12:21:51.263317 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.004s	sys 0.049s
I20260709 12:21:51.253623 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 282) took 1318 ms. Trace:
I20260709 12:21:51.264062 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:50.924201 20698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.002s	sys 0.019s
W20260709 12:21:51.265153 20698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.002s	sys 0.026s
W20260709 12:21:51.139036 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.007s	sys 0.017s
I20260709 12:21:51.251466 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
W20260709 12:21:51.265913 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.010s	sys 0.021s
I20260709 12:21:51.266705 20883 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.237679 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:50.943468 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:51.263988 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 1.00507e+06us
I20260709 12:21:50.741957 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:50.604349 22293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.027s	sys 0.072s
I20260709 12:21:51.275404 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1332) took 1124 ms. Trace:
I20260709 12:21:51.276186 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:50.795445 22233 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.012s
I20260709 12:21:51.270315 20735 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:51.275835 20877 rpcz_store.cc:276] 0709 12:21:50.150919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.151287 (+   368us) service_pool.cc:224] Handling call
0709 12:21:50.182386 (+ 31099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:50.384779 (+202393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:51.273734 (+888955us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:51.273879 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464112512}
I20260709 12:21:51.270434 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:51.270394 20810 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.279688 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:50.906607 20875 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:21:50.489248 22311 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.001s
I20260709 12:21:51.282307 20884 raft_consensus.cc:2368] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:21:51.268510 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:51.278215 20876 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:50.728715 22459 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.273838 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:54488 (request call id 1333) took 1116 ms. Trace:
I20260709 12:21:50.601291 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:50.506239 22450 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.006544 22491 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:50.885885 22457 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.427521 22461 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:50.870738 22467 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.634676 22309 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:50.607606 22361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.002s	sys 0.018s
I20260709 12:21:50.619045 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.239310 22447 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:50.503782 22451 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:51.284338 20727 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:51.235807 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:51.584748 20728 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:50.990150 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.587075 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:51.589337 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:51.285745 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:50.826265 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.276722 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.597837 22401 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.599330 22401 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.600065 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:50.825940 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.264387 20878 rpcz_store.cc:276] 0709 12:21:49.935010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.935129 (+   119us) service_pool.cc:224] Handling call
0709 12:21:49.935438 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:49.935617 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:21:49.996831 (+ 61214us) log.cc:844] Serialized 14 byte log entry
0709 12:21:50.288286 (+291455us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b180036eda0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:21:51.253306 (+965020us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:51.253361 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:51.253399 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:51.253458 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:51.253559 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1166,"prepare.queue_time_us":170930,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":105191,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":21995,"spinlock_wait_cycles":6610176,"thread_start_us":1310859,"threads_started":2}
I20260709 12:21:51.602602 22451 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.286927 20883 rpcz_store.cc:276] 0709 12:21:50.152484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.152676 (+   192us) service_pool.cc:224] Handling call
0709 12:21:51.267203 (+1114527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:51.606163 22489 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.275259 22360 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.612228 22360 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:51.613096 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 43581.4us
I20260709 12:21:51.614665 22490 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.614750 22493 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:51.615708 22490 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.490684 22375 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:21:51.279909 22293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.268s	user 0.043s	sys 0.105s
I20260709 12:21:51.618462 22375 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.620419 22293 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:51.622238 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:50.785223 22377 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.873911 22336 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.623799 22377 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.624158 22336 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.267827 22389 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.283810 22384 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.627988 22389 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:50.463655 22397 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:51.631242 22397 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.632246 21429 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:50.963830 22393 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.606961 22489 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.299736 22309 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.633844 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 476) took 1034 ms. Trace:
I20260709 12:21:51.633760 22393 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.634389 21429 rpcz_store.cc:276] 0709 12:21:50.599577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.600996 (+  1419us) service_pool.cc:224] Handling call
0709 12:21:51.633507 (+1032511us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54007b2ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:51.633694 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782629120}
I20260709 12:21:50.427894 22406 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:50.413100 22419 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.015s	sys 0.025s
I20260709 12:21:51.637036 22336 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:51.637143 22419 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.783s	user 0.033s	sys 0.025s
I20260709 12:21:51.606158 20810 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:51.614966 22346 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:50.634676 22420 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.638439 22389 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.639324 22389 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.639632 22348 raft_consensus.cc:1064] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:21:51.639395 22420 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:51.640183 22389 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.641050 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:50.864379 22388 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.642335 22420 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.239495 22448 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.642494 22389 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.642992 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:51.642618 22388 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.643577 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:51.643551 22448 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:51.644578 20728 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:51.645394 22503 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.646176 22388 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.646440 22448 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:50.729225 22449 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:50.463891 22442 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.858s
I20260709 12:21:50.619277 22478 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.272264 22497 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.647835 22449 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.648283 22478 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.375445 22452 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:50.753748 22454 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.296810 22457 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.293337 22450 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.295176 22491 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.288337 22311 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.299144 22461 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.762151 22491 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:51.298882 22361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.258s	user 0.002s	sys 0.029s
I20260709 12:21:51.762823 22461 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.565349 22455 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.299082 22467 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:51.567909 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 303766us
I20260709 12:21:51.569777 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:51.570257 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:51.579701 22500 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:51.580808 22488 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:51.581511 22476 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.615849 22493 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.595716 20802 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:51.635877 22489 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.636296 22393 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.622880 22375 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.624178 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 285) took 1268 ms. Trace:
I20260709 12:21:51.648555 22497 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.780022 22497 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.780150 22467 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.365520 22492 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.780719 22497 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.781446 22497 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.651639 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:51.782469 22476 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.684587 22451 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.685536 22502 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.619052 22490 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.636204 22309 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.693219 20733 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:51.694280 20735 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
W20260709 12:21:51.699061 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 85649.5us
I20260709 12:21:51.794308 22505 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.795167 22505 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.796140 22397 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.796741 22233 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.797019 22490 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.797730 22505 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.797462 22233 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.797819 22490 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.702845 22501 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:51.702983 20806 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:51.768028 22488 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.799369 22501 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.799642 22488 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:51.799821 22501 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:51.703521 22478 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.800647 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1653) took 1034 ms. Trace:
I20260709 12:21:51.703927 21426 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
W20260709 12:21:51.705600 20775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.704s (SENT) [suppressed 4 similar messages]
I20260709 12:21:51.691617 22452 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.686051 22401 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.692085 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.812338 20801 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:51.812822 22497 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.717267 22377 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.721128 20730 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:51.721134 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:51.723001 22449 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.724839 22504 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.729058 22457 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.737082 22462 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.736423 22348 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.692461 22360 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.741683 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:51.757594 22406 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.646203 22503 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.761197 22454 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.761757 22336 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:21:51.564527 22484 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.022s	sys 0.012s
I20260709 12:21:51.762960 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:51.767277 20732 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:51.766952 22455 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:51.768004 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 493584us
I20260709 12:21:51.768615 22517 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.769122 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:51.768476 22284 raft_consensus.cc:359] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:51.753420 22481 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.758262 22448 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.777609 20875 rpcz_store.cc:276] 0709 12:21:50.356016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.359035 (+  3019us) service_pool.cc:224] Handling call
0709 12:21:51.623570 (+1264535us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b540022f6d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:51.623914 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995802880}
I20260709 12:21:51.781296 22492 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.293574 22459 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:51.673051 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58846.5us
I20260709 12:21:51.762171 22311 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.770915 22373 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:51.801276 20806 rpcz_store.cc:276] 0709 12:21:50.766410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.772413 (+  6003us) service_pool.cc:224] Handling call
0709 12:21:51.800383 (+1027970us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b54002aadd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:51.800566 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110183296}
I20260709 12:21:51.802305 22467 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.759588 22346 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.817190 20728 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.817242 22478 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.822369 22490 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.824115 22461 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.836902 22502 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.808817 20699 leader_election.cc:400] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:21:51.862330 20804 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:21:51.866850 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97887.2us
I20260709 12:21:51.887846 20699 leader_election.cc:403] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:21:51.887034 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1438) took 1217 ms. Trace:
W20260709 12:21:51.856523 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151086us
I20260709 12:21:51.888346 20730 rpcz_store.cc:276] 0709 12:21:50.666696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.666854 (+   158us) service_pool.cc:224] Handling call
0709 12:21:50.752427 (+ 85573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:51.816567 (+1064140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:51.878707 (+ 62140us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54000e7ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:51.883608 (+  4901us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222348544}
W20260709 12:21:51.889611 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 126872us
I20260709 12:21:51.867509 22448 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.837291 22500 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.891480 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:51.864126 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:51.867950 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 474) took 1282 ms. Trace:
I20260709 12:21:51.895474 22476 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.880216 22519 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.895316 21426 rpcz_store.cc:276] 0709 12:21:50.585651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.600026 (+ 14375us) service_pool.cc:224] Handling call
0709 12:21:51.867633 (+1267607us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54007201d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:51.867849 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808058624}
I20260709 12:21:51.899755 22519 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:51.900307 22462 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.901137 22462 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.864997 22517 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.903662 22384 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.903868 22500 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.904429 22517 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.904313 22384 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.904712 22448 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.900364 22519 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.880823 22377 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.891860 22467 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.874264 22311 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:51.866751 22284 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:21:51.866433 22397 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.876232 22450 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.924904 22517 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.979867 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:21:51.927822 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64059.3us
I20260709 12:21:51.930028 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.980643 21428 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.980895 21432 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:51.930405 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1652) took 1164 ms. Trace:
I20260709 12:21:51.982384 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:51.982463 20804 rpcz_store.cc:276] 0709 12:21:50.766087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.767846 (+  1759us) service_pool.cc:224] Handling call
0709 12:21:51.930147 (+1162301us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003c6c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:51.930326 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2429732224}
I20260709 12:21:51.983865 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:51.985737 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.982714 20728 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:51.931968 22488 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.933673 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.935302 22393 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.935637 22233 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.937112 22476 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:51.866900 22464 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.939492 22346 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.940469 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.940809 22477 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:51.867097 22481 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.860046 22406 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.861582 22504 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.947257 20803 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:51.857532 22336 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.948441 22311 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.948714 22478 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.011453 22464 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.024782 22311 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.024888 22464 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.948144 22377 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.025115 22478 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.949781 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:51.950135 22462 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.950297 22492 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.874861 22503 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:51.950644 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.930588 22501 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.950809 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.942983 20628 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 194) took 2937 ms. Trace:
I20260709 12:21:51.951457 22392 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.952921 22512 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.906925 22373 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.955359 22467 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.957015 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.887267 22457 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.960058 22459 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:51.958050 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66701.3us
I20260709 12:21:51.952281 20633 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:21:51.962927 22490 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.908401 22455 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.963745 22448 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:51.964403 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:51.967718 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:51.968180 22493 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:51.967737 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:51.873119 22449 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:51.970638 22502 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:51.972579 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:51.973917 22454 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:52.047039 22457 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.042466 22449 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.972692 22519 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:51.958663 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 150275us
I20260709 12:21:52.056643 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:51.975978 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:51.948110 22497 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:51.976382 22284 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:51.960776 22397 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:51.960776 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.977995 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.979663 22517 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:51.927404 22360 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:51.980019 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:51.931026 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.931314 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.983865 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:51.986526 22491 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:51.987022 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:51.984686 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:51.906507 22384 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.099319 21435 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:51.927456 22500 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.993366 21428 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:51.993989 21431 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:52.001513 21427 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:52.003118 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.001502 22506 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:21:52.005939 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.009079 22346 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:52.010138 22393 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.011724 22477 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:52.027601 22377 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.034166 22336 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:52.039937 22406 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.040275 20879 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:21:52.040520 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1656) took 1270 ms. Trace:
I20260709 12:21:52.030424 20628 rpcz_store.cc:276] 0709 12:21:49.005295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.005752 (+   457us) service_pool.cc:224] Handling call
0709 12:21:49.015987 (+ 10235us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b5400002490. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7b0b2 00007f9e24af4759 00007f9e24a8ca76 00007f9e24a5dd58 00007f9e24b5c04e 00007f9e2301d2a4 00007f9e2301d232 00007f9e2301d161 00007f9e2301d0dc
0709 12:21:51.942702 (+2926715us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:49.441023 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:49.580177 (+139154us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:49.584339 (+  4162us) write_op.cc:435] Acquired schema lock
0709 12:21:49.585813 (+  1474us) tablet.cc:662] Decoding operations
0709 12:21:49.654874 (+ 69061us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:49.671720 (+ 16846us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:49.677552 (+  5832us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:49.703449 (+ 25897us) tablet.cc:701] Row locks acquired
0709 12:21:49.710755 (+  7306us) write_op.cc:260] PREPARE: finished
0709 12:21:49.754677 (+ 43922us) write_op.cc:270] Start()
0709 12:21:49.754808 (+   131us) write_op.cc:276] Timestamp: P: 1783599709754619 usec, L: 0
0709 12:21:49.754831 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:21:49.755598 (+   767us) log.cc:844] Serialized 536 byte log entry
0709 12:21:50.084839 (+329241us) op_driver.cc:464] REPLICATION: finished
0709 12:21:50.905503 (+820664us) write_op.cc:301] APPLY: starting
0709 12:21:50.905614 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:50.905743 (+   129us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:50.923553 (+ 17810us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:50.923871 (+   318us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:50.971259 (+ 47388us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:21:50.990309 (+ 19050us) write_op.cc:312] APPLY: finished
0709 12:21:51.006123 (+ 15814us) log.cc:844] Serialized 23 byte log entry
0709 12:21:51.091109 (+ 84986us) spinlock_profiling.cc:243] Waited 930 us on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1bb950ac 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b
0709 12:21:51.858104 (+766995us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:51.858293 (+   189us) write_op.cc:454] Released schema lock
0709 12:21:51.858385 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:51.936251 (+ 77866us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:51.936304 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":60727,"reported_tablets":4,"spinlock_wait_cycles":17716864,"child_traces":[["op",{"apply.queue_time_us":809789,"apply.run_cpu_time_us":40289,"apply.run_wall_time_us":987558,"num_ops":1,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":11611,"prepare.run_wall_time_us":325502,"replication_time_us":251609,"spinlock_wait_cycles":2138496,"thread_start_us":1220379,"threads_started":2,"wal-append.queue_time_us":558653}]]}
I20260709 12:21:52.046661 20876 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:21:52.047498 22526 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:21:52.047901 22488 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.044723 20809 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:52.048458 22448 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.051402 22512 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.052425 22490 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.032107 21426 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
W20260709 12:21:52.071676 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:52.069803 22392 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:52.077308 20681 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
I20260709 12:21:52.077797 22311 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:52.078545 22497 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:52.079722 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.099593 20804 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:52.096967 20878 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:21:51.985421 22461 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.104714 22500 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.104764 21430 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:52.105561 21434 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:52.106894 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.252826 20884 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:52.221066 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.166566 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
W20260709 12:21:52.113484 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 126592us
I20260709 12:21:52.106907 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.107803 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.165208 20877 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:21:52.158977 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.258370 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.080600 22459 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:52.057471 20801 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:52.262321 21431 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:52.100651 22309 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.113143 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:52.280894 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:52.289274 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.290102 20878 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:52.251986 22461 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:52.259831 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 99432.2us
I20260709 12:21:52.264741 22457 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:52.158697 22384 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:52.221596 22449 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:52.291055 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 285267us
I20260709 12:21:52.300393 20882 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:52.291527 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.291584 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.218621 22397 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.301252 22233 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.301874 22233 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.304282 22233 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.292629 20880 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:52.304744 22451 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:52.305214 22451 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:52.306337 22451 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:52.258215 22500 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.107604 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.295926 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.160939 22504 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.295115 22461 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.259044 22517 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.253283 22284 ts_tablet_manager.cc:1434] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 1.481s	user 0.048s	sys 0.057s
I20260709 12:21:52.295397 22377 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.257318 22477 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:52.299420 20681 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for f15b5f1ac1444b63bad60a89fa9f8a29 already in progress
I20260709 12:21:52.308650 21433 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:52.311761 20809 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:52.257988 20803 rpcz_store.cc:276] 0709 12:21:50.769995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.772247 (+  2252us) service_pool.cc:224] Handling call
0709 12:21:52.040257 (+1268010us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54003c9954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:52.040440 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678347136}
I20260709 12:21:52.312139 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:52.292982 22309 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.311522 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.310838 22481 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.160612 22360 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.299216 22501 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.320389 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.322333 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.333575 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.309293 22348 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:52.308961 22519 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.259212 21432 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:52.327235 20804 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:52.327816 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:52.329721 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.313920 22449 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.331482 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.329802 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.331966 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.312952 22293 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.315686 22397 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.332892 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.335640 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.338578 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.334199 20809 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:21:52.324893 21430 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.313138 22384 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:52.331611 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:52.338186 20735 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:52.333811 20733 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:52.339861 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.321146 22377 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.313166 22406 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:52.338725 20802 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:52.327097 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.345953 21433 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:52.328724 22309 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.337841 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.345257 20728 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.349553 20727 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:52.349826 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.350092 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:52.350661 20731 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:52.346138 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 185725us
I20260709 12:21:52.346940 20732 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:21:52.334625 20681 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for f15b5f1ac1444b63bad60a89fa9f8a29 already in progress
W20260709 12:21:52.324868 22284 ts_tablet_manager.cc:1467] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Tablet startup took 1555ms
I20260709 12:21:52.352533 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.353446 20727 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:52.318337 22459 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.356572 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.354727 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.342684 22377 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.359632 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:52.360435 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:52.358372 20729 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:52.346886 22505 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:52.351275 22488 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.364359 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.365674 20734 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:52.368710 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.367691 20730 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:52.369262 20732 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:52.370667 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.373346 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.374951 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.376860 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.377996 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.378077 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1472) took 1089 ms. Trace:
I20260709 12:21:52.381785 20728 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:52.382710 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.384450 22505 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.349848 22293 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.387652 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:52.388685 20726 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:52.389621 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.390254 20726 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:52.392351 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.366137 22501 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:52.367830 22519 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:21:52.396344 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44324us
I20260709 12:21:52.350442 22348 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.397490 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.372370 22525 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.397493 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.358457 22449 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.397035 22518 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.382932 20807 rpcz_store.cc:276] 0709 12:21:51.288563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.731025 (+442462us) service_pool.cc:224] Handling call
0709 12:21:51.731709 (+   684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:52.034833 (+303124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:52.043578 (+  8745us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:52.044216 (+   638us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:52.060009 (+ 15793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:52.377766 (+317757us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540006ccd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:52.377810 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:52.377926 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":687610112}
I20260709 12:21:52.407776 22518 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:52.399024 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:52.408619 20730 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:52.361407 22406 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.391172 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.400679 22525 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.355351 22497 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.368234 22397 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.422127 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.413309 22501 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:52.398218 22491 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.423141 22491 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.423530 22491 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.423709 22397 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.423938 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.408300 22518 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.424680 20732 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:52.425786 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.420219 22525 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.407644 22384 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.428103 22348 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.420463 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.434212 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.434870 20734 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:52.435820 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:52.436373 20734 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.426987 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.440546 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.400472 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.441145 21431 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:52.441521 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.445598 20731 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
W20260709 12:21:52.442500 22523 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.435s	user 0.002s	sys 0.004s
I20260709 12:21:52.421079 22393 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.451496 20807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "626209a224524080801f3a9ad7307fbe"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:52.455030 22406 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.459357 22348 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.460038 22348 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.462298 22348 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.451802 22451 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.438211 22392 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:52.463680 22448 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.464439 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1467) took 1188 ms. Trace:
I20260709 12:21:52.465485 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.396870 22488 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.472285 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.473245 20731 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:52.474470 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:21:52.464829 20805 rpcz_store.cc:276] 0709 12:21:51.276179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.601907 (+325728us) service_pool.cc:224] Handling call
0709 12:21:51.602423 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:51.749775 (+147352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:51.749824 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:51.750581 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 12:21:51.886625 (+136044us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:51.887666 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:51.887759 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:52.419441 (+531682us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:21:52.464026 (+ 44585us) raft_consensus.cc:1685] finished
0709 12:21:52.464051 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:52.464272 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67439,"raft.queue_time_us":99937,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":194,"spinlock_wait_cycles":11709696,"thread_start_us":183808,"threads_started":2,"wal-append.queue_time_us":167987}
I20260709 12:21:52.478260 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:52.478888 20733 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.483726 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.483976 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:52.485652 22406 raft_consensus.cc:2764] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:21:52.486672 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.487322 20732 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:52.454911 22491 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.488162 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.489264 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.489580 22491 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.490223 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.485118 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.496409 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.496031 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:52.500504 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102527us
I20260709 12:21:52.501161 20733 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:52.500222 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:52.502214 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.503616 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.504175 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.496035 22461 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.505451 22459 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.506982 22461 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:52.505191 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.508091 21431 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:52.448992 22490 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:52.510820 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45418.2us
I20260709 12:21:52.510885 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.511622 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.513329 22361 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.513710 22361 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.514045 22361 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.514520 22459 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.507092 22449 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.514948 22459 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.516222 22459 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.516142 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.516866 21431 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:52.518819 22495 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:52.519443 22495 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.521611 22495 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:52.522281 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.524458 22518 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.490725 20726 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:52.526652 22518 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.530030 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.530570 22505 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.530753 21435 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:52.531076 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.531848 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:52.532431 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.532713 21435 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:52.533061 21432 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:52.533771 22373 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.533931 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.534256 22512 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.525844 22459 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.537730 22459 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.539398 22462 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:52.542510 22462 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:52.543253 22501 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.543725 22462 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.541565 22488 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.546625 22293 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.528069 20807 raft_consensus.cc:606] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:52.548182 20807 raft_consensus.cc:614] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:52.547835 21431 raft_consensus.cc:2955] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:21:52.552536 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68912.8us
I20260709 12:21:52.472556 22500 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.553371 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:52.554446 22500 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.555199 20808 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3.
I20260709 12:21:52.555061 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.463063 22377 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.555704 20884 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:52.556807 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.557420 22377 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.447059 22519 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.557422 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:52.562026 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.562597 20878 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:52.563544 22501 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:52.563977 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.564112 22501 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.565646 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.566210 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:52.567127 20805 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5.
I20260709 12:21:52.566216 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.538939 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:52.568152 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:52.568477 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.568917 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:52.569181 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.569774 22491 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:52.569835 20803 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:21:52.570118 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.570842 22346 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:52.571120 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.571386 22346 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.570384 22491 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:52.572280 22346 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:52.573661 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:52.575358 22461 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.575721 22461 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.576160 22461 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.572901 22491 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.590859 20882 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:21:52.579237 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.580036 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.580971 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.596583 20805 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:52.597839 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.598683 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.598635 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.600826 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.585912 20734 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:52.578876 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:21:52.592554 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 90665.7us
I20260709 12:21:52.594255 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.609371 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.610642 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:52.610841 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.495566 22525 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.619613 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.610841 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.621306 20809 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:52.625700 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.626670 22502 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.627149 22502 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.627547 22502 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.627652 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.626703 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.628764 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:52.626875 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73661.1us
I20260709 12:21:52.594676 22519 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.645978 22309 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.649614 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.649984 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.631256 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.633383 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.652762 20876 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:52.653061 20808 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:52.633594 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:52.634440 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:52.636708 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.659323 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:21:52.624840 22392 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.633421 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.624356 22293 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.663231 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:52.652026 22501 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.661590 22392 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.661844 20728 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:52.663995 20804 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
W20260709 12:21:52.667498 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 164087us
I20260709 12:21:52.658252 22393 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.674329 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:52.675482 20878 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:52.663662 22293 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.663998 21426 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.667627 22392 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.664772 22360 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.688696 22309 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.676419 22393 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.700280 22309 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:52.686949 22293 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.700959 22309 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.711608 22393 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.724072 22309 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.725649 22448 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.726159 22448 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.726521 22448 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.694096 22455 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.727594 22455 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:52.728001 22455 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.755287 22346 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.757921 22346 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:52.759090 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.767339 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.767899 20878 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:52.766250 22346 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:21:52.770967 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:52.771730 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:52.781940 20875 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:52.783672 22519 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:52.784693 22501 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:52.785184 22501 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:52.785841 22502 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.785950 22501 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.754101 22448 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.769325 22346 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.788683 22525 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.796577 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.797829 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.798111 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:52.836547 20884 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.799328 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:52.837819 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 228517us
I20260709 12:21:52.838316 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.839079 20877 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:52.839931 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.840639 20877 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:52.841054 20806 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:52.800494 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.801041 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.801750 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.807292 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.845424 20883 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:52.842885 20877 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:52.844084 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.851116 20731 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:52.813896 22462 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.815348 21426 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:21:52.353137 22284 ts_tablet_manager.cc:1469] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Trace:
0709 12:21:42.903865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.904004 (+   139us) service_pool.cc:224] Handling call
0709 12:21:43.551015 (+647011us) ts_tablet_manager.cc:934] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
0709 12:21:47.015916 (+3464901us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:21:47.262345 (+246429us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:47.270263 (+  7918us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:21:47.677480 (+407217us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:21:47.677757 (+   277us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:47.678068 (+   311us) ts_tablet_manager.cc:989] Replica registered.
0709 12:21:47.678145 (+    77us) inbound_call.cc:177] Queueing success response
0709 12:21:48.617819 (+939674us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:21:50.772441 (+2154622us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:21:51.071953 (+299512us) tablet_replica.cc:261] Starting instrumentation
0709 12:21:51.607605 (+535652us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":244,"mutex_wait_us":7572,"raft.queue_time_us":27328,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":65413,"tablet-copy.queue_time_us":427503,"tablet-download-pool.queue_time_us":483,"tablet-download-pool.run_cpu_time_us":1206,"tablet-download-pool.run_wall_time_us":289803,"thread_start_us":1484633,"threads_started":2}
I20260709 12:21:52.764508 22293 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.818465 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:52.755013 22477 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:52.885685 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.750927 22361 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.831858 22501 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:52.889529 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:52.879873 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 78207.2us
I20260709 12:21:52.892513 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:52.894140 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:52.818994 22393 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.897709 22393 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:52.899794 22488 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.900496 22488 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:52.901074 22488 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.903398 22377 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:52.905079 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:52.920447 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:52.920919 22309 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:52.921418 22377 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:52.921875 22377 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:52.921607 22309 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.922283 22377 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.934237 22361 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.935563 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:52.936514 20731 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:21:52.940750 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.820982 22455 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.883108 20876 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:52.761986 22392 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.960595 22293 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:52.964715 21430 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.002s
I20260709 12:21:52.877269 22519 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:52.966634 20881 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
W20260709 12:21:52.969961 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:52.971057 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 392234us
I20260709 12:21:52.966800 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:52.965390 21430 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
W20260709 12:21:52.972025 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 352446us
I20260709 12:21:52.975090 20884 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:52.947364 22497 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:52.976617 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:52.972071 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:52.980738 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.978433 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:52.983940 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:52.990145 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:52.992295 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.995123 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:52.995736 21431 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:52.981775 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:53.003643 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.004464 20878 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:53.007356 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:52.966527 22309 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:52.968743 22518 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.015043 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:53.018646 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46671.6us
I20260709 12:21:53.020035 21431 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:52.966868 22519 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.021236 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.021236 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:53.023264 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 218550us
I20260709 12:21:53.039549 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:52.945731 22488 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:52.981954 22497 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.998725 22504 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:52.997620 22360 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.055941 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.004096 22457 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:53.064183 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.056630 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:53.065068 21430 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.072714 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.069844 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.078090 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:53.021672 22459 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:52.963660 22462 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:53.085711 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:53.036383 22502 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.049271 22519 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:53.043757 22518 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.049659 22497 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:53.091295 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 288) took 1101 ms. Trace:
I20260709 12:21:53.064965 22517 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.072851 22457 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.101558 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.099720 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.064657 22504 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.071787 20882 raft_consensus.cc:2955] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:21:53.065661 22360 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.120513 22504 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:53.120674 22360 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.121477 21430 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:21:53.090114 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108401us
I20260709 12:21:53.122699 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.123509 21435 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.090736 22501 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:53.083774 22377 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.115999 22397 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:53.096767 22491 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:53.114106 22457 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:53.118388 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 372473us
I20260709 12:21:53.094523 22540 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:53.096582 22459 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:53.097541 22518 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.119992 20731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:53.125283 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:53.088181 22309 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:53.151762 22491 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:53.151608 22309 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:53.097350 22461 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.004s
I20260709 12:21:53.127614 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.096889 22336 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.095649 21434 rpcz_store.cc:276] 0709 12:21:51.989650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.534744 (+545094us) service_pool.cc:224] Handling call
0709 12:21:52.535074 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:52.535434 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:21:52.947755 (+412321us) log.cc:844] Serialized 14 byte log entry
0709 12:21:53.090311 (+142556us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:53.090941 (+   630us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:53.090989 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:53.091050 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:53.091240 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5896,"raft.queue_time_us":6047,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6192,"spinlock_wait_cycles":2304,"thread_start_us":442092,"threads_started":2}
I20260709 12:21:53.141007 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:53.141287 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101862us
W20260709 12:21:53.119987 22545 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:21:53.143253 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 108106us
I20260709 12:21:53.147020 20731 raft_consensus.cc:606] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:53.148643 22346 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.142341 21429 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:53.149886 22455 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.131575 22525 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:53.160617 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:53.198561 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.167447 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:53.202489 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:53.168381 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:53.154060 20875 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:53.191396 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.141984 22501 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:53.190507 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.209993 21433 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:53.211066 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.213099 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:53.213871 20734 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:53.159418 22373 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:53.222189 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:53.166409 22397 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:53.231587 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:53.232211 20735 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.233976 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:53.234515 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66244.6us
I20260709 12:21:53.235247 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:53.236244 20729 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:21:53.237697 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:53.239374 20727 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:53.240146 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:53.240765 20730 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.166111 22336 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.243907 20849 leader_election.cc:400] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5.
I20260709 12:21:53.244254 20849 leader_election.cc:403] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:21:53.245625 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.219725 22506 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:53.215256 22384 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.220774 22346 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.166394 22461 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.250052 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.251545 22397 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.253190 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:53.235455 22477 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:53.237939 22525 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:53.230913 22392 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.260334 22377 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.263383 22459 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.264535 22501 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.161638 22540 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.235105 22451 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:53.266882 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:53.305507 20884 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:53.269168 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:53.271935 20727 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:21:53.273154 20877 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.254339 20726 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:53.265857 22462 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:53.152815 22491 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.255918 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.243427 22542 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.240864 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.241379 22551 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:21:53.282502 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 204316us
I20260709 12:21:53.270794 22506 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.286947 20881 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:53.289134 22336 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:53.280292 22553 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5.
I20260709 12:21:53.293012 22461 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.293565 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:53.339570 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:53.270437 22346 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:53.340544 22346 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.341863 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:53.342439 20804 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.276026 22397 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:53.347173 22397 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.347432 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:53.315063 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:53.350378 20805 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.350728 21435 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:53.317113 20730 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:21:53.317421 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48375.1us
I20260709 12:21:53.320855 20878 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:53.355219 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.320438 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:53.320742 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64907.6us
I20260709 12:21:53.265982 22384 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:53.335346 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:53.361289 20808 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:53.304966 22451 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.369390 22451 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:53.308525 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:53.355973 20883 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.325950 22477 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:53.357932 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:53.347891 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.359800 22346 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.365468 22448 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.367533 22384 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.367971 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 297) took 1023 ms. Trace:
I20260709 12:21:53.368896 22449 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.392467 21429 rpcz_store.cc:276] 0709 12:21:52.344307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.545895 (+201588us) service_pool.cc:224] Handling call
0709 12:21:52.561347 (+ 15452us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:21:53.094001 (+532654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:53.195880 (+101879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:53.367760 (+171880us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54005de150. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:53.367907 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229840128}
I20260709 12:21:53.392705 22449 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.363916 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.356679 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.376001 20804 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:53.329923 22377 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.332063 20893 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-f15b5f1ac1444b63bad60a89fa9f8a29 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:21:53.391999 22336 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:53.387030 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:53.403749 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.380463 22461 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:21:53.405746 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49139.2us
I20260709 12:21:53.406994 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.379675 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.408898 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.411442 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:53.406039 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:53.414247 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.423713 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.382666 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.430044 22540 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.394531 22488 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.389077 22552 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.432227 22552 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.432775 22457 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.434303 22552 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.434823 22457 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.409842 22502 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:53.435297 20893 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-f15b5f1ac1444b63bad60a89fa9f8a29 on tablet f15b5f1ac1444b63bad60a89fa9f8a29 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.435465 22558 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.408540 22461 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.400561 22377 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:21:53.439452 22539 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.324s	user 0.000s	sys 0.008s
I20260709 12:21:53.439697 22451 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.443562 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.390651 22525 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.409008 22477 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:53.482199 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68017.8us
W20260709 12:21:53.482800 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:53.482960 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:53.439507 22512 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.485294 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.405395 22392 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:21:53.486910 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:53.492429 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:53.469164 22500 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.469343 22559 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:53.481696 21428 raft_consensus.cc:2368] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:53.496704 20879 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:53.475745 22488 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:21:53.494024 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84567.7us
I20260709 12:21:53.477772 22551 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:53.473035 22502 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.480427 22461 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.514781 22497 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.473385 22558 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.500267 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.501454 20884 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:53.496191 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.480235 22377 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.476171 22552 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.510728 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:53.485424 22517 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:53.510994 20849 leader_election.cc:400] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:21:53.520406 20849 leader_election.cc:403] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:21:53.511032 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:53.487397 22348 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:53.511117 20878 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:53.521281 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:53.512789 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:53.514232 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:53.517242 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:53.495836 22373 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:53.518959 20809 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:53.547880 22449 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.548018 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 290) took 1526 ms. Trace:
W20260709 12:21:53.548834 22549 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:21:53.549335 22360 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.552349 22497 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.552785 22506 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.519639 20879 raft_consensus.cc:2368] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:53.491024 22540 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:53.493192 22293 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:21:53.523156 22448 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:53.518458 22488 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:53.524525 20881 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:53.528364 22451 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:53.592374 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75530.3us
W20260709 12:21:53.592554 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 191788us
I20260709 12:21:53.593250 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.508380 22508 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:53.529011 22519 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.537302 20877 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:53.547003 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:53.554380 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:53.554883 20805 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:53.555466 20883 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
W20260709 12:21:53.518270 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 154474us
I20260709 12:21:53.534524 22504 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.553254 20733 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:53.516395 22502 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:53.553349 21432 rpcz_store.cc:276] 0709 12:21:52.022115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.534794 (+512679us) service_pool.cc:224] Handling call
0709 12:21:52.563282 (+ 28488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:52.597609 (+ 34327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:52.597642 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:52.626928 (+ 29286us) log.cc:844] Serialized 26 byte log entry
0709 12:21:52.953943 (+327015us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:52.954008 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:52.954130 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:53.438858 (+484728us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:21:53.535637 (+ 96779us) raft_consensus.cc:1685] finished
0709 12:21:53.535658 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:53.547950 (+ 12292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7020,"raft.queue_time_us":18837,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":12421,"spinlock_wait_cycles":20325888,"thread_start_us":321435,"threads_started":1,"wal-append.queue_time_us":416273}
I20260709 12:21:53.536300 22500 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.520018 22558 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.607721 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:53.600080 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 172456us
I20260709 12:21:53.605121 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:21:53.608487 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61611.4us
I20260709 12:21:53.608944 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:53.609047 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.609531 20882 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.609784 20805 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:21:53.605245 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63446.2us
I20260709 12:21:53.554352 22559 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:53.531024 22461 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:53.594930 22552 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.614370 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.617638 20808 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:53.617395 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:53.553268 22517 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.600502 22477 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:21:53.560688 22542 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:21:53.597293 22508 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:53.596199 22448 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:53.591373 22566 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:53.596895 22540 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:53.597741 22567 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:53.646842 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.652418 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:53.650507 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.657516 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:53.661410 21432 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.661446 20808 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:53.616787 22556 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:53.616586 22559 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:53.619848 20807 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:53.620671 22392 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:53.649919 20801 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:53.667411 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:53.648747 22346 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:53.618579 22568 raft_consensus.cc:2764] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:21:53.617179 22552 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:53.688051 22552 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.654484 22566 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.651701 22491 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:53.675715 22392 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:53.701435 22392 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:53.683771 22508 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.691771 20882 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:21:53.683563 22309 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.702543 22508 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.703455 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:53.700058 22559 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.706141 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.711089 22568 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:21:53.682232 22518 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.510s	user 0.003s	sys 0.006s
I20260709 12:21:53.619764 22373 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.696882 22556 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:53.731019 22568 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:53.741703 22448 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.742239 22568 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.744382 22556 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.678851 22542 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:53.732818 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:53.705950 22566 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.701915 22491 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.750316 22384 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:53.748973 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45600us
I20260709 12:21:53.731228 22575 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:53.755223 20876 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:53.757398 22517 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.620378 22551 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.757918 22540 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:21:53.758908 22574 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:21:53.681828 22488 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.759974 22504 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.755787 22495 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:53.761232 22348 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:21:53.762166 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 266067us
I20260709 12:21:53.762698 22506 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.765977 20776 leader_election.cc:400] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:53.793009 20776 leader_election.cc:403] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:21:53.749174 22564 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:21:53.770458 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.732216 22397 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.739710 22518 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:53.773011 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:53.773581 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:53.747426 22542 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:21:53.760030 22533 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.660s	user 0.003s	sys 0.000s
I20260709 12:21:53.760763 22519 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.791697 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:21:53.801637 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 191588us
W20260709 12:21:53.797297 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 276295us
I20260709 12:21:53.797363 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.804549 21431 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:53.815714 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49902.6us
I20260709 12:21:53.787662 22508 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:21:53.817474 22576 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:53.817865 22508 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:53.818270 22508 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:53.771142 22575 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:53.820205 20810 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:53.819082 22508 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:53.828815 22555 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.256s	user 0.001s	sys 0.001s
I20260709 12:21:53.828722 22449 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:21:53.829953 22573 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:21:53.829872 22348 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:53.796218 22461 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:53.831894 20774 leader_election.cc:400] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:53.800289 22495 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:53.796411 22561 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:21:53.971136 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:53.996778 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 172746us
I20260709 12:21:53.997339 20733 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.940506 20804 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:53.852885 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.975944 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:53.823002 21432 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:53.999610 20774 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:54.010561 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:54.011168 20774 leader_election.cc:403] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:21:53.925923 22518 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:54.013202 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 221572us
I20260709 12:21:54.014303 22508 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:54.014734 22534 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.028s	sys 0.036s
I20260709 12:21:54.012382 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:54.014632 22392 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.057122 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:54.031100 22393 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:54.031459 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:53.851089 22397 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:54.015445 22553 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.033002 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.997876 22491 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:54.019829 22459 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.884398 20803 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:53.812786 22517 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.207s
I20260709 12:21:54.038769 22497 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:21:54.041808 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44193.3us
I20260709 12:21:54.044260 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:53.841096 22575 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.032119 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:53.921685 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.137055 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:54.138523 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:53.926492 22360 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:53.928864 22384 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:53.973711 22377 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.067821 20733 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:54.071564 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.165068 20802 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.063488 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.165684 20810 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:54.060487 22477 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:54.077243 22512 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:54.065137 22455 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:54.060976 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:54.065713 22293 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:21:54.079320 22579 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:21:54.053774 22534 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.028s	sys 0.036s
I20260709 12:21:54.136485 22392 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.055984 22449 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
W20260709 12:21:54.200184 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 139273us
I20260709 12:21:54.108791 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:54.111285 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:54.117362 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73176.1us
I20260709 12:21:54.118850 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:54.119450 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 467071us
I20260709 12:21:54.119887 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:54.202239 20809 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:54.116746 22569 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.115963 22497 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:54.170542 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:54.170943 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.171614 20805 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:54.171844 20879 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:54.172441 22377 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:54.175379 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.125782 22309 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:54.081898 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.139793 22517 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:54.152537 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:54.176010 22512 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:54.150452 22519 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.101058 22397 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.201797 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:54.222738 22397 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.223721 22461 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.224152 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:54.225566 20804 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:21:54.225951 22540 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:54.227136 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.227558 22540 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:54.227949 22540 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:54.228204 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.228892 20804 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:54.228801 22540 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.204686 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:54.229702 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.230209 22540 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.230718 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.230232 20804 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:54.231308 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.233101 20802 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:53.876609 22542 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.209774 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.239060 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.213033 21432 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:21:54.210009 22502 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.592s	user 0.004s	sys 0.007s
I20260709 12:21:54.216112 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.216507 22459 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.210213 22309 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:54.210355 22517 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:54.203414 22455 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:54.208866 22579 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:54.225435 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:54.176039 22477 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.235561 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.242225 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.242642 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.245958 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:54.159739 22495 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.218007 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.269724 20802 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:54.211103 22448 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:21:54.270289 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44914.2us
I20260709 12:21:54.279541 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.271575 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.271921 22512 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.273453 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:54.244035 22558 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.629s	user 0.004s	sys 0.007s
I20260709 12:21:54.281224 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.283107 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.283340 20728 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.282291 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:54.284696 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45693.8us
I20260709 12:21:54.230458 22569 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.288576 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:54.286573 20879 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:54.244498 22360 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:54.267776 22361 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.200443 22552 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.311520 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.314719 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.311523 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.315467 22552 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:54.315768 20809 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:21:54.316947 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.317405 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.266754 22459 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.312870 22393 raft_consensus.cc:2764] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:21:54.318132 20804 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:54.318935 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:54.319175 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.275323 22346 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:54.313196 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.321451 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.322719 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.291380 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.268555 22309 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.317643 20809 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:54.319630 20804 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:54.315896 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.235067 22519 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.330067 20775 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.274497 22542 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:54.340623 20733 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:54.344673 20735 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:54.344853 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.349423 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.350910 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.345510 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.352133 20727 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:21:54.267416 22525 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:21:54.276733 22448 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:54.274006 22590 raft_consensus.cc:2764] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:21:54.319473 22459 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:54.347810 20732 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:54.353749 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.290323 22360 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:54.464753 22360 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.464959 22461 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:54.465144 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:54.466149 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 112517us
I20260709 12:21:54.466465 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.244953 22502 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.466898 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.468777 22502 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.470003 22504 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:54.469997 22518 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.376101 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.472044 21430 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:54.471906 22360 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.472939 21429 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:54.473256 20732 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:54.377738 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.329694 22517 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.352680 22393 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:54.475627 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:54.340773 22392 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:54.352952 22566 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.002s
I20260709 12:21:54.356189 22346 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:54.373850 22448 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:54.380319 22586 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:21:54.378821 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:54.383051 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:54.385423 22584 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 12:21:54.290012 22558 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.346547 22512 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.388003 22397 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:54.389621 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.514760 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.390930 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.392971 20801 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.399940 20879 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:21:54.400679 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:54.402397 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.403128 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.404397 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.290442 22348 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:21:54.340660 22552 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:54.414584 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.416174 22560 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:21:54.525957 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 203326us
I20260709 12:21:54.418077 20728 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:54.423434 22579 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.425678 22519 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:54.427076 22569 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:54.427281 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.427637 22455 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:54.430861 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.431972 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:54.436256 22477 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.354925 22576 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:54.354614 22559 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.442595 20726 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:21:54.447304 20804 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:54.447834 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.375494 22377 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.356457 22449 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:21:54.374015 22594 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:21:54.454209 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.461807 20882 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:54.282104 22561 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:21:54.376833 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.291221 22497 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.480968 20732 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.488545 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:21:54.492027 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 172914us
I20260709 12:21:54.389753 22575 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:54.499291 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 212931us
I20260709 12:21:54.519634 20733 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:21:54.547432 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93345.1us
I20260709 12:21:54.483440 22393 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:54.515010 22517 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:54.526866 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 124529us
I20260709 12:21:54.487484 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:54.417697 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.531092 21431 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:54.498921 22361 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:54.544610 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.520140 20727 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:54.547035 21432 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:54.548317 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:54.550184 20735 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:54.550981 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62527.1us
I20260709 12:21:54.564208 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.564678 20879 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:54.527799 22560 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:54.554327 22361 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.500066 22566 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.569229 21433 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:54.537961 22563 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.541858 22502 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:54.572398 21434 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:54.568511 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.573421 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.546005 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:54.550458 22552 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:54.552551 22393 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.581812 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.514380 22346 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.584554 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.566372 22518 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.567679 22567 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:54.575068 22361 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:54.570575 22558 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.588560 22502 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:54.590853 22554 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.592188 20808 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:21:54.592923 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.591459 22567 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.604532 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.569216 22397 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:21:54.586143 22576 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.594838 22551 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.600812 22583 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:21:54.600643 22502 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:54.580525 22568 raft_consensus.cc:2764] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:21:54.606613 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:54.606568 22554 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.574894 22501 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:21:54.606813 22560 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.608256 22566 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:54.608527 22575 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.608716 22603 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:54.606601 22567 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:21:54.610927 20850 leader_election.cc:336] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.922s (SENT)
I20260709 12:21:54.594872 22508 raft_consensus.cc:2764] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:21:54.646051 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.616441 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:54.617805 20881 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:54.583529 22495 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:21:54.625540 22560 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.625747 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:54.621865 22459 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:54.632660 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103901us
I20260709 12:21:54.659077 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:54.638113 22373 raft_consensus.cc:993] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:54.632746 22603 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:54.627981 22606 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.617113 22360 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.617311 20801 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:54.641178 20735 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:54.648481 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:54.648758 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:54.649572 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:54.650506 20884 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:54.627079 22501 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:54.652509 22601 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:54.653239 20882 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:54.660286 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:21:54.672391 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66560.2us
W20260709 12:21:54.673815 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62915.2us
I20260709 12:21:54.651136 22561 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.711822 21400 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3.
I20260709 12:21:54.712435 21400 leader_election.cc:403] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:21:54.661909 22449 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:54.661933 22603 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:54.783067 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 122822us
I20260709 12:21:54.787957 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.672909 22373 raft_consensus.cc:1081] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:54.667985 22583 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:54.671564 22501 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:21:54.797318 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 124265us
I20260709 12:21:54.801139 20731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:54.801708 20731 raft_consensus.cc:606] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership
I20260709 12:21:54.803953 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.809340 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.789248 22508 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3.
I20260709 12:21:54.678896 22566 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 12:21:54.788679 22601 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:54.788761 22576 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.792361 22556 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:54.809499 22551 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.833190 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.820497 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:54.835296 22556 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.835465 22501 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.835855 22601 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.836618 22501 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.792515 22569 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.829473 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:54.838032 22607 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:21:54.813604 22583 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.843784 22397 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:54.844480 22397 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.845312 22583 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:54.847776 22610 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:21:54.668790 22606 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.800061 20880 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:21:54.859475 20727 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.860954 20732 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.862363 20733 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.865738 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.846894 22397 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:54.874382 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.877337 20729 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:54.837769 22455 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:54.892238 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.811434 22448 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.899245 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:54.899652 22309 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.001s
I20260709 12:21:54.900056 22512 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.900691 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:54.906597 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:54.910506 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.915148 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:54.915328 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.915865 20882 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:54.916980 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:54.917687 20882 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:54.918264 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.922317 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.922997 21428 raft_consensus.cc:2376] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3. Re-sending same reply.
I20260709 12:21:54.924947 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:54.927276 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:54.927392 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.928216 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.930213 21431 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:54.924283 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.937022 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.940770 21430 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:54.941720 21432 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:54.951995 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.952433 21435 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:54.952579 21432 raft_consensus.cc:2376] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3. Re-sending same reply.
I20260709 12:21:54.954872 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.955657 21432 raft_consensus.cc:2376] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3. Re-sending same reply.
I20260709 12:21:54.932257 22517 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:54.961295 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:54.904712 22455 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.962586 21431 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:21:54.963163 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 285249us
I20260709 12:21:54.939721 22397 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:54.966238 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:54.966732 20732 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:54.967633 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:21:54.959534 22614 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:21:54.909087 22309 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:54.977480 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.957091 22556 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.931329 22512 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:54.965013 22542 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.003s
I20260709 12:21:54.969301 22346 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:54.980810 22346 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:54.982939 22346 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:54.983438 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:54.985338 20880 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.986305 20880 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:21:54.986869 20880 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:54.962917 22517 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:54.964383 22615 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:21:54.999825 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.000880 22517 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.001266 22459 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.001926 22504 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:55.002372 22504 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:54.966889 22397 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:55.003037 22397 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:54.994378 22455 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.006383 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:55.007210 20731 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:54.993036 22617 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:55.008471 22617 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.008535 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 315) took 1209 ms. Trace:
I20260709 12:21:55.008786 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:55.008857 22617 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.008967 21434 rpcz_store.cc:276] 0709 12:21:53.799403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.799522 (+   119us) service_pool.cc:224] Handling call
0709 12:21:53.799797 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.583424 (+783627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:55.008240 (+424816us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b540050c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:55.008435 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776414080}
I20260709 12:21:55.010258 20802 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.009944 22617 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:54.998651 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.961318 22569 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:21:55.011976 22542 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:55.012218 22569 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:55.012651 22569 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.012526 22542 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.013134 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 611) took 1243 ms. Trace:
I20260709 12:21:55.013985 22309 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.013631 22569 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.014837 22542 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.015357 20727 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.999361 22616 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:55.002022 20732 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:55.017755 20728 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:55.018538 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.018811 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1490) took 1265 ms. Trace:
I20260709 12:21:54.996069 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.013477 21433 rpcz_store.cc:276] 0709 12:21:53.769841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.769944 (+   103us) service_pool.cc:224] Handling call
0709 12:21:53.773702 (+  3758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.576177 (+802475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:55.012928 (+436751us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b540058d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:55.013082 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1829079808}
I20260709 12:21:55.021960 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:55.025260 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.023399 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.027768 22309 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.028168 22448 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.029532 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:55.014928 22569 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:55.031942 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:55.032049 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.032373 21432 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:55.021862 20802 rpcz_store.cc:276] 0709 12:21:53.753649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.343626 (+589977us) service_pool.cc:224] Handling call
0709 12:21:54.374585 (+ 30959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:55.010637 (+636052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:55.011148 (+   511us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400127854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:55.018732 (+  7584us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461456128}
I20260709 12:21:55.037568 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.043264 21433 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:55.039535 21435 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:55.041201 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:55.045065 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:54.996562 20880 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.055445 20875 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:21:55.032696 22603 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.057228 22540 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:55.019624 21426 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:55.012573 22512 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.060531 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.062019 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.130934 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:55.031255 22617 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.134919 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:55.114257 20875 pending_rounds.cc:85] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 2
I20260709 12:21:55.047695 22488 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.113539 22448 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:55.118480 22616 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.008585 22393 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.048118 22579 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 12:21:55.060938 22518 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.048484 22455 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
W20260709 12:21:55.160225 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43967.2us
I20260709 12:21:55.060391 22461 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 12:21:55.059703 22600 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:55.143105 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:55.175976 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:55.176586 20730 raft_consensus.cc:606] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership
I20260709 12:21:55.177248 22393 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.177587 21432 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.178753 22517 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.180739 20880 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.181469 22517 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.147784 20875 pending_rounds.cc:107] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:21:55.183454 20875 raft_consensus.cc:2889] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:55.161517 20726 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:21:55.153491 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:21:55.145445 22361 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.538s	user 0.005s	sys 0.005s
I20260709 12:21:55.113329 22377 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 12:21:55.143561 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:55.148514 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86566us
I20260709 12:21:55.136654 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.163057 20775 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4.
I20260709 12:21:55.019357 22504 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.198482 22504 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:55.166432 20728 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:21:55.150391 22604 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.195807 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:21:55.197183 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43796.8us
I20260709 12:21:55.206183 20878 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 12:21:55.131888 22551 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:21:55.210438 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.162837 22448 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:55.219724 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:55.224874 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:55.235598 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:55.198484 22613 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.244360 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:21:55.228955 22621 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:21:55.246578 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:55.190704 22605 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.221s
I20260709 12:21:55.271780 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.228986 22600 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:55.204815 22361 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.279176 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:55.279923 20876 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.281365 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.239494 22517 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:55.237565 22568 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.288405 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.290418 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.237322 22360 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:21:55.294731 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 178962us
I20260709 12:21:55.260603 22615 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.278097 22600 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:55.257895 22604 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.276011 22626 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:55.282709 22618 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.277907 22560 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:21:55.290280 22348 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:55.294240 22517 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:55.288347 21427 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
W20260709 12:21:55.329231 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95010.9us
I20260709 12:21:55.330607 22361 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.299832 22605 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.297946 22459 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:55.301563 22600 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.301934 22626 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:55.347970 22603 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.327913 22542 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.355834 20732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37260
I20260709 12:21:55.357411 20732 raft_consensus.cc:606] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership
W20260709 12:21:55.352823 22501 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.516s	user 0.000s	sys 0.010s
W20260709 12:21:55.353443 22624 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:21:55.352095 22626 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:55.377488 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 87169.6us
I20260709 12:21:55.326691 22448 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:55.376080 22623 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:21:55.351902 21429 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:55.380851 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:55.381413 22348 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:55.357856 22542 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.385589 22616 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.385072 22348 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.393503 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:21:55.389423 22625 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:21:55.390530 22497 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.360494 22632 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:21:55.381870 22612 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:21:55.350618 22629 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 12:21:55.395023 22567 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.742s	user 0.003s	sys 0.009s
I20260709 12:21:55.401026 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.406364 20802 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.409457 22373 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.409741 20810 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.409897 22373 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.399334 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.401446 20809 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:55.394673 22449 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:55.398234 22293 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.399832 22361 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:21:55.419999 20801 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:55.423506 20731 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:55.422142 22500 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 12:21:55.424007 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1686) took 1812 ms (client timeout 1968 ms). Trace:
I20260709 12:21:55.410872 22373 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:55.424418 20808 rpcz_store.cc:269] 0709 12:21:53.611876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.667113 (+ 55237us) service_pool.cc:224] Handling call
0709 12:21:55.423813 (+1756700us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54003c6c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:55.423948 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096394496}
I20260709 12:21:55.425069 22373 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.426716 20809 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.404310 22616 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.436347 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1485) took 1774 ms. Trace:
I20260709 12:21:55.442873 20805 rpcz_store.cc:276] 0709 12:21:53.654069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.346689 (+692620us) service_pool.cc:224] Handling call
0709 12:21:54.347038 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.525458 (+178420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:54.525488 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:54.525525 (+    37us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:54.525554 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:55.427028 (+901474us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54003d68d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:55.427076 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:55.427933 (+   857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409751424}
I20260709 12:21:55.458885 22552 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.429005 22605 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:55.459414 22552 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:55.459671 22605 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.435300 22293 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:21:55.460636 22293 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:55.460497 22552 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.461118 22293 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.460758 22605 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.461921 22293 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.462944 22605 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.463066 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1498) took 1316 ms. Trace:
I20260709 12:21:55.463059 22293 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.463587 20807 rpcz_store.cc:276] 0709 12:21:54.146878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.384760 (+237882us) service_pool.cc:224] Handling call
0709 12:21:54.393206 (+  8446us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:21:54.393401 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.580722 (+187321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:54.580752 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:54.580795 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:54.580821 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:55.462842 (+882021us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54003d5ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:55.462888 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:55.463010 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445206144}
I20260709 12:21:55.464673 21431 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEARNER]: Advancing to term 4
I20260709 12:21:55.471328 20876 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:55.472970 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:55.420917 22449 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.474423 20877 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:55.475071 22449 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.475646 22449 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:55.476080 22449 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:55.476920 22449 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.426668 22397 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.480952 22397 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.482033 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:55.482302 21431 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:21:55.469139 22348 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.487161 20632 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 4, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 4 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.489007 20632 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 4, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 4 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:21:55.469384 22542 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.416565 22567 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.470772 22616 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.496501 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:55.497489 22519 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:21:55.498381 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.498759 22616 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.505699 20729 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:55.505924 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:55.521289 20878 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.507263 22504 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:55.507941 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:55.523408 20882 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.510169 20884 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.524971 20882 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:55.511525 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.512405 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.514302 22567 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.515049 22617 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:55.515467 22606 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.515937 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:55.516408 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.556772 20879 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:21:55.518535 20803 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:55.549618 22504 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.548652 22613 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:55.577241 22613 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:55.577592 22613 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:55.578189 22504 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.528578 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.529471 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.578689 22568 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:55.544912 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:55.582242 21431 raft_consensus.cc:2376] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4. Re-sending same reply.
I20260709 12:21:55.545939 21434 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:55.547547 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.548467 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.548089 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.540494 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:55.549160 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.583869 20884 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:21:55.560786 22552 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.556309 22455 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.564687 20809 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:55.569500 20807 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:21:55.548498 21427 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.423300 22517 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.527534 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.520504 21426 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:21:55.579797 20808 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:55.578548 22613 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.574908 20875 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:55.553903 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.540145 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.581121 22542 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.422559 22638 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.129s
I20260709 12:21:55.472596 22569 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 12:21:55.558660 22502 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.585309 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:55.595922 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:55.585963 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.628350 21434 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:55.628515 20804 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:21:55.586835 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:55.607612 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.618782 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:55.629328 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75565.6us
I20260709 12:21:55.607173 20884 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:21:55.615162 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.587148 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.607987 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.624368 20805 raft_consensus.cc:2368] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:21:55.631310 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:55.586238 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.634141 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:55.635102 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.639995 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:55.643447 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.644608 20805 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:21:55.608562 21429 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:55.649829 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.584223 22568 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:55.653100 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:55.608415 20880 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.661432 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.642567 22542 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.662279 20806 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:55.663669 22542 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.664857 22361 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:55.594331 22455 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.668218 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:55.646368 22502 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.668536 22455 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.631750 22558 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.646067 22293 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:21:55.670110 22502 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.654312 22631 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.671972 20726 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:55.651450 22508 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:21:55.642205 22346 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:21:55.615962 22373 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:21:55.677920 22373 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:55.678292 22373 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.646224 22617 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.654871 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:55.680681 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.656705 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.681876 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:55.682255 21434 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:21:55.682489 20732 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:55.646216 22632 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:55.685110 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:55.691565 22563 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:55.692137 22563 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.687726 22618 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:55.648408 22495 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.699066 22495 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:55.699575 22525 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:55.699985 22495 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.646416 22461 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.691483 22632 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:55.697135 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:55.692497 22563 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:55.656351 22579 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.609421 22626 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.711004 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.655372 22556 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:55.712646 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.708287 22346 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.681608 22600 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.714465 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.726058 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:21:55.695477 22613 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.659941 22540 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:21:55.679567 22570 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:21:55.679128 22373 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.708391 22569 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:55.716161 22502 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:21:55.759457 21434 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:55.723160 22618 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.693413 22617 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.761528 20734 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.762172 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.762507 20884 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:55.709796 22631 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.756798 22455 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:55.731500 22461 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.776166 20810 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:21:55.776489 20810 pending_rounds.cc:85] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:21:55.776697 20810 pending_rounds.cc:107] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:21:55.776880 20810 raft_consensus.cc:2889] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:55.722921 22558 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:21:55.763526 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.778544 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:55.781538 20884 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.782217 22600 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.763693 20875 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:21:55.783856 20884 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.789140 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.789745 20810 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.796746 21431 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:21:55.804327 21431 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:21:55.780213 22373 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:55.811883 20810 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:55.811903 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.813838 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:55.815080 21427 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 12:21:55.791501 22455 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:55.820719 21427 pending_rounds.cc:85] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 3
I20260709 12:21:55.821046 21427 pending_rounds.cc:107] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 12:21:55.803653 22631 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.831311 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.831766 20804 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.831581 22631 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.832111 20810 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:55.832783 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.815008 22393 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:21:55.833508 20804 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:55.833573 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.834513 20804 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.834717 20810 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:55.808007 22603 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:21:55.825006 20875 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:55.825421 22617 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.806692 22632 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.816715 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:55.836848 20810 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.837180 21426 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.828039 22601 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.838465 22601 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.839376 22601 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.828981 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.821462 22455 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.840858 20808 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:21:55.830425 22348 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.841440 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.833913 22393 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:21:55.836503 22652 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:21:55.768163 22568 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.842310 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.842301 20804 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.842693 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.803876 22461 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:21:55.821368 21427 raft_consensus.cc:2917] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 12:21:55.851933 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.853147 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.777817 22618 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:55.834029 22631 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:55.849571 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.858008 20808 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:55.858870 20881 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:55.851583 20810 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:55.831338 22626 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.859938 20808 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:55.846182 22449 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.870949 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.879180 22449 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.879827 20809 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:55.843741 22393 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:55.880467 20633 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 298) took 6396 ms. Trace:
I20260709 12:21:55.881207 22393 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.882477 20804 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:21:55.882560 22393 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.813311 22636 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:21:55.844277 22360 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.855538 22639 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.869971 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.888711 22508 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.872208 22512 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:21:55.870014 22601 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.883821 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.885219 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.890458 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:55.889073 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.872102 22495 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:55.892316 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.877607 21427 raft_consensus.cc:2889] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:55.894454 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.895236 22500 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.894932 22501 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:55.880856 20633 rpcz_store.cc:276] 0709 12:21:49.483632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.484440 (+   808us) service_pool.cc:224] Handling call
0709 12:21:55.880397 (+6395957us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:52.393817 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:52.452567 (+ 58750us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:52.452618 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:21:52.452636 (+    18us) tablet.cc:662] Decoding operations
0709 12:21:52.453097 (+   461us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:52.453178 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:52.453217 (+    39us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:52.453521 (+   304us) tablet.cc:701] Row locks acquired
0709 12:21:52.453563 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:21:52.453722 (+   159us) write_op.cc:270] Start()
0709 12:21:52.453854 (+   132us) write_op.cc:276] Timestamp: P: 1783599712453661 usec, L: 0
0709 12:21:52.453892 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:21:52.454611 (+   719us) log.cc:844] Serialized 599 byte log entry
0709 12:21:53.102002 (+647391us) op_driver.cc:464] REPLICATION: finished
0709 12:21:53.648545 (+546543us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1bb950ac 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e226d5ef0
0709 12:21:54.175907 (+527362us) write_op.cc:301] APPLY: starting
0709 12:21:54.176040 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:54.203074 (+ 27034us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:54.207802 (+  4728us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:54.351931 (+144129us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:54.378962 (+ 27031us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:21:54.379017 (+    55us) write_op.cc:312] APPLY: finished
0709 12:21:54.379315 (+   298us) log.cc:844] Serialized 23 byte log entry
0709 12:21:54.579049 (+199734us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:54.585096 (+  6047us) write_op.cc:454] Released schema lock
0709 12:21:54.586463 (+  1367us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:54.614075 (+ 27612us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:54.616072 (+  1997us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30171,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":1009493,"apply.run_cpu_time_us":19224,"apply.run_wall_time_us":435316,"num_ops":1,"prepare.queue_time_us":311984,"prepare.run_cpu_time_us":3277,"prepare.run_wall_time_us":78931,"raft.queue_time_us":167901,"raft.run_cpu_time_us":35579,"raft.run_wall_time_us":56310,"replication_time_us":632515,"spinlock_wait_cycles":27652352,"thread_start_us":1547955,"threads_started":4,"wal-append.queue_time_us":252823}]]}
I20260709 12:21:55.895740 22508 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:21:55.902352 22508 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:55.902685 22508 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:55.903482 22508 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.903694 22500 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.904755 21432 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.905977 22500 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.906342 20632 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 289) took 6189 ms. Trace:
I20260709 12:21:55.906643 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.906940 22603 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:21:55.907881 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 94148.6us
I20260709 12:21:55.909653 22570 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:55.910228 22570 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.909804 20630 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.900158 22567 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:21:55.913594 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:55.913537 20731 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:21:55.914198 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:55.915127 20805 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:21:55.904583 22508 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:55.918081 20810 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:55.924715 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:55.929292 20732 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:21:55.930637 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:55.932621 22497 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.933019 22497 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:55.921561 22501 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.934046 22497 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:55.938678 21435 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:55.940006 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:55.915558 20630 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.943775 20731 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:55.946244 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:55.946813 20735 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:55.893164 22618 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.943583 20630 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.950232 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.958719 20731 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:55.951834 20882 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.946791 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:55.960058 20735 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.941355 21435 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:55.965620 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:55.966869 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.970031 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:55.942070 22397 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:55.958467 22569 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:55.906733 20632 rpcz_store.cc:276] 0709 12:21:49.717343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.720096 (+  2753us) service_pool.cc:224] Handling call
0709 12:21:55.906284 (+6186188us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:55.542435 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:55.543209 (+   774us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:55.543256 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:21:55.543284 (+    28us) tablet.cc:662] Decoding operations
0709 12:21:55.543796 (+   512us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:55.543892 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:55.543933 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:21:55.544330 (+   397us) tablet.cc:701] Row locks acquired
0709 12:21:55.544355 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:21:55.544534 (+   179us) write_op.cc:270] Start()
0709 12:21:55.544654 (+   120us) write_op.cc:276] Timestamp: P: 1783599715544484 usec, L: 0
0709 12:21:55.544674 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:21:55.545342 (+   668us) log.cc:844] Serialized 929 byte log entry
0709 12:21:55.715087 (+169745us) op_driver.cc:464] REPLICATION: finished
0709 12:21:55.821205 (+106118us) write_op.cc:301] APPLY: starting
0709 12:21:55.821294 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:55.821406 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:55.822718 (+  1312us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:55.823207 (+   489us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:55.823363 (+   156us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:21:55.823404 (+    41us) write_op.cc:312] APPLY: finished
0709 12:21:55.823587 (+   183us) log.cc:844] Serialized 29 byte log entry
0709 12:21:55.823758 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:55.823839 (+    81us) write_op.cc:454] Released schema lock
0709 12:21:55.823883 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:55.824240 (+   357us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:55.824266 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":906578,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":96058,"apply.run_cpu_time_us":2104,"apply.run_wall_time_us":4137,"num_ops":2,"prepare.queue_time_us":50266,"prepare.run_cpu_time_us":3226,"prepare.run_wall_time_us":3226,"replication_time_us":170262,"thread_start_us":101326,"threads_started":2}]]}
I20260709 12:21:55.958940 22637 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.999683 22501 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:55.999766 21426 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:56.005982 22497 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:56.009569 22579 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.980671 20735 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.014258 20735 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.958298 22500 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:21:55.974282 22377 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:55.945140 22360 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:21:55.971217 22495 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:21:56.019676 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:55.980008 22449 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:55.994326 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:55.999589 22619 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:56.001037 22517 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.021225 20807 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:56.032471 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 228967us
I20260709 12:21:56.012820 22618 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:56.020291 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:56.042258 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.028136 22377 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.045219 22613 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:56.050377 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.051175 22613 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:56.060173 22613 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.061628 22613 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:56.066138 22619 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:56.083982 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.005642 22397 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:55.944747 22570 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:56.095224 22650 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:21:56.079633 22605 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.096053 22605 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.086792 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.091988 20735 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:56.051358 22632 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.100647 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.093544 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1511) took 1894 ms. Trace:
I20260709 12:21:56.051333 21428 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:56.094310 22449 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:56.049039 22377 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.086608 20801 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:56.044989 22563 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.156849 22563 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.158087 21434 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:56.101480 20735 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:56.047266 22618 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:56.116808 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:56.086046 22540 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:56.116416 20803 rpcz_store.cc:276] 0709 12:21:54.199697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.386742 (+187045us) service_pool.cc:224] Handling call
0709 12:21:54.387012 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:55.573125 (+1186113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.093259 (+520134us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003a0554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:56.093469 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2601399040}
I20260709 12:21:56.087661 22502 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.124573 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.044047 22517 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.212949 22517 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.131413 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:21:56.213939 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 82646.8us
I20260709 12:21:56.132507 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:56.147325 22605 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:56.152472 22397 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.101763 22569 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.219846 22373 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:56.067015 22619 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.112313 21430 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:56.186822 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:56.188413 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:56.222477 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90091.9us
I20260709 12:21:56.190730 22504 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.087597 22556 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:21:56.097378 22629 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:21:56.203981 22631 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:56.204468 22461 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:56.206326 22618 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:56.159719 22552 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.001s
I20260709 12:21:56.241557 20802 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:21:56.220431 22373 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:56.244743 20728 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 12:21:56.245359 20728 pending_rounds.cc:85] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 3
I20260709 12:21:56.245708 20728 pending_rounds.cc:107] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
W20260709 12:21:56.246443 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:56.248071 20728 raft_consensus.cc:2917] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 12:21:56.248692 20728 raft_consensus.cc:2889] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:56.235533 22631 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:21:56.202011 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 115294us
I20260709 12:21:56.208659 20727 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:56.214864 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.223724 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:56.227116 22504 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.229809 21432 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:21:56.225672 22397 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.158165 22449 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.263769 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:56.264640 20803 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:56.265827 22449 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.267098 22590 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.267292 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:56.268807 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:56.274151 20849 leader_election.cc:400] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:21:56.274480 20849 leader_election.cc:403] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:21:56.274369 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59568.5us
I20260709 12:21:56.276204 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.281492 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.285423 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:56.286600 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:56.288234 20733 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:21:56.282711 22397 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:56.289584 22397 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:56.289839 22637 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:56.290216 22637 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:56.290499 22397 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.294327 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.295289 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:56.266343 22377 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.295001 20728 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:56.300827 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:56.309999 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.311553 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:56.312134 22361 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.314004 22361 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.299227 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:56.283531 22629 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:21:56.280370 22552 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:56.281102 22556 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:56.323123 22461 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:56.324900 22461 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.325443 22461 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:56.325968 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.327132 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.327459 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:56.328212 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.328529 21435 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:56.329113 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:56.329754 21430 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:56.283407 22348 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:56.325088 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.267716 22590 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.336164 22590 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:56.336498 21426 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:56.328819 20735 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:56.345971 20735 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:21:56.341251 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.308635 22502 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:56.308028 22605 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.360713 22659 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:56.355695 22605 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.331370 22556 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:56.369544 20735 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:21:56.370606 21431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.003s	sys 0.022s
W20260709 12:21:56.371109 21431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.003s	sys 0.022s
I20260709 12:21:56.323356 22540 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:21:56.349989 22601 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:21:56.396091 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:21:56.291127 22637 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.398556 22637 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:56.389526 22506 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:56.371248 22556 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:56.369109 22459 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:21:56.358567 22629 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 12:21:56.423288 20681 auto_rebalancer.cc:893] Removing replace marker failed: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:21:56.423730 20681 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 7ccae903e8484f8d8c3af847e2c4c3d4 already in progress
W20260709 12:21:56.424023 20681 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 7ccae903e8484f8d8c3af847e2c4c3d4 already in progress
I20260709 12:21:56.330835 22631 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.353144 22373 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.406888 22638 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:21:56.488921 20730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.008s	sys 0.002s
W20260709 12:21:56.489530 20730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.009s	sys 0.002s
I20260709 12:21:56.498615 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:56.408665 22506 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:56.423027 22605 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:56.487520 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:56.517076 22377 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:56.524034 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:56.524730 21430 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:56.533033 22665 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:21:56.534900 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:21:56.536571 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:56.548678 22525 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:56.549400 22525 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:56.550503 22525 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.557610 22461 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.558971 22502 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:56.559825 22540 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:56.577657 22540 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:21:56.500759 22664 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.580530 22664 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.568001 22506 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.508625 22631 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:56.588590 22616 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.605378 22638 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:56.613714 22638 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.600672 22616 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.620540 22616 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:56.606170 21427 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:56.623606 22506 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:56.599123 22568 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:56.632579 22504 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:56.640355 21430 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:21:56.647989 22501 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.648583 22501 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.578955 22540 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.651436 22501 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:56.655459 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 1754) took 1638 ms. Trace:
I20260709 12:21:56.655969 20733 rpcz_store.cc:276] 0709 12:21:55.016864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.016993 (+   129us) service_pool.cc:224] Handling call
0709 12:21:55.481825 (+464832us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:21:56.655227 (+1173402us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54001d6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:56.655356 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1809309696}
I20260709 12:21:56.492976 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:56.667256 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:56.669975 22517 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:56.670554 22517 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.671377 22377 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.671855 22377 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.674315 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:21:56.679823 22567 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:56.595763 22664 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:56.680420 22567 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.684571 22377 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:56.687278 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:56.692384 22559 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.694587 22559 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.698673 20630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 269) took 4735 ms. Trace:
I20260709 12:21:56.706736 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:56.703910 22590 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:21:56.715271 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76808.8us
W20260709 12:21:56.718747 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64071.6us
I20260709 12:21:56.717288 22556 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.720300 20875 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:56.720330 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:56.721496 22563 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.722002 22563 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.722393 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.724261 22563 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:56.725019 22517 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.725746 22567 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:56.726321 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55169.3us
I20260709 12:21:56.727497 22459 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.728150 22459 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:56.730638 21433 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:56.733072 22590 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:21:56.737000 22309 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:56.739818 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:56.741925 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:56.730839 22459 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.756661 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:56.700835 20630 rpcz_store.cc:276] 0709 12:21:51.963008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.963264 (+   256us) service_pool.cc:224] Handling call
0709 12:21:56.698596 (+4735332us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:56.033743 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:56.059615 (+ 25872us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:56.059653 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:21:56.059669 (+    16us) tablet.cc:662] Decoding operations
0709 12:21:56.060321 (+   652us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:56.060379 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:56.060411 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:21:56.088188 (+ 27777us) tablet.cc:701] Row locks acquired
0709 12:21:56.097535 (+  9347us) write_op.cc:260] PREPARE: finished
0709 12:21:56.120136 (+ 22601us) write_op.cc:270] Start()
0709 12:21:56.155256 (+ 35120us) write_op.cc:276] Timestamp: P: 1783599716112376 usec, L: 0
0709 12:21:56.155284 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:21:56.155894 (+   610us) log.cc:844] Serialized 1365 byte log entry
0709 12:21:56.191584 (+ 35690us) op_driver.cc:464] REPLICATION: finished
0709 12:21:56.192095 (+   511us) write_op.cc:301] APPLY: starting
0709 12:21:56.192176 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:56.192317 (+   141us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:56.192431 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:56.192914 (+   483us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:56.193115 (+   201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:21:56.193163 (+    48us) write_op.cc:312] APPLY: finished
0709 12:21:56.193398 (+   235us) log.cc:844] Serialized 35 byte log entry
0709 12:21:56.193571 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:56.193683 (+   112us) write_op.cc:454] Released schema lock
0709 12:21:56.193733 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:56.194095 (+   362us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:56.194130 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12296,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":289,"apply.run_cpu_time_us":1952,"apply.run_wall_time_us":1952,"num_ops":3,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":3394,"prepare.run_wall_time_us":137530,"replication_time_us":36154}]]}
I20260709 12:21:56.753140 22637 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.764556 22348 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:56.778287 22348 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:56.753319 22559 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.761394 22561 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.778756 22348 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:56.763166 22606 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:56.790426 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:56.751252 22570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.062s	sys 0.070s
W20260709 12:21:56.806380 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50757.3us
I20260709 12:21:56.814664 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:56.771277 22567 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.830202 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:56.834590 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:56.834933 22618 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.835512 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:56.835651 22618 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.838266 22618 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:56.838423 21429 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:56.839321 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:56.839803 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:56.840013 20881 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:56.840420 20806 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:56.841248 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:56.843246 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:56.844748 20806 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:21:56.821564 22570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.062s	sys 0.070s
I20260709 12:21:56.850657 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:21:56.851675 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:56.854846 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:56.826865 22461 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:56.891752 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:21:56.899309 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48761us
I20260709 12:21:56.762488 22590 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.869972 22559 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:56.894387 20806 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 1.
I20260709 12:21:56.957619 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:56.895829 22455 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.008s	sys 0.039s
I20260709 12:21:56.901577 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:56.897368 22397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.007s	sys 0.027s
W20260709 12:21:56.869014 22569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.018s	sys 0.013s
I20260709 12:21:56.905128 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:56.958343 20735 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:21:56.910343 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.962404 20731 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
W20260709 12:21:56.965198 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60185.2us
I20260709 12:21:56.910888 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.968892 20727 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:56.787734 22637 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.969519 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.893778 22517 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:56.869235 22626 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:21:56.888626 22604 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.010s	sys 0.016s
I20260709 12:21:56.896340 22459 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:56.964622 22397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.007s	sys 0.027s
W20260709 12:21:56.912904 22619 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.000s	sys 0.049s
I20260709 12:21:56.982622 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:56.901377 22500 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.904086 22639 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.847113 22567 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:56.888828 22508 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:21:56.843941 22660 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:21:56.846781 22561 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 12:21:56.995019 21433 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:21:56.965649 22455 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.008s	sys 0.040s
W20260709 12:21:56.971104 22569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.018s	sys 0.013s
I20260709 12:21:56.965946 22559 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:56.993937 22604 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.010s	sys 0.016s
W20260709 12:21:56.959992 22579 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.006s	sys 0.022s
I20260709 12:21:56.978878 22616 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:56.912539 22566 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 12:21:56.981634 22517 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.053717 22517 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:56.993768 22293 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.061254 22293 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.061811 22517 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:56.978704 22563 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:56.990553 22626 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:57.004629 22619 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.000s	sys 0.049s
I20260709 12:21:57.049643 20729 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:21:57.063607 22563 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.066489 22563 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.040506 22397 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.069809 22397 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.071069 22517 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.072016 22397 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.075457 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1561) took 1126 ms. Trace:
I20260709 12:21:57.076452 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.076967 20726 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:57.075929 20879 rpcz_store.cc:276] 0709 12:21:55.949500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.949628 (+   128us) service_pool.cc:224] Handling call
0709 12:21:55.950043 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:55.950489 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:55.950530 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:55.964949 (+ 14419us) log.cc:844] Serialized 26 byte log entry
0709 12:21:56.424936 (+459987us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:56.424984 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.425054 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:57.075152 (+650098us) raft_consensus.cc:1685] finished
0709 12:21:57.075179 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:57.075366 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6373,"thread_start_us":408714,"threads_started":1,"wal-append.queue_time_us":983344}
I20260709 12:21:57.079581 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1765) took 1169 ms. Trace:
I20260709 12:21:57.080958 22293 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.080032 20730 rpcz_store.cc:276] 0709 12:21:55.909705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.911296 (+  1591us) service_pool.cc:224] Handling call
0709 12:21:55.911684 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:55.912042 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:55.912068 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:55.912554 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:21:56.135753 (+223199us) spinlock_profiling.cc:243] Waited 9.34 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:21:56.490153 (+354400us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:56.490217 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.490320 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:57.079243 (+588923us) raft_consensus.cc:1685] finished
0709 12:21:57.079269 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:57.079517 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33188,"raft.run_cpu_time_us":3157,"raft.run_wall_time_us":189505,"spinlock_wait_cycles":21488384,"thread_start_us":554744,"threads_started":1,"wal-append.queue_time_us":1045918}
I20260709 12:21:57.060694 22637 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.063637 22626 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:57.097151 22455 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:57.097869 22455 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.101195 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:57.103511 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:57.041124 22461 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:57.104836 20730 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:57.160929 22461 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.042348 22639 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.047411 22556 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:57.050606 22559 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:57.172281 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66890.1us
I20260709 12:21:57.165565 22455 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:57.095564 22579 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.006s	sys 0.022s
I20260709 12:21:57.183651 22613 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:57.184038 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:57.184719 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:57.106895 22309 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:57.242377 22600 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:57.243155 22600 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.163334 22508 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:57.244643 22309 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:57.243084 22461 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:57.275218 20878 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 331) took 1513 ms (client timeout 1575 ms). Trace:
W20260709 12:21:57.275635 20878 rpcz_store.cc:269] 0709 12:21:55.761843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.761948 (+   105us) service_pool.cc:224] Handling call
0709 12:21:57.274969 (+1513021us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:21:57.275133 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2946330880}
I20260709 12:21:57.276536 22567 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.277523 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1763) took 1368 ms. Trace:
I20260709 12:21:57.277894 20729 rpcz_store.cc:276] 0709 12:21:55.909191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.909300 (+   109us) service_pool.cc:224] Handling call
0709 12:21:55.909622 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.064385 (+1154763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.277295 (+212910us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000c49d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:57.277469 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2621229312}
I20260709 12:21:57.278812 22461 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:57.279572 22461 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:57.281229 22631 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.000s
I20260709 12:21:57.282091 22461 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:57.274887 22568 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.000s
I20260709 12:21:57.274422 22309 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.272819 22616 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.315433 22616 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.286751 22556 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:57.326969 22556 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.296022 22348 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.237815 22613 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.337378 22613 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:57.339699 20881 pending_rounds.cc:85] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 4
I20260709 12:21:57.340197 20881 pending_rounds.cc:107] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:21:57.340534 20881 raft_consensus.cc:2889] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:57.343451 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:57.351119 22626 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.352660 20734 raft_consensus.cc:2368] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
I20260709 12:21:57.355496 22626 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.307729 22639 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.309136 22508 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:21:57.410122 22552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.014s	sys 0.026s
I20260709 12:21:57.322363 21428 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:57.306847 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:21:57.461213 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 493218us
I20260709 12:21:57.461539 22705 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:57.297523 22600 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.462112 22705 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.356489 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:57.463054 20878 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:57.463986 22560 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.464533 22560 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:57.387439 20884 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.181s	user 0.023s	sys 0.044s
W20260709 12:21:57.397183 21434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.168s	user 0.028s	sys 0.039s
W20260709 12:21:57.467799 21434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.239s	user 0.029s	sys 0.040s
W20260709 12:21:57.399319 20850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.008s	sys 0.049s
W20260709 12:21:57.471446 20850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.008s	sys 0.049s
W20260709 12:21:57.471835 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.07477e+06us
W20260709 12:21:57.395915 22393 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.025s	sys 0.041s
W20260709 12:21:57.410985 20804 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.988s	user 0.013s	sys 0.027s
W20260709 12:21:57.510820 20804 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.014s	sys 0.027s
I20260709 12:21:57.386134 22613 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.306322 22606 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:57.386495 22377 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:57.385795 22567 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:21:57.311517 22638 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.697s	user 0.004s	sys 0.008s
W20260709 12:21:57.429644 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.159s	user 0.019s	sys 0.022s
W20260709 12:21:57.547091 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.019s	sys 0.022s
W20260709 12:21:57.547539 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.27739e+06us
W20260709 12:21:57.510392 22393 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.314s	user 0.026s	sys 0.041s
I20260709 12:21:57.441128 22508 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:57.458029 20808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.014s	sys 0.018s
W20260709 12:21:57.550923 20808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.015s	sys 0.019s
W20260709 12:21:57.459869 20803 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.010s	sys 0.031s
W20260709 12:21:57.461355 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 154609us
I20260709 12:21:57.332059 22348 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.335039 22556 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.462531 22705 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.593957 22560 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.596458 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.597388 20880 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:57.551348 22638 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:57.475430 20878 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:21:57.467406 20884 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.261s	user 0.023s	sys 0.044s
W20260709 12:21:57.457253 22525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.008s	sys 0.025s
I20260709 12:21:57.511785 22639 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:21:57.441147 20732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.401s	user 0.000s	sys 0.072s
I20260709 12:21:57.604701 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:21:57.607285 20732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.568s	user 0.001s	sys 0.072s
I20260709 12:21:57.552348 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:57.601444 22525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.008s	sys 0.026s
I20260709 12:21:57.466730 22512 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.551625 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 634) took 1323 ms. Trace:
I20260709 12:21:57.608798 22512 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.341027 22559 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.510598 22570 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:57.518431 22361 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.001s	sys 0.041s
I20260709 12:21:57.588945 22508 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.589232 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:57.589847 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.590169 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.591070 20882 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:57.552987 22377 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.601718 22638 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.603379 21432 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:57.605490 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:57.606532 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:21:57.619514 22361 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.002s	sys 0.041s
I20260709 12:21:57.603018 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:21:57.582649 20803 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.011s	sys 0.031s
I20260709 12:21:57.599159 20880 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:57.518577 22631 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:21:57.624518 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:57.518816 22552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.015s	sys 0.026s
I20260709 12:21:57.548130 22613 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.610757 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:21:57.637535 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.646886 20876 raft_consensus.cc:2368] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
I20260709 12:21:57.619578 22606 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:57.633057 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:57.618108 22556 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:57.647330 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.608916 21428 rpcz_store.cc:276] 0709 12:21:56.226376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.229931 (+  3555us) service_pool.cc:224] Handling call
0709 12:21:56.237384 (+  7453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:57.460659 (+1223275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.479626 (+ 18967us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:21:57.549285 (+ 69659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487445888}
I20260709 12:21:57.649147 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.658751 20876 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:21:57.602348 22567 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.652347 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:57.653236 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:57.663734 20877 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:21:57.640861 22615 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:57.615361 22616 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:57.649271 22346 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.022s	sys 0.029s
I20260709 12:21:57.618391 22639 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.691041 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.618257 22461 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:21:57.692276 20882 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:57.710413 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:57.680132 22569 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:57.686767 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:57.716471 21435 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:57.687644 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:57.686662 22346 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.022s	sys 0.029s
I20260709 12:21:57.680476 22348 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:21:57.715476 20809 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.010s	sys 0.005s
I20260709 12:21:57.717137 20729 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
W20260709 12:21:57.721383 20809 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.010s	sys 0.005s
I20260709 12:21:57.682323 21429 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
W20260709 12:21:57.719023 20810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.013s	sys 0.002s
I20260709 12:21:57.719323 20728 raft_consensus.cc:2368] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 3. Current term is 4.
W20260709 12:21:57.724782 20810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.013s	sys 0.002s
W20260709 12:21:57.712512 20802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.022s	user 0.015s	sys 0.009s
I20260709 12:21:57.621111 22570 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.658149 22556 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.711238 22615 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:21:57.715890 22569 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.684955 22664 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:21:57.725523 20802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.015s	sys 0.009s
W20260709 12:21:57.723978 22558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.383s	user 0.005s	sys 0.072s
I20260709 12:21:57.723250 22525 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.690248 22705 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.724841 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:57.723675 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:57.727249 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:57.725281 22348 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.736498 22568 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.738812 22569 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.760138 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:57.753638 20878 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:57.771301 20878 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:21:57.780702 21399 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.183s	user 0.015s	sys 0.015s
W20260709 12:21:57.781131 21399 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.015s	sys 0.015s
W20260709 12:21:57.781442 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.18806e+06us
I20260709 12:21:57.753067 22631 raft_consensus.cc:1064] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
W20260709 12:21:57.780696 20805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.009s	sys 0.054s
W20260709 12:21:57.860996 20805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.159s	user 0.009s	sys 0.054s
W20260709 12:21:57.871783 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 83322.4us
I20260709 12:21:57.745750 22570 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.746726 22512 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:57.882711 20878 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:57.894511 20878 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:57.754992 22616 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.752667 22556 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:57.871337 22461 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.751921 22559 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:57.751389 22639 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.888315 22638 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:57.897778 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:57.991827 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:57.907914 22348 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:57.909725 20698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.007s	sys 0.017s
W20260709 12:21:57.912774 22604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.020s	sys 0.030s
W20260709 12:21:57.742733 22558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.411s	user 0.005s	sys 0.073s
I20260709 12:21:57.753832 22613 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.888563 22377 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:57.755748 22525 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:58.005972 20733 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.008s	sys 0.025s
W20260709 12:21:58.006667 20733 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.008s	sys 0.025s
I20260709 12:21:57.935699 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:58.010191 21432 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:21:58.010390 20727 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.015s	sys 0.017s
W20260709 12:21:58.010953 20727 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.015s	sys 0.018s
W20260709 12:21:57.889389 22631 raft_consensus.cc:1068] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Unable to promote non-voter 50c1ab13351d4427b836fbb449c6d7bb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:21:57.889504 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:57.998752 22604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.020s	sys 0.030s
W20260709 12:21:57.986289 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.000s	sys 0.030s
W20260709 12:21:57.986908 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.778367 22568 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:57.925972 20807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.230s	user 0.005s	sys 0.065s
W20260709 12:21:58.022991 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.000s	sys 0.030s
W20260709 12:21:57.997381 20698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.007s	sys 0.017s
I20260709 12:21:57.986591 22293 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.893872 22615 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:57.772661 22455 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:58.041707 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:57.890239 20801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.128s	user 0.013s	sys 0.031s
W20260709 12:21:57.995344 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.655s	user 0.008s	sys 0.027s
W20260709 12:21:58.061920 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.722s	user 0.009s	sys 0.028s
W20260709 12:21:58.062376 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.72227e+06us
W20260709 12:21:57.922333 20806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.961s	user 0.015s	sys 0.010s
W20260709 12:21:58.063391 20806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.019s	sys 0.010s
W20260709 12:21:58.030506 21430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.002s	sys 0.041s
I20260709 12:21:58.006184 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:58.025005 20807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.329s	user 0.005s	sys 0.069s
I20260709 12:21:57.925446 22461 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:58.065480 21430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.225s	user 0.002s	sys 0.041s
I20260709 12:21:58.070930 20878 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 3 index: 6. (term mismatch)
I20260709 12:21:58.071084 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:21:58.075095 20878 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:21:58.000175 22638 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:58.064322 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1541) took 1314 ms. Trace:
I20260709 12:21:58.026185 22567 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.075888 20878 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:21:58.076972 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.088451 20878 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:58.076864 20806 rpcz_store.cc:276] 0709 12:21:56.749983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.960122 (+210139us) service_pool.cc:224] Handling call
0709 12:21:56.960440 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:58.063859 (+1103419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:58.063905 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:58.063967 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.064012 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.064102 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.064256 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2047,"raft.queue_time_us":41844,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":31335,"thread_start_us":1102941,"threads_started":1}
I20260709 12:21:58.082737 21432 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:21:58.060379 22677 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.069403 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.069145 22455 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:58.061798 20801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.426s	user 0.014s	sys 0.032s
W20260709 12:21:58.043641 22551 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.399s	user 0.037s	sys 0.046s
I20260709 12:21:58.020083 22377 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 12:21:58.098173 20804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.004s	sys 0.012s
I20260709 12:21:58.098866 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:58.123867 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:58.100373 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.108152 20728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:21:58.127130 20728 raft_consensus.cc:606] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
W20260709 12:21:58.129562 20804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.005s	sys 0.012s
I20260709 12:21:58.130640 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1535) took 1717 ms. Trace:
W20260709 12:21:58.100682 22540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.346s	user 0.015s	sys 0.021s
W20260709 12:21:58.095117 22373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.421s	user 0.024s	sys 0.055s
W20260709 12:21:58.060166 22590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.010s	sys 0.025s
I20260709 12:21:58.134886 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:58.136161 20806 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:58.131098 20804 rpcz_store.cc:276] 0709 12:21:56.412865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.413093 (+   228us) service_pool.cc:224] Handling call
0709 12:21:56.413492 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.413751 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:21:57.511325 (+1097574us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.130226 (+618901us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:58.130288 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.130339 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.130423 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.130578 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1334321,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":1707159,"threads_started":2}
I20260709 12:21:58.158104 20728 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:21:58.158996 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:58.159494 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:21:58.113979 21426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.491s	user 0.012s	sys 0.047s
W20260709 12:21:58.147446 20879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.010s	sys 0.028s
W20260709 12:21:58.164824 20879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.010s	sys 0.028s
W20260709 12:21:58.147832 20881 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.019s	sys 0.020s
I20260709 12:21:58.040841 22293 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:58.150456 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:21:58.165920 20730 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:58.096606 22637 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:58.152040 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:21:58.118180 22551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.474s	user 0.037s	sys 0.046s
I20260709 12:21:58.155418 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:21:58.174930 20729 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:58.133220 22348 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:58.148985 22540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.394s	user 0.016s	sys 0.021s
I20260709 12:21:58.112205 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:58.164253 20728 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:21:58.157526 22373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.484s	user 0.024s	sys 0.056s
I20260709 12:21:58.095498 22682 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:21:58.168804 20735 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:58.118782 22618 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.163831 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:58.160621 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:58.128674 20875 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:58.182750 20806 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:58.095976 22631 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:21:58.130452 22600 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:58.133674 22567 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:58.101259 22677 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:58.166818 20881 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.020s	sys 0.020s
W20260709 12:21:58.165870 22590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.010s	sys 0.025s
W20260709 12:21:58.178339 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66261.9us
I20260709 12:21:58.214236 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:58.148334 22660 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.001s	sys 0.036s
W20260709 12:21:58.148125 22579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.018s	sys 0.018s
I20260709 12:21:58.266607 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:21:58.203550 20729 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:21:58.200692 22638 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.197088 22455 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:58.164556 21426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.555s	user 0.012s	sys 0.048s
W20260709 12:21:58.239874 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.215411 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:58.197504 22309 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:58.187589 22348 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:58.259732 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
W20260709 12:21:58.287964 22579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.019s	sys 0.019s
I20260709 12:21:58.260275 20804 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
W20260709 12:21:58.264779 22660 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.002s	sys 0.041s
I20260709 12:21:58.195864 22377 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:58.288725 20806 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:58.200407 22373 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:58.201422 22525 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:58.177313 22683 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 12:21:58.303174 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 87858.7us
I20260709 12:21:58.418471 20775 leader_election.cc:400] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
I20260709 12:21:58.418807 20775 leader_election.cc:403] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:21:58.212888 22600 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:58.287474 22705 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:58.301638 22293 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:58.293922 20726 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:58.213840 22567 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:58.193094 22618 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:58.416416 20730 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:21:58.424959 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:58.442091 20808 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.027s	sys 0.004s
I20260709 12:21:58.446492 20806 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:21:58.447475 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:21:58.448254 20884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.018s	sys 0.010s
W20260709 12:21:58.448655 20884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.018s	sys 0.010s
W20260709 12:21:58.425470 21434 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.014s	sys 0.013s
I20260709 12:21:58.418303 22568 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:58.459170 20809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.020s	sys 0.010s
W20260709 12:21:58.459784 20809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.021s	sys 0.010s
W20260709 12:21:58.462489 20808 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.028s	sys 0.004s
W20260709 12:21:58.462507 21434 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.014s	sys 0.013s
I20260709 12:21:58.463495 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:58.423094 22525 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:21:58.424122 22393 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.008s	sys 0.022s
I20260709 12:21:58.403102 22377 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:58.487622 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 341) took 2259 ms. Trace:
I20260709 12:21:58.426695 22703 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:58.439443 22559 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:21:58.465481 22361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.017s	sys 0.019s
W20260709 12:21:58.448189 21398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.016s	sys 0.011s
I20260709 12:21:58.417601 22373 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:58.478910 22393 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.009s	sys 0.022s
I20260709 12:21:58.496609 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 633) took 2251 ms. Trace:
I20260709 12:21:58.597016 21434 rpcz_store.cc:276] 0709 12:21:56.225929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.226081 (+   152us) service_pool.cc:224] Handling call
0709 12:21:56.227522 (+  1441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.227950 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.227997 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.228644 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 12:21:57.468316 (+1239672us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.465254 (+996938us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.477343 (+ 12089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.477439 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.477498 (+    59us) raft_consensus.cc:1685] finished
0709 12:21:58.477517 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.477698 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11011,"prepare.queue_time_us":225986,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":16804,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":18256,"thread_start_us":2151723,"threads_started":2,"wal-append.queue_time_us":1662663}
I20260709 12:21:58.614820 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:58.647358 20804 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:58.476025 22525 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:21:58.501914 22361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.017s	sys 0.019s
I20260709 12:21:58.474464 22556 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:58.487710 22508 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:58.492901 22703 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:58.595974 21398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.016s	sys 0.012s
I20260709 12:21:58.501498 22559 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.492656 22293 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:21:58.624904 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.007s	sys 0.035s
W20260709 12:21:58.748303 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.007s	sys 0.035s
W20260709 12:21:58.748694 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.07457e+06us
I20260709 12:21:58.749581 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:21:58.631085 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.007s	sys 0.030s
W20260709 12:21:58.750384 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.037s	user 0.007s	sys 0.030s
W20260709 12:21:58.751971 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.037s	sys 0.019s
W20260709 12:21:58.752733 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.038s	sys 0.020s
I20260709 12:21:58.588902 22677 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:58.590265 22604 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:21:58.634948 22552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.014s	sys 0.039s
W20260709 12:21:58.712932 21431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.036s	sys 0.033s
W20260709 12:21:58.630604 22689 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:21:58.712857 22525 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:58.518179 22704 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
W20260709 12:21:58.630879 22675 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:21:58.729889 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:58.616968 22686 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:58.713065 22346 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.006s	sys 0.037s
W20260709 12:21:58.715422 21429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.021s	sys 0.016s
W20260709 12:21:58.754484 22551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.002s	sys 0.004s
W20260709 12:21:58.756363 21431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.036s	sys 0.033s
I20260709 12:21:58.739460 22703 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:58.762806 20733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.037s	sys 0.040s
W20260709 12:21:58.776782 20733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.037s	sys 0.040s
I20260709 12:21:58.777844 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1519) took 1875 ms. Trace:
W20260709 12:21:58.763211 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.019s	sys 0.024s
W20260709 12:21:58.755354 22701 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.531s	user 0.000s	sys 0.001s
W20260709 12:21:58.760653 22552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.014s	sys 0.040s
I20260709 12:21:58.745991 22508 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:58.749426 22559 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:58.796205 22700 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
W20260709 12:21:58.769832 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:58.763476 22346 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.007s	sys 0.037s
I20260709 12:21:58.750633 22348 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.778298 20733 rpcz_store.cc:276] 0709 12:21:56.907581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.907743 (+   162us) service_pool.cc:224] Handling call
0709 12:21:56.968883 (+ 61140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.969120 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:21:58.007282 (+1038162us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.777362 (+770080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:58.777445 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:58.777500 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.777643 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.777779 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":578235,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":82,"thread_start_us":1783160,"threads_started":2}
W20260709 12:21:58.780059 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.020s	sys 0.024s
I20260709 12:21:58.759006 22293 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:58.766278 21429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.022s	sys 0.016s
W20260709 12:21:58.795207 22715 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:21:58.762607 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:21:58.768095 20805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.009s	sys 0.033s
W20260709 12:21:58.778250 22699 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.482s	user 0.000s	sys 0.001s
W20260709 12:21:58.776036 22697 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.480s	user 0.000s	sys 0.001s
W20260709 12:21:58.868338 20805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.010s	sys 0.033s
W20260709 12:21:58.769078 22551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.003s	sys 0.004s
W20260709 12:21:58.869567 20727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.040s	sys 0.051s
I20260709 12:21:58.759630 22604 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:58.873664 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.008s	sys 0.082s
I20260709 12:21:58.764509 22677 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
I20260709 12:21:58.880870 22570 raft_consensus.cc:993] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
W20260709 12:21:58.877025 20727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.041s	sys 0.051s
W20260709 12:21:58.883729 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.008s	sys 0.082s
W20260709 12:21:58.889720 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 756115us
W20260709 12:21:58.870525 22707 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.455s	user 0.000s	sys 0.001s
I20260709 12:21:58.891327 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:58.889206 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1721) took 2184 ms. Trace:
I20260709 12:21:58.894909 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 243) took 1899 ms. Trace:
I20260709 12:21:58.896034 20810 rpcz_store.cc:276] 0709 12:21:56.691768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.691910 (+   142us) service_pool.cc:224] Handling call
0709 12:21:56.692275 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.692523 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.692570 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.699728 (+  7158us) log.cc:844] Serialized 26 byte log entry
0709 12:21:57.725377 (+1025649us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:57.725424 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.725487 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.876191 (+1150704us) raft_consensus.cc:1685] finished
0709 12:21:58.876212 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.876372 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1013488,"threads_started":1,"wal-append.queue_time_us":1478546}
I20260709 12:21:58.901757 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 283) took 1623 ms. Trace:
I20260709 12:21:58.902143 20810 rpcz_store.cc:276] 0709 12:21:57.278680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.900938 (+1622258us) service_pool.cc:224] Handling call
0709 12:21:58.901243 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:58.901433 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:58.901464 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:58.901509 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.901535 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.901592 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.901701 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:58.897143 20727 rpcz_store.cc:276] 0709 12:21:56.995517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.995677 (+   160us) service_pool.cc:224] Handling call
0709 12:21:56.996039 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.996283 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:21:58.019479 (+1023196us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.894587 (+875108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:58.894639 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:58.894682 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.894744 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.894855 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4976,"prepare.queue_time_us":593194,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"thread_start_us":1854242,"threads_started":2}
W20260709 12:21:58.906563 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.016s	sys 0.028s
W20260709 12:21:58.908573 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.016s	sys 0.028s
I20260709 12:21:58.910090 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.915547 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1583) took 1576 ms. Trace:
W20260709 12:21:58.894219 22694 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.488s	user 0.001s	sys 0.001s
W20260709 12:21:58.909940 21433 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.393s	user 0.021s	sys 0.062s
I20260709 12:21:58.766903 22686 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:58.920125 21433 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.404s	user 0.021s	sys 0.064s
I20260709 12:21:58.916014 20881 rpcz_store.cc:276] 0709 12:21:57.338827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.339024 (+   197us) service_pool.cc:224] Handling call
0709 12:21:57.339467 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.341430 (+  1963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:57.341479 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:57.341956 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.211832 (+869876us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:58.211891 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.211992 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.910009 (+698017us) raft_consensus.cc:1685] finished
0709 12:21:58.910027 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.915484 (+  5457us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":869538,"threads_started":1,"wal-append.queue_time_us":1262737}
I20260709 12:21:58.903244 22705 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.899518 22348 raft_consensus.cc:1081] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:21:58.928718 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1720) took 2237 ms. Trace:
I20260709 12:21:58.907208 22677 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:58.937397 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.940070 20882 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:58.938071 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1719) took 2248 ms. Trace:
I20260709 12:21:58.940649 20802 rpcz_store.cc:276] 0709 12:21:56.689145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.689334 (+   189us) service_pool.cc:224] Handling call
0709 12:21:56.689657 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.689973 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.689997 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.690415 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 12:21:57.739770 (+1049355us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:57.739812 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.739874 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.937778 (+1197904us) raft_consensus.cc:1685] finished
0709 12:21:58.937807 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.937981 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4093,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":1048832,"threads_started":1,"wal-append.queue_time_us":1486976}
I20260709 12:21:58.929632 20807 rpcz_store.cc:276] 0709 12:21:56.691068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.691286 (+   218us) service_pool.cc:224] Handling call
0709 12:21:56.694740 (+  3454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.695200 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.695242 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.695817 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.065191 (+1369374us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:58.065248 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.065335 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.928538 (+863203us) raft_consensus.cc:1685] finished
0709 12:21:58.928557 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.928666 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5370,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":4970,"thread_start_us":1368932,"threads_started":1,"wal-append.queue_time_us":1748985}
W20260709 12:21:58.950057 20773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.021s	sys 0.035s
W20260709 12:21:58.950429 20773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.021s	sys 0.035s
W20260709 12:21:58.952627 22716 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:21:58.952978 22604 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.954437 22686 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:58.955446 22714 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:21:58.955857 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 276) took 2262 ms. Trace:
W20260709 12:21:58.956664 22569 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.163s	user 0.007s	sys 0.004s
I20260709 12:21:58.956831 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1581) took 1700 ms. Trace:
I20260709 12:21:58.957155 22569 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:58.957270 20879 rpcz_store.cc:276] 0709 12:21:57.256898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.257046 (+   148us) service_pool.cc:224] Handling call
0709 12:21:57.257418 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.257681 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:57.257717 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:57.258270 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.165281 (+907011us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.165318 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.165385 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.956589 (+791204us) raft_consensus.cc:1685] finished
0709 12:21:58.956616 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.956768 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":906575,"threads_started":1,"wal-append.queue_time_us":1329462}
I20260709 12:21:58.959304 20734 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:21:58.960006 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 352) took 1301 ms. Trace:
I20260709 12:21:58.960135 22723 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:58.960767 22723 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:58.961163 22712 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:21:58.962216 22726 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:58.962842 22293 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.919677 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.945055 22705 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:21:58.965045 22704 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:58.956300 20805 rpcz_store.cc:276] 0709 12:21:56.693458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.693608 (+   150us) service_pool.cc:224] Handling call
0709 12:21:56.696600 (+  2992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.700609 (+  4009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.700670 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.702189 (+  1519us) log.cc:844] Serialized 26 byte log entry
0709 12:21:56.728039 (+ 25850us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:21:57.861447 (+1133408us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:57.897901 (+ 36454us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.875018 (+977117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.875124 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.955506 (+ 80382us) raft_consensus.cc:1685] finished
0709 12:21:58.955616 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.955769 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15729,"prepare.queue_time_us":9727,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":202570,"raft.run_cpu_time_us":2683,"raft.run_wall_time_us":41944,"spinlock_wait_cycles":8835968,"thread_start_us":2123458,"threads_started":2,"wal-append.queue_time_us":1485991}
I20260709 12:21:58.965421 22704 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:21:58.966120 22705 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.966290 22704 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:58.967724 22693 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 12:21:58.968129 20698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.000s	sys 0.023s
I20260709 12:21:58.968003 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:58.968453 20698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.000s	sys 0.023s
I20260709 12:21:58.968663 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.969074 22705 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:58.969312 22710 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:21:58.969626 22705 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:58.969985 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 523) took 2367 ms. Trace:
W20260709 12:21:58.971479 22668 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.931s	user 0.012s	sys 0.010s
I20260709 12:21:58.970480 21426 rpcz_store.cc:276] 0709 12:21:56.602734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.607489 (+  4755us) service_pool.cc:224] Handling call
0709 12:21:56.607956 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.608440 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.608495 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.609127 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.376533 (+1767406us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.402817 (+ 26284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.418547 (+ 15730us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.969738 (+551191us) raft_consensus.cc:1685] finished
0709 12:21:58.969769 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.969916 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3310,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":1688492,"threads_started":1,"wal-append.queue_time_us":2139837}
I20260709 12:21:58.972675 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1766) took 3062 ms. Trace:
I20260709 12:21:58.963758 22393 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.977193 22393 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:58.979138 22309 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.000s
I20260709 12:21:58.973414 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.979257 22393 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:21:58.979622 22711 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 12:21:58.979976 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 348) took 1194 ms. Trace:
I20260709 12:21:58.980334 22393 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.980435 20883 rpcz_store.cc:276] 0709 12:21:57.785659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.914362 (+128703us) service_pool.cc:224] Handling call
0709 12:21:58.973260 (+1058898us) spinlock_profiling.cc:243] Waited 9.61 ms on lock 0x7b5800003698. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:21:58.979893 (+  6633us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22099200}
W20260709 12:21:58.974318 22702 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 12:21:58.973058 20732 rpcz_store.cc:276] 0709 12:21:55.909988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.931797 (+ 21809us) service_pool.cc:224] Handling call
0709 12:21:55.932205 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:55.994599 (+ 62394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:55.994623 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.015013 (+ 20390us) log.cc:844] Serialized 26 byte log entry
0709 12:21:57.607693 (+1592680us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:57.607739 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.607808 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.970873 (+1363065us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:21:58.972356 (+  1483us) raft_consensus.cc:1685] finished
0709 12:21:58.972381 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.972575 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48551,"raft.queue_time_us":68941,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":116,"spinlock_wait_cycles":213759872,"thread_start_us":1567894,"threads_started":1,"wal-append.queue_time_us":1894647}
I20260709 12:21:58.967659 22373 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.971767 22705 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.962945 22377 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:58.922749 22560 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.982677 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 344) took 2148 ms. Trace:
I20260709 12:21:58.974287 22723 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:58.960539 21429 rpcz_store.cc:276] 0709 12:21:57.658619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.748763 (+ 90144us) service_pool.cc:224] Handling call
0709 12:21:57.749128 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.749381 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:57.749411 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:57.749935 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.876494 (+1126559us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:58.890490 (+ 13996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.900323 (+  9833us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.959735 (+ 59412us) raft_consensus.cc:1685] finished
0709 12:21:58.959754 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.959902 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1115881,"threads_started":1,"wal-append.queue_time_us":1208200}
I20260709 12:21:58.983065 21430 rpcz_store.cc:276] 0709 12:21:56.833825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.839342 (+  5517us) service_pool.cc:224] Handling call
0709 12:21:56.839695 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.840106 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.840136 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.840599 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.066000 (+1225401us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.095094 (+ 29094us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.096429 (+  1335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.096684 (+   255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.982487 (+885803us) raft_consensus.cc:1685] finished
0709 12:21:58.982508 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.982624 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18976,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":2256,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":6326,"thread_start_us":1225018,"threads_started":1,"wal-append.queue_time_us":1588990}
W20260709 12:21:58.753063 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 943764us
I20260709 12:21:58.980883 22393 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:58.986188 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:21:59.163424 20808 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.008s	sys 0.034s
W20260709 12:21:59.560883 20808 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.094s	user 0.016s	sys 0.040s
I20260709 12:21:59.561568 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1537) took 3135 ms. Trace:
W20260709 12:21:58.982960 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.011s	sys 0.027s
I20260709 12:21:59.562044 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:59.089044 20809 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.000s	sys 0.011s
I20260709 12:21:59.563041 21430 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:21:59.563279 20809 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.007s	sys 0.017s
I20260709 12:21:59.043565 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:59.613817 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:58.944578 22677 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:59.052372 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:21:59.038471 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
W20260709 12:21:59.142970 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.000s	sys 0.011s
W20260709 12:21:59.562534 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.466s	user 0.012s	sys 0.040s
W20260709 12:21:58.949721 20803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.263s	user 0.003s	sys 0.035s
W20260709 12:21:59.349969 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.000s	sys 0.023s
W20260709 12:21:59.595872 21427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.394s	user 0.008s	sys 0.018s
W20260709 12:21:58.949930 22615 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.006s	sys 0.033s
I20260709 12:21:59.083269 22618 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:58.962814 22726 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:59.354645 21432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.000s	sys 0.024s
I20260709 12:21:59.644315 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:59.396399 22703 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.596s	user 0.007s	sys 0.002s
W20260709 12:21:59.566079 21431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.016s	sys 0.007s
I20260709 12:21:59.645802 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:21:59.655774 21431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.016s	sys 0.017s
I20260709 12:21:59.656582 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 349) took 2001 ms. Trace:
I20260709 12:21:59.613325 22309 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.639130 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:59.614025 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.010s	sys 0.023s
I20260709 12:21:59.658531 21430 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:21:59.658900 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.011s	sys 0.023s
W20260709 12:21:59.638340 21427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.429s	user 0.008s	sys 0.018s
W20260709 12:21:59.312992 22717 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
W20260709 12:21:59.235620 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.632s (SENT) [suppressed 23 similar messages]
W20260709 12:21:59.398204 22660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.006s	sys 0.012s
I20260709 12:21:58.973526 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1533) took 2623 ms. Trace:
W20260709 12:21:59.624253 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.52805e+06us
I20260709 12:21:59.672345 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:21:59.624817 20881 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:21:59.625651 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.003s	sys 0.023s
I20260709 12:21:59.674229 20881 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:59.674319 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.24932e+06us
W20260709 12:21:59.613705 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.011s	sys 0.016s
W20260709 12:21:59.675854 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.011s	sys 0.016s
W20260709 12:21:59.676167 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 913552us
I20260709 12:21:59.677071 21400 leader_election.cc:400] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 3. Current term is 4.
I20260709 12:21:59.677345 21400 leader_election.cc:403] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:21:59.627543 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.006s	sys 0.026s
W20260709 12:21:59.677934 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 928338us
I20260709 12:21:59.679121 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:59.561939 20808 rpcz_store.cc:276] 0709 12:21:56.426414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.426821 (+   407us) service_pool.cc:224] Handling call
0709 12:21:56.427251 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.427704 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.427744 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.486856 (+ 59112us) log.cc:844] Serialized 26 byte log entry
0709 12:21:57.551320 (+1064464us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.463006 (+911686us) log.cc:844] Serialized 14 byte log entry
0709 12:21:59.561279 (+1098273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.561355 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:59.561396 (+    41us) raft_consensus.cc:1685] finished
0709 12:21:59.561407 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:59.561517 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20228,"prepare.queue_time_us":197562,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":465861,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":74469,"thread_start_us":2941726,"threads_started":3,"wal-append.queue_time_us":1395228}
I20260709 12:21:59.679586 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:59.038153 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:59.385152 22740 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:59.524246 22361 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:59.628867 22677 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.612224 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1717) took 2923 ms. Trace:
I20260709 12:21:59.234542 22569 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:59.612735 22552 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:21:59.655288 21432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.000s	sys 0.042s
I20260709 12:21:59.695503 20881 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:59.393666 22559 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.000s
I20260709 12:21:59.701233 22559 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:59.702311 20881 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:59.703328 22559 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:59.704133 22559 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.704751 22559 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:59.196583 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.656893 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:21:59.627327 20803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.941s	user 0.017s	sys 0.035s
I20260709 12:21:59.711445 21435 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:21:59.614643 20882 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:21:59.711665 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1534) took 3360 ms. Trace:
I20260709 12:21:58.993674 22560 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:59.712795 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:59.712118 20803 rpcz_store.cc:276] 0709 12:21:56.351022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.353499 (+  2477us) service_pool.cc:224] Handling call
0709 12:21:56.394040 (+ 40541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.408477 (+ 14437us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:21:57.651683 (+1243206us) log.cc:844] Serialized 14 byte log entry
0709 12:21:59.711243 (+2059560us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:59.711295 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:59.711350 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.711440 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:59.711591 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1316979,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"thread_start_us":3156760,"threads_started":2,"wal-append.queue_time_us":1555425}
I20260709 12:21:59.052352 22732 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:59.613703 22293 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:59.714354 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:21:59.714757 22732 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:59.715288 22732 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:59.664644 21429 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:59.716041 22552 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.382236 22723 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.047490 22704 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:59.719275 22560 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:59.721329 22723 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:59.728991 22723 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:21:59.566480 22616 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:59.671566 22660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.269s	user 0.006s	sys 0.018s
W20260709 12:21:59.672132 22568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.023s	sys 0.014s
I20260709 12:21:59.696687 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:59.697645 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:59.662159 22618 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.656939 21431 rpcz_store.cc:276] 0709 12:21:57.654838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.655284 (+   446us) service_pool.cc:224] Handling call
0709 12:21:57.688434 (+ 33150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.688834 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:57.688874 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:57.711336 (+ 22462us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.769271 (+1057935us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:58.794654 (+ 25383us) log.cc:844] Serialized 14 byte log entry
0709 12:21:59.656161 (+861507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.656247 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:59.656306 (+    59us) raft_consensus.cc:1685] finished
0709 12:21:59.656326 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:59.656488 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25631,"prepare.run_cpu_time_us":464,"prepare.run_wall_time_us":2137,"raft.queue_time_us":275646,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":57477,"thread_start_us":1897627,"threads_started":2,"wal-append.queue_time_us":1235272}
I20260709 12:21:59.714906 22293 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:21:59.656111 22615 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.568s	user 0.025s	sys 0.039s
I20260709 12:21:59.715092 21435 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:59.733991 20881 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:59.729904 20809 rpcz_store.cc:276] 0709 12:21:56.688565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.691170 (+  2605us) service_pool.cc:224] Handling call
0709 12:21:56.703275 (+ 12105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.703508 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:21:57.721849 (+1018341us) log.cc:844] Serialized 14 byte log entry
0709 12:21:59.611870 (+1890021us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:59.611928 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:59.611978 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.612043 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:59.612167 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1045173,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":459941,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":11574,"spinlock_wait_cycles":12870016,"thread_start_us":2852298,"threads_started":3,"wal-append.queue_time_us":1217503}
I20260709 12:21:59.680768 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:59.671056 22567 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:59.660475 22703 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.689117 22742 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:59.665968 22309 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:59.696414 20880 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:21:59.746887 21430 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:21:59.751565 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:21:59.736129 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 526) took 2221 ms. Trace:
W20260709 12:21:59.735214 21398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.029s	sys 0.051s
I20260709 12:21:59.698422 22740 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:59.795527 21434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.010s	sys 0.043s
W20260709 12:21:59.835104 21434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.010s	sys 0.049s
W20260709 12:21:59.748728 22568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.023s	sys 0.019s
I20260709 12:21:59.832113 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:59.842011 21430 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
W20260709 12:21:59.739353 22604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.003s	sys 0.044s
I20260709 12:21:59.733244 22559 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:59.835557 20806 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:21:59.793185 21433 rpcz_store.cc:276] 0709 12:21:57.514336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.515089 (+   753us) service_pool.cc:224] Handling call
0709 12:21:57.515450 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.515806 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:57.515834 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:57.516312 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.924766 (+1408454us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:58.924802 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.924864 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:59.735921 (+811057us) raft_consensus.cc:1685] finished
0709 12:21:59.735942 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:59.736069 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3158,"raft.run_cpu_time_us":7353,"raft.run_wall_time_us":58569,"thread_start_us":1408136,"threads_started":1,"wal-append.queue_time_us":1654556}
I20260709 12:21:59.834237 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:21:59.752426 22723 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.856168 21431 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:59.857295 22723 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:21:59.844306 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:59.831964 21398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.033s	sys 0.055s
I20260709 12:21:59.744379 22733 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:59.753376 22749 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:21:59.864979 20804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.967s	user 0.018s	sys 0.011s
W20260709 12:21:59.865450 20804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.018s	sys 0.011s
W20260709 12:21:59.850227 22604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.010s	sys 0.072s
I20260709 12:21:59.744738 22616 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:59.752002 22618 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.732131 22361 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:59.853734 22568 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.731165 22732 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.794250 22567 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.881317 22682 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:59.888504 22293 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:21:59.851284 22747 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:59.896442 21435 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:21:59.749437 22719 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:59.872664 22743 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:21:59.871568 22393 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:59.872797 22749 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.912390 22747 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.921581 20809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:21:59.922343 20809 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:21:59.896607 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:21:59.925098 22747 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:59.916513 22525 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.147s	user 0.006s	sys 0.007s
I20260709 12:21:59.906117 22754 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:59.914305 22755 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.921432 22719 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.937449 22755 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.938187 22749 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:21:59.873100 22733 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.943308 22719 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:59.925474 22746 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:21:59.913022 22750 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 3. Current term is 4.
I20260709 12:21:59.898962 22568 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:21:59.926149 22726 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:59.926054 22752 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:59.950132 20810 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:21:59.954905 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 272900us
I20260709 12:21:59.966406 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:21:59.951613 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 273098us
I20260709 12:21:59.952108 22746 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:59.967706 22660 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:59.968410 22660 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:21:59.969724 22746 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:59.971130 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69362.2us
I20260709 12:21:59.936868 22525 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:21:59.975245 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 739601us
W20260709 12:21:59.972446 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 292899us
I20260709 12:21:59.976399 22740 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:21:59.979314 22525 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:21:59.976287 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 297168us
I20260709 12:21:59.985937 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:21:59.994184 22740 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:59.994642 22740 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:21:59.995083 22740 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.000051 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:59.997083 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:21:59.995715 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:00.004941 20805 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:22:00.003729 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:00.014541 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:21:59.971233 22293 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:00.025574 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:21:59.973723 22604 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:21:59.942473 22754 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.029767 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:00.035421 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 125454us
I20260709 12:22:00.035890 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.038336 20809 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:22:00.007844 22703 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.007076 22741 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:59.671929 20801 rpcz_store.cc:276] 0709 12:21:56.350502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.351299 (+   797us) service_pool.cc:224] Handling call
0709 12:21:56.633805 (+282506us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:21:56.634075 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.634559 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.634603 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.635264 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.137772 (+1502508us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:58.151070 (+ 13298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.165333 (+ 14263us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:58.973221 (+807888us) raft_consensus.cc:1685] finished
0709 12:21:58.973249 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.973461 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4793,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":8855,"spinlock_wait_cycles":42098944,"thread_start_us":1480683,"threads_started":1,"wal-append.queue_time_us":1571908}
I20260709 12:22:00.015472 22748 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:00.045548 20774 leader_election.cc:400] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3.
W20260709 12:22:00.048666 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 254110us
I20260709 12:22:00.026906 22704 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:00.017521 22746 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:22:00.011140 22525 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:22:00.059294 22461 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.060330 22732 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.041334 22309 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:00.047216 22744 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:22:00.063513 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:00.062804 22626 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:00.041085 22293 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:00.061785 22751 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:22:00.060408 22754 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:00.057133 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.041919 22604 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.064917 22748 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:00.067202 22618 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.069617 22626 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.098879 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:00.099773 21435 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:00.101415 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:00.087072 22758 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:22:00.086035 22723 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:22:00.147022 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 180686us
I20260709 12:22:00.148890 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.069320 22703 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:22:00.155177 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.100692 22626 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:00.159219 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.083760 22753 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:22:00.089561 20803 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:00.169387 22733 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.085705 22704 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:22:00.164645 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:00.174584 20882 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.167526 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:00.175345 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.176506 20881 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:00.177515 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 155710us
I20260709 12:22:00.151587 22723 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:00.181269 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 156258us
I20260709 12:22:00.168354 22293 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.169301 22604 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:00.185985 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 182803us
I20260709 12:22:00.179741 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:00.188763 20881 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:21:58.489648 20884 rpcz_store.cc:276] 0709 12:21:56.205809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.205965 (+   156us) service_pool.cc:224] Handling call
0709 12:21:56.206263 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.206463 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:21:57.601495 (+1395032us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.454621 (+853126us) log.cc:844] Serialized 14 byte log entry
0709 12:21:58.464295 (+  9674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:58.464345 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:58.464388 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:58.464473 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:58.464651 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14413,"prepare.queue_time_us":451075,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":34425,"spinlock_wait_cycles":55100672,"thread_start_us":2241835,"threads_started":2}
I20260709 12:22:00.187889 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.191838 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:00.192317 20877 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:00.194554 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:00.195109 21435 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.196820 20879 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:00.193737 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:00.203316 22756 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 12:22:00.203118 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 1588) took 2337 ms (client timeout 1910 ms). Trace:
W20260709 12:22:00.203569 22767 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:22:00.186657 22723 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:00.207321 22719 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:00.210335 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:00.216140 20734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 245) took 2565 ms (client timeout 1820 ms). Trace:
W20260709 12:22:00.245525 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51918.3us
I20260709 12:22:00.246552 21435 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:00.178004 22568 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:00.235441 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53161.2us
I20260709 12:22:00.235499 20882 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:00.203955 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:00.240392 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45447.7us
I20260709 12:22:00.247138 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:00.245446 20734 rpcz_store.cc:269] 0709 12:21:57.651074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.652073 (+   999us) service_pool.cc:224] Handling call
0709 12:22:00.215876 (+2563803us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003b5454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:00.216025 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2937396480}
I20260709 12:22:00.247270 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:00.247771 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83121.3us
W20260709 12:22:00.236362 20850 connection.cc:256] RPC call timeout handler was delayed by -1.2571771360003368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
I20260709 12:22:00.251530 20883 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 2: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:00.257531 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:00.263820 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:00.216150 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 527) took 2701 ms. Trace:
I20260709 12:22:00.279553 21435 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:22:00.285800 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:00.292493 20881 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:00.185292 22761 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:22:00.295245 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:00.191227 22677 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.250800 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:00.323369 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:00.324014 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:00.326718 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 75994.8us
I20260709 12:22:00.327872 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:00.244446 22604 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.251735 22770 raft_consensus.cc:2764] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:22:00.254709 22759 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.334767 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 532) took 1574 ms. Trace:
I20260709 12:22:00.253012 22771 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:00.335469 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:00.287142 22560 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:00.339663 21432 rpcz_store.cc:276] 0709 12:21:58.760654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.760771 (+   117us) service_pool.cc:224] Handling call
0709 12:21:58.761083 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:59.699599 (+938516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.699644 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:59.700273 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:21:59.962677 (+262404us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:59.962723 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.962800 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:00.334555 (+371755us) raft_consensus.cc:1685] finished
0709 12:22:00.334574 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:00.334710 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":218475,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":1199999,"threads_started":2,"wal-append.queue_time_us":369100}
I20260709 12:22:00.353243 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 282) took 3291 ms. Trace:
I20260709 12:22:00.353678 20804 rpcz_store.cc:276] 0709 12:21:57.062132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.770315 (+1708183us) service_pool.cc:224] Handling call
0709 12:21:58.770733 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:58.771113 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:58.771157 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:58.771746 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 12:21:59.865980 (+1094234us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:59.866057 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.866147 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:00.348135 (+481988us) raft_consensus.cc:1685] finished
0709 12:22:00.348165 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:00.353182 (+  5017us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":189114,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":129,"thread_start_us":968580,"threads_started":1,"wal-append.queue_time_us":1194230}
I20260709 12:22:00.357076 20734 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:00.297631 22745 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.359318 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:00.360023 20732 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.360437 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:00.350970 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:00.249586 22293 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 12:22:00.353175 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:00.249522 22764 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:22:00.238884 20875 rpcz_store.cc:269] 0709 12:21:57.865785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.918033 (+ 52248us) service_pool.cc:224] Handling call
0709 12:22:00.202888 (+2284855us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:00.203011 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61452160}
I20260709 12:22:00.251685 22719 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.331904 22765 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:00.368553 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.381789 20730 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:22:00.336876 22612 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:00.372510 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88615.1us
I20260709 12:22:00.372599 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.385371 20729 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:00.370628 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.386554 20734 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:00.375870 20727 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:00.333444 22754 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:00.340660 22753 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:00.379047 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:00.388888 21432 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:00.338289 22732 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:00.366806 21435 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:00.195968 22748 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.389940 22612 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.362365 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.464473 20733 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:00.465987 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:00.336944 22726 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:00.387538 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.387544 20732 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:00.469296 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.470604 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.389295 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.471486 20727 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:00.472446 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.473603 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.474387 20727 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:00.350693 22759 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:00.418042 22769 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:22:00.419045 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.477583 20729 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:22:00.478279 22759 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.340499 22604 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.480446 22677 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:00.482237 22781 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:22:00.483901 22747 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.005s
I20260709 12:22:00.391326 22732 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.385738 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.489910 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:00.492280 22732 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:00.453370 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68326.9us
I20260709 12:22:00.454012 21432 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:00.419837 22719 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.332065 22568 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:00.384164 22765 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:00.454885 22785 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:00.388720 22754 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.475703 22726 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.492182 22626 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.478528 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:00.485270 20732 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 2: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:00.485527 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:00.514268 20727 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:00.515030 20849 connection.cc:256] RPC call timeout handler was delayed by -3.4236003430000892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 12:22:00.515116 20727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:00.413708 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:00.484514 22742 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.001s
I20260709 12:22:00.501946 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.452842 22704 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:00.452158 22629 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:00.452561 22753 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:00.505229 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.505151 22752 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:00.511672 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.535946 20733 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:00.536217 20883 raft_consensus.cc:2368] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5.
I20260709 12:22:00.506263 22754 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:00.484361 22747 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:00.516485 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102929us
I20260709 12:22:00.514249 22461 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.517045 22615 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:00.517572 20729 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.492219 20726 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:00.519505 22604 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.514029 22785 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.520820 22552 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.515810 20727 raft_consensus.cc:606] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:00.530310 20879 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:00.506562 22765 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:00.516896 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:00.540591 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:00.555601 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:00.527793 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:00.556710 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:00.557036 22604 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.557499 22604 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.557880 22604 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.559746 22778 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:00.560338 22778 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:00.561235 22778 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.535362 22629 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.565172 22629 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:00.542968 22461 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.566694 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:00.567277 22461 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.547966 22612 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.567533 22759 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.567979 22759 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:00.568454 22759 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.570497 22747 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.534708 22742 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:00.571977 22753 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.512293 22568 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.578008 20729 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:00.291981 21427 rpcz_store.cc:276] 0709 12:21:57.514650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.518338 (+  3688us) service_pool.cc:224] Handling call
0709 12:21:57.591560 (+ 73222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.625472 (+ 33912us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:21:57.632444 (+  6972us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:57.640445 (+  8001us) log.cc:844] Serialized 26 byte log entry
0709 12:21:58.201429 (+560984us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b1800360e20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:21:59.673457 (+1472028us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:59.689358 (+ 15901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.696976 (+  7618us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:00.164284 (+467308us) raft_consensus.cc:1685] finished
0709 12:22:00.164313 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:00.201608 (+ 37295us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5827,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5547,"spinlock_wait_cycles":13152512,"thread_start_us":1461570,"threads_started":1,"wal-append.queue_time_us":2146157}
I20260709 12:22:00.580519 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:00.590157 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:00.590301 22779 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:00.591176 22779 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:00.591516 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:00.592958 22783 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:00.593536 22783 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.594014 22783 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:00.604542 20882 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:22:00.575431 22788 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:00.576794 22740 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.005s
I20260709 12:22:00.534976 22704 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:22:00.612691 22792 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:22:00.614414 22795 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:00.607452 22552 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:00.603801 20880 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:00.603271 22615 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:00.603514 22748 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:00.627393 22800 raft_consensus.cc:2764] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:22:00.628124 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72609.4us
I20260709 12:22:00.539552 22752 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.611986 22742 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:00.634264 22783 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.607672 22779 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.608704 22629 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:22:00.615168 22795 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.611963 22740 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.645275 22740 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.659150 22629 raft_consensus.cc:1064] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
W20260709 12:22:00.659672 22629 raft_consensus.cc:1068] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Unable to promote non-voter 50c1ab13351d4427b836fbb449c6d7bb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:22:00.665153 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71949.1us
I20260709 12:22:00.626180 22552 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.639302 22742 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:00.668769 22742 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:00.668841 22783 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.669137 22552 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.671476 22629 raft_consensus.cc:1064] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:22:00.672760 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:00.673429 21426 raft_consensus.cc:2376] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4. Re-sending same reply.
I20260709 12:22:00.674016 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 638) took 1112 ms. Trace:
I20260709 12:22:00.673621 22723 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.673727 22629 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:00.678133 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1552) took 2609 ms (client timeout 2192 ms). Trace:
W20260709 12:22:00.678625 20806 rpcz_store.cc:269] 0709 12:21:58.068397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.447238 (+378841us) service_pool.cc:224] Handling call
0709 12:22:00.677891 (+2230653us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540019a0d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:00.678032 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3190193920}
I20260709 12:22:00.642498 22795 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.674371 21426 rpcz_store.cc:276] 0709 12:21:59.561913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.565761 (+  3848us) service_pool.cc:224] Handling call
0709 12:22:00.672644 (+1106883us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aeee 000000000036eb39 00007f9e23bee1e0 00007f9e23cb70c4 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:00.673947 (+  1303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6714624}
I20260709 12:22:00.680202 22568 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:00.681954 20849 connection.cc:256] RPC call timeout handler was delayed by -2.2581168809997507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
W20260709 12:22:00.689582 20849 connection.cc:256] RPC call timeout handler was delayed by -2.1555168810000396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
I20260709 12:22:00.682981 22723 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.690888 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:00.685092 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:00.686544 22631 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.428s	user 0.031s	sys 0.088s
I20260709 12:22:00.686406 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.688331 22626 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.606815 22788 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:00.684124 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:00.637495 22748 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:00.680280 22726 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.695235 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.693853 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:00.702750 22788 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:00.694615 22631 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.470s	user 0.031s	sys 0.088s
I20260709 12:22:00.689631 22637 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:00.716038 22704 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.717752 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.718391 20734 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:22:00.719605 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.720206 20734 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:22:00.721540 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.700043 22800 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.724027 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:00.726836 22752 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.726552 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.700220 22785 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.728523 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:00.730420 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:00.731530 22723 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:00.731863 22600 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.733016 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.733633 20806 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:00.713670 22795 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:00.733987 22618 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:00.734433 22618 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:00.735119 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.737264 22748 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:00.737568 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:22:00.723158 22637 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.738543 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.740008 22637 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:00.740267 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.741564 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.742199 20803 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:00.743044 22759 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.745638 20727 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [2.4-4.5]
W20260709 12:22:00.745966 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:00.743543 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.720471 22765 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.752497 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.816998 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.850637 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:00.853747 20806 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:00.808888 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.855204 20803 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:22:00.760681 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.797974 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.857844 20807 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:00.858810 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.859457 20807 raft_consensus.cc:2368] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4.
I20260709 12:22:00.799767 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.854800 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.729367 22800 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:00.864466 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:00.969506 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:00.970263 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.851686 20734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:00.975586 20734 raft_consensus.cc:606] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:00.976626 20734 raft_consensus.cc:614] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:00.977138 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.978780 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.976881 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.983786 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.984604 20802 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:00.993557 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:00.748546 22704 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:00.995966 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:01.001515 20734 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:01.001720 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 137368us
I20260709 12:22:01.002550 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:00.823626 22745 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:00.863788 22748 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:00.731943 22723 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:00.833135 22781 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:22:01.016816 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:00.976329 22786 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.003690 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.023694 22765 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:00.976312 22800 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.003660 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:00.983515 22525 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:00.981072 22785 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:22:00.833953 22604 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:00.845155 22732 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.050315 20734 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:01.050514 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.060108 20805 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:01.035815 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:00.837145 22802 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.319s
I20260709 12:22:00.990129 22787 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.334s
I20260709 12:22:01.028599 22723 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:01.017004 22629 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:01.013676 22704 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:01.018589 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:01.052426 22770 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:01.091899 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.055299 20803 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:01.026360 22748 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:01.094105 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58374.8us
W20260709 12:22:01.094339 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.052861 20801 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:01.096848 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:01.089347 20805 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:01.098266 22508 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:01.098611 22754 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.000s
I20260709 12:22:01.082410 22629 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:01.093097 22704 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:00.754902 22618 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.098938 22785 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:01.102247 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:01.098842 22732 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:01.101795 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.141475 20807 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:01.142230 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.097242 21429 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:01.092926 22759 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.066466 22525 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.148459 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.048307 22786 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.101863 22770 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:01.165072 20805 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:01.149808 20726 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:01.142392 22629 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.161439 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:01.149195 22802 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:01.174376 20732 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:01.149595 22785 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:01.188527 20805 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:01.189910 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.115736 22754 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.167973 22764 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:01.190584 22785 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.200901 21432 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:22:01.206601 20732 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:01.207650 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:01.208535 22783 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.197785 22796 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:01.196270 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:01.222127 20731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1529) took 3101 ms (client timeout 1617 ms). Trace:
W20260709 12:22:01.222536 20731 rpcz_store.cc:269] 0709 12:21:58.121159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.123547 (+  2388us) service_pool.cc:224] Handling call
0709 12:22:01.221936 (+3098389us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54001d7c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:01.222043 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778915200}
I20260709 12:22:01.212796 20732 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:01.213235 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:01.215531 22806 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:22:01.216929 22810 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.230226 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.198105 22764 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.231894 22796 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:01.235176 22796 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:01.213470 22787 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:22:01.223574 22783 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.246400 22783 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:01.247730 22788 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:01.248157 22525 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.218595 22765 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.249082 22765 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.213096 22770 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.252557 22754 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:01.211946 22813 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:01.255007 22813 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:01.255491 22813 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.213960 22552 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:01.258366 22552 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.251467 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.260257 20729 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:01.262849 20805 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:22:01.264353 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.265086 20805 raft_consensus.cc:2368] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:22:01.265810 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.248164 22788 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:01.276463 22788 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.277554 22788 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.288038 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:01.268137 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 122552us
I20260709 12:22:01.304775 20732 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
W20260709 12:22:01.310199 22808 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:22:01.310076 22742 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:01.269315 22802 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:01.327775 20731 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:01.328548 20849 leader_election.cc:400] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4.
I20260709 12:22:01.260870 22604 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.331099 20803 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:01.339438 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.340421 22816 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:01.345865 20731 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:01.345237 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.358374 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.359776 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.266999 22810 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.335286 22802 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.360459 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.360747 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.361491 22748 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.349951 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.371836 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.372973 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:01.373752 22770 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:01.374168 22770 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:01.374225 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:01.374567 22770 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:01.370147 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.375737 20731 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:01.375099 22606 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:01.376550 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:01.375352 22604 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:01.378907 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.382409 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:01.365748 22742 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:01.386241 22742 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.386627 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.387311 20809 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:01.388564 22742 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.388736 22810 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.368499 22525 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.389420 22525 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.390262 22759 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:01.391534 22525 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:01.383098 22373 raft_consensus.cc:2764] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:22:01.381171 22604 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:01.393774 22813 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.396296 22813 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:01.393782 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.397771 22604 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:01.398716 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.391558 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 641) took 1761 ms. Trace:
I20260709 12:22:01.394238 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.400930 20808 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:01.402081 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.402752 20808 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.399410 21433 rpcz_store.cc:276] 0709 12:21:59.630317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.854413 (+224096us) service_pool.cc:224] Handling call
0709 12:21:59.854810 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:59.943265 (+ 88455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.943295 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:59.943338 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:59.943370 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:01.391338 (+1447968us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:01.391379 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:01.391504 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8521,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":20671,"spinlock_wait_cycles":193301888}
I20260709 12:22:01.378120 22606 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:01.410392 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 65234.6us
I20260709 12:22:01.412467 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.413856 20700 leader_election.cc:400] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:22:01.414252 20700 leader_election.cc:403] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:01.384106 22764 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.420388 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:01.410457 22606 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.359953 22816 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.411288 22604 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:01.413908 22615 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:01.417483 22810 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:01.418089 22759 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:22:01.430226 22552 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.438925 22604 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.425148 22525 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.412533 22629 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.428692 22626 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:01.424926 21429 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:01.437450 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.448526 22816 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.454396 22810 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:01.451879 20809 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:01.439638 22800 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.468077 22629 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:01.480615 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.481599 22810 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.483212 22765 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:01.493376 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:01.495451 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1557) took 1816 ms. Trace:
I20260709 12:22:01.495990 20810 rpcz_store.cc:276] 0709 12:21:59.679010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.680441 (+  1431us) service_pool.cc:224] Handling call
0709 12:22:01.485533 (+1805092us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002b6bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:01.495367 (+  9834us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374834048}
I20260709 12:22:01.497259 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.498584 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.502266 20810 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 1.
I20260709 12:22:01.617491 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.618472 20809 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:01.625475 22615 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:01.546221 22816 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:22:01.626827 22615 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:01.629810 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 196998us
I20260709 12:22:01.632755 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.633080 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.633574 20732 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:01.633551 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.634986 20731 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:01.636435 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.637352 20731 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:01.640343 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.646149 20810 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:01.651659 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.648765 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.649160 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.652743 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:01.653478 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:01.653923 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.655143 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:01.656122 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.656695 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:01.657756 20802 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:01.658663 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:01.659523 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:01.660055 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:01.661255 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46401ecb9b443118647ee361081a01e"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:22:01.661818 20802 raft_consensus.cc:606] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:01.662855 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.663460 20810 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:01.559114 22821 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.537535 22788 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:01.664759 20810 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:01.647784 22723 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.556s	user 0.002s	sys 0.007s
I20260709 12:22:01.666150 20809 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:01.492970 22704 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.680162 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:01.681119 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:01.682305 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:01.682804 20884 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:01.686291 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:01.675235 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:01.688366 20732 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:01.676128 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:01.506851 22785 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.691133 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:01.692987 20877 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:01.694316 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:01.706607 20876 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:01.666801 22759 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:01.665840 22788 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:01.711741 20879 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:22:01.713284 20879 pending_rounds.cc:85] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 2
I20260709 12:22:01.714159 20879 pending_rounds.cc:107] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:22:01.714944 20879 raft_consensus.cc:2889] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:01.716135 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.713414 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:01.717481 20883 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:01.720281 20805 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:01.712008 22810 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:01.733032 22810 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.733580 22810 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.734870 22606 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.738544 20808 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:01.739228 22827 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:01.677738 22822 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.748307 22822 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.748914 22785 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:01.671655 22780 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.749414 20884 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:01.746863 22525 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:01.706102 22797 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.796299 22822 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.797792 22755 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:01.720476 22788 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.749776 22780 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.751827 22802 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.807660 20883 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:01.739022 22606 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:01.830091 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 181368us
I20260709 12:22:01.677337 22723 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.755952 22759 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:01.806782 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:01.862262 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:01.870500 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 239876us
I20260709 12:22:01.679368 22821 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.875698 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:01.866771 22800 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:01.876008 22759 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:01.815590 22525 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:01.813802 22797 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:01.927917 22823 raft_consensus.cc:993] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
W20260709 12:22:01.942432 22828 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:22:01.951287 22780 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:01.870589 22802 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.985620 22802 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:01.992529 22837 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.998754 22837 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:01.989496 22823 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:02.002503 22838 raft_consensus.cc:1081] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:22:02.022076 22821 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:02.022888 22823 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:02.024137 22823 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.018955 22837 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:02.034825 22837 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:02.035502 22837 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.037989 22837 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:02.038709 22837 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:02.070724 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 281741us
W20260709 12:22:02.078475 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 424450us
I20260709 12:22:02.087036 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:02.090520 20729 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37282
I20260709 12:22:02.091630 20729 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:02.092476 20729 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:01.814388 20808 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:02.103000 20804 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:02.116788 20809 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:02.131665 20884 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:02.118877 20809 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:22:02.205385 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.206647 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:02.207058 22660 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.207886 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.208333 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.209018 20880 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:02.209163 20882 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:02.210129 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:02.211771 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.212606 20880 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:02.212798 22704 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:02.213850 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.214303 22723 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.210278 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:02.184247 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 308455us
W20260709 12:22:02.003242 22796 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.000s
I20260709 12:22:02.237402 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.238935 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:02.239135 20882 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:02.237655 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:02.039132 22837 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:01.992043 22797 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:02.271857 22559 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.009s	sys 0.008s
W20260709 12:22:02.272491 22559 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.009s	sys 0.009s
W20260709 12:22:02.273746 20733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1548) took 1861 ms (client timeout 1535 ms). Trace:
W20260709 12:22:02.277201 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 371) took 1308 ms (client timeout 1577 ms). Trace:
W20260709 12:22:02.277813 21432 rpcz_store.cc:269] 0709 12:22:00.968195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.969163 (+   968us) service_pool.cc:224] Handling call
0709 12:22:02.276534 (+1307371us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:02.276890 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521430272}
W20260709 12:22:02.278388 22552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.005s	sys 0.013s
W20260709 12:22:02.279006 22552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.005s	sys 0.013s
I20260709 12:22:02.280534 22846 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:02.281629 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:02.274385 20733 rpcz_store.cc:269] 0709 12:22:00.411915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.465630 (+ 53715us) service_pool.cc:224] Handling call
0709 12:22:02.273130 (+1807500us) spinlock_profiling.cc:243] Waited 69.1 ms on lock 0x7b54000e6ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:02.273504 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158928640}
I20260709 12:22:02.299149 20731 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:02.303570 22837 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:02.306852 22770 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.890s	user 0.005s	sys 0.007s
I20260709 12:22:02.307613 22770 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:02.308619 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:02.310590 20883 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:02.311780 22796 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.314647 22796 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:02.318327 22849 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:22:02.321745 20877 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:02.323915 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 361) took 2694 ms. Trace:
I20260709 12:22:02.324903 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 251) took 1639 ms. Trace:
I20260709 12:22:02.345976 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.323904 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1582) took 1226 ms. Trace:
I20260709 12:22:02.325495 20726 rpcz_store.cc:276] 0709 12:22:00.685750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.686119 (+   369us) service_pool.cc:224] Handling call
0709 12:22:02.324429 (+1638310us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54001d7754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:02.324719 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905563904}
I20260709 12:22:02.356398 20802 rpcz_store.cc:276] 0709 12:22:01.097819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.672304 (+574485us) service_pool.cc:224] Handling call
0709 12:22:01.672539 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:01.672707 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:02.093635 (+420928us) log.cc:844] Serialized 14 byte log entry
0709 12:22:02.270219 (+176584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:02.304520 (+ 34301us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:02.323347 (+ 18827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.323489 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.323671 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193018,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":546442,"threads_started":2}
I20260709 12:22:02.324471 21430 rpcz_store.cc:276] 0709 12:21:59.629336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.842754 (+213418us) service_pool.cc:224] Handling call
0709 12:21:59.864944 (+ 22190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:59.933266 (+ 68322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.933315 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:59.933371 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:59.933421 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.293670 (+2360249us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b5400645ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:02.323409 (+ 29739us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.323650 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156547328}
I20260709 12:22:02.365589 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1747) took 1021 ms. Trace:
I20260709 12:22:02.366125 20802 rpcz_store.cc:276] 0709 12:22:01.343963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.362189 (+1018226us) service_pool.cc:224] Handling call
0709 12:22:02.362855 (+   666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.365070 (+  2215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:02.365123 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:02.365196 (+    73us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:02.365245 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.365331 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.365498 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:02.263371 22844 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:02.281744 22846 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.406512 20729 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.436103 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.443392 20882 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:02.445538 22844 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:02.456800 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93207.6us
W20260709 12:22:02.445204 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96414.8us
I20260709 12:22:02.437194 22796 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.472455 20806 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:02.478204 20731 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:02.482436 20729 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.488430 22844 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:02.490216 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:02.491143 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1724) took 1619 ms (client timeout 1913 ms). Trace:
I20260709 12:22:02.491570 22846 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:02.494140 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 214617us
W20260709 12:22:02.491572 20806 rpcz_store.cc:269] 0709 12:22:00.857481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.986576 (+129095us) service_pool.cc:224] Handling call
0709 12:22:02.281150 (+1294574us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:02.477181 (+196031us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13115776}
W20260709 12:22:02.494761 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.017s	sys 0.043s
W20260709 12:22:02.499307 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 283247us
W20260709 12:22:02.500944 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.018s	sys 0.043s
I20260709 12:22:02.500308 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:02.517836 22772 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 2.274s	user 0.009s	sys 0.042s
I20260709 12:22:02.519703 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 645) took 2824 ms. Trace:
I20260709 12:22:02.520134 21431 rpcz_store.cc:276] 0709 12:21:59.695636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.856876 (+161240us) service_pool.cc:224] Handling call
0709 12:21:59.857169 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:59.857504 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.857530 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:59.858064 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:22:00.083686 (+225622us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:00.083731 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:00.083810 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:02.366777 (+2282967us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:02.519260 (+152483us) raft_consensus.cc:1685] finished
0709 12:22:02.519282 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.519536 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85838,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":104,"spinlock_wait_cycles":11447168,"thread_start_us":225090,"threads_started":1,"wal-append.queue_time_us":320287}
W20260709 12:22:02.526793 22810 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.793s	user 0.012s	sys 0.004s
W20260709 12:22:02.530678 22854 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:22:02.511178 22796 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.534878 22796 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:02.537335 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:02.538187 20878 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:02.544199 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1729) took 1685 ms (client timeout 1829 ms). Trace:
W20260709 12:22:02.544727 20807 rpcz_store.cc:269] 0709 12:22:00.858879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.976934 (+118055us) service_pool.cc:224] Handling call
0709 12:22:02.543823 (+1566889us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003ca854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:02.544038 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376417536}
I20260709 12:22:02.548147 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:02.549901 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 295) took 1467 ms. Trace:
I20260709 12:22:02.550336 20803 rpcz_store.cc:276] 0709 12:22:01.090217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.547566 (+1457349us) service_pool.cc:224] Handling call
0709 12:22:02.549797 (+  2231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:02.550827 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:02.552084 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 639) took 2956 ms (client timeout 2622 ms). Trace:
W20260709 12:22:02.552572 21428 rpcz_store.cc:269] 0709 12:21:59.595786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.613364 (+ 17578us) service_pool.cc:224] Handling call
0709 12:22:02.550411 (+2937047us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:02.551962 (+  1551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35791744}
I20260709 12:22:02.555342 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.555446 20882 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:02.556977 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.558413 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:02.560640 22616 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.561792 20878 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:02.560197 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:02.511636 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 836313us
I20260709 12:22:02.566435 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.506732 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.567749 20878 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:02.565876 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58926.2us
I20260709 12:22:02.569082 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.569489 22861 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.572342 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.573136 20884 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:02.574406 22616 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.575359 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.577421 22836 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:02.578275 22836 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.578539 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:02.565531 22810 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.581121 22836 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:02.581889 22810 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:02.582548 22626 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:02.583760 22740 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.583458 22861 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.584514 22740 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.586282 22861 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.587997 22569 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:02.587013 22740 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:02.588963 22740 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:02.588778 22569 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.589599 22740 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:02.589769 22757 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 2.544s	user 0.007s	sys 0.003s
I20260709 12:22:02.591655 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:02.591284 22740 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.593469 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:02.594394 20803 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:22:02.593603 22461 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.586833 22770 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:02.583129 22626 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:02.599310 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 531) took 3839 ms. Trace:
I20260709 12:22:02.599478 20878 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:02.599555 20879 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:02.599984 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:02.598824 22569 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.598611 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.604640 22786 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.607815 21434 rpcz_store.cc:276] 0709 12:21:58.760216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.760378 (+   162us) service_pool.cc:224] Handling call
0709 12:21:58.773664 (+ 13286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:58.774072 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:58.774110 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:58.774650 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:21:59.852759 (+1078109us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:59.853648 (+   889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.853749 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:02.589182 (+2735433us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b24002f14b8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf109d 00007f9e1e82310e 00007f9e1e83a88c 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221
0709 12:22:02.597300 (+  8118us) raft_consensus.cc:1685] finished
0709 12:22:02.597322 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.599053 (+  1731us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":206152,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"spinlock_wait_cycles":21267200,"thread_start_us":929532,"threads_started":1,"wal-append.queue_time_us":1153895}
I20260709 12:22:02.614614 20807 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:02.616695 20805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1577) took 1565 ms (client timeout 1809 ms). Trace:
I20260709 12:22:02.616798 20807 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:02.617272 20805 rpcz_store.cc:269] 0709 12:22:01.051687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.292513 (+240826us) service_pool.cc:224] Handling call
0709 12:22:02.616310 (+1323797us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b540027bd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:02.616526 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866258816}
I20260709 12:22:02.617594 22763 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:02.618026 20807 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.619117 22763 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:02.619565 20807 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:02.619839 22763 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:02.607635 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:02.575208 22616 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:02.610176 22604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.000s	sys 0.014s
W20260709 12:22:02.660830 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:02.662933 22393 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:02.663591 22393 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:02.664074 22393 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:02.668136 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:22:02.668550 22788 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.696s	user 0.016s	sys 0.001s
I20260709 12:22:02.669039 22631 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:02.669605 22631 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:02.673702 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:02.675163 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:02.675083 22616 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:02.622287 22461 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.678493 22461 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:02.680817 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:02.683845 22540 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:02.684332 22540 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:02.685550 22540 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.689157 22745 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.690531 22745 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:22:02.690683 22825 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.065s	user 0.008s	sys 0.016s
I20260709 12:22:02.669168 22788 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.683857 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:02.691085 22745 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:02.661665 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144988us
I20260709 12:22:02.621676 22763 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.686812 22540 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:02.716946 22361 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.717609 22788 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:02.717856 20881 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:02.622062 22626 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:02.753513 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 126182us
W20260709 12:22:02.819787 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:02.819998 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:02.821331 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102349us
I20260709 12:22:02.823436 22764 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:02.823671 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:02.824129 22764 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:02.825546 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:02.825369 22764 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.671017 22631 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.873416 20807 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:02.906155 22461 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:02.907393 22788 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:02.907819 22788 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:02.908958 22788 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.910234 22788 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:02.839666 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:02.833246 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:02.922842 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1591) took 1266 ms. Trace:
I20260709 12:22:02.829741 22626 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:02.931587 22747 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.935115 20876 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:02.939765 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1479) took 1223 ms. Trace:
I20260709 12:22:02.936410 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:02.940356 20876 rpcz_store.cc:276] 0709 12:22:01.712663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.713003 (+   340us) service_pool.cc:224] Handling call
0709 12:22:02.839262 (+1126259us) spinlock_profiling.cc:243] Waited 67.3 ms on lock 0x7b5800003698. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:02.935771 (+ 96509us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154837760}
I20260709 12:22:02.837934 22393 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:02.947093 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 371311us
I20260709 12:22:02.832113 22837 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.932562 22747 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.945112 22559 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:02.961958 22631 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:02.968466 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1566) took 2194 ms (client timeout 2393 ms). Trace:
W20260709 12:22:02.969065 20801 rpcz_store.cc:269] 0709 12:22:00.751841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.752941 (+  1100us) service_pool.cc:224] Handling call
0709 12:22:02.924194 (+2171253us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:02.946707 (+ 22513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410324352}
I20260709 12:22:02.970712 22703 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:02.971223 22703 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:02.971702 22703 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:02.949996 22861 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:02.973893 22393 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:02.975323 22857 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:02.975965 22857 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:02.977007 22857 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:02.978072 22747 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:02.979552 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 146427us
I20260709 12:22:02.970333 22567 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:22:02.984630 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1476) took 1291 ms. Trace:
I20260709 12:22:02.985051 20877 rpcz_store.cc:276] 0709 12:22:01.693082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.715518 (+ 22436us) service_pool.cc:224] Handling call
0709 12:22:01.715872 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.322390 (+606518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.984334 (+661944us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b5400268254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:02.984554 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392374528}
I20260709 12:22:02.987843 20729 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:02.989477 22618 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:02.992192 22837 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:02.995005 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 296) took 1504 ms. Trace:
I20260709 12:22:02.995528 20802 rpcz_store.cc:276] 0709 12:22:01.490530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.403879 (+913349us) service_pool.cc:224] Handling call
0709 12:22:02.436000 (+ 32121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:02.436413 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:02.436441 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:02.475867 (+ 39426us) log.cc:844] Serialized 26 byte log entry
0709 12:22:02.808620 (+332753us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:02.808674 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.808769 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:02.994726 (+185957us) raft_consensus.cc:1685] finished
0709 12:22:02.994754 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.994930 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":316608,"threads_started":1,"wal-append.queue_time_us":456422}
I20260709 12:22:02.968585 20731 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:22:03.003238 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44113.4us
W20260709 12:22:03.018355 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 342307us
I20260709 12:22:03.020466 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.021517 20734 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:22:03.025709 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.027612 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.027796 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.028388 20807 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:03.029127 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.030023 20802 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:03.030321 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:03.031154 20807 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:03.031704 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:03.033545 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.034364 20876 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 2: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:03.035929 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:02.963294 22559 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:03.036867 20884 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:03.036697 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:03.036779 22559 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:03.040854 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 124992us
W20260709 12:22:03.060125 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 633169us
W20260709 12:22:03.064453 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61748.8us
I20260709 12:22:03.075623 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.075814 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:03.077003 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.067725 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.063092 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.102281 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.102840 20805 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.102401 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.103444 20882 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:03.107244 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1746) took 1601 ms. Trace:
I20260709 12:22:03.107730 20808 rpcz_store.cc:276] 0709 12:22:01.343721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.335363 (+991642us) service_pool.cc:224] Handling call
0709 12:22:02.413105 (+ 77742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.432080 (+ 18975us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:02.458187 (+ 26107us) log.cc:844] Serialized 14 byte log entry
0709 12:22:02.943406 (+485219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:02.943483 (+    77us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:02.943558 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.943672 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.945046 (+  1374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12753,"prepare.queue_time_us":24528,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":6512,"raft.queue_time_us":19139,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":7337,"spinlock_wait_cycles":1424768,"thread_start_us":429091,"threads_started":1,"wal-append.queue_time_us":585701}
I20260709 12:22:03.109653 22833 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:03.111306 22793 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:03.111543 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:03.111820 22793 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:03.116216 22552 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:03.116781 22552 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:03.022804 22703 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:03.117277 22552 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.158743 22745 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.159989 22745 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.164959 20735 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:03.167565 22838 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:03.168198 22838 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:03.168678 22838 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.175736 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 1617) took 1378 ms. Trace:
I20260709 12:22:03.176433 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.179255 22814 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:03.179769 22814 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:03.176215 20881 rpcz_store.cc:276] 0709 12:22:01.797372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.400503 (+603131us) service_pool.cc:224] Handling call
0709 12:22:03.175468 (+774965us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:03.175650 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645639552}
I20260709 12:22:03.211053 20805 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
W20260709 12:22:03.223775 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:03.249366 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 370) took 2380 ms. Trace:
I20260709 12:22:03.251991 22813 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:03.252621 22813 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:03.254077 20734 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:03.252777 21399 leader_election.cc:400] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5.
I20260709 12:22:03.255002 21399 leader_election.cc:403] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:22:03.280010 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50149.7us
I20260709 12:22:03.298553 22765 raft_consensus.cc:993] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:22:03.299465 22765 raft_consensus.cc:1081] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
W20260709 12:22:03.302481 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:03.335400 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1561) took 2600 ms (client timeout 2209 ms). Trace:
I20260709 12:22:03.335649 20810 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:03.405464 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 124397us
I20260709 12:22:03.181026 22814 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:02.925203 20806 rpcz_store.cc:276] 0709 12:22:01.656209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.545179 (+888970us) service_pool.cc:224] Handling call
0709 12:22:02.545604 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.545944 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:02.546388 (+   444us) log.cc:844] Serialized 14 byte log entry
0709 12:22:02.922403 (+376015us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:02.922491 (+    88us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:02.922549 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.922630 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.922772 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46660,"prepare.queue_time_us":2448,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":60,"raft.queue_time_us":55442,"thread_start_us":321869,"threads_started":1}
I20260709 12:22:03.353802 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:03.421344 20804 rpcz_store.cc:269] 0709 12:22:00.734797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.738305 (+  3508us) service_pool.cc:224] Handling call
0709 12:22:03.334993 (+2596688us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:03.335266 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3135022336}
W20260709 12:22:03.432448 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 357039us
I20260709 12:22:03.433357 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1745) took 2060 ms. Trace:
I20260709 12:22:03.404491 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.437368 20883 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:03.390559 22703 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:03.448868 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:03.463992 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 530051us
I20260709 12:22:03.466318 21434 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.438126 22704 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:03.433821 20809 rpcz_store.cc:276] 0709 12:22:01.343415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.205586 (+862171us) service_pool.cc:224] Handling call
0709 12:22:02.206295 (+   709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:02.484897 (+278602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:02.484935 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:02.509498 (+ 24563us) log.cc:844] Serialized 26 byte log entry
0709 12:22:02.948040 (+438542us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:02.948078 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.948157 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:03.402428 (+454271us) raft_consensus.cc:1685] finished
0709 12:22:03.402459 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.404151 (+  1692us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":177754,"raft.queue_time_us":27661,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":52071,"thread_start_us":625725,"threads_started":2,"wal-append.queue_time_us":689113}
W20260709 12:22:03.495718 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 243209us
W20260709 12:22:03.493222 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 392788us
I20260709 12:22:03.498822 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.507236 21431 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:03.514304 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:03.525641 22773 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5.
I20260709 12:22:03.527647 22765 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:22:03.539855 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:03.551482 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44757.8us
I20260709 12:22:03.539366 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:03.557785 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:03.559346 22742 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.043s
W20260709 12:22:03.560810 20699 connection.cc:256] RPC call timeout handler was delayed by -1.027992845999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
I20260709 12:22:03.561544 22742 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:03.561967 22742 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:03.563894 22783 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.564523 22783 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:03.490872 22704 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.511363 20726 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:03.506417 22813 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:03.511606 22814 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:03.568156 22813 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.569799 22800 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:03.570456 22800 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:03.572736 22800 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:03.561573 20699 connection.cc:256] RPC call timeout handler was delayed by -1.0000928459999159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
I20260709 12:22:03.515318 22857 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:03.592156 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.595527 20805 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:22:03.594067 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71525.2us
I20260709 12:22:03.606446 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.601856 21434 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:03.609812 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.610564 21431 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:03.567071 22783 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:03.584283 22763 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.605791 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.615950 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:03.623179 20806 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.623440 22755 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:03.623916 20805 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:03.625066 21428 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:03.622895 22827 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:03.626645 22827 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:03.629771 22827 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:03.623831 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:03.631356 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.631680 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:03.632349 20808 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:03.632354 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:03.633771 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 131247us
I20260709 12:22:03.634763 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:03.636619 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:03.636629 22639 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:03.637951 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1464) took 1959 ms. Trace:
I20260709 12:22:03.632367 21431 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.639357 22639 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:03.639835 22639 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.251351 21427 rpcz_store.cc:276] 0709 12:22:00.868747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.967837 (+ 99090us) service_pool.cc:224] Handling call
0709 12:22:02.612719 (+1644882us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ae9c 00007f9e1bb7aa5b 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:02.613021 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.613464 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:02.913546 (+300082us) log.cc:844] Serialized 14 byte log entry
0709 12:22:03.248747 (+335201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:03.248831 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:03.248905 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.248999 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.249166 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":371833,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":1215,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"spinlock_wait_cycles":24735488,"thread_start_us":499083,"threads_started":2}
I20260709 12:22:03.611443 22773 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:03.641191 22773 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:03.641348 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:03.641590 22773 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:03.643652 20775 leader_election.cc:336] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.535s (SENT)
I20260709 12:22:03.644178 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:03.645090 22703 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:03.647656 22857 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.648464 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:03.649824 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:03.650245 22857 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:03.650657 22872 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:03.626181 22631 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.657s	user 0.005s	sys 0.007s
I20260709 12:22:03.650848 20727 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:03.651528 22872 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.651907 22618 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:03.652997 20727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 1794) took 2907 ms (client timeout 1579 ms). Trace:
W20260709 12:22:03.653440 20727 rpcz_store.cc:269] 0709 12:22:00.745414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.746529 (+  1115us) service_pool.cc:224] Handling call
0709 12:22:03.652755 (+2906226us) spinlock_profiling.cc:243] Waited 2.63 s on lock 0x7b54003ae8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:03.652901 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6058931072}
I20260709 12:22:03.654016 20809 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:03.655089 22615 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.638345 20875 rpcz_store.cc:276] 0709 12:22:01.678603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.690524 (+ 11921us) service_pool.cc:224] Handling call
0709 12:22:02.716794 (+1026270us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b5800003698. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:02.717083 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:02.717658 (+   575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:02.717699 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:02.718288 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 12:22:02.947429 (+229141us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:03.473012 (+525583us) log.cc:844] Serialized 14 byte log entry
0709 12:22:03.506129 (+ 33117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.512139 (+  6010us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:03.637462 (+125323us) raft_consensus.cc:1685] finished
0709 12:22:03.637588 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.637742 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7128,"prepare.queue_time_us":685680,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":19386,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":5736,"spinlock_wait_cycles":86967296,"thread_start_us":644880,"threads_started":2,"wal-append.queue_time_us":855654}
I20260709 12:22:03.655615 22615 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.656198 22560 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:03.656517 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:03.657574 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:03.659106 22293 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:03.659559 22705 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:03.660480 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:03.659575 22293 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:03.661139 22293 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.661459 22705 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:03.662068 22705 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:03.663160 22705 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.665612 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.667026 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:03.663784 22552 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 12:22:03.667692 22552 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.668254 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.668977 20807 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:03.669047 21433 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:03.669003 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.669478 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.669996 22552 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:03.670212 21426 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:03.670619 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.663878 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:03.670064 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:03.673812 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:03.674561 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:03.675134 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:03.675511 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:03.678544 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.678488 21433 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:03.679518 20806 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:03.685269 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:03.688266 20809 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:03.692350 20776 connection.cc:256] RPC call timeout handler was delayed by -1.2722777829999359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 12:22:03.692832 20877 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:03.563041 22742 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:03.696470 20876 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:03.701848 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:03.702521 20804 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:03.703483 22771 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:03.703832 22837 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:03.704381 22837 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:03.703826 22793 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.704806 22837 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.705096 22793 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.705672 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1739) took 2670 ms. Trace:
I20260709 12:22:03.706061 20810 rpcz_store.cc:276] 0709 12:22:01.035800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.666654 (+630854us) service_pool.cc:224] Handling call
0709 12:22:03.335337 (+1668683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.438393 (+103056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.705434 (+267041us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54002aa8d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:03.705605 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3836407168}
I20260709 12:22:03.683250 22795 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.711045 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:03.703969 22771 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:03.712388 22771 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:03.696801 22827 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:03.696399 22857 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.717180 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:03.717849 20776 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.617s (SENT)
I20260709 12:22:03.715629 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.725019 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d46401ecb9b443118647ee361081a01e"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54488
I20260709 12:22:03.725617 20881 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.725972 20881 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:03.735088 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:03.734561 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:03.739029 21431 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:03.713768 22615 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:03.741777 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.743860 20804 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:22:03.744064 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80340.4us
W20260709 12:22:03.748768 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:03.751117 21435 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:03.719106 22827 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:03.758148 21434 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:03.717594 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:03.759881 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:03.760324 22838 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.001s
I20260709 12:22:03.763792 21433 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:03.764030 22838 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:03.766418 20729 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [2.4-4.5]
I20260709 12:22:03.766530 22838 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:03.760555 22559 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.705s	user 0.013s	sys 0.000s
I20260709 12:22:03.767510 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.767570 22559 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.766950 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.769187 20731 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:03.769155 20735 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:03.779933 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.780659 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.743005 22783 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:03.781651 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.749970 22742 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:03.709915 22795 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:03.784693 20734 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:22:03.789573 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:03.810127 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:03.791592 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.795517 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.795709 22704 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:03.796267 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 99292.8us
I20260709 12:22:03.829624 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:03.796960 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.830549 20804 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.797349 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:03.797765 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:03.834789 21431 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:03.798731 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.795537 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.830703 20735 raft_consensus.cc:2376] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3. Re-sending same reply.
I20260709 12:22:03.803503 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.803983 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:03.772822 22827 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:03.828881 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.766582 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.839145 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.835204 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.838050 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:03.847967 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.854511 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.758535 22857 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:03.866976 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.867806 20734 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:03.869263 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.870330 20735 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:03.871599 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.876886 22618 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.831370 22783 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:03.881461 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.866195 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:03.888883 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.890404 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.893424 20730 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:22:03.901693 20729 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.901553 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.907752 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.862212 22703 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:03.911569 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.912608 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.929118 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:03.862769 20726 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:03.931401 20728 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:03.988461 20727 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:03.989393 20730 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:03.989950 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:03.992375 20731 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:03.994664 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.995482 21431 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:03.996555 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:03.997248 21431 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:22:03.999228 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 133067us
I20260709 12:22:03.911947 20875 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 2: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:04.018550 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.005633 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 94993.3us
I20260709 12:22:03.890009 22783 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.004660 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.030356 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:04.030738 20850 leader_election.cc:336] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.555s (SENT)
I20260709 12:22:04.009153 20733 raft_consensus.cc:2376] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3. Re-sending same reply.
I20260709 12:22:04.036437 20735 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:04.039036 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:04.047154 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.048147 21431 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:04.049157 20734 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:04.056954 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:04.064474 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 346610us
I20260709 12:22:04.057922 20809 raft_consensus.cc:2376] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4. Re-sending same reply.
W20260709 12:22:04.086205 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.035862 22723 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:04.097288 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.060909 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:04.101539 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 271237us
I20260709 12:22:04.102913 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:04.104597 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 316664us
I20260709 12:22:04.107573 20733 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:04.110311 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:04.111707 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111683us
I20260709 12:22:04.040517 22704 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:04.116421 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.118412 20876 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
W20260709 12:22:04.126495 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65711.3us
I20260709 12:22:04.120008 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:04.126725 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 96035.6us
I20260709 12:22:04.127168 20733 raft_consensus.cc:2376] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2. Re-sending same reply.
I20260709 12:22:04.111092 21431 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:04.121218 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:04.121634 22837 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:04.127662 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.128496 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:04.125972 20876 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:22:04.133060 22795 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:04.134662 22771 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.032214 22765 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 12:22:04.137557 20733 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:04.137010 22639 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.138192 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:04.141571 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.143532 20806 raft_consensus.cc:2376] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2. Re-sending same reply.
I20260709 12:22:04.156651 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:04.145763 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.040s	sys 0.085s
I20260709 12:22:04.157480 20728 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
W20260709 12:22:04.157799 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.040s	sys 0.085s
I20260709 12:22:04.162036 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:04.162593 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.162698 20808 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:04.143497 20876 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 12:22:04.166083 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.166410 20876 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:04.170674 21435 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:04.137945 22704 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.174049 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.175993 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:04.177076 20805 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
W20260709 12:22:04.178447 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.179248 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:04.147233 22552 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:04.182485 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.185160 20806 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:04.187515 21431 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
I20260709 12:22:04.202318 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:04.203496 20809 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:22:04.203660 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.206902 20801 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:04.210251 22857 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:04.221102 22872 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.003s
I20260709 12:22:04.221562 22872 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:04.222234 22872 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:04.223250 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.223383 20728 raft_consensus.cc:2376] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3. Re-sending same reply.
I20260709 12:22:04.224011 21431 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:04.224625 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.225425 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:04.226470 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.232366 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:04.233496 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.234477 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.243793 21427 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:04.246480 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.244953 20880 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:04.247610 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.247951 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.235483 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:04.251641 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:22:04.250106 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58524.2us
I20260709 12:22:04.253002 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:04.255542 20729 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:04.256474 20734 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:04.262524 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:04.234393 22763 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.000s
I20260709 12:22:04.266099 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.267561 20808 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
W20260709 12:22:04.236927 22293 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.006s
I20260709 12:22:04.268663 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:04.237793 22814 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.669s	user 0.013s	sys 0.009s
I20260709 12:22:04.277184 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:04.278223 21431 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:04.279428 20807 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
W20260709 12:22:04.280999 20881 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.555s	user 0.018s	sys 0.007s
W20260709 12:22:04.281330 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.195518 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:04.282693 20804 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:04.283708 20810 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:04.282924 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:04.281389 20881 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.235863 22639 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:04.292305 20882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.005s	sys 0.021s
I20260709 12:22:04.290882 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.291685 22559 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.240437 22837 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.293149 20728 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:04.293861 20882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.006s	sys 0.021s
W20260709 12:22:04.293990 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.295504 20881 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:04.276455 22813 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.000s
I20260709 12:22:04.297240 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:04.297848 20881 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:04.298852 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.299669 20729 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:04.313412 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:04.297441 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.315081 21431 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:04.312741 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.318405 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.319862 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:04.320644 20876 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:04.293216 20875 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:04.288218 22833 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:04.326808 20884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.018s	sys 0.012s
W20260709 12:22:04.327275 20884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.018s	sys 0.012s
I20260709 12:22:04.298987 22795 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.332432 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:04.328604 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.334084 20729 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:04.303668 22788 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:04.336582 20775 leader_election.cc:336] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.708s (SENT)
I20260709 12:22:04.280580 22293 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:04.342404 20878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.019s	sys 0.048s
I20260709 12:22:04.339695 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:04.343051 20878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.020s	sys 0.049s
I20260709 12:22:04.341540 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:04.343672 20808 raft_consensus.cc:2376] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4. Re-sending same reply.
I20260709 12:22:04.344089 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:04.350926 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.351162 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.361821 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:04.342573 22742 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.007s
I20260709 12:22:04.365077 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.367409 20727 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:04.376132 20879 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.013s	sys 0.019s
W20260709 12:22:04.377475 20879 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.013s	sys 0.020s
I20260709 12:22:04.339619 22795 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:04.364583 21429 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.062s	sys 0.070s
W20260709 12:22:04.383492 20849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 2.638s (SENT) [suppressed 55 similar messages]
I20260709 12:22:04.384552 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:04.385442 20878 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:04.348145 22559 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
W20260709 12:22:04.390904 20849 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.579s (SENT)
I20260709 12:22:04.397516 20729 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:04.391050 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:04.393925 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.399314 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.387223 22773 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.745s	user 0.007s	sys 0.006s
W20260709 12:22:04.387394 21429 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.063s	sys 0.071s
I20260709 12:22:04.404688 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:04.413663 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:04.405440 20881 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
W20260709 12:22:04.405157 22618 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.008s
I20260709 12:22:04.432228 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:04.433880 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:04.419994 22793 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.714s	user 0.017s	sys 0.010s
I20260709 12:22:04.405007 22771 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.468519 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:04.467470 20850 leader_election.cc:336] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.812s (SENT)
W20260709 12:22:04.466499 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.470901 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:04.440572 22639 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:04.482695 22639 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.483141 22639 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.439543 22837 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:04.438324 22618 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:04.434792 22742 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:04.490702 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:04.419495 22773 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.495333 20878 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.503105 20733 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:04.503353 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:04.493404 22891 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:22:04.495033 22837 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.512804 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 537) took 1005 ms. Trace:
I20260709 12:22:04.493330 22814 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.518211 20732 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:22:04.523084 20775 leader_election.cc:336] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.837s (SENT)
I20260709 12:22:04.525506 21433 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:04.526186 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 382923us
W20260709 12:22:04.528136 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.527375 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 480068us
I20260709 12:22:04.540740 21431 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:04.542346 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 351125us
I20260709 12:22:04.513789 22618 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:04.546252 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:04.521765 22837 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.500941 22742 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.554214 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:04.539237 22293 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.572350 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:04.575928 20775 leader_election.cc:336] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.882s (SENT)
I20260709 12:22:04.576572 20775 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:04.583132 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 300249us
I20260709 12:22:04.586465 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.537871 22723 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:04.552438 21428 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:04.541962 22793 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:04.589135 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:04.531095 22814 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.591380 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:04.601778 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:04.541664 22540 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:22:04.601238 20877 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:04.597064 20775 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.601s (SENT)
I20260709 12:22:04.603626 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:04.567866 22705 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:04.605208 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 432861us
I20260709 12:22:04.607689 21431 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:04.611924 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 91194.5us
I20260709 12:22:04.607102 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:04.557628 20776 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.771s (SENT)
I20260709 12:22:04.613385 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:04.629381 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.613968 21400 connection.cc:256] RPC call timeout handler was delayed by -1.4876242899999852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
I20260709 12:22:04.635223 20878 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:04.590780 22618 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:04.642084 20807 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.586486 22763 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:04.647296 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 55926us
W20260709 12:22:04.645576 20850 leader_election.cc:336] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.517s (SENT)
I20260709 12:22:04.652683 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:04.653527 20808 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:04.606570 22705 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.660499 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.665027 20701 leader_election.cc:400] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3.
W20260709 12:22:04.669100 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 123478us
I20260709 12:22:04.615563 22908 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:04.642299 22703 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:04.666657 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:04.679725 20776 connection.cc:256] RPC call timeout handler was delayed by -1.098577782999655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
W20260709 12:22:04.680641 20776 connection.cc:256] RPC call timeout handler was delayed by -1.093852871999843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2622s for a response
I20260709 12:22:04.649295 22765 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:04.663481 22783 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.002s
I20260709 12:22:04.662159 22795 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:04.706667 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:04.707517 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54077us
I20260709 12:22:04.666399 22705 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:04.710889 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1494) took 1017 ms. Trace:
W20260709 12:22:04.711244 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54862.6us
I20260709 12:22:04.714113 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:04.721045 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.694890 22773 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.716900 20879 rpcz_store.cc:276] 0709 12:22:03.693639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.734846 (+ 41207us) service_pool.cc:224] Handling call
0709 12:22:03.757294 (+ 22448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.757555 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:04.381205 (+623650us) log.cc:844] Serialized 14 byte log entry
0709 12:22:04.709257 (+328052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:04.709339 (+    82us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:04.710608 (+  1269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.710694 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.710827 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9842,"prepare.queue_time_us":83144,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"thread_start_us":867592,"threads_started":3}
I20260709 12:22:04.707810 22783 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.726877 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:04.723174 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:04.729580 20776 leader_election.cc:336] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.622s (SENT)
W20260709 12:22:04.744390 22897 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:22:04.744150 22618 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.746414 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 541) took 1082 ms. Trace:
I20260709 12:22:04.749120 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 653) took 1020 ms. Trace:
I20260709 12:22:04.749570 21434 rpcz_store.cc:276] 0709 12:22:03.728328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.775312 (+ 46984us) service_pool.cc:224] Handling call
0709 12:22:03.808622 (+ 33310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:03.834053 (+ 25431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:03.834109 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:04.197606 (+363497us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b18002c1040. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a
0709 12:22:04.198325 (+   719us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:04.199127 (+   802us) log.cc:844] Serialized 52 byte log entry
0709 12:22:04.495359 (+296232us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:04.495411 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.495556 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:04.748868 (+253312us) raft_consensus.cc:1685] finished
0709 12:22:04.748889 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.749054 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1611,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":15657,"spinlock_wait_cycles":65186432,"thread_start_us":214784,"threads_started":1,"wal-append.queue_time_us":383808}
I20260709 12:22:04.751063 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:04.751545 22893 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 12:22:04.752754 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.752866 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1490) took 1075 ms. Trace:
I20260709 12:22:04.723685 22906 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:04.754125 21435 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.753280 20884 rpcz_store.cc:276] 0709 12:22:03.677877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.677980 (+   103us) service_pool.cc:224] Handling call
0709 12:22:03.690181 (+ 12201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:03.714299 (+ 24118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:03.714330 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:03.718401 (+  4071us) log.cc:844] Serialized 26 byte log entry
0709 12:22:04.344519 (+626118us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:04.344580 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.344663 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:04.752671 (+408008us) raft_consensus.cc:1685] finished
0709 12:22:04.752691 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.752813 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5481,"raft.queue_time_us":15620,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":15299,"thread_start_us":554027,"threads_started":1,"wal-append.queue_time_us":747193}
I20260709 12:22:04.755168 22913 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:04.755633 22913 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.755963 22913 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:04.757965 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56272.7us
I20260709 12:22:04.759500 20876 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:04.725247 22892 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 12:22:04.731308 22898 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 12:22:04.725956 22886 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:22:04.725394 22827 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.844s	user 0.004s	sys 0.008s
I20260709 12:22:04.722875 22908 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:04.743184 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:04.763717 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:04.724958 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:04.771013 20808 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:04.762203 21435 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:04.705937 22765 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.792171 21435 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:04.792426 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1640) took 1705 ms. Trace:
I20260709 12:22:04.761075 22918 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:04.800804 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71314.2us
W20260709 12:22:04.801783 20776 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 3.079s (SENT)
I20260709 12:22:04.804999 22837 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.806173 22560 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:04.792990 22833 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.807869 22833 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:04.809320 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86296.3us
I20260709 12:22:04.765677 22906 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:04.811753 22742 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:04.741721 22723 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.812652 22920 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:04.813064 21426 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:04.780814 22919 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.792866 20883 rpcz_store.cc:276] 0709 12:22:03.087078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.467675 (+380597us) service_pool.cc:224] Handling call
0709 12:22:03.506334 (+ 38659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:03.523963 (+ 17629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:03.524014 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:03.524601 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:22:04.158425 (+633824us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:04.158491 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.158702 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:04.792204 (+633502us) raft_consensus.cc:1685] finished
0709 12:22:04.792227 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.792364 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":633391,"threads_started":1,"wal-append.queue_time_us":980977}
I20260709 12:22:04.785753 22827 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.811581 22906 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.786144 22857 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:04.816543 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.811616 20726 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.003s
I20260709 12:22:04.813086 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:04.814896 22929 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:04.820019 21435 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.789202 21434 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:04.816898 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 203620us
I20260709 12:22:04.747016 21427 rpcz_store.cc:276] 0709 12:22:03.664121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.670332 (+  6211us) service_pool.cc:224] Handling call
0709 12:22:04.746118 (+1075786us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400746b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:04.746326 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197524224}
I20260709 12:22:04.829586 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:04.829654 22827 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.831423 22919 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.832459 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:04.832614 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:04.829778 22552 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.830299 22773 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:04.810913 22833 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:04.810765 22639 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:04.851205 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.862241 22837 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.862921 22837 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:04.842218 20850 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.004s (SENT)
I20260709 12:22:04.863739 22919 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.864187 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:04.865077 22742 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:04.867663 21432 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.029s	sys 0.052s
W20260709 12:22:04.868099 21432 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.029s	sys 0.053s
I20260709 12:22:04.868343 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:04.868965 20883 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.861392 22814 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:04.873637 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 538) took 1351 ms. Trace:
I20260709 12:22:04.838244 20875 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:04.878389 20878 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:04.879261 22639 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.879665 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:04.880448 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:04.880103 22908 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.887065 20883 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:04.825479 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:04.888226 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 1809) took 1124 ms. Trace:
I20260709 12:22:04.888299 22765 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:04.888700 22894 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:22:04.888666 20726 rpcz_store.cc:276] 0709 12:22:03.764186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.766246 (+  2060us) service_pool.cc:224] Handling call
0709 12:22:04.888156 (+1121910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:04.891345 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1487) took 1214 ms. Trace:
I20260709 12:22:04.895180 21431 raft_consensus.cc:2368] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:04.826398 22929 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:04.860109 22763 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:04.861394 22773 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:04.891837 20882 rpcz_store.cc:276] 0709 12:22:03.677045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.677153 (+   108us) service_pool.cc:224] Handling call
0709 12:22:03.683699 (+  6546us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:03.696189 (+ 12490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:03.696223 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:03.715633 (+ 19410us) log.cc:844] Serialized 52 byte log entry
0709 12:22:04.294348 (+578715us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:04.294386 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.294464 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:04.891077 (+596613us) raft_consensus.cc:1685] finished
0709 12:22:04.891106 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.891272 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4191,"raft.queue_time_us":8345,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2899,"thread_start_us":506814,"threads_started":1,"wal-append.queue_time_us":797565}
I20260709 12:22:04.908598 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:04.909641 22912 raft_consensus.cc:2764] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:22:04.910279 22615 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:04.910599 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 543) took 1245 ms. Trace:
I20260709 12:22:04.910640 22837 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:04.911963 22704 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.911695 22929 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.913221 21434 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:04.912949 22931 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:04.874136 21432 rpcz_store.cc:276] 0709 12:22:03.517970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.518130 (+   160us) service_pool.cc:224] Handling call
0709 12:22:03.524221 (+  6091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.524533 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:03.848127 (+323594us) log.cc:844] Serialized 14 byte log entry
0709 12:22:04.868592 (+1020465us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:04.868659 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:04.868706 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.868783 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.868901 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":600527,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1208,"thread_start_us":1337672,"threads_started":2,"wal-append.queue_time_us":691591}
I20260709 12:22:04.893308 22631 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.930301 20883 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.514107 21430 rpcz_store.cc:276] 0709 12:22:03.500724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.500875 (+   151us) service_pool.cc:224] Handling call
0709 12:22:03.511562 (+ 10687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.555554 (+ 43992us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:04.014521 (+458967us) log.cc:844] Serialized 14 byte log entry
0709 12:22:04.500064 (+485543us) log.cc:844] Serialized 14 byte log entry
0709 12:22:04.500236 (+   172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:04.500279 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:04.505396 (+  5117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.505478 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.505667 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":620949,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":5425,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5529,"thread_start_us":845841,"threads_started":2}
I20260709 12:22:04.935550 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:04.945988 21434 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:04.912969 22919 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:04.871161 22552 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.944865 22763 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:22:04.963907 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 657) took 1106 ms. Trace:
I20260709 12:22:04.964082 22765 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:04.964483 22765 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:04.965420 22765 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.965907 22552 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:04.966431 22765 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:04.890787 22705 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.968112 22552 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.968695 22552 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.969158 22705 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:04.969901 21435 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:04.950788 22814 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:22:04.970763 22924 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:04.971341 22919 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.972692 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:04.973507 22919 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:04.973693 22924 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:04.974215 22924 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:04.976475 22552 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:04.976642 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:04.977599 22540 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:04.977151 20883 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:04.978145 22540 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:04.978807 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.975101 22924 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:04.948155 22833 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:22:04.960170 22872 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:04.960770 22783 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:04.961263 21430 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.962007 22929 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:04.955081 22704 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 12:22:04.962689 22915 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:04.977449 22742 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:04.948550 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166767us
I20260709 12:22:04.896831 22913 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.987463 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.989838 21430 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:04.964289 21431 rpcz_store.cc:276] 0709 12:22:03.857264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.645481 (+788217us) service_pool.cc:224] Handling call
0709 12:22:04.963690 (+318209us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:04.963844 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492471680}
I20260709 12:22:04.929203 22773 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:04.971024 22814 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:04.999038 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:04.983021 22872 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:04.945551 22615 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:05.002547 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:05.017073 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.001360 21431 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:22:05.063846 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.993808 22773 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.076910 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:05.078857 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:04.994287 22814 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:04.993935 22763 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.980226 22631 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.078526 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:05.083865 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75808us
W20260709 12:22:05.079744 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62771.8us
I20260709 12:22:04.979102 22540 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:04.988917 22793 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:22:05.110128 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:05.000640 22927 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.111066 22927 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:05.111825 22923 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:22:05.112908 22936 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:05.113380 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 109006us
I20260709 12:22:05.113474 22936 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.114048 22927 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:05.114324 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.115586 22793 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:05.116078 22793 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:05.115435 22814 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:05.117035 22793 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.118299 22793 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.118206 22763 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:05.119349 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 140793us
I20260709 12:22:05.117372 22814 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:04.992982 22813 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.094s
I20260709 12:22:05.138706 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:05.005304 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:05.139724 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:05.144002 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:05.160468 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.149202 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:05.164099 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49875us
I20260709 12:22:05.149682 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:05.189435 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.211496 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:05.212875 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68986.7us
I20260709 12:22:05.145057 22908 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:22:05.208025 22927 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.217981 22927 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.145491 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:05.166375 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:05.219372 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:05.214468 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.148387 22837 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:05.212229 21430 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 1.
I20260709 12:22:05.236478 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:05.237510 21430 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:05.146353 22919 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:05.224220 20730 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:05.231575 21431 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:05.223629 20729 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:05.205104 22705 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:05.249105 22540 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:05.258564 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.252741 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:05.255615 20850 leader_election.cc:336] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.680s (SENT)
I20260709 12:22:05.223863 22833 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:05.258693 20849 leader_election.cc:400] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:22:05.262203 20849 leader_election.cc:403] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:05.247689 22784 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:05.264757 21430 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:22:05.230763 22813 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:05.261495 22919 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:05.244827 22928 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:05.255169 22837 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:05.280416 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 447834us
I20260709 12:22:05.244971 22931 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:22:05.257599 22783 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:05.287719 22540 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3.
I20260709 12:22:04.944432 21429 rpcz_store.cc:276] 0709 12:22:03.664724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.667340 (+  2616us) service_pool.cc:224] Handling call
0709 12:22:03.667690 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.668039 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:04.437101 (+769062us) log.cc:844] Serialized 14 byte log entry
0709 12:22:04.900832 (+463731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:04.903400 (+  2568us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:04.910294 (+  6894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.910394 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.910537 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":525497,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":92032,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":207,"thread_start_us":1105086,"threads_started":3,"wal-append.queue_time_us":343437}
I20260709 12:22:05.288601 22783 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:05.289661 22783 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.279633 22813 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:05.270119 22930 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:05.264335 22833 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:05.292912 22615 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:05.296440 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:05.324810 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:05.352520 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95884.3us
I20260709 12:22:05.353160 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1597) took 1239 ms. Trace:
I20260709 12:22:05.353578 20810 rpcz_store.cc:276] 0709 12:22:04.113942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.114062 (+   120us) service_pool.cc:224] Handling call
0709 12:22:04.283462 (+169400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.284203 (+   741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.352944 (+1068741us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54003d5ed0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:05.353097 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347964672}
W20260709 12:22:05.349903 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53176.5us
I20260709 12:22:05.356523 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.221357 22872 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.360313 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1577) took 1293 ms. Trace:
I20260709 12:22:05.281957 22927 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:05.358287 22838 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:05.364799 22838 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:05.365828 22913 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:05.365777 22838 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:05.367213 22838 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:05.367858 22939 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:22:05.371021 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.372604 22763 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:05.375379 21429 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:05.375811 21429 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:05.376066 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.376603 21429 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.377686 21429 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:05.377235 22837 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.292943 22813 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:05.384092 22935 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:22:05.293351 22833 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:05.388475 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 92096.8us
I20260709 12:22:05.362092 20734 rpcz_store.cc:276] 0709 12:22:04.062807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.063086 (+   279us) service_pool.cc:224] Handling call
0709 12:22:04.256244 (+193158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.264479 (+  8235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.355803 (+1091324us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54003b4050. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:05.355931 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350294912}
I20260709 12:22:05.394469 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:05.395939 22908 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.396668 22940 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:05.396878 22813 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:05.396673 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.399401 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.403688 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.408595 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:05.412843 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.407111 22540 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:22:05.416160 22615 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:05.395246 22930 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:05.402947 22908 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.406275 22813 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:05.438627 22913 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:05.455816 21429 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:05.456723 22793 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.457146 22936 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:05.449366 22615 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:22:05.407112 22928 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 12:22:05.465412 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61186us
I20260709 12:22:05.459762 22913 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:05.450039 22945 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.409765 22948 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
W20260709 12:22:05.469053 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.450879 22813 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:05.466436 22921 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.741s	user 0.008s	sys 0.005s
I20260709 12:22:05.472258 22913 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:05.485705 22813 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:05.442683 22919 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.579921 20807 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:05.565201 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:05.565443 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 563) took 1763 ms (client timeout 2236 ms). Trace:
I20260709 12:22:05.566314 21432 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:22:05.566823 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 163202us
I20260709 12:22:05.568194 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.568948 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:05.571798 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:05.572405 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.577916 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:05.583778 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:05.587179 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:05.587620 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.580945 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.584374 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:05.581736 21433 rpcz_store.cc:269] 0709 12:22:03.801599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.246170 (+444571us) service_pool.cc:224] Handling call
0709 12:22:05.490068 (+1243898us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:05.565381 (+ 75313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616087936}
I20260709 12:22:05.585248 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.585453 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.585832 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.588927 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.590058 20806 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.591814 20809 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:05.592141 20881 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:22:05.593631 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:05.591725 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.596769 20881 pending_rounds.cc:85] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:22:05.597349 20879 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:22:05.597860 20805 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 1.
I20260709 12:22:05.565065 22615 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:05.580164 22929 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.586s	user 0.014s	sys 0.004s
I20260709 12:22:05.600579 20883 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:05.578114 22940 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:05.604519 22615 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:05.604908 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:05.625850 20877 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:05.605448 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:05.605844 22952 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.606535 20801 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:05.608800 22908 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.591692 22937 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:05.610970 20884 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:05.611567 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:05.612216 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47122.3us
I20260709 12:22:05.631642 20875 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:05.635399 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.635584 20875 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:05.636696 20875 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.637892 20875 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.612730 22903 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:05.639093 22903 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:05.640071 22903 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.641412 22937 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:05.642333 22937 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:05.643365 22954 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:05.602572 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:05.645628 20882 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:22:05.645876 20877 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:05.616014 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 373) took 1305 ms. Trace:
W20260709 12:22:05.616772 20732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 1815) took 1848 ms (client timeout 1905 ms). Trace:
I20260709 12:22:05.597114 20881 pending_rounds.cc:107] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.6
W20260709 12:22:05.633315 22902 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.909s	user 0.004s	sys 0.005s
I20260709 12:22:05.617599 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:05.646853 20732 rpcz_store.cc:269] 0709 12:22:03.768474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.778310 (+  9836us) service_pool.cc:224] Handling call
0709 12:22:05.616597 (+1838287us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54002986d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:05.616704 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931248896}
I20260709 12:22:05.648100 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.648435 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1571) took 1775 ms. Trace:
I20260709 12:22:05.648834 20731 rpcz_store.cc:276] 0709 12:22:03.873486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.993281 (+119795us) service_pool.cc:224] Handling call
0709 12:22:03.993631 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.993901 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:04.504061 (+510160us) log.cc:844] Serialized 14 byte log entry
0709 12:22:05.416923 (+912862us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:05.647863 (+230940us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b1800313660. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b
0709 12:22:05.648037 (+   174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:05.648087 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:05.648162 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.648239 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:05.648382 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49834,"prepare.queue_time_us":296491,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":100760,"spinlock_wait_cycles":41641728,"thread_start_us":561507,"threads_started":3,"wal-append.queue_time_us":200150}
I20260709 12:22:05.618294 22953 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:05.619645 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:05.651125 20883 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:05.592681 22944 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.142s	user 0.000s	sys 0.010s
I20260709 12:22:05.620616 20735 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:05.621984 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.624536 20806 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:05.646450 20878 rpcz_store.cc:276] 0709 12:22:04.311085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.636134 (+325049us) service_pool.cc:224] Handling call
0709 12:22:04.878216 (+242082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.878756 (+   540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.615830 (+737074us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54001ffc50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:05.615953 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555590400}
I20260709 12:22:05.632295 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.614434 20630 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 2 to 3, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:22:05.627672 22763 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:22:05.580260 22773 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.618908 21434 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.646998 20881 raft_consensus.cc:2917] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Aborting config change with OpId 4.6: Aborted: Op aborted by new leader
I20260709 12:22:05.650161 22813 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:05.666289 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44395.6us
I20260709 12:22:05.668470 20881 raft_consensus.cc:2889] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:05.668537 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.667655 22838 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:05.669945 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.676405 22615 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:05.660430 22936 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:05.701745 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.680991 20630 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 4, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 4 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:22:05.666326 22928 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:05.687595 21434 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:05.691497 20850 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.114s (SENT)
I20260709 12:22:05.691780 20877 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:05.651762 22930 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:05.692603 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.692972 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.645776 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:05.694150 22773 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:05.694765 22813 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:05.713584 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46047.7us
I20260709 12:22:05.708809 22615 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:05.714139 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 386) took 1577 ms. Trace:
I20260709 12:22:05.695147 20882 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:05.714406 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.646648 22788 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:05.675954 22940 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.695922 20878 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:05.678119 22924 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:05.705929 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:05.706768 20883 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:05.709651 20881 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:05.710280 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:05.702276 22948 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:05.708411 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 123033us
I20260709 12:22:05.720067 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.742708 20630 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 4, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 4 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:05.733633 22937 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.746059 20630 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 3, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:05.747490 20630 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 2 to 4, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 4 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:05.748250 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.751684 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1578) took 1651 ms. Trace:
I20260709 12:22:05.708563 22838 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:05.744621 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:05.753929 20877 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
W20260709 12:22:05.744872 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106350us
I20260709 12:22:05.766072 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:05.769155 22813 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.748083 22827 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:05.755244 21434 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:05.722673 22788 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:05.752986 20735 rpcz_store.cc:276] 0709 12:22:04.100443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.135505 (+ 35062us) service_pool.cc:224] Handling call
0709 12:22:05.620434 (+1484929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:05.667220 (+ 46786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.751478 (+ 84258us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54002986d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:05.751628 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3414189952}
I20260709 12:22:05.793486 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 673) took 1574 ms. Trace:
I20260709 12:22:05.793879 21434 rpcz_store.cc:276] 0709 12:22:04.218686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.869210 (+650524us) service_pool.cc:224] Handling call
0709 12:22:04.896001 (+ 26791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:05.618529 (+722528us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b4c001bd7c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1cf6ba20 00007f9e1cf7ca29 00007f9e1cf42644 00007f9e1cf69f2d 00007f9e1cf69b93 00007f9e1ec9a18b 00007f9e1ec9acb5 00007f9e1ec95f30 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416
0709 12:22:05.793088 (+174559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.793422 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3834240}
W20260709 12:22:05.795253 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 103799us
I20260709 12:22:05.743677 22559 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.760185 22615 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:05.798928 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.710896 22908 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:22:05.714576 21426 rpcz_store.cc:276] 0709 12:22:04.136363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.796229 (+659866us) service_pool.cc:224] Handling call
0709 12:22:04.796553 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.813529 (+ 16976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.713897 (+900368us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:05.714080 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37485952}
I20260709 12:22:05.715096 22929 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:22:05.803260 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:05.803992 22928 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.804493 22833 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.721098 22927 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:22:05.760617 22838 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:05.815980 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1830) took 2014 ms. Trace:
I20260709 12:22:05.816407 20727 rpcz_store.cc:276] 0709 12:22:03.801741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.991075 (+189334us) service_pool.cc:224] Handling call
0709 12:22:04.035070 (+ 43995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.367831 (+332761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.815733 (+1447902us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b540028d554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:05.815916 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763858688}
W20260709 12:22:05.817404 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51480.6us
I20260709 12:22:05.817638 20876 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:05.772459 22956 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:05.799170 22639 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 12:22:05.831440 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:05.814384 22814 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.013s
I20260709 12:22:05.797202 22958 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.838013 20727 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:05.764912 22948 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.841360 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:05.723104 22930 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.811306 22615 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:05.846501 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.817078 22945 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.849269 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:05.823021 22956 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:05.829191 22813 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:05.835896 22907 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.108s	user 0.006s	sys 0.008s
I20260709 12:22:05.855991 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:05.831707 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:05.858489 20806 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:05.858647 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.865347 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.866204 20806 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:05.856019 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:05.858752 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.859575 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.870934 20810 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:05.860350 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.860831 20808 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:05.861353 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:05.875236 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60946.9us
I20260709 12:22:05.873610 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.876680 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:05.876778 20806 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:05.878424 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.879279 20805 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:05.881803 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.883718 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.884641 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.885830 20807 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:05.887425 22833 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:05.887712 20801 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:05.887748 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:05.888439 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:05.888829 21435 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:05.889166 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:05.864373 22903 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.853915 22956 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:05.888010 22813 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:05.891499 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:05.892050 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.816594 22928 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.901347 21433 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:05.902581 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:05.902843 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:05.871968 22964 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.908780 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.913465 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:05.934834 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:05.917344 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.937330 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.928478 21431 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:05.939412 20808 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.940618 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.930776 20807 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:05.903811 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:05.920042 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.954701 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.938843 20880 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:22:05.933331 22952 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.922022 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.927472 20875 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:22:05.963301 20875 pending_rounds.cc:85] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:22:05.963639 20875 pending_rounds.cc:107] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:22:05.963912 20875 raft_consensus.cc:2889] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:05.964214 22953 raft_consensus.cc:1064] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:22:05.932626 22964 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:05.924731 22615 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:05.954986 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.955698 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.890288 22903 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.945294 22945 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:05.954799 22962 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:05.922933 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.960758 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.955533 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:05.984390 20807 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.961721 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:05.932235 22913 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.945096 22742 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:05.965593 20875 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:05.909693 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:05.970975 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.007411 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:05.980351 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:05.878196 22838 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:05.981676 22963 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:06.009578 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:05.957821 22958 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:06.011140 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:05.924492 22948 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.002539 20807 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:06.002797 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:05.987778 22957 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:22:05.972412 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:05.954473 22814 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:22:05.984083 22962 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.013314 22952 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:06.013638 22928 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.014261 22969 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:06.014555 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:06.015403 20804 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:06.020552 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 306193us
W20260709 12:22:06.021354 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49061.1us
I20260709 12:22:06.012364 22771 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.024724 22964 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:06.025100 20880 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:06.006496 20803 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:06.017951 20801 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:06.016620 21429 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:06.016072 22945 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:06.035113 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:06.020368 22742 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:06.022758 22962 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:06.042196 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:06.051699 22945 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:06.059453 22771 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:06.059832 22945 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:06.036149 22958 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.052079 22953 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.067325 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.022892 22813 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.032039 22948 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:06.077302 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.042243 22833 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:06.010851 22559 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:22:06.048451 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.055590 20730 raft_consensus.cc:606] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:06.071223 20734 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:06.073153 20878 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:06.084033 20730 raft_consensus.cc:614] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:22:06.085210 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51137.9us
I20260709 12:22:06.078054 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:06.037464 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.036576 22952 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:06.086287 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.085393 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:06.081712 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.086920 20728 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:06.083575 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:06.036386 22969 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.054751 22913 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.053785 22783 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:06.092350 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.077490 22948 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.098182 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.053748 22903 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.094986 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.101691 20733 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:06.093577 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:06.099395 22918 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:22:06.104187 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.104455 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.104799 21433 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:06.093407 22919 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:22:06.105264 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.047489 22928 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:22:06.107180 22903 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.107612 22928 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:06.107440 22952 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.108081 22928 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:06.107798 22903 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.109037 22928 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.110369 22928 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:06.110960 21426 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:06.100445 22752 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:06.091449 22958 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:22:06.098898 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.095513 22920 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:06.115623 22918 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:06.116276 22918 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:06.105135 21431 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:06.095978 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:06.117242 22918 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.118027 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:06.119550 22961 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.114415 22903 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:06.120326 20879 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:06.120705 20879 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:06.100796 22924 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:06.121150 22924 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:06.121579 22924 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.121465 20879 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.122390 20879 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:06.088299 22922 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.354s	user 0.007s	sys 0.000s
I20260709 12:22:06.123697 22963 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.052647 22742 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.106134 22919 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:06.078107 22784 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.095142 22964 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:06.138577 20733 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:06.098130 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.099084 22948 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:06.114383 20734 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 2.
I20260709 12:22:06.113948 22752 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:06.092583 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 59335.6us
I20260709 12:22:06.168684 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.171676 22975 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:06.172536 22975 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.173058 22975 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:06.127424 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:06.177488 20775 connection.cc:256] RPC call timeout handler was delayed by -1.2992100819997177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
W20260709 12:22:06.178087 20775 connection.cc:256] RPC call timeout handler was delayed by -1.293142705000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
W20260709 12:22:06.178522 20775 connection.cc:256] RPC call timeout handler was delayed by -1.2542100820000996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
W20260709 12:22:06.179005 20775 connection.cc:256] RPC call timeout handler was delayed by -1.1831100819999847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
W20260709 12:22:06.179531 20775 connection.cc:256] RPC call timeout handler was delayed by -1.1653120229998422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 12:22:06.180186 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:06.180598 20775 connection.cc:256] RPC call timeout handler was delayed by -1.0681120230001397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 12:22:06.181041 20775 connection.cc:256] RPC call timeout handler was delayed by -1.0410366329997487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 12:22:06.125066 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:06.129756 20730 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:06.096166 22788 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.139302 20733 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:06.185801 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60808.8us
I20260709 12:22:06.131110 22958 consensus_queue.cc:1243] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:22:06.196120 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.166815 22936 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.142643 22964 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:06.201035 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 83082us
I20260709 12:22:06.202843 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:06.205000 20730 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.130601 22919 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:06.207815 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.208056 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:06.216512 22963 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:06.217111 22963 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.217517 22963 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:06.195600 22615 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:06.218153 22615 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:06.218294 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:06.220289 22615 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.221076 20734 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:06.222383 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.223115 20734 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:06.223740 20728 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.203897 20733 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.202885 22913 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.141418 22784 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.231825 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:06.232134 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:06.232345 22969 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.309258 22969 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:06.221120 22978 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.311493 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.236028 22752 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:06.314184 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.315263 21434 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEARNER]: Advancing to term 5
I20260709 12:22:06.316350 22984 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:06.294624 22903 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:22:06.233379 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:06.322569 20734 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:06.234269 22919 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.227954 22927 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:22:06.235925 20879 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:06.240335 22788 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.242539 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:06.242919 22945 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.245505 22975 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:06.246560 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 76066.9us
I20260709 12:22:06.205879 22964 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:06.272367 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.314620 22924 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.333006 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:06.333732 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:06.285071 20726 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:06.288358 22966 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:06.299530 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:06.309301 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:06.306182 21431 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:06.306664 21429 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:06.307904 22784 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.345432 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:06.279016 22960 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.547s	user 0.003s	sys 0.000s
W20260709 12:22:06.324723 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106507us
I20260709 12:22:06.280849 22773 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:06.216517 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:06.340459 21434 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:06.342691 20878 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:06.354482 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.343494 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:06.346694 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:06.347963 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.350854 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:06.351789 21430 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:06.351995 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109508us
I20260709 12:22:06.359844 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:06.367451 20882 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:06.367761 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:06.369342 20806 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:06.369973 21431 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:22:06.371018 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 154614us
I20260709 12:22:06.365912 21433 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:06.371817 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:06.352994 22773 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:06.349790 22919 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:06.374104 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 145370us
I20260709 12:22:06.304932 22962 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.372627 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.324553 22913 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.379243 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.385807 21434 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:06.389238 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:06.396055 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45332.7us
I20260709 12:22:06.397348 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:06.370777 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:06.403488 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.404585 21433 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:06.376447 21426 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.403453 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:06.409372 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.378260 22986 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:06.414567 22961 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.406761 22953 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:06.426834 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.426208 21434 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.412338 22742 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:06.412227 22788 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:06.414852 20880 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:06.421404 22986 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.435401 21433 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:06.407516 22993 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:06.415409 20875 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:06.420466 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 542) took 2756 ms. Trace:
I20260709 12:22:06.407053 22765 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:22:06.444597 22975 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.426802 22953 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.421618 22928 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:22:06.491460 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.442504 22838 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.447163 22631 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.443889 22924 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:06.490373 22956 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.556s	user 0.004s	sys 0.008s
I20260709 12:22:06.446444 22833 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:06.498075 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92914.1us
I20260709 12:22:06.435125 22985 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:06.479676 22986 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.482128 22993 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:06.516907 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.498003 22953 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.489002 21428 rpcz_store.cc:276] 0709 12:22:03.664454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.677688 (+ 13234us) service_pool.cc:224] Handling call
0709 12:22:04.511033 (+833345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.595784 (+ 84751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:06.403051 (+1807267us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b5400592250. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:06.414415 (+ 11364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882863488}
I20260709 12:22:06.549425 22631 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:06.560469 22631 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:06.560832 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.561566 22631 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.562780 22631 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.562987 20726 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:06.435316 22992 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.119s
I20260709 12:22:06.588030 20730 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:06.525527 22993 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.519742 22956 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.489601 22963 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.561964 22991 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.518146 20801 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:06.620122 22765 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:06.620669 22765 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:06.621066 22765 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.621595 20804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:22:06.622247 20804 raft_consensus.cc:606] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:22:06.622545 20804 raft_consensus.cc:614] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:06.623492 22975 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:22:06.624673 22987 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:22:06.625114 22962 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:06.630028 20809 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:06.630683 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.632056 20879 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:22:06.632261 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 108210us
I20260709 12:22:06.634902 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:06.635145 20877 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:06.631556 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 263929us
W20260709 12:22:06.636592 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 257605us
I20260709 12:22:06.637516 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:06.605157 22945 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:06.639855 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:06.639053 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:06.627272 22918 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.629913 22966 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:06.644271 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.644589 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.645016 20804 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:06.648604 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.633201 22985 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:06.617168 22948 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.638375 22913 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:06.641281 22784 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.675819 22784 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.642155 22908 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:06.678382 22903 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:06.678647 22908 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:06.678876 22903 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.679181 22908 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:06.679281 22903 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:06.678642 22784 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:06.653347 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:06.654838 20727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:06.683327 20727 raft_consensus.cc:606] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:06.683442 20804 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.648761 22827 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:22:06.652655 23006 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:22:06.664222 22988 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 12:22:06.702346 20806 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:06.707445 23006 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:06.708034 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 675) took 2025 ms. Trace:
I20260709 12:22:06.708775 23006 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.708487 21432 rpcz_store.cc:276] 0709 12:22:04.682294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.948755 (+266461us) service_pool.cc:224] Handling call
0709 12:22:05.565144 (+616389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:05.586281 (+ 21137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:06.707825 (+1121544us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54001e69d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:06.707971 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337297152}
I20260709 12:22:06.664459 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:06.666435 22948 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.667625 22925 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:06.672350 23009 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:06.672910 22963 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.674180 22956 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:06.695767 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:06.717713 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64478.3us
I20260709 12:22:06.714573 21435 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:06.654079 22945 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:06.722157 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:06.726049 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.757908 21431 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:06.758188 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 113662us
I20260709 12:22:06.628290 22991 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.648300 22918 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.662539 22552 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:06.721175 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:06.750422 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.754042 20805 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.754338 22953 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.727725 22925 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:06.757095 20808 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:06.754559 22765 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.764269 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.767483 20804 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:06.769644 21432 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:06.768355 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 123503us
I20260709 12:22:06.771431 20805 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:06.769297 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:06.777323 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:06.764037 22945 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.779466 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.655164 23002 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 12:22:06.781298 22903 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.756451 23005 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.718509 22964 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.768955 22552 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:06.797614 22552 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.798882 22552 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.786582 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:06.777199 22976 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:06.791613 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.785113 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:06.789347 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:06.789593 20804 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:06.793054 22703 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:06.785627 22945 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.807103 22703 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.810523 23007 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:06.810683 22703 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:06.811286 23007 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:06.811740 22999 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:22:06.795071 20806 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:22:06.796612 22900 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 2.002s	user 0.004s	sys 0.002s
I20260709 12:22:06.794076 22991 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:06.813350 20802 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:06.779220 22784 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:22:06.784322 22838 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:22:06.817813 22742 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:06.811671 23007 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:06.857875 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72824.1us
I20260709 12:22:06.822283 22930 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.824818 22872 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.824589 22703 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.827832 22765 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:06.827930 22978 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.000s
I20260709 12:22:06.809397 22925 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.804777 22976 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:06.829627 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.883947 21435 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.831418 20805 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.847749 21431 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:06.854367 20879 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:22:06.821182 23003 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:22:06.853477 21432 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:06.770851 23009 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:22:06.786644 22913 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:22:06.858690 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:06.892755 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:06.859586 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:06.880465 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:06.896164 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:06.882104 20877 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:06.897475 20879 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:06.887176 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.826699 22986 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.869413 22903 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:06.807250 22985 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:06.868264 22991 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:06.859820 22948 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:22:06.860426 23007 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.899315 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:06.920799 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:06.917217 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:06.911487 20805 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:06.881510 23005 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:06.924060 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.925035 21431 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 2: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:06.885109 22978 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:06.888206 22976 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.932665 22918 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.912379 22964 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.904386 22765 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:06.942654 22293 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:06.941424 20879 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:06.940652 21435 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
W20260709 12:22:06.951121 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.951121 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:06.948472 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 96133.9us
I20260709 12:22:06.902997 22742 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:06.903604 23009 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.904147 22703 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:06.956133 22293 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:06.951234 20881 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:06.961714 20802 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:06.927021 22962 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:06.963125 20878 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:06.947520 22925 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.964550 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:06.965240 22838 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:06.927706 23002 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:06.926843 23005 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:06.965713 22293 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:06.970221 20879 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:06.970125 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:06.955493 22956 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:06.958921 22765 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:06.957171 22742 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:06.986313 22293 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:06.980501 22919 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 12:22:06.990196 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:06.962646 22975 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:22:06.980420 22953 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:06.993028 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:06.978721 22964 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.995528 21435 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.999470 20882 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:06.977691 22962 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:07.001919 20879 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:07.003616 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:07.004820 20882 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:06.987975 22742 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:06.986266 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.014012 20882 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:07.019862 23005 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.021256 23005 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.007998 22956 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:07.030781 21435 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:07.031996 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:07.022439 21429 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:07.030920 22560 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.050458 22560 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:07.051270 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.052026 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.031039 22994 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:07.055853 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:07.036543 22953 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:07.029489 22765 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.075712 22986 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:07.039593 22956 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:07.086575 20730 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:07.104591 23010 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.128939 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:07.130510 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:07.134629 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:07.129860 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:07.133680 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:07.141760 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:07.144951 20731 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:07.145905 20804 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.146819 20728 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:07.148411 20729 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.137804 20735 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:07.142905 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:07.150885 20734 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:07.151752 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.154726 20802 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:07.141808 22765 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:07.161072 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.165714 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1782) took 1305 ms. Trace:
W20260709 12:22:07.135167 22793 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:22:07.122960 22540 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:07.168184 20802 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:07.126995 22978 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:07.170334 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 185342us
W20260709 12:22:07.187006 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.166167 20810 rpcz_store.cc:276] 0709 12:22:05.860569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.897676 (+ 37107us) service_pool.cc:224] Handling call
0709 12:22:05.928161 (+ 30485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:05.934809 (+  6648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:05.934842 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:06.944184 (+1009342us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:06.949981 (+  5797us) log.cc:844] Serialized 26 byte log entry
0709 12:22:07.023440 (+ 73459us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:07.023479 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.023555 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:07.165348 (+141793us) raft_consensus.cc:1685] finished
0709 12:22:07.165374 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.165648 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48936,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":14744,"spinlock_wait_cycles":9930496,"thread_start_us":45638,"threads_started":1,"wal-append.queue_time_us":203199}
I20260709 12:22:07.168717 23010 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.174106 20805 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:07.127074 22953 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:07.191423 22953 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.195253 23010 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:07.202667 20728 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:07.206750 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:07.209270 20804 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:07.211154 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 316) took 1288 ms. Trace:
I20260709 12:22:07.213616 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:07.173367 22948 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.175525 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:07.176426 22540 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.214632 23024 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.203956 22978 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.216042 23013 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:07.204877 22986 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:07.241436 22986 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.243615 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.244418 20805 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
W20260709 12:22:07.245455 22908 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 12:22:07.245882 22908 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.246433 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.246913 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.247524 20802 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:07.247596 22994 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:07.248091 22994 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:07.248844 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:07.249442 20802 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:07.250026 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:07.247983 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:07.125706 22945 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.136168 22962 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.179834 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:07.263901 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:07.243861 22986 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:07.217617 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:07.267474 22293 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:22:07.268049 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50517.2us
I20260709 12:22:07.217795 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.269914 22963 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:07.272497 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:07.217870 20807 rpcz_store.cc:276] 0709 12:22:05.922854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.019323 (+ 96469us) service_pool.cc:224] Handling call
0709 12:22:06.019728 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:06.020014 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:06.034596 (+ 14582us) log.cc:844] Serialized 14 byte log entry
0709 12:22:07.077129 (+1042533us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b1800401680. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:22:07.210826 (+133697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:07.210879 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:07.210923 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.210990 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.211099 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17445,"prepare.queue_time_us":14367,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":843,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":9134,"spinlock_wait_cycles":4804864,"thread_start_us":132608,"threads_started":1}
I20260709 12:22:07.220522 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:07.253710 22929 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:07.274325 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.220659 20726 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:07.247943 22993 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.000s
W20260709 12:22:07.221256 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 319) took 1291 ms (client timeout 1626 ms). Trace:
W20260709 12:22:07.283921 20808 rpcz_store.cc:269] 0709 12:22:05.929579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.940369 (+ 10790us) service_pool.cc:224] Handling call
0709 12:22:07.221076 (+1280707us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b540019aad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:07.221203 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874527616}
I20260709 12:22:07.284468 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.273465 20884 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:07.227850 21431 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:07.234690 20877 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:07.279939 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:07.255848 20810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad57e68a6a114847ac252579307658f7"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:22:07.232906 21434 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:07.285161 20807 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:07.235289 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1781) took 1375 ms. Trace:
I20260709 12:22:07.287474 20809 rpcz_store.cc:276] 0709 12:22:05.860274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.881413 (+ 21139us) service_pool.cc:224] Handling call
0709 12:22:07.235017 (+1353604us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:07.235141 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716231552}
I20260709 12:22:07.287205 20810 raft_consensus.cc:606] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:07.289328 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.290234 20805 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:07.211386 22615 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.256294 20880 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:07.293454 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:07.253516 22872 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:07.221761 22955 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:22:07.219340 22948 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.220678 22793 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:07.221355 22540 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.306022 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.278226 21429 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.307523 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:07.224124 22795 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:07.229848 22978 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:07.233520 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 670) took 3015 ms. Trace:
I20260709 12:22:07.218106 22560 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:07.282502 22937 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:07.294737 22927 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:07.292896 22552 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.318655 22784 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:07.276129 22765 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:22:07.271111 22952 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:22:07.225883 22814 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
W20260709 12:22:07.359491 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52094.4us
I20260709 12:22:07.393221 22937 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:07.304883 22969 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:07.302799 22991 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:07.299625 22963 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:07.362491 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48805.6us
I20260709 12:22:07.364010 20877 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:07.223805 23024 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.319253 23025 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:22:07.297101 22993 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.391050 20884 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:07.300099 22929 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:22:07.301789 22763 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:22:07.305034 23006 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:22:07.231506 22956 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.308772 23010 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:22:07.308291 22975 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:22:07.302997 22615 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.316061 22793 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.357154 22913 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:22:07.375550 22927 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.361148 21427 rpcz_store.cc:276] 0709 12:22:04.217844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.828859 (+611015us) service_pool.cc:224] Handling call
0709 12:22:04.829302 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:06.089683 (+1260381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:06.089726 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:06.097972 (+  8246us) log.cc:844] Serialized 26 byte log entry
0709 12:22:06.202123 (+104151us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:06.202188 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.233083 (+1030895us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540029ca54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:07.233138 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:07.233189 (+    51us) raft_consensus.cc:1685] finished
0709 12:22:07.233206 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.233456 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23252,"raft.queue_time_us":26587,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5802,"spinlock_wait_cycles":2810445184,"thread_start_us":95316,"threads_started":1,"wal-append.queue_time_us":122742,"wal-append.run_cpu_time_us":1508,"wal-append.run_wall_time_us":1003688}
I20260709 12:22:07.416086 22773 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.413589 22639 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:22:07.403146 20801 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 2. Current term is 4. Ops: []
I20260709 12:22:07.418457 22903 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:07.405802 20733 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:07.400813 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.518656 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:07.309882 22918 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 12:22:07.383970 22930 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:07.499431 22991 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:07.277980 23023 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 12:22:07.428619 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.432170 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.445626 22784 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:07.448832 22945 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:07.451105 23011 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:07.454061 22976 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.456614 22966 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.458281 22994 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.503316 22872 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:22:07.412778 22948 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:07.412240 23013 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:07.493911 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 216628us
I20260709 12:22:07.464682 22969 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.501003 22560 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:07.506718 20881 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:07.478780 22908 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:07.438709 22552 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.371126 22742 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:07.651417 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:07.590459 23011 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:07.576442 22975 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.493432 22783 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:22:07.503639 22765 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:07.384503 22964 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:07.623672 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1612) took 1387 ms. Trace:
W20260709 12:22:07.563429 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 142903us
W20260709 12:22:07.547457 23007 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.005s
W20260709 12:22:07.581549 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 191588us
I20260709 12:22:07.547852 22927 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:07.576392 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:07.556316 22978 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.503511 20726 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:07.419044 22838 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:07.565994 22930 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:07.626376 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52686.6us
I20260709 12:22:07.578162 22784 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:07.655349 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:07.559289 22293 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:22:07.623803 22966 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:07.701257 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 197291us
I20260709 12:22:07.666395 22925 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:07.703019 22293 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:07.703445 22293 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:07.656847 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 160147us
W20260709 12:22:07.668076 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.647235 23013 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:07.710836 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55622.9us
I20260709 12:22:07.654222 22952 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:22:07.573501 22903 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:07.717456 22293 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.685966 20880 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:07.655269 23006 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:07.688233 20879 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:07.688624 23041 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:07.591707 22963 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.689072 22838 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.689997 22930 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:07.692654 23031 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
W20260709 12:22:07.693488 23033 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:22:07.694068 22763 raft_consensus.cc:993] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:22:07.694536 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:07.643658 22929 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:22:07.659149 22765 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:07.677320 20803 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:07.665906 20801 rpcz_store.cc:276] 0709 12:22:06.203339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.402563 (+1199224us) service_pool.cc:224] Handling call
0709 12:22:07.402936 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.562752 (+159816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.590954 (+ 28202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:07.720016 22925 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:07.706792 20733 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:07.712538 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:07.802688 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1856) took 1674 ms. Trace:
W20260709 12:22:07.802969 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 90490.9us
I20260709 12:22:07.803104 20733 rpcz_store.cc:276] 0709 12:22:06.127968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.128113 (+   145us) service_pool.cc:224] Handling call
0709 12:22:06.138379 (+ 10266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.404774 (+1266395us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400351c20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec14a1d 00007f9e1ebfb1a4 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419
0709 12:22:07.802381 (+397607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.802626 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262205696}
I20260709 12:22:07.803934 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:07.649878 22969 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.656158 22948 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:22:07.710253 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:07.722363 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 161383us
I20260709 12:22:07.667408 22975 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.726014 22293 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.711253 22912 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:07.678282 23007 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.709097 22908 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:07.685298 22978 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.717105 22814 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:07.559496 22937 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.700186 22540 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.725629 23041 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:07.802047 23045 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:07.814440 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 104260us
I20260709 12:22:07.714641 23011 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.824007 21431 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:07.816048 22795 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:07.812585 22618 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:22:07.759581 22993 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:07.728968 23006 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.793376 23024 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:07.845947 22872 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:07.714655 22783 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:22:07.853682 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:07.751418 22962 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:07.722173 22991 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.818693 23039 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.815646 22912 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:07.819311 22908 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:07.813086 22986 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:07.844811 22918 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.847508 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.804975 22784 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.853055 22814 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:07.906121 21430 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:07.865036 23044 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.874375 22293 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.912462 22293 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.893548 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:07.872030 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68191.8us
W20260709 12:22:07.871608 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 141865us
I20260709 12:22:07.868706 23040 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:07.919603 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.920421 22293 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:07.922848 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.890573 22559 raft_consensus.cc:1081] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:22:07.875208 22872 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.925951 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:07.890458 23052 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.870421 23048 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.921074 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.852773 22765 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.905606 22827 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:07.934773 22293 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:07.939556 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.942065 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.912338 23053 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.906211 22552 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:07.936185 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.930970 22986 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:07.914616 22742 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:07.890671 22962 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.906644 22918 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:07.929687 23035 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 12:22:07.963485 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69989.2us
I20260709 12:22:07.926357 20801 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:07.969738 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.893508 22795 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:22:07.972780 21430 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:07.863384 23045 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:07.943940 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75986.7us
I20260709 12:22:07.997182 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.997926 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:08.000121 20733 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:07.909116 22837 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:22:07.945683 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:07.941542 22872 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.886901 22925 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.983855 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:07.967319 22827 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.012199 22827 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.913229 23051 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
W20260709 12:22:08.015641 23046 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 12:22:07.961061 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:07.991266 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:07.988682 21426 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:07.988801 22765 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.929991 23040 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.961761 22783 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:08.030853 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:07.968884 22784 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:07.994647 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:08.035285 20809 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:07.974534 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:07.891004 23039 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:07.962159 23048 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:08.009819 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.988070 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.016896 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1663) took 1686 ms. Trace:
I20260709 12:22:08.064368 20876 rpcz_store.cc:276] 0709 12:22:06.330879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.331288 (+   409us) service_pool.cc:224] Handling call
0709 12:22:06.344914 (+ 13626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:06.356653 (+ 11739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:06.356685 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:06.366715 (+ 10030us) log.cc:844] Serialized 26 byte log entry
0709 12:22:07.461092 (+1094377us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b1800375b80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:22:07.641690 (+180598us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:07.641728 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.642038 (+   310us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:08.016627 (+374589us) raft_consensus.cc:1685] finished
0709 12:22:08.016647 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.016837 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5315,"raft.queue_time_us":10349,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5919,"spinlock_wait_cycles":2923264,"thread_start_us":180400,"threads_started":1,"wal-append.queue_time_us":1343807}
I20260709 12:22:07.991966 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:08.022310 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.023842 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:07.994400 22903 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:08.002246 21429 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.002s
I20260709 12:22:07.994228 22838 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.084107 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:07.921722 23044 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.037158 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:08.037498 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:08.010481 22742 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:22:07.966325 22908 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.050414 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:08.063241 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:08.098946 20727 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.099227 20732 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:08.085595 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.027730 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:08.008947 22952 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:22:08.091702 20728 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:08.092114 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54704us
I20260709 12:22:08.041430 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:08.106611 20730 raft_consensus.cc:606] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:08.092808 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.063877 22540 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.080294 23007 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:07.987769 22986 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.004069 22814 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.084503 21429 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:08.101382 23055 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:22:08.104234 23056 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:22:08.105286 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.062933 22978 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.119496 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:08.094293 20877 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:07.973733 22912 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.062299 23051 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:08.084237 23047 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.091572 22837 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.206552 22837 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.101203 23063 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:08.180114 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.080168 22783 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:07.962321 23052 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:08.134557 23042 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
I20260709 12:22:07.974989 23053 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:08.216148 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.206943 23041 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.215507 22978 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.245949 23052 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.282080 20727 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:08.221535 20726 raft_consensus.cc:2410] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:08.291244 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.296496 22926 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.298875 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.202386 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 585) took 1100 ms. Trace:
I20260709 12:22:08.215226 23063 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.312613 20728 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:08.216979 23047 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.319715 23062 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.321831 22986 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.285292 23006 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.220978 23051 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.305002 23041 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.286685 22978 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.357789 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1677) took 1079 ms. Trace:
I20260709 12:22:08.359618 20884 rpcz_store.cc:276] 0709 12:22:07.278811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.589106 (+310295us) service_pool.cc:224] Handling call
0709 12:22:07.630287 (+ 41181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:07.658174 (+ 27887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.658209 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:07.675418 (+ 17209us) log.cc:844] Serialized 26 byte log entry
0709 12:22:07.912190 (+236772us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:07.912235 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.912309 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:08.357528 (+445219us) raft_consensus.cc:1685] finished
0709 12:22:08.357552 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.357722 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16187,"raft.queue_time_us":11452,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":20160,"thread_start_us":198264,"threads_started":1,"wal-append.queue_time_us":300118}
I20260709 12:22:08.343822 20805 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:08.304714 21429 rpcz_store.cc:276] 0709 12:22:07.081346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.104221 (+ 22875us) service_pool.cc:224] Handling call
0709 12:22:08.181245 (+1077024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:08.352073 23024 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.377737 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.327967 23063 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:08.388074 20877 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:08.335139 22962 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.388625 20877 pending_rounds.cc:85] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 1
I20260709 12:22:08.390226 20877 pending_rounds.cc:107] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:08.390529 20877 raft_consensus.cc:2889] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:08.384225 22952 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.384055 23045 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.393978 23040 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.403605 23072 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.405356 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.406365 20877 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:08.397763 22918 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.339371 22926 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.387805 23024 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:08.412942 23029 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.024s	user 0.005s	sys 0.000s
I20260709 12:22:08.416700 20877 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:08.404217 23072 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.358937 23062 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.405032 22952 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.419351 23053 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:08.702822 23074 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.290s	user 0.000s	sys 0.000s
I20260709 12:22:08.712857 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:08.487147 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:08.617576 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:08.619410 20701 connection.cc:256] RPC call timeout handler was delayed by -1.1058477779997702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
W20260709 12:22:08.540876 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:08.715468 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.716056 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:08.717123 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:08.425817 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 134251us
I20260709 12:22:08.725270 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.726949 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.729076 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.729410 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.730242 20810 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:08.731003 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.731729 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.732060 20805 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:08.732584 20810 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:08.667513 22827 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.002s
W20260709 12:22:08.609920 23071 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:22:08.411221 23048 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.524930 22978 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.487648 23039 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:08.530948 22966 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.816s	user 0.000s	sys 0.009s
I20260709 12:22:08.486775 20801 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:22:08.682475 23024 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.422997 22783 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.742414 22936 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:08.735514 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:08.582998 23077 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:22:08.773401 23068 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:22:08.735520 22969 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.896s	user 0.002s	sys 0.011s
I20260709 12:22:08.774612 22969 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.714831 23010 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.752663 23085 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:22:08.755875 22615 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.756842 22978 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:08.714768 22926 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.733664 23053 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.734159 22955 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.681609 23040 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:08.524811 23047 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.431488 23044 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.628042 23005 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:08.763559 20801 pending_rounds.cc:85] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 6
I20260709 12:22:08.741866 22993 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:08.714967 23051 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.771466 22783 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:08.778452 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 391) took 1113 ms. Trace:
I20260709 12:22:08.806427 20878 rpcz_store.cc:276] 0709 12:22:07.665214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.052341 (+387127us) service_pool.cc:224] Handling call
0709 12:22:08.052753 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.053153 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.053193 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:08.081106 (+ 27913us) log.cc:844] Serialized 26 byte log entry
0709 12:22:08.296122 (+215016us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:08.296172 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.296246 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:08.778227 (+481981us) raft_consensus.cc:1685] finished
0709 12:22:08.778247 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.778394 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":32633,"thread_start_us":158988,"threads_started":1,"wal-append.queue_time_us":253372}
I20260709 12:22:08.808844 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.809934 20878 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:08.779181 20875 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:08.748265 22827 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.796243 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1528) took 1906 ms. Trace:
I20260709 12:22:08.812109 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.811945 20879 rpcz_store.cc:276] 0709 12:22:06.889843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.011909 (+122066us) service_pool.cc:224] Handling call
0709 12:22:07.015654 (+  3745us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:07.722127 (+706473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.795910 (+1073783us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b5400059e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:08.796168 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546115968}
I20260709 12:22:08.787034 23048 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:08.784147 22969 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.804061 20801 pending_rounds.cc:107] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:22:08.825834 20879 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:08.814960 20878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad57e68a6a114847ac252579307658f7"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54488
I20260709 12:22:08.827838 20878 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:22:08.805320 22930 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.050s	user 0.011s	sys 0.002s
I20260709 12:22:08.839418 22930 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.841509 22976 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.766577 22966 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.844628 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 392) took 1120 ms. Trace:
I20260709 12:22:08.844940 22966 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.845012 20884 rpcz_store.cc:276] 0709 12:22:07.724094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.389877 (+665783us) service_pool.cc:224] Handling call
0709 12:22:08.402733 (+ 12856us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.402952 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.405129 (+  2177us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:08.412101 (+  6972us) log.cc:844] Serialized 26 byte log entry
0709 12:22:08.785043 (+372942us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:08.785076 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.785141 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:08.844391 (+ 59250us) raft_consensus.cc:1685] finished
0709 12:22:08.844417 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.844565 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":292785,"threads_started":1,"wal-append.queue_time_us":359304}
W20260709 12:22:08.808640 22784 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.003s
I20260709 12:22:08.811506 22993 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.847793 20807 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:08.819653 20877 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:08.849617 22993 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.829618 20801 raft_consensus.cc:2889] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:08.850939 22952 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.821595 22827 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.822297 20734 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:08.781352 22936 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:08.804800 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.805136 23025 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.795325 23005 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:08.830811 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.788883 22955 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.786828 23053 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.834870 20878 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.792778 23040 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:08.861814 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 123289us
I20260709 12:22:08.865234 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:08.865798 20877 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:08.834935 23062 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.870301 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.877391 22936 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.877847 22793 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.832010 23048 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.879706 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.880328 20879 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:08.876412 20878 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.883291 22955 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.883919 23005 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.884985 23005 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:08.886085 22773 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.871891 22962 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:22:08.887748 20878 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.862586 23025 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.890866 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:08.892483 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:08.875321 22925 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:22:08.895690 22955 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:08.872941 23063 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.898658 20876 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:22:08.879170 22872 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.853s	user 0.008s	sys 0.004s
I20260709 12:22:08.899001 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:08.901100 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.901741 20878 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:08.904235 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.904959 20810 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:08.906518 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:08.907068 20879 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.907136 20877 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:08.908267 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.910526 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.877883 23053 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.911149 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:08.890457 23025 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.913331 20810 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:08.873051 22993 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:22:08.897125 22962 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:08.915051 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.915130 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:08.915683 20810 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:08.917191 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:08.917389 20805 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:08.917773 20877 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:08.918292 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:08.918866 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:08.899398 23072 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:08.906277 22955 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:08.906906 22872 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:08.925280 23027 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.560s	user 0.000s	sys 0.003s
I20260709 12:22:08.926193 22872 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.926550 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1673) took 2002 ms. Trace:
I20260709 12:22:08.926905 20882 rpcz_store.cc:276] 0709 12:22:06.924101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.017815 (+ 93714us) service_pool.cc:224] Handling call
0709 12:22:07.018223 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:07.018713 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.018755 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:07.022940 (+  4185us) log.cc:844] Serialized 26 byte log entry
0709 12:22:07.141631 (+118691us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:07.141694 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.141789 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:08.924693 (+1782904us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:08.926248 (+  1555us) raft_consensus.cc:1685] finished
0709 12:22:08.926270 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.926487 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7130,"spinlock_wait_cycles":111834880,"thread_start_us":98323,"threads_started":1,"wal-append.queue_time_us":240402}
I20260709 12:22:08.928671 20729 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:08.921916 20810 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:08.936652 23054 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.994s	user 0.005s	sys 0.001s
I20260709 12:22:08.934885 20810 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:08.938479 23041 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.003s
I20260709 12:22:08.940356 20879 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:08.941170 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:08.926386 22955 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.941493 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:08.944670 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:08.929282 23091 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.945955 23053 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:08.940327 22927 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.240s	user 0.009s	sys 0.004s
I20260709 12:22:08.948113 20877 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:22:08.937878 23072 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.949815 22927 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:08.951959 23072 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:08.953586 22927 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:08.941843 23013 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.219s	user 0.012s	sys 0.000s
I20260709 12:22:08.949159 22962 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:08.938632 23048 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.960139 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.943379 22955 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.963320 20879 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:08.953950 23072 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:08.948837 22993 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:08.983422 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1613) took 2779 ms. Trace:
I20260709 12:22:08.983880 20803 rpcz_store.cc:276] 0709 12:22:06.204391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.797406 (+593015us) service_pool.cc:224] Handling call
0709 12:22:06.797751 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.809484 (+1011733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.983131 (+1173647us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b5400126e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:08.983336 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1960071424}
I20260709 12:22:08.984474 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.984760 21428 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:08.985397 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:08.986056 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 693) took 2351 ms. Trace:
W20260709 12:22:08.987257 23093 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:22:08.962510 22962 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.987913 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:08.987824 21428 rpcz_store.cc:276] 0709 12:22:06.634534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.635865 (+  1331us) service_pool.cc:224] Handling call
0709 12:22:06.649424 (+ 13559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:08.985199 (+2335775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.985809 (+   610us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b540042ed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:08.985995 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5369716096}
I20260709 12:22:09.000463 21433 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:09.001114 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 588) took 1190 ms. Trace:
I20260709 12:22:09.001430 21433 rpcz_store.cc:276] 0709 12:22:07.810814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.810978 (+   164us) service_pool.cc:224] Handling call
0709 12:22:08.999951 (+1188973us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:09.000138 (+   187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:09.000863 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.001060 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11497344}
I20260709 12:22:08.993832 20810 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:08.969014 20879 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:08.995551 20882 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:09.005627 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:09.007647 22793 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:09.007733 23063 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:08.950078 23053 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.009759 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:08.982062 22993 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.003539 22962 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.986382 23045 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.963569 23013 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:09.027583 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 309899us
I20260709 12:22:09.008280 22793 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:09.039495 20879 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:09.034859 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 317775us
I20260709 12:22:09.033382 22964 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.032497 23040 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:09.037164 22993 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:09.054716 20806 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:09.053321 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:09.058928 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1881) took 1025 ms. Trace:
I20260709 12:22:09.059369 20729 rpcz_store.cc:276] 0709 12:22:08.036380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.036520 (+   140us) service_pool.cc:224] Handling call
0709 12:22:08.064787 (+ 28267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.929072 (+864285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.058677 (+129605us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54002986d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:09.058867 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1986113408}
I20260709 12:22:09.060089 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1675) took 2099 ms. Trace:
I20260709 12:22:09.060412 23053 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:09.060902 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1531) took 2015 ms. Trace:
I20260709 12:22:09.060484 20880 rpcz_store.cc:276] 0709 12:22:06.960714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.382389 (+421675us) service_pool.cc:224] Handling call
0709 12:22:07.404461 (+ 22072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.751358 (+346897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.059836 (+1308478us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:09.060028 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632350080}
I20260709 12:22:09.061383 20881 rpcz_store.cc:276] 0709 12:22:07.045492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.459457 (+413965us) service_pool.cc:224] Handling call
0709 12:22:07.459866 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.646903 (+187037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.060696 (+1413793us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:09.060850 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107105024}
I20260709 12:22:09.062754 22837 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:09.062920 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1811) took 1055 ms. Trace:
I20260709 12:22:09.063614 20801 rpcz_store.cc:276] 0709 12:22:08.007822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.048822 (+ 41000us) service_pool.cc:224] Handling call
0709 12:22:08.079736 (+ 30914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:08.850499 (+770763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.062683 (+212184us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540027a954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:09.062859 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865716480}
I20260709 12:22:09.051867 23072 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:22:09.052708 22927 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 12:22:09.068853 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:09.069350 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:09.069859 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:09.070547 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:09.072418 23075 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.948s	user 0.002s	sys 0.000s
I20260709 12:22:09.072844 22929 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:09.072952 22993 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:09.073378 22929 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:09.073932 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1876) took 1085 ms. Trace:
I20260709 12:22:09.074949 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1877) took 1045 ms. Trace:
I20260709 12:22:09.075048 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:09.074956 22784 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:09.075539 20734 rpcz_store.cc:276] 0709 12:22:08.029567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.029680 (+   113us) service_pool.cc:224] Handling call
0709 12:22:08.029992 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.859512 (+829520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.074706 (+215194us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54002981d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:09.074891 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821515648}
I20260709 12:22:09.076773 22784 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:09.077525 22784 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:09.077687 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:09.079615 22827 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:09.080076 22953 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:09.082607 23047 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:09.083034 23047 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.083396 23047 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.084196 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:09.085058 21429 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:09.074388 20733 rpcz_store.cc:276] 0709 12:22:08.029226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.030356 (+  1130us) service_pool.cc:224] Handling call
0709 12:22:08.059366 (+ 29010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.059851 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.059917 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:08.082514 (+ 22597us) log.cc:844] Serialized 564 byte log entry
0709 12:22:08.100553 (+ 18039us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:08.100610 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.100713 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:09.071727 (+971014us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:09.073610 (+  1883us) raft_consensus.cc:1685] finished
0709 12:22:09.073630 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:09.073873 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26602880,"thread_start_us":15933,"threads_started":1,"wal-append.queue_time_us":40474}
I20260709 12:22:09.086485 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:09.052971 22872 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:22:09.088227 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.050987 22964 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.098172 20733 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:09.097321 22906 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:09.100313 22793 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.077884 22784 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:09.104127 22964 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:09.103989 22784 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.105212 22936 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.106199 22936 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.106613 22784 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:09.046763 23045 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.095943 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.107620 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.147871 20881 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:09.108297 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.108906 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.117291 20728 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:09.147907 21432 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:09.118058 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:09.130692 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:09.153045 22993 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:09.154393 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:09.155040 20805 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:09.156105 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:09.157763 23024 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.158435 23024 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:09.164681 22991 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:09.169766 22964 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:09.169809 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:09.172111 21432 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:09.189751 22937 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.190613 22937 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.178375 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:09.201283 23039 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:09.201828 23039 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.202278 23039 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.102769 23047 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:09.243512 22925 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.247859 20730 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:09.484144 22969 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:09.460147 23053 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:09.149489 22927 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.487218 22969 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.247574 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.488539 20727 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:09.489583 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.489827 22969 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:09.490312 20727 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:09.490546 22925 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.248574 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.493199 20735 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:09.494102 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.494431 22925 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:09.425668 23041 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.495383 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.495153 23041 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.496483 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.245522 20728 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:09.342486 20630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 195) took 8994 ms. Trace:
I20260709 12:22:09.506702 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.508178 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.508795 20728 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:09.506567 20630 rpcz_store.cc:276] 0709 12:22:00.348222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.348364 (+   142us) service_pool.cc:224] Handling call
0709 12:22:09.342382 (+8994018us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:06.043647 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:06.046419 (+  2772us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:06.046476 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:22:06.046511 (+    35us) tablet.cc:662] Decoding operations
0709 12:22:06.049042 (+  2531us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:06.049151 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:06.049180 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:22:06.049912 (+   732us) tablet.cc:701] Row locks acquired
0709 12:22:06.049956 (+    44us) write_op.cc:260] PREPARE: finished
0709 12:22:06.050118 (+   162us) write_op.cc:270] Start()
0709 12:22:06.091939 (+ 41821us) write_op.cc:276] Timestamp: P: 1783599726050053 usec, L: 0
0709 12:22:06.094143 (+  2204us) op_driver.cc:348] REPLICATION: starting
0709 12:22:06.203471 (+109328us) log.cc:844] Serialized 2195 byte log entry
0709 12:22:06.935823 (+732352us) op_driver.cc:464] REPLICATION: finished
0709 12:22:07.032997 (+ 97174us) write_op.cc:301] APPLY: starting
0709 12:22:07.036308 (+  3311us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:07.043600 (+  7292us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:07.043823 (+   223us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:07.044981 (+  1158us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:07.045321 (+   340us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:22:07.045394 (+    73us) write_op.cc:312] APPLY: finished
0709 12:22:07.045810 (+   416us) log.cc:844] Serialized 47 byte log entry
0709 12:22:07.046121 (+   311us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:07.046338 (+   217us) write_op.cc:454] Released schema lock
0709 12:22:07.046416 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:07.046998 (+   582us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:07.047027 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":96892,"apply.run_cpu_time_us":3953,"apply.run_wall_time_us":14003,"num_ops":5,"prepare.queue_time_us":159254,"prepare.run_cpu_time_us":34274,"prepare.run_wall_time_us":297438,"raft.queue_time_us":224253,"raft.run_cpu_time_us":32427,"raft.run_wall_time_us":60431,"replication_time_us":841506,"thread_start_us":330275,"threads_started":4,"wal-append.queue_time_us":162429,"wal-append.run_cpu_time_us":48427,"wal-append.run_wall_time_us":1653729}]]}
I20260709 12:22:09.154044 23096 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.346s
I20260709 12:22:09.250810 23063 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.249513 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.492470 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.512457 23047 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:09.512072 23063 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.512899 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.247939 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.357374 22837 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.513787 20734 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:09.514153 22837 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.515334 20881 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:09.515823 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.249917 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:09.497202 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:09.488032 23053 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.517098 20729 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:09.488014 22927 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.498426 23041 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:09.296002 22918 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:09.517957 23041 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:09.518252 22918 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.491328 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.305543 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.520325 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:09.521262 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.498215 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:09.523439 20734 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:22:09.244231 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.518553 23041 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.487547 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:09.527509 20805 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:09.501281 20878 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:22:09.435402 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.564s (SENT) [suppressed 27 similar messages]
W20260709 12:22:09.528734 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93381.7us
I20260709 12:22:09.530301 20727 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:09.535681 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.542186 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:09.542909 21426 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:09.543138 22962 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:09.511865 22913 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:09.514487 23063 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:09.568328 22913 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:09.214690 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.516337 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:09.244561 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.569324 22913 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:09.524535 20884 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:09.524272 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.544139 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:09.544575 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:09.544708 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:09.299790 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.623667 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.705798 20728 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:10.048820 20734 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
W20260709 12:22:09.581427 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.020927 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:10.058921 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:10.059134 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.017740 22837 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.061564 20734 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:22:09.588282 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:09.537495 22936 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:09.971446 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.012s	sys 0.000s
W20260709 12:22:10.000519 22784 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.893s	user 0.013s	sys 0.000s
W20260709 12:22:09.837476 23039 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.635s	user 0.000s	sys 0.014s
I20260709 12:22:10.060185 22991 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:10.074774 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.042938 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:10.037367 21433 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.003s	sys 0.004s
I20260709 12:22:10.019177 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:09.535096 22975 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:09.657795 20877 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:09.888145 22962 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:10.017736 22927 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.021806 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:09.970587 23040 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.005s	sys 0.008s
I20260709 12:22:10.077425 20729 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:09.939160 22955 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.958s	user 0.004s	sys 0.007s
W20260709 12:22:09.963315 22742 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.003s	sys 0.008s
I20260709 12:22:10.077621 22918 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.079488 23039 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:10.076782 21433 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.003s	sys 0.005s
I20260709 12:22:10.079335 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.081077 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.081270 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 708) took 1001 ms. Trace:
I20260709 12:22:10.081832 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.078770 20727 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.058238 20735 raft_consensus.cc:2376] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3. Re-sending same reply.
I20260709 12:22:09.846242 23108 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:10.079419 22742 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.004s	sys 0.010s
I20260709 12:22:10.083593 23108 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.085700 23108 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.083060 23041 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.086476 23039 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.086751 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:10.043200 22793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.012s	sys 0.009s
W20260709 12:22:10.044030 22773 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.016s	sys 0.007s
I20260709 12:22:10.087551 20735 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:10.087730 22773 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.017s	sys 0.008s
I20260709 12:22:10.081670 21433 rpcz_store.cc:276] 0709 12:22:09.080214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.082928 (+  2714us) service_pool.cc:224] Handling call
0709 12:22:09.083251 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:09.083438 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:09.288325 (+204887us) log.cc:844] Serialized 14 byte log entry
0709 12:22:10.080951 (+792626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:10.081005 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:10.081050 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.081111 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.081218 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":487251,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":4600,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":996780,"threads_started":2,"wal-append.queue_time_us":796235}
W20260709 12:22:10.051390 23095 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.483s	user 0.000s	sys 0.013s
I20260709 12:22:10.088514 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:09.570046 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:10.048841 20805 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:10.076375 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:10.078723 23040 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.006s	sys 0.010s
W20260709 12:22:10.049934 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 269365us
I20260709 12:22:10.079394 22955 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.079771 22991 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.082309 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.093521 20729 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:10.094394 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:10.042459 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.003s	sys 0.020s
W20260709 12:22:10.095359 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.003s	sys 0.023s
I20260709 12:22:10.095520 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.086412 20734 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:10.096568 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.097204 20884 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:10.097360 22955 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:10.071573 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.015s	sys 0.000s
I20260709 12:22:10.059629 20882 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:10.048205 20804 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:10.107362 20727 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:10.108505 20727 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:10.109550 20727 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:10.094980 22991 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.113363 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:10.114387 20804 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:10.114642 22837 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:10.096625 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.115209 22837 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.115705 22837 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.116721 20734 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:10.097219 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:10.087570 22793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.013s	sys 0.009s
I20260709 12:22:10.119501 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:10.119782 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.089223 20735 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:10.121655 20729 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:10.130142 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:09.572739 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:10.153795 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.154723 20734 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.140010 23041 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:10.163791 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1646) took 1439 ms. Trace:
I20260709 12:22:10.164379 20806 rpcz_store.cc:276] 0709 12:22:08.727687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.728746 (+  1059us) service_pool.cc:224] Handling call
0709 12:22:10.163513 (+1434767us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54003d6dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:10.163705 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746578176}
I20260709 12:22:10.165907 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:10.166705 20806 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.171176 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:10.171991 20804 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.184589 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1640) took 2142 ms. Trace:
I20260709 12:22:10.185199 20808 rpcz_store.cc:276] 0709 12:22:08.041785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.066865 (+ 25080us) service_pool.cc:224] Handling call
0709 12:22:08.067369 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.248828 (+181459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.248877 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:08.249558 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:22:08.382183 (+132625us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:08.382235 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.176394 (+1794159us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:10.176446 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:10.176491 (+    45us) raft_consensus.cc:1685] finished
0709 12:22:10.176506 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.184490 (+  7984us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":193183,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":9475,"spinlock_wait_cycles":66127488,"thread_start_us":132194,"threads_started":1,"wal-append.queue_time_us":132575,"wal-append.run_cpu_time_us":14456,"wal-append.run_wall_time_us":1646094}
I20260709 12:22:10.188335 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.159874 23041 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.193936 20734 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:10.194952 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.168853 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1654) took 1058 ms. Trace:
I20260709 12:22:10.190248 20808 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:10.196573 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.198347 20804 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:10.178370 22962 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.198655 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:10.203608 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.204413 20734 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:10.190747 23041 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.206465 20806 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:10.217445 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.218117 20734 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:10.220389 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:10.222590 22976 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:10.205667 23039 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:10.223387 23039 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:10.223555 22966 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:10.223770 23039 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.224085 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 1689) took 1241 ms. Trace:
I20260709 12:22:10.224088 22966 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.224663 20877 rpcz_store.cc:276] 0709 12:22:08.982553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.984231 (+  1678us) service_pool.cc:224] Handling call
0709 12:22:10.223871 (+1239640us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b5400305f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:10.224011 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546435456}
I20260709 12:22:10.226387 22966 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.231805 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1534) took 2543 ms. Trace:
I20260709 12:22:10.232208 20876 rpcz_store.cc:276] 0709 12:22:07.688622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.181605 (+492983us) service_pool.cc:224] Handling call
0709 12:22:08.220146 (+ 38541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.900408 (+680262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.231541 (+1331133us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:10.231745 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559847808}
I20260709 12:22:10.237677 20735 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
W20260709 12:22:10.238221 23113 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:10.196519 20801 rpcz_store.cc:276] 0709 12:22:09.110453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.572401 (+461948us) service_pool.cc:224] Handling call
0709 12:22:10.161977 (+589576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:10.243961 22994 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.245671 22929 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.246903 22929 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:10.249441 22837 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:10.252318 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1813) took 1527 ms (client timeout 1792 ms). Trace:
W20260709 12:22:10.252864 20807 rpcz_store.cc:269] 0709 12:22:08.724803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.725014 (+   211us) service_pool.cc:224] Handling call
0709 12:22:10.252053 (+1527039us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54003b6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:10.252241 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279425408}
I20260709 12:22:10.254719 20802 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:22:10.258404 21434 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:10.261540 23049 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:10.261977 23049 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:10.262993 23049 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.263899 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1666) took 3636 ms. Trace:
I20260709 12:22:10.264293 23049 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.264361 20875 rpcz_store.cc:276] 0709 12:22:06.627560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.628346 (+   786us) service_pool.cc:224] Handling call
0709 12:22:08.418683 (+1790337us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b5800003698. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7af8c 00007f9e1bb7aef6 000000000036eb39 00007f9e23bee1e0 00007f9e23cb70c4 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:08.778901 (+360218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:08.815489 (+ 36588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.263643 (+1448154us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b540022f6d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:10.263811 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831678080}
W20260709 12:22:10.269315 23105 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.191s	user 0.002s	sys 0.000s
I20260709 12:22:10.270514 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:10.271587 23116 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.194s	user 0.002s	sys 0.000s
I20260709 12:22:10.276739 23011 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:10.279700 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 395) took 1173 ms. Trace:
I20260709 12:22:10.280321 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.281342 20801 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:10.280217 20880 rpcz_store.cc:276] 0709 12:22:09.105887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.106017 (+   130us) service_pool.cc:224] Handling call
0709 12:22:09.106347 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:09.106680 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:09.106709 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:09.107291 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:22:09.526253 (+418962us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:09.526335 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.526534 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:10.268646 (+742112us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:10.279352 (+ 10706us) raft_consensus.cc:1685] finished
0709 12:22:10.279380 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.279628 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":137778,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":24498432,"thread_start_us":415881,"threads_started":1,"wal-append.queue_time_us":970670}
I20260709 12:22:10.282531 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.285493 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.275903 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.286347 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.286983 20735 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:10.295984 22964 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.334299 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.387025 20878 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:10.386284 22964 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.287714 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.390360 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.392189 20735 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:22:10.393167 20805 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:10.393648 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.396045 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.336001 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.336225 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:10.400249 20880 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:10.337673 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.402489 20731 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:22:10.402959 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1600) took 1306 ms. Trace:
I20260709 12:22:10.403352 20731 rpcz_store.cc:276] 0709 12:22:09.096749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.115376 (+ 18627us) service_pool.cc:224] Handling call
0709 12:22:10.337517 (+1222141us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:10.402889 (+ 65372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5697280}
I20260709 12:22:10.404752 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.405364 20731 raft_consensus.cc:2376] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5. Re-sending same reply.
I20260709 12:22:10.405989 22908 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.406239 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.312811 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.320478 22814 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:10.408411 22814 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.410701 22814 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.368335 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1647) took 1640 ms. Trace:
I20260709 12:22:10.411697 23024 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.411763 20802 rpcz_store.cc:276] 0709 12:22:08.728143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.728295 (+   152us) service_pool.cc:224] Handling call
0709 12:22:10.254492 (+1526197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:10.255122 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.368075 (+112953us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54003b6d50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:10.368258 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3509011840}
I20260709 12:22:10.335594 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.326061 23013 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.413437 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:10.357537 20810 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:10.414304 20876 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:10.414585 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1650) took 1305 ms. Trace:
I20260709 12:22:10.317407 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.414952 20810 rpcz_store.cc:276] 0709 12:22:09.109045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.110937 (+  1892us) service_pool.cc:224] Handling call
0709 12:22:09.111380 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:10.357249 (+1245869us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b540011f6a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca11c0 00007f9e1eca2231 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c
0709 12:22:10.414324 (+ 57075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.414532 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100088704}
I20260709 12:22:10.416139 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.417138 20733 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:22:10.299055 23120 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:22:10.290768 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.418919 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:10.420188 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:10.421242 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:10.305148 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.423781 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:10.419116 20734 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:10.427606 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:10.427913 20734 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:10.417881 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.433619 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:10.438181 20883 raft_consensus.cc:2368] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:10.204780 22962 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.436319 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.439842 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.445132 20883 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:10.446084 20806 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:10.440505 20734 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:22:10.440852 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.439231 20733 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.441727 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:10.294952 20730 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:10.433255 23119 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:10.445876 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.447000 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.449996 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:10.450281 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:10.450963 20805 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:10.423592 22993 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:10.362076 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1656) took 1210 ms. Trace:
I20260709 12:22:10.452656 22955 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:10.453498 22955 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:10.453945 22955 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:10.282691 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 106147us
W20260709 12:22:10.447583 21400 connection.cc:256] RPC call timeout handler was delayed by -2.0285210809997807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 12:22:10.455579 21400 connection.cc:256] RPC call timeout handler was delayed by -2.0285210809997807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 12:22:10.452077 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:10.710314 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 258301us
I20260709 12:22:10.447531 20734 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 5
I20260709 12:22:10.695497 23049 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:10.711153 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:10.691465 22827 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.711417 20734 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:22:10.714239 20734 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:10.696605 23039 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.719035 23039 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.453199 20803 rpcz_store.cc:276] 0709 12:22:09.151643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.089842 (+938199us) service_pool.cc:224] Handling call
0709 12:22:10.090140 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:10.120783 (+ 30643us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:10.121411 (+   628us) log.cc:844] Serialized 14 byte log entry
0709 12:22:10.202407 (+ 80996us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:10.205023 (+  2616us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:10.207865 (+  2842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.360631 (+152766us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b18003c7480. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:22:10.361759 (+  1128us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:10.361810 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.361992 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7831,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":1599,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2744,"spinlock_wait_cycles":75789056,"thread_start_us":73834,"threads_started":1,"wal-append.queue_time_us":131049}
I20260709 12:22:10.712646 22993 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:10.710887 22937 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.720950 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:10.586714 23110 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:10.723009 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.723621 20732 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:10.724395 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.715620 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.726107 23117 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:10.726495 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:10.726794 23117 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:10.727581 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.728643 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.728394 23118 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.731504 23118 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.733489 23118 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:10.734046 23118 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:10.711884 23049 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.737596 23049 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.738498 22827 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.592422 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:10.587136 22837 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.740656 22827 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.514146 22742 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.182s
I20260709 12:22:10.586597 23053 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:10.591794 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:10.592079 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:10.665225 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:10.550905 22293 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:10.745734 22293 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.746206 22293 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.555148 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:10.752099 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 197048us
I20260709 12:22:10.546289 23011 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:10.549561 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107910us
I20260709 12:22:10.435770 22814 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:10.579730 23130 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:22:10.516181 22618 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.697371 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:10.512849 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83488.7us
I20260709 12:22:10.427692 22986 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.591233 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:10.587911 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1615) took 1341 ms. Trace:
I20260709 12:22:10.545236 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 717) took 1012 ms. Trace:
W20260709 12:22:10.513641 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.455010 22918 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:10.576320 23107 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.491s	user 0.000s	sys 0.003s
W20260709 12:22:10.570277 20775 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.753s (SENT)
W20260709 12:22:10.775822 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 205557us
W20260709 12:22:10.776576 20775 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.766s (SENT)
W20260709 12:22:10.599900 23057 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 2.714s	user 0.009s	sys 0.010s
I20260709 12:22:10.777097 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:10.223127 22976 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.779857 22784 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:10.781567 22293 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.782570 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 703) took 3113 ms. Trace:
I20260709 12:22:10.433516 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:10.783807 22293 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.435792 23038 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:10.785136 23038 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:10.787763 23038 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.518733 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.788515 20881 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:10.578995 22962 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.515107 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:10.789825 20876 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.545228 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.795995 20880 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
W20260709 12:22:10.700490 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.548240 23041 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.705134 20733 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:10.797973 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.798724 20733 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:10.799086 23041 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.585470 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1616) took 1339 ms. Trace:
I20260709 12:22:10.799746 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.800597 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:10.800045 20728 rpcz_store.cc:276] 0709 12:22:09.246375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.049379 (+803004us) service_pool.cc:224] Handling call
0709 12:22:10.070371 (+ 20992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:10.205096 (+134725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:10.205125 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:10.215953 (+ 10828us) log.cc:844] Serialized 26 byte log entry
0709 12:22:10.384875 (+168922us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:10.384930 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.385009 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:10.585248 (+200239us) raft_consensus.cc:1685] finished
0709 12:22:10.585273 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.585400 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21231,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":123,"thread_start_us":175684,"threads_started":2,"wal-append.queue_time_us":228412}
I20260709 12:22:10.801868 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.545109 23119 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:10.803532 20729 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:10.804173 23119 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.521871 20731 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:10.287483 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:10.691864 20701 connection.cc:256] RPC call timeout handler was delayed by -1.3199170170000798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 12:22:10.448817 23124 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.454847 22955 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.421713 22964 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.324793 22930 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:10.585297 23126 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.332s	user 0.000s	sys 0.000s
I20260709 12:22:10.449409 20808 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:10.583150 22912 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.583619 22929 raft_consensus.cc:993] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:22:10.365298 23045 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:10.591490 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:10.583809 23129 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:22:10.453316 20877 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:10.665952 22913 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.729254 20734 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 2.
I20260709 12:22:10.754148 20732 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:10.744863 21434 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:10.768149 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:10.768431 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:10.769040 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.763686 20730 rpcz_store.cc:276] 0709 12:22:09.246149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.512668 (+266519us) service_pool.cc:224] Handling call
0709 12:22:10.587709 (+1075041us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b54003ad9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:10.587841 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1745269120}
I20260709 12:22:10.764456 21432 rpcz_store.cc:276] 0709 12:22:09.533250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.533399 (+   149us) service_pool.cc:224] Handling call
0709 12:22:09.533733 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:09.534074 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:09.534106 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:09.534569 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:22:09.656737 (+122168us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:09.656782 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.656871 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:10.545025 (+888154us) raft_consensus.cc:1685] finished
0709 12:22:10.545044 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.545180 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34891,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":121790,"threads_started":1,"wal-append.queue_time_us":607679}
W20260709 12:22:10.772158 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74857.2us
I20260709 12:22:10.767620 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.773643 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.812012 22929 raft_consensus.cc:1081] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:22:10.813596 20876 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:10.783033 21435 rpcz_store.cc:276] 0709 12:22:07.668803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.669161 (+   358us) service_pool.cc:224] Handling call
0709 12:22:07.669621 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:07.685069 (+ 15448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.685108 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:07.715239 (+ 30131us) log.cc:844] Serialized 26 byte log entry
0709 12:22:10.168337 (+2453098us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b18001a64a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1
0709 12:22:10.169185 (+   848us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:10.169241 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.169337 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:10.584002 (+414665us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:10.782310 (+198308us) raft_consensus.cc:1685] finished
0709 12:22:10.782338 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.782500 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24739,"raft.queue_time_us":138201,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":34134,"spinlock_wait_cycles":16297088,"thread_start_us":109359,"threads_started":1,"wal-append.queue_time_us":157860}
I20260709 12:22:10.815883 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:10.816540 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:10.817003 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:10.818018 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:10.781934 22976 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.449599 22793 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:10.818156 22560 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:10.818542 22293 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:10.818939 22793 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:10.774492 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.819408 22560 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:10.819378 23124 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.819900 22793 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.820976 20727 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:10.822412 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1637) took 2803 ms. Trace:
I20260709 12:22:10.822911 20809 rpcz_store.cc:276] 0709 12:22:08.019607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.040070 (+ 20463us) service_pool.cc:224] Handling call
0709 12:22:08.040366 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.040709 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:09.536758 (+1496049us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b1800423400. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:22:10.034766 (+498008us) log.cc:844] Serialized 14 byte log entry
0709 12:22:10.163174 (+128408us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:10.163798 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:22:10.163972 (+   174us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:10.168494 (+  4522us) log.cc:844] Serialized 14 byte log entry
0709 12:22:10.170058 (+  1564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.170142 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:10.259487 (+ 89345us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b1800423400. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:10.822155 (+562668us) raft_consensus.cc:1685] finished
0709 12:22:10.822187 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:10.822338 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212937,"prepare.queue_time_us":1506471,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":22473,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":20697,"spinlock_wait_cycles":49969280,"thread_start_us":588555,"threads_started":2,"wal-append.queue_time_us":150924}
I20260709 12:22:10.823834 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:10.826529 20733 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:10.826571 20801 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:10.754201 22937 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:10.829661 22293 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:22:10.830579 21433 raft_consensus.cc:2368] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4.
W20260709 12:22:10.832391 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.802886 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:10.832953 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.833563 20728 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:10.829205 21435 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:10.778946 20775 leader_election.cc:336] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.938s (SENT)
W20260709 12:22:10.836700 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57776.7us
W20260709 12:22:10.837594 20775 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.578s (SENT)
I20260709 12:22:10.838053 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:10.758857 21427 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:10.778885 23053 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:10.839452 22293 consensus_queue.cc:1243] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:22:10.863332 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:10.865132 23153 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:10.865188 22937 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:22:10.878541 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 168529us
I20260709 12:22:10.879329 20632 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:10.861243 23038 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:10.880934 23124 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:10.881323 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:10.762149 23011 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:10.774830 23049 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:22:10.828163 23091 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:10.839298 23137 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:22:10.839260 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:10.830281 23133 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:10.771502 22618 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:10.866605 23053 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.980264 20730 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:10.857831 22964 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:10.880458 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:10.982604 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 269207us
I20260709 12:22:10.983424 21435 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:10.952533 21400 leader_election.cc:336] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 2.663s (SENT)
W20260709 12:22:10.982246 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 144669us
I20260709 12:22:10.984535 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:10.839108 23052 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:22:10.952065 20632 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 4 to 5, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:10.931517 23153 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:10.965498 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 184141us
W20260709 12:22:10.904069 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 190469us
I20260709 12:22:10.999202 23153 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.999900 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.000115 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:10.963691 21426 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.965029 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:10.997311 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:10.982771 23053 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:10.986042 20775 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.665s (SENT)
I20260709 12:22:10.996084 20632 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:22:11.038341 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 270292us
I20260709 12:22:11.006605 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:11.040124 20881 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:11.006760 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.007647 22618 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:11.007753 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55262.3us
I20260709 12:22:10.996568 22964 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.010941 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:11.014446 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 196141us
W20260709 12:22:11.026849 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.001605 23144 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:11.040859 20880 raft_consensus.cc:2410] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:10.965102 23091 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.054744 20734 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:11.053896 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:11.038196 22814 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:11.043155 20632 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 5, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:10.995206 20803 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:11.044040 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.013176 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:11.039408 23041 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:10.982916 23133 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:11.052835 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.036798 22918 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:22:11.045878 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:11.061261 22955 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.094924 20632 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.110405 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:11.108958 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63138.2us
I20260709 12:22:11.111109 21431 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:11.041577 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55576.8us
I20260709 12:22:11.099490 20875 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.056257 22993 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:11.099812 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.111459 20632 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:11.052160 22986 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.087527 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.118613 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.106361 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:11.109421 21430 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:11.119786 21435 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.052301 22930 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:11.122918 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.111560 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:11.041517 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:11.056617 22964 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.065896 23011 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.114318 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:11.127982 20809 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.115057 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.115803 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:11.128935 20808 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:22:11.116665 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.130051 21432 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:11.097509 22814 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.118733 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.119098 23157 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:11.119297 21434 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:11.117707 20632 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 3, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.053762 23143 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:11.125437 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83969.3us
I20260709 12:22:11.122605 22930 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:11.131086 22986 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:11.123265 23024 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:22:11.155900 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:11.157476 22814 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:11.112941 22742 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:11.156867 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.131400 21435 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:11.156965 20632 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 4 to 5, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.158332 23157 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:11.164482 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.129042 21431 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.169939 20735 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:11.132050 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:11.171095 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43542us
I20260709 12:22:11.173581 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.116173 22993 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:11.176613 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:11.198148 20632 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 2 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:11.132943 23154 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:11.243541 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67005.7us
I20260709 12:22:11.127310 21427 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:11.259351 20632 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 4, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 4 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.259284 23154 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:11.284992 22618 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:11.290516 23154 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.276526 23091 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:11.271124 20875 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:11.243371 21428 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:11.290930 20632 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:11.249349 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.296532 20632 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 3, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:11.251518 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:11.299443 21434 raft_consensus.cc:2376] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5. Re-sending same reply.
I20260709 12:22:11.299669 23011 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:11.254925 23045 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:11.300202 23011 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:11.298019 20632 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 4, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 4 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.260017 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:11.307641 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47684.9us
I20260709 12:22:11.308372 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:11.245967 20804 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
W20260709 12:22:11.255826 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 65357.6us
I20260709 12:22:11.177192 23133 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:11.285506 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 241779us
I20260709 12:22:11.096586 23144 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.272153 20801 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:11.273114 22993 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.318878 22993 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.319208 22991 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:11.319819 23123 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:11.198743 23157 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:11.323239 22991 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:11.323624 23045 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:11.324085 23045 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:11.323699 23157 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.324219 22991 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.325026 23157 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.324990 23045 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.326297 23045 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.272625 22930 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.328356 22930 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.301126 23011 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.284883 23049 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:11.333729 23049 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:11.334683 23049 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.335963 23049 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.258953 22927 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.272217 21429 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:22:11.339484 22927 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.179499 22742 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.340775 22742 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:11.271821 23013 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:11.284385 23124 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:11.303300 20803 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:11.304672 21426 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:11.196097 22964 raft_consensus.cc:2764] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:22:11.232312 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.179538 20726 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:11.310242 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:11.381207 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71034.5us
I20260709 12:22:11.382220 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.285526 23147 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:11.309805 20632 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 4, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 4 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.311062 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:11.384763 20632 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:11.385331 23052 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:11.385802 23052 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.386129 23052 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:11.386070 20632 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:11.387643 23167 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.388159 23167 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:11.390408 23167 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:11.392000 22784 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:11.393183 20701 leader_election.cc:400] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4.
W20260709 12:22:11.393277 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 82319.3us
I20260709 12:22:11.393499 20701 leader_election.cc:403] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:11.245391 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:11.395743 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 150414us
I20260709 12:22:11.396044 21431 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:11.394127 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:11.393692 20632 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:11.399924 20632 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.315408 20809 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:11.316412 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:11.317818 22919 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:11.177428 23143 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:11.333312 23011 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.370841 20732 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:22:11.342554 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84107.7us
W20260709 12:22:11.373883 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 191243us
W20260709 12:22:11.285146 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 287915us
I20260709 12:22:11.407430 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:11.408286 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:11.408685 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 92350.3us
I20260709 12:22:11.409531 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.409698 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:11.341873 23091 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.410763 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:11.416079 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:11.417497 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:11.418087 20734 raft_consensus.cc:2376] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3. Re-sending same reply.
I20260709 12:22:11.418319 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.419564 20735 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:11.420259 20730 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:22:11.420608 20730 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 3
I20260709 12:22:11.420855 20730 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:22:11.421118 20730 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:11.408977 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.423632 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.425072 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:11.425609 20730 raft_consensus.cc:2376] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2. Re-sending same reply.
I20260709 12:22:11.426870 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.427364 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:11.427997 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.428282 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.429018 20734 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:11.429090 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.430037 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.430583 20735 raft_consensus.cc:2376] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5. Re-sending same reply.
I20260709 12:22:11.431367 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.431939 21431 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 12:22:11.377475 23013 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:11.441057 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:11.383289 22986 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:11.436410 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.449123 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.271047 23053 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:11.440666 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.275727 23153 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.444402 20801 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:11.446156 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.446480 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.447253 20806 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:22:11.450824 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.442955 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.452762 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.453135 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.452843 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:11.373108 23124 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.404137 23147 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.406695 23048 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:11.499019 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56126us
I20260709 12:22:11.516305 23048 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:11.517040 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.517820 23085 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:11.517503 23048 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.520289 20877 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:11.521080 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.521275 22784 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.521725 22784 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.522356 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.524948 20876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:22:11.525502 20876 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:11.525815 22784 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:11.525853 20876 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.528533 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.530911 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.531633 20880 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:11.533387 23049 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.533927 23049 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.536026 23049 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.536541 23049 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:11.537667 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:11.421108 23123 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:11.423261 22975 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:11.421314 23167 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:11.538421 23123 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.538589 22975 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.538869 23123 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.538784 23167 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:11.500218 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.540222 23167 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.540741 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.541512 20730 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:11.500232 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.542605 20729 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:11.543910 20729 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:22:11.544051 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:11.544808 22936 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:11.545156 20729 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:11.546607 22793 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:11.502604 20733 raft_consensus.cc:2368] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:11.503239 21431 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:22:11.495229 22929 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:11.434580 22618 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:11.440104 22872 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:11.548992 22872 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.549376 22872 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:11.505630 22919 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.550612 22919 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.551306 22793 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.551808 22793 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.552870 22919 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:11.433144 22962 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.553779 22793 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:11.554293 22793 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:11.555225 22793 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.432597 23125 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:11.432431 22912 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:11.440860 23133 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:11.545195 22936 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.562944 22936 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:11.538249 20876 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.571383 20883 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:11.539139 22975 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.570971 22793 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.542496 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.575548 20730 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:11.576507 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.573546 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:11.505296 23044 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:11.578189 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:11.582296 22912 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:11.582736 22912 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.583097 22912 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.584333 23167 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:11.585186 23167 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.585719 22975 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.587298 20805 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:11.515620 21430 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:11.568763 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:11.406430 23024 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:11.586532 23167 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.499495 20728 raft_consensus.cc:2376] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4. Re-sending same reply.
I20260709 12:22:11.588850 23024 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.566772 20735 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:11.589769 23024 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:11.595690 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.591804 20876 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.432307 23163 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:11.601053 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:11.606168 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 209452us
W20260709 12:22:11.607612 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69943.1us
I20260709 12:22:11.498461 22913 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:11.616815 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.425015 23091 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.620365 23091 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:11.454584 23135 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.157s
I20260709 12:22:11.511451 21428 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:11.599031 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:11.454489 23039 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.153s
W20260709 12:22:11.606686 23168 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:22:11.631630 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.616815 20734 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:11.636626 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:11.686990 20728 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:11.687387 20728 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:11.636188 23013 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:11.690223 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 281289us
I20260709 12:22:11.612025 23163 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.685931 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.620870 23060 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:11.637034 22872 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.598434 23067 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 12:22:11.608050 23038 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:22:11.698894 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.639786 23135 consensus_queue.cc:1243] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:22:11.633112 22784 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:11.695869 21426 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.704970 20802 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:11.706708 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 289) took 1257 ms. Trace:
I20260709 12:22:11.706583 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.703778 20728 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.709575 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.737210 20727 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:11.731055 22929 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:11.747925 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49106.5us
W20260709 12:22:11.762797 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149092us
I20260709 12:22:11.707917 23163 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.754487 20732 rpcz_store.cc:276] 0709 12:22:10.448905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.824273 (+375368us) service_pool.cc:224] Handling call
0709 12:22:11.356219 (+531946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:11.405818 (+ 49599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.706501 (+300683us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400226ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:11.706644 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222484864}
I20260709 12:22:11.815904 20728 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.737109 22913 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.734998 23041 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
W20260709 12:22:11.806005 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 99486.3us
I20260709 12:22:11.708501 22918 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:22:11.808777 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:11.625756 23143 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.731452 22991 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:22:11.733136 23133 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:11.709751 23091 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 12:22:11.823799 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.845134 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.825711 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:11.775944 23135 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:11.828821 22913 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.834561 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:11.791059 23144 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.836202 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:11.791690 22975 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.807854 23117 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.925868 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.885929 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.756901 22784 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.887501 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.893692 22293 raft_consensus.cc:1064] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:22:11.894398 23181 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
W20260709 12:22:11.887470 23160 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.381s	user 0.000s	sys 0.001s
I20260709 12:22:11.895068 23179 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:11.884289 23044 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.887645 23174 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.164s
I20260709 12:22:11.895920 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 288) took 1454 ms. Trace:
I20260709 12:22:11.956642 20727 rpcz_store.cc:276] 0709 12:22:10.441666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.821583 (+379917us) service_pool.cc:224] Handling call
0709 12:22:11.056676 (+235093us) spinlock_profiling.cc:243] Waited 8.76 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:11.056874 (+   198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:11.806991 (+750117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.895683 (+ 88692us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54000c49d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:11.895857 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579226368}
I20260709 12:22:11.958719 20727 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:11.896121 22618 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:11.896309 23169 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:22:11.887853 23125 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:11.897557 23045 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.000s
I20260709 12:22:11.897876 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.888123 23175 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:11.913060 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76970.4us
I20260709 12:22:11.909848 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:11.991931 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:11.908414 23133 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:11.825731 22962 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.910917 23124 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:11.706073 22986 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.920933 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:11.929733 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43927.4us
I20260709 12:22:12.022867 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:11.932698 20728 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:11.877877 23135 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:12.028355 23135 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:11.890153 23048 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.953658 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:11.954022 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:11.822463 22929 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:12.030395 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76805.4us
I20260709 12:22:12.030622 22918 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:11.934894 23038 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:12.030400 20881 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:11.889066 23053 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:11.888185 23164 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.933108 22837 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:12.032203 23164 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:12.033265 22929 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:12.034653 23164 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:11.988453 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:11.963169 23170 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 12:22:11.963289 23171 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
I20260709 12:22:11.951328 23153 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:11.961771 20727 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:11.962985 22618 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:12.047811 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:12.044206 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:12.049520 21433 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:12.052457 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:12.053365 21429 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.977485 22930 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:12.073976 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:12.074312 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51549.4us
I20260709 12:22:12.074893 20732 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:12.075151 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:12.063227 23180 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:22:12.078974 22618 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:11.963474 23024 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.080346 22618 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:12.081355 22872 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.082017 23024 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:12.082245 22742 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.741s	user 0.009s	sys 0.004s
I20260709 12:22:11.974562 23123 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:12.084573 20804 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:12.085119 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.017s	sys 0.017s
W20260709 12:22:12.085572 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.017s	sys 0.017s
W20260709 12:22:12.086310 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 645276us
I20260709 12:22:12.087595 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:12.088788 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:12.057775 23154 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.756s	user 0.005s	sys 0.009s
W20260709 12:22:12.073926 22936 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.001s
I20260709 12:22:12.048687 23038 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:12.066963 22913 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:12.136214 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:12.082728 22742 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:12.092017 23167 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 12:22:12.067762 22962 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:12.148329 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52503.3us
W20260709 12:22:12.226992 20806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.007s	sys 0.012s
W20260709 12:22:12.231714 20806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.007s	sys 0.012s
I20260709 12:22:12.155097 20774 leader_election.cc:400] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:12.226418 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:12.233273 20774 leader_election.cc:403] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:22:12.089912 23048 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:12.243968 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:12.267668 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 173384us
W20260709 12:22:12.269397 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 277601us
W20260709 12:22:12.125053 23052 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.738s	user 0.015s	sys 0.000s
I20260709 12:22:12.268375 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:12.093027 23044 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:12.125201 22908 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.008s	sys 0.004s
I20260709 12:22:12.288798 20877 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:12.073544 23164 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.212s
I20260709 12:22:12.196646 23187 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.311506 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:12.300994 22908 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.013s	sys 0.007s
W20260709 12:22:12.311961 22912 consensus_meta.cc:287] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.728s	user 0.008s	sys 0.003s
I20260709 12:22:12.090842 22930 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.190721 23123 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:12.234633 22962 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:12.284555 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:12.348351 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 369943us
I20260709 12:22:12.123646 23181 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.178s
W20260709 12:22:12.305681 21400 connection.cc:256] RPC call timeout handler was delayed by -1.7463642829998207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3251s for a response
W20260709 12:22:12.356460 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87733.5us
I20260709 12:22:12.288784 23182 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:12.288543 22913 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:12.348464 22784 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:12.168676 22936 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:12.291566 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 607) took 1035 ms. Trace:
W20260709 12:22:12.375933 21400 leader_election.cc:336] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.642s (SENT)
I20260709 12:22:12.235644 23154 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:12.266968 23167 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:12.375604 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:12.290469 23052 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.382081 22930 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.387346 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:12.366575 22962 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:12.377599 21428 rpcz_store.cc:276] 0709 12:22:11.253096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.296229 (+ 43133us) service_pool.cc:224] Handling call
0709 12:22:11.403767 (+107538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:11.637703 (+233936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:12.230008 (+592305us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400690fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:12.288886 (+ 58878us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231837568}
I20260709 12:22:12.397440 22936 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:12.364676 23187 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.399555 23052 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.400282 20730 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:22:12.362897 22912 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.399566 23187 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.391045 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:12.405939 23191 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:22:12.384712 23038 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.880870 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1914) took 1330 ms. Trace:
I20260709 12:22:12.432857 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:12.898711 20732 rpcz_store.cc:276] 0709 12:22:11.550611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.075718 (+525107us) service_pool.cc:224] Handling call
0709 12:22:12.076343 (+   625us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:12.090364 (+ 14021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:12.090400 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:22:12.124158 (+ 33758us) log.cc:844] Serialized 616 byte log entry
0709 12:22:12.379268 (+255110us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:12.379327 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:12.379423 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:12.879758 (+500335us) raft_consensus.cc:1685] finished
0709 12:22:12.879857 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:12.880462 (+   605us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5145,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":176017,"threads_started":1,"wal-append.queue_time_us":248797}
W20260709 12:22:12.885759 20735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.001s	sys 0.005s
W20260709 12:22:12.901222 20735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.001s	sys 0.005s
I20260709 12:22:12.902469 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:12.903466 20875 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
W20260709 12:22:12.580268 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 202500us
W20260709 12:22:12.915023 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:12.916904 21400 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.798s (SENT)
W20260709 12:22:12.918844 21400 leader_election.cc:336] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.829s (SENT)
W20260709 12:22:12.920316 21400 connection.cc:256] RPC call timeout handler was delayed by -1.6065937570001552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2555s for a response
W20260709 12:22:12.922963 21400 connection.cc:256] RPC call timeout handler was delayed by -1.2222937570004433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2982s for a response
W20260709 12:22:12.923372 21400 connection.cc:256] RPC call timeout handler was delayed by -1.1043937570002527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3113s for a response
I20260709 12:22:12.926029 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:12.929143 22784 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.930419 22784 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.932044 22872 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.933015 22872 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.935612 22913 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.936435 22930 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.936502 22913 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:12.937295 22930 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:12.939400 22872 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.935568 22908 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.939987 22930 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:12.940424 22908 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.942830 22908 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.943998 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:12.944242 23044 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:12.945055 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:12.939117 22913 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:12.946285 20880 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:12.947141 23044 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:12.940014 22872 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.947366 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:12.950141 22872 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:12.950680 22742 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.420265 22927 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.952821 22872 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.948181 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:12.935025 22784 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.953989 20875 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
W20260709 12:22:12.726174 21426 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.906s	user 0.007s	sys 0.004s
I20260709 12:22:12.606694 22912 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:12.746910 22962 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:12.608395 23011 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.193s	user 0.014s	sys 0.000s
W20260709 12:22:12.736219 23147 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.124s	user 0.011s	sys 0.000s
I20260709 12:22:12.993831 23147 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4.
I20260709 12:22:12.564294 22936 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:12.547657 23190 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 12:22:12.725245 22618 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.002s
W20260709 12:22:13.012293 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 86627.1us
W20260709 12:22:12.631498 21433 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.581s	user 0.008s	sys 0.004s
W20260709 12:22:12.728514 21429 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.674s	user 0.006s	sys 0.006s
I20260709 12:22:13.015465 21429 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:13.015460 21433 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
W20260709 12:22:12.547657 23193 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:22:13.020328 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1709) took 1196 ms. Trace:
I20260709 12:22:13.016646 22837 raft_consensus.cc:2764] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:22:12.607077 23163 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.765s	user 0.005s	sys 0.003s
W20260709 12:22:12.548983 22955 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.432s	user 0.010s	sys 0.002s
I20260709 12:22:13.016646 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 735) took 1024 ms. Trace:
W20260709 12:22:12.623021 23157 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.297s	user 0.011s	sys 0.002s
I20260709 12:22:13.020443 20803 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:13.016444 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 736) took 1024 ms. Trace:
I20260709 12:22:13.022915 21433 rpcz_store.cc:276] 0709 12:22:11.992076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.992220 (+   144us) service_pool.cc:224] Handling call
0709 12:22:13.016555 (+1024335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:13.025430 20732 raft_consensus.cc:2368] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3.
I20260709 12:22:12.609548 23167 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.025960 23185 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.026012 22936 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.028936 23185 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.020929 20876 rpcz_store.cc:276] 0709 12:22:11.822715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.822825 (+   110us) service_pool.cc:224] Handling call
0709 12:22:11.823133 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:11.826150 (+  3017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:11.826196 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:11.826793 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 12:22:12.093641 (+266848us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:12.093702 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:12.093802 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:13.005145 (+911343us) raft_consensus.cc:1685] finished
0709 12:22:13.007131 (+  1986us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:13.019420 (+ 12289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2614,"raft.queue_time_us":6170,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":264077,"threads_started":1,"wal-append.queue_time_us":540804}
I20260709 12:22:12.605175 23154 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:13.032768 23118 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.031323 23163 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.024863 21426 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:13.036334 20883 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:22:13.036245 23053 raft_consensus.cc:2764] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:13.034315 23118 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.024223 21429 rpcz_store.cc:276] 0709 12:22:11.992396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.003702 (+ 11306us) service_pool.cc:224] Handling call
0709 12:22:13.016286 (+1012584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:13.038617 23185 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.041596 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:13.059005 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.059572 22962 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.059962 20732 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.064085 22936 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:13.066394 20807 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:22:13.044620 21426 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 415) took 1590 ms (client timeout 1769 ms). Trace:
W20260709 12:22:13.075302 21426 rpcz_store.cc:269] 0709 12:22:11.450383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.455382 (+  4999us) service_pool.cc:224] Handling call
0709 12:22:13.041143 (+1585761us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:13.077382 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:13.078637 22618 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:13.079362 22618 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:13.080981 22618 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.089756 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.094540 23163 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:13.094666 23118 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.099005 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1910) took 1655 ms. Trace:
I20260709 12:22:13.099565 20735 rpcz_store.cc:276] 0709 12:22:11.443461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.978238 (+534777us) service_pool.cc:224] Handling call
0709 12:22:11.978611 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:12.226359 (+247748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:12.226410 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:12.301306 (+ 74896us) log.cc:844] Serialized 52 byte log entry
0709 12:22:12.902106 (+600800us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:12.902247 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:12.902456 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:13.097693 (+195237us) raft_consensus.cc:1685] finished
0709 12:22:13.097721 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:13.098931 (+  1210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66942,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":27434,"thread_start_us":784727,"threads_started":2,"wal-append.queue_time_us":682647}
I20260709 12:22:13.101804 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.102730 20735 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:13.104528 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:13.104735 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.105674 20727 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:13.107753 22919 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.108491 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1905) took 1670 ms. Trace:
I20260709 12:22:13.108913 20730 rpcz_store.cc:276] 0709 12:22:11.437444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.577352 (+139908us) service_pool.cc:224] Handling call
0709 12:22:11.577657 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:12.400809 (+823152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.107895 (+707086us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54003b3b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:13.108275 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891120896}
I20260709 12:22:13.109628 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:13.110671 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1847) took 1645 ms (client timeout 1679 ms). Trace:
I20260709 12:22:13.110870 20730 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:13.105749 23185 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:13.112294 23185 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.112710 23185 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:13.111059 20808 rpcz_store.cc:269] 0709 12:22:11.464999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.499435 (+ 34436us) service_pool.cc:224] Handling call
0709 12:22:13.109215 (+1609780us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:13.110543 (+  1328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13850368}
I20260709 12:22:13.115432 20732 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:13.111764 20727 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:22:13.112145 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84599.6us
I20260709 12:22:13.124159 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1920) took 1291 ms. Trace:
I20260709 12:22:13.125315 20727 rpcz_store.cc:276] 0709 12:22:11.832088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.106501 (+1274413us) service_pool.cc:224] Handling call
0709 12:22:13.107123 (+   622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:13.123473 (+ 16350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.123739 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:13.124208 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1921) took 1089 ms. Trace:
I20260709 12:22:13.126471 20730 rpcz_store.cc:276] 0709 12:22:12.034040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.110066 (+1076026us) service_pool.cc:224] Handling call
0709 12:22:13.110488 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:13.115797 (+  5309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.123445 (+  7648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:13.128755 23135 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.132966 22919 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:22:13.135464 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 424) took 1214 ms (client timeout 1615 ms). Trace:
W20260709 12:22:13.137573 20877 rpcz_store.cc:269] 0709 12:22:11.920484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.920622 (+   138us) service_pool.cc:224] Handling call
0709 12:22:13.134959 (+1214337us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54001fde54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:13.135268 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612027648}
I20260709 12:22:13.136663 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1856) took 1493 ms. Trace:
I20260709 12:22:13.138906 22837 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:13.139859 23044 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:13.140390 23044 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:13.140988 22837 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:13.141541 23044 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.143270 23044 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.141724 22837 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:13.146143 20726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 1891) took 3073 ms (client timeout 1971 ms). Trace:
W20260709 12:22:13.146692 20726 rpcz_store.cc:269] 0709 12:22:10.072914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.081581 (+  8667us) service_pool.cc:224] Handling call
0709 12:22:13.145701 (+3064120us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001d7c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:13.145904 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503213440}
I20260709 12:22:13.147496 22837 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:13.143491 23048 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:13.148294 21433 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:13.150470 20731 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:13.151708 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:13.152516 21433 raft_consensus.cc:2376] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5. Re-sending same reply.
I20260709 12:22:13.153301 23118 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:13.154032 23118 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.143884 23175 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:13.138959 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1907) took 1700 ms. Trace:
I20260709 12:22:13.157034 23118 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.157315 20729 rpcz_store.cc:276] 0709 12:22:11.438037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.953299 (+515262us) service_pool.cc:224] Handling call
0709 12:22:11.954997 (+  1698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:11.972413 (+ 17416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:11.972473 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:12.005813 (+ 33340us) log.cc:844] Serialized 26 byte log entry
0709 12:22:12.347969 (+342156us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:12.348027 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.138079 (+790052us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:13.138139 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:13.138203 (+    64us) raft_consensus.cc:1685] finished
0709 12:22:13.138217 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:13.138774 (+   557us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39280384,"thread_start_us":255999,"threads_started":1,"wal-append.queue_time_us":327987}
W20260709 12:22:13.162737 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 885843us
I20260709 12:22:13.164227 23163 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:13.164480 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.165125 23163 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.166108 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.166925 21430 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:13.162897 23118 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:13.168021 23163 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:13.154285 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:13.169004 23192 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.821s	user 0.013s	sys 0.019s
I20260709 12:22:13.168578 23118 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.169242 21433 raft_consensus.cc:2376] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3. Re-sending same reply.
I20260709 12:22:13.170065 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:13.170723 21432 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.171028 21433 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:13.171752 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.144824 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.172916 21426 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:13.165500 21428 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:13.171870 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.174654 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.175308 21430 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:13.175447 21434 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:13.173583 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:13.176712 21427 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.177127 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:13.177866 21435 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.135092 23185 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.171037 23118 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.181397 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.182277 21430 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:13.182812 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:13.182592 23185 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:13.183317 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.184844 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.138695 20804 rpcz_store.cc:276] 0709 12:22:11.643469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.689468 (+ 45999us) service_pool.cc:224] Handling call
0709 12:22:13.136216 (+1446748us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400127354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:13.136486 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638409600}
I20260709 12:22:13.198199 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.162750 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:13.204421 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:13.205561 23118 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:13.206483 22936 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:13.207019 22936 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.207461 22936 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.209285 23167 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:13.210500 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.195960 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.215118 21433 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:13.217650 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.222499 21432 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:13.222680 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.224340 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.217190 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:13.231868 21430 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:13.231699 21435 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:13.233453 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.236617 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.238155 20731 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:13.221756 22742 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:22:13.239929 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:13.240906 21428 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:13.242033 21427 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:13.237283 20727 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:13.243326 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.243499 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.244132 20728 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:13.244534 23085 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:13.244974 23085 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.245350 23085 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.223997 23118 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:13.256728 20731 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:13.235920 22955 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:13.257654 20732 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:13.242801 22936 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.236515 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.261068 20726 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.263398 22991 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:13.263875 22991 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:13.264307 22991 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.266705 20730 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:13.267562 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 950103us
I20260709 12:22:13.267797 23118 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.269896 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:13.271333 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:13.271697 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 895995us
I20260709 12:22:13.271356 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:13.272693 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:13.273371 23155 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:13.273851 23155 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:13.274241 23155 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.277362 23024 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:13.277458 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:13.278026 23024 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:13.278575 22929 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:13.278806 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:13.279181 22929 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:13.280428 22929 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.281282 23060 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:13.281769 23060 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:13.282182 23060 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.283532 21400 leader_election.cc:400] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3.
I20260709 12:22:13.283667 23123 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:13.283856 21400 leader_election.cc:403] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:22:13.284255 23123 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:13.284618 23045 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:13.287606 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 114658us
I20260709 12:22:13.287479 23045 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.295934 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.296857 20729 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:13.296921 22936 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.298065 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.298144 23045 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:13.299590 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.287842 23185 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3.
I20260709 12:22:13.292616 22742 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:13.314005 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:13.285637 23024 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.315690 23189 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:13.316305 23189 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:13.319372 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:13.324196 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.324765 20732 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:13.324568 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:13.325548 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.305804 22742 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:13.327809 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.328416 20732 raft_consensus.cc:2376] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4. Re-sending same reply.
I20260709 12:22:13.329066 20727 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:22:13.328600 22742 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.333837 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:13.290901 23123 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.336830 23167 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:13.337846 23174 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:13.340086 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:13.340219 20810 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:13.341470 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.340736 20810 pending_rounds.cc:85] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 1
I20260709 12:22:13.343966 20810 pending_rounds.cc:107] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:13.344314 20810 raft_consensus.cc:2889] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:13.344115 23179 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 12:22:13.344965 22293 raft_consensus.cc:1068] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Unable to promote non-voter 50c1ab13351d4427b836fbb449c6d7bb: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:22:13.330078 22936 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:13.345965 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1846) took 1895 ms. Trace:
I20260709 12:22:13.345851 22936 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.346362 20810 rpcz_store.cc:276] 0709 12:22:11.450356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.450681 (+   325us) service_pool.cc:224] Handling call
0709 12:22:11.892406 (+441725us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:11.892601 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:13.344947 (+1452346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.345653 (+   706us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54002b70d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:13.345892 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3331807872}
I20260709 12:22:13.348130 22936 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:13.340384 23167 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:13.352370 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1839) took 1916 ms. Trace:
W20260709 12:22:13.353014 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 189023us
I20260709 12:22:13.352900 20803 rpcz_store.cc:276] 0709 12:22:11.435678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.442434 (+  6756us) service_pool.cc:224] Handling call
0709 12:22:11.599627 (+157193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:13.025270 (+1425643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.352033 (+326763us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540027b354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:13.352311 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3210931968}
I20260709 12:22:13.355458 22975 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:13.357240 22975 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:13.359390 23174 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:13.360452 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1904) took 1899 ms. Trace:
I20260709 12:22:13.360584 23174 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.361215 23163 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:13.362200 23041 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:13.362802 23041 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:13.363862 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1851) took 1751 ms. Trace:
I20260709 12:22:13.364142 23041 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:13.338721 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:13.367213 23163 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:13.367699 23038 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.368244 23038 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.368387 23163 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.369742 23163 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.370316 23038 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:13.370852 23038 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.352404 23167 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.371855 22964 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:13.372582 22964 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:13.373637 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:13.364259 20807 rpcz_store.cc:276] 0709 12:22:11.612699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.612908 (+   209us) service_pool.cc:224] Handling call
0709 12:22:11.613221 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:13.067247 (+1454026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.363531 (+296284us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54003ca354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:13.363792 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3310633344}
I20260709 12:22:13.360836 20733 rpcz_store.cc:276] 0709 12:22:11.437155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.547649 (+110494us) service_pool.cc:224] Handling call
0709 12:22:11.994434 (+446785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:11.994623 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:11.994647 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:11.994690 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:11.994732 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.994783 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:13.329833 (+1335050us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54002e4fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:13.336981 (+  7148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026838400}
I20260709 12:22:13.374507 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:13.384519 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.385151 20732 raft_consensus.cc:2376] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6. Re-sending same reply.
I20260709 12:22:13.337410 23185 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:13.391928 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:13.317727 23189 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.395850 20876 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:13.402551 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68789.2us
I20260709 12:22:13.407634 20730 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:13.403064 22986 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:13.380132 23045 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:13.423197 22986 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.392683 23185 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:13.405767 22964 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:13.403810 23199 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:13.421715 23038 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.435691 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.591447 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.428722 22986 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:13.530596 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98137.9us
W20260709 12:22:13.577906 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 222845us
I20260709 12:22:13.429636 23185 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.591053 23044 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:13.600239 23044 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.591333 23199 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:13.602741 23199 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:13.603585 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:13.604517 23124 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.605266 23124 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.606601 23067 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:13.460594 23189 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.463274 23181 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.602807 23044 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.607254 23067 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.612326 20730 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:22:13.603914 23199 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.461496 22814 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.462144 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:13.463481 23198 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.598081 23182 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:13.609462 23135 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.431995 23045 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.623240 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:13.621377 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:13.630595 20727 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.652467 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:13.615733 23179 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.604444 23060 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.619261 20726 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:13.662075 23179 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.666914 23044 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:13.667009 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58229.5us
I20260709 12:22:13.619071 23147 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.615957 23085 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.627614 23181 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:13.669817 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 242478us
I20260709 12:22:13.668251 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:13.631631 23135 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.676816 23182 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:13.677551 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129331us
I20260709 12:22:13.683668 23147 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.619416 22742 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.654210 22962 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 12:22:13.720690 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52557.5us
W20260709 12:22:13.684258 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67527.8us
I20260709 12:22:13.731914 20731 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:13.725695 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:13.726917 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:13.660375 23124 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:13.739847 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 134204us
I20260709 12:22:13.743255 20729 raft_consensus.cc:2368] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5.
W20260709 12:22:13.717984 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58094us
I20260709 12:22:13.762465 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:13.759027 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:13.725693 23044 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:13.762988 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:13.753914 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:13.665558 22814 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.699684 23198 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:13.824378 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58224.2us
I20260709 12:22:13.740327 22964 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:13.759330 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:13.814559 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:13.784337 22991 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.003s
I20260709 12:22:13.851157 20808 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:22:13.835799 23044 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.838371 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:13.886441 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:13.887403 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 137070us
W20260709 12:22:13.887856 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62124.1us
I20260709 12:22:13.888020 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:13.888283 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:13.851828 22955 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:13.879805 20876 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:13.883348 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:13.884435 23049 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:13.885169 23038 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:13.851593 23124 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:13.849494 22930 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:13.737843 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37c593b286274096a8c431979866be8a"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
W20260709 12:22:13.891119 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 128891us
I20260709 12:22:13.926245 20880 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:13.866494 20726 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:13.875456 23044 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.939770 20730 raft_consensus.cc:606] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:13.940858 20730 raft_consensus.cc:614] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:13.934250 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:13.947547 23045 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:13.947937 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:13.948601 20881 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.889909 23060 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:13.952224 22742 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:13.952301 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:13.952744 23228 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:22:13.953183 20884 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.957603 23038 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.918166 22964 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.867411 22991 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.964900 23085 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:13.939684 23124 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.967773 21427 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:13.968959 23045 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:13.969753 23045 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:13.971390 22930 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:13.972133 20880 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:13.929870 22929 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.973448 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:13.928937 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:13.997117 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57583.1us
I20260709 12:22:13.992970 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:13.995328 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 243668us
I20260709 12:22:13.939570 22955 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:14.006311 20882 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:13.999940 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.060191 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:13.949708 23041 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:14.066387 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.061280 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.071686 20877 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:22:14.074239 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 343025us
I20260709 12:22:14.003968 23067 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:14.099027 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 220750us
I20260709 12:22:14.099094 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:14.078862 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:14.106007 20879 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:14.108419 20883 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:14.111511 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.084604 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:14.082716 23155 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.808s	user 0.013s	sys 0.003s
I20260709 12:22:14.116976 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:14.112142 20802 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
W20260709 12:22:14.119508 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44510.1us
I20260709 12:22:14.122465 20729 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:14.123307 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 130384us
I20260709 12:22:14.092723 22930 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:14.084230 23167 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:14.122555 20876 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:14.129416 20876 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:13.979990 23181 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:14.137475 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 249297us
W20260709 12:22:14.135895 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51552.2us
I20260709 12:22:14.141163 20882 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:14.142658 20731 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:14.145332 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:14.082015 22929 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:14.172852 20727 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.001s
I20260709 12:22:14.179296 23250 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:14.152190 20806 pending_rounds.cc:85] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:22:14.153932 20876 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.182992 20806 pending_rounds.cc:107] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:22:14.183459 20876 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:14.183430 20806 raft_consensus.cc:2889] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:14.154829 23198 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:14.155519 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:14.155659 22814 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:14.146925 22930 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.156317 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:14.156610 23147 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:14.157248 23123 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.158524 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:14.159502 23045 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:14.167160 23239 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:22:14.167847 23232 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:22:14.174293 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:14.177428 23179 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:14.122978 23155 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:14.180037 23163 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.002s
W20260709 12:22:14.180163 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.243867 23038 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:14.244347 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:14.244465 23038 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.178007 20727 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:14.151481 23220 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:14.150719 23231 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:22:14.192677 20884 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:14.183640 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:14.196560 23250 consensus_queue.cc:1243] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:14.196810 20802 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:14.239928 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 97581.3us
I20260709 12:22:14.239493 22930 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.240442 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:14.243331 23155 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:14.180577 23247 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:14.248458 20731 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
W20260709 12:22:14.252194 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106912us
I20260709 12:22:14.259987 20727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:14.260759 20727 raft_consensus.cc:606] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:14.261210 20727 raft_consensus.cc:614] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:14.268389 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:14.262015 23247 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:14.269553 20775 leader_election.cc:400] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:14.222076 22742 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:14.272269 20881 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:14.278704 20807 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
W20260709 12:22:14.279973 23242 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:22:14.245965 23045 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:22:14.280753 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:14.281311 23045 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:14.281739 23045 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:14.285324 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:14.285959 20876 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:22:14.286820 22986 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.001s
I20260709 12:22:14.287459 22986 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:14.287978 20729 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:14.289408 20729 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:14.289921 23135 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:14.290235 20875 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:14.291213 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:14.279973 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:14.292109 20805 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:14.292234 20807 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:14.292981 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:14.293190 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:14.294086 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:14.194064 22955 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:14.296118 23049 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:14.296550 23049 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:14.296876 23049 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:14.205489 23044 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.294875 20807 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:14.286501 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:14.282891 23045 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.262991 23123 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.304564 23155 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.304737 23045 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:14.303817 22742 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:14.305157 23155 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:14.305878 23123 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:14.305944 22742 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:14.306668 23123 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:14.307471 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:14.308568 23038 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:14.310874 23044 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:14.270182 23085 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:14.311949 23085 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:14.312306 23085 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:14.305176 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:14.286832 23251 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:14.269630 23247 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:14.289958 22986 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:14.315549 20810 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:14.316509 22930 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:14.321346 23198 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:14.262367 22814 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:14.325138 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1934) took 1048 ms. Trace:
I20260709 12:22:14.326371 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.327339 20727 raft_consensus.cc:2368] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:14.328431 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:14.329457 20727 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:14.330353 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:14.324723 23185 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.697s	user 0.005s	sys 0.005s
I20260709 12:22:14.331758 23185 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.332685 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1935) took 1055 ms. Trace:
I20260709 12:22:14.333139 20735 rpcz_store.cc:276] 0709 12:22:13.277047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.277194 (+   147us) service_pool.cc:224] Handling call
0709 12:22:13.277536 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:13.277999 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:13.349708 (+ 71709us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:22:13.408716 (+ 59008us) log.cc:844] Serialized 14 byte log entry
0709 12:22:13.882406 (+473690us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:13.952052 (+ 69646us) log.cc:844] Serialized 26 byte log entry
0709 12:22:13.977902 (+ 25850us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:13.977968 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.978082 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:14.332369 (+354287us) raft_consensus.cc:1685] finished
0709 12:22:14.332468 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:14.332599 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23731,"prepare.queue_time_us":300254,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":9055,"raft.run_cpu_time_us":2223,"raft.run_wall_time_us":71720,"spinlock_wait_cycles":27297920,"thread_start_us":541726,"threads_started":2,"wal-append.queue_time_us":648975}
I20260709 12:22:14.335536 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:14.323907 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 149740us
I20260709 12:22:14.341502 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:14.342716 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 191798us
I20260709 12:22:14.342988 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:14.348524 20732 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:14.351267 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.352562 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.321979 23198 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:14.307062 23123 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:14.354287 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:14.358170 20735 raft_consensus.cc:2368] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:14.355413 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.358353 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.358916 20732 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:14.360493 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.356837 20876 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:14.340137 23185 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:14.373369 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 217093us
W20260709 12:22:14.375583 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.377343 23174 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:14.379709 20881 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:14.380641 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:14.370539 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.325620 20728 rpcz_store.cc:276] 0709 12:22:13.276697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.283327 (+  6630us) service_pool.cc:224] Handling call
0709 12:22:13.283728 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:13.284097 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:13.284124 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:13.284672 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:22:13.359573 (+ 74901us) spinlock_profiling.cc:243] Waited 6.25 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:22:13.614307 (+254734us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:13.614352 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.614560 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:14.322345 (+707785us) raft_consensus.cc:1685] finished
0709 12:22:14.322371 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:14.325024 (+  2653us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3535,"raft.run_cpu_time_us":1541,"raft.run_wall_time_us":72234,"spinlock_wait_cycles":14376704,"thread_start_us":328868,"threads_started":1,"wal-append.queue_time_us":468859}
I20260709 12:22:14.382252 20882 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:14.359009 20729 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:14.344866 22814 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:14.384836 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:14.389750 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.392324 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:14.395520 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:14.360698 23123 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:14.356235 23067 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:14.410730 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:14.414983 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.414896 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:14.423390 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 306253us
I20260709 12:22:14.432722 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.435698 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.437157 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:14.437336 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:14.438905 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.440037 20729 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:14.408229 23243 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.441854 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:14.414259 23067 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.452812 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:14.446393 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:14.415827 23185 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:14.432742 21429 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:14.455461 20879 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:14.455628 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.413287 22962 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:14.441072 23220 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.462757 21435 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:14.461802 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:14.463075 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73753.8us
W20260709 12:22:14.464757 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:14.468914 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 110911us
I20260709 12:22:14.455906 23024 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:14.455533 23135 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:14.468264 20728 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:14.479568 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.460489 22986 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:14.467929 21427 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:14.482635 22814 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.486016 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.487346 23198 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:14.492204 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:14.488585 23198 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:14.478788 23067 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.478224 23220 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:14.499902 23060 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:14.500602 23125 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:22:14.524629 20728 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:14.499768 23182 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:14.469573 23243 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:14.532917 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 148172us
I20260709 12:22:14.534616 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:14.535777 21400 leader_election.cc:400] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5.
I20260709 12:22:14.536142 21400 leader_election.cc:403] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:22:14.507613 22986 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:14.542928 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 104463us
I20260709 12:22:14.550477 23179 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.550604 22991 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:14.550995 23067 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:14.551134 23179 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:14.524144 23181 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:22:14.542343 23185 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:14.550927 23251 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:14.546099 23240 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.550s	user 0.005s	sys 0.000s
I20260709 12:22:14.548898 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:14.528741 22962 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:22:14.559307 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:14.565078 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:14.549880 23038 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:14.593868 23024 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:15.211197 23024 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:15.211557 20804 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:14.568614 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:14.594360 23252 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:22:14.681084 20809 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:14.652840 23266 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.682442 23243 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:14.562462 23185 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:14.688375 23199 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.063s	user 0.011s	sys 0.006s
W20260709 12:22:14.617702 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82085.3us
I20260709 12:22:14.688822 23253 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:14.691176 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 196022us
I20260709 12:22:14.691653 23085 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:14.692150 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 317) took 1233 ms. Trace:
I20260709 12:22:14.563050 23067 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:22:14.693169 23179 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:14.693624 21427 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:14.694059 22962 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:14.616329 23163 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.698403 20806 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:14.725711 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 302105us
W20260709 12:22:14.755343 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62509us
I20260709 12:22:14.686003 23049 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:14.757927 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.270095 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:14.915515 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.683907 23247 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:14.698935 23267 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:15.199756 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 634648us
I20260709 12:22:15.208078 20730 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:15.200826 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1570) took 1483 ms (client timeout 1526 ms). Trace:
I20260709 12:22:15.217348 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.218590 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 349) took 1351 ms. Trace:
W20260709 12:22:15.248875 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 689578us
I20260709 12:22:15.200019 23182 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:15.218281 23189 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.596s	user 0.044s	sys 0.135s
W20260709 12:22:15.270751 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70528us
I20260709 12:22:15.227404 23185 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:14.898999 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.554s (SENT) [suppressed 42 similar messages]
W20260709 12:22:15.353729 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 454754us
I20260709 12:22:15.354772 20727 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:15.320209 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:15.328225 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57474.4us
I20260709 12:22:15.244120 20733 rpcz_store.cc:276] 0709 12:22:13.458867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.511591 (+ 52724us) service_pool.cc:224] Handling call
0709 12:22:13.565170 (+ 53579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:13.593542 (+ 28372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:13.593592 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:13.621316 (+ 27724us) log.cc:844] Serialized 26 byte log entry
0709 12:22:13.822928 (+201612us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:13.822987 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.823083 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:14.488479 (+665396us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:14.691825 (+203346us) raft_consensus.cc:1685] finished
0709 12:22:14.691848 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:14.692081 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10032,"raft.queue_time_us":22806,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":9943,"spinlock_wait_cycles":15327616,"thread_start_us":161421,"threads_started":1,"wal-append.queue_time_us":215687}
I20260709 12:22:15.342597 20804 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:15.243070 23253 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:15.407856 23265 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:15.343848 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:15.343673 20875 rpcz_store.cc:269] 0709 12:22:13.717562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.719739 (+  2177us) service_pool.cc:224] Handling call
0709 12:22:14.737076 (+1017337us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:15.200702 (+463626us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035552000}
I20260709 12:22:15.255564 23273 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:15.417847 23265 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.418562 23273 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.419313 23147 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:15.419251 23267 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.419811 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.345973 20809 rpcz_store.cc:276] 0709 12:22:13.866861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.928963 (+ 62102us) service_pool.cc:224] Handling call
0709 12:22:14.595707 (+666744us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ae9c 00007f9e1bb7aa5b 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:14.680705 (+ 84998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:15.214706 (+534001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.218527 (+  3821us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4938752}
I20260709 12:22:15.421478 23267 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.417279 23253 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:15.420014 23147 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.422289 23147 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:15.422991 23198 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.920s	user 0.011s	sys 0.008s
I20260709 12:22:15.345180 22814 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.423741 22814 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.425279 23189 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:15.425873 22814 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.328482 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:15.427855 23251 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.843s	user 0.006s	sys 0.010s
I20260709 12:22:15.427915 20806 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:15.428434 23251 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5.
W20260709 12:22:15.429242 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152489us
I20260709 12:22:15.429548 23253 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.349746 23199 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:22:15.430414 23147 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.351881 23182 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.431303 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:15.431998 23253 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:15.432175 23268 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:15.363004 20729 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:15.420552 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:15.429139 23251 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:15.457834 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:15.431691 23182 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:15.276139 23163 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.358419 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.484005 23174 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:15.484211 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.489781 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:15.484576 23147 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.493144 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.484548 23174 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.495807 23281 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:15.496012 23174 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:15.496353 23085 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.438905 23271 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:22:15.459276 23189 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:22:15.504799 23271 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:15.505300 23271 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:15.509068 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.509755 20809 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:15.510672 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.511373 20809 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:15.511652 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.512172 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.512557 20804 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:15.512846 20809 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:15.513852 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.514657 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.515070 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:15.515391 20805 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:15.515606 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.514065 20809 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:15.516378 20807 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:15.517380 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.519059 22929 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.520977 23167 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.521759 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:15.522750 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1675) took 1116 ms. Trace:
I20260709 12:22:15.506372 23271 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.258077 23266 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.522949 23167 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:22:15.432731 23253 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 12:22:15.524827 23123 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.093s	user 0.019s	sys 0.000s
I20260709 12:22:15.589268 23123 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.526513 23265 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.529306 23247 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.534155 23155 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.547271 23049 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.549376 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1680) took 1089 ms. Trace:
I20260709 12:22:15.599658 20729 rpcz_store.cc:276] 0709 12:22:14.459690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.468360 (+  8670us) service_pool.cc:224] Handling call
0709 12:22:15.549181 (+1080821us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54000e7ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:15.549318 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2005831040}
W20260709 12:22:15.550307 23227 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.672s	user 0.000s	sys 0.003s
W20260709 12:22:15.649127 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.654042 22986 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:15.563089 23241 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.571s	user 0.001s	sys 0.002s
I20260709 12:22:15.657166 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 320) took 2052 ms. Trace:
I20260709 12:22:15.657725 23124 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.524426 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1575) took 1758 ms. Trace:
I20260709 12:22:15.540127 23271 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:15.659081 22964 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.658890 20878 rpcz_store.cc:276] 0709 12:22:13.765803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.765939 (+   136us) service_pool.cc:224] Handling call
0709 12:22:15.524176 (+1758237us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b540026d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:15.524342 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264734976}
I20260709 12:22:15.660521 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:15.661638 20727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1660) took 1334 ms (client timeout 1710 ms). Trace:
W20260709 12:22:15.662041 20727 rpcz_store.cc:269] 0709 12:22:14.327135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.330083 (+  2948us) service_pool.cc:224] Handling call
0709 12:22:15.661421 (+1331338us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001d7c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:15.661577 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354001280}
I20260709 12:22:15.662583 23163 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.551447 23267 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:15.557093 21429 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:15.531883 23285 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:15.564754 20726 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:15.542593 23266 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.632573 23135 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.550587 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1666) took 1172 ms. Trace:
I20260709 12:22:15.633754 23255 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:15.634938 20803 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:15.634931 23123 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.636308 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:15.632082 21399 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.615s (SENT)
I20260709 12:22:15.523180 20731 rpcz_store.cc:276] 0709 12:22:14.406340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.479219 (+ 72879us) service_pool.cc:224] Handling call
0709 12:22:15.521600 (+1042381us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ae9c 00007f9e1bb7aa5b 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:15.522658 (+  1058us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3076864}
I20260709 12:22:15.664765 20727 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:22:15.657545 20734 rpcz_store.cc:276] 0709 12:22:13.605024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.613285 (+  8261us) service_pool.cc:224] Handling call
0709 12:22:13.617273 (+  3988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:13.637396 (+ 20123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:13.637427 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:13.669663 (+ 32236us) log.cc:844] Serialized 26 byte log entry
0709 12:22:13.839573 (+169910us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:13.879076 (+ 39503us) log.cc:844] Serialized 14 byte log entry
0709 12:22:13.933107 (+ 54031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.933268 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:15.562560 (+1629292us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b24001855b8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf109d 00007f9e1e82310e 00007f9e1e83a88c 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221
0709 12:22:15.656779 (+ 94219us) raft_consensus.cc:1685] finished
0709 12:22:15.656872 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.656991 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49716,"prepare.queue_time_us":38602,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":40324,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":5527,"spinlock_wait_cycles":16600320,"thread_start_us":91033,"threads_started":1,"wal-append.queue_time_us":143195}
I20260709 12:22:15.669312 23273 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.671207 23220 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:15.692891 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 381423us
I20260709 12:22:15.699815 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 753) took 1108 ms. Trace:
W20260709 12:22:15.738963 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106924us
I20260709 12:22:15.739238 21429 rpcz_store.cc:276] 0709 12:22:14.563938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.566432 (+  2494us) service_pool.cc:224] Handling call
0709 12:22:15.556347 (+989915us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:15.556624 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:15.672061 (+115437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.672326 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19625856}
I20260709 12:22:15.725080 20728 rpcz_store.cc:276] 0709 12:22:14.377932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.529178 (+151246us) service_pool.cc:224] Handling call
0709 12:22:14.529521 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:14.541243 (+ 11722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:14.541274 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:14.552221 (+ 10947us) log.cc:844] Serialized 52 byte log entry
0709 12:22:14.819689 (+267468us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:15.229244 (+409555us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.229438 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.229509 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:15.550361 (+320852us) raft_consensus.cc:1685] finished
0709 12:22:15.550380 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.550531 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1156,"prepare.queue_time_us":407582,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":33083,"raft.queue_time_us":198537,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":16758,"thread_start_us":266831,"threads_started":1,"wal-append.queue_time_us":980698}
I20260709 12:22:15.735926 20729 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:15.671991 23267 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:15.693543 23285 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:15.738456 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1679) took 1289 ms. Trace:
I20260709 12:22:15.745000 20727 rpcz_store.cc:276] 0709 12:22:14.446025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.662974 (+1216949us) service_pool.cc:224] Handling call
0709 12:22:15.664326 (+  1352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:15.735087 (+ 70761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.735371 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:15.724462 20700 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.507s (SENT)
I20260709 12:22:15.726987 22964 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.741396 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1681) took 1264 ms. Trace:
I20260709 12:22:15.745898 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:15.749174 20729 rpcz_store.cc:276] 0709 12:22:14.477302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.703235 (+1225933us) service_pool.cc:224] Handling call
0709 12:22:15.732709 (+ 29474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:15.741076 (+  8367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.741326 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:15.728585 23255 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:15.751950 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1671) took 1372 ms. Trace:
I20260709 12:22:15.751456 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:15.755348 20810 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:22:15.755474 20730 rpcz_store.cc:276] 0709 12:22:14.379285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.438849 (+1059564us) service_pool.cc:224] Handling call
0709 12:22:15.439276 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:15.481619 (+ 42343us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:15.493550 (+ 11931us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.751604 (+258054us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:15.751657 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:15.751711 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.751771 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.751887 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8220,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"thread_start_us":257619,"threads_started":1}
I20260709 12:22:15.740540 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1673) took 1360 ms. Trace:
I20260709 12:22:15.769899 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.770836 20804 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 2.
I20260709 12:22:15.741415 21428 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:15.740119 23135 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:15.778213 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53770us
W20260709 12:22:15.772326 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 341041us
I20260709 12:22:15.804380 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.805862 20804 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:15.807009 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:15.766387 20726 rpcz_store.cc:276] 0709 12:22:14.379854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.563926 (+1184072us) service_pool.cc:224] Handling call
0709 12:22:15.564421 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:15.702590 (+138169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.737874 (+ 35284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:15.783237 23125 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:15.817982 23125 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:15.819363 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70342.6us
I20260709 12:22:15.820537 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:15.806283 23257 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:22:15.822218 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:15.823256 23287 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:15.823717 23287 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:15.806147 23199 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:22:15.824138 23287 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:15.824919 23135 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:15.826746 23024 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.615s	user 0.005s	sys 0.005s
I20260709 12:22:15.827220 23038 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:15.827631 23038 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:15.828056 23038 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.829469 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:15.829532 23174 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.830343 23243 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:15.831045 23243 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:15.833993 20810 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:15.835270 20878 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:15.835610 20875 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:15.759581 22964 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.836505 23189 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:15.837077 23189 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.837669 22964 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.823441 23285 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.840278 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.841112 20875 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:15.845136 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.845918 20810 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:15.765558 23266 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:15.857321 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1664) took 1480 ms. Trace:
I20260709 12:22:15.861181 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:15.861495 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:15.864879 20774 connection.cc:256] RPC call timeout handler was delayed by -1.1659208290002425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 12:22:15.865855 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.867002 20878 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:22:15.783349 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:15.868659 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 353970us
I20260709 12:22:15.870859 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.872380 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.873057 20883 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:15.869058 23258 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:15.875073 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.874763 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:15.875036 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.875823 20774 leader_election.cc:400] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:15.887349 20774 leader_election.cc:403] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:15.827167 23024 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.865825 20732 rpcz_store.cc:276] 0709 12:22:14.377261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.380002 (+  2741us) service_pool.cc:224] Handling call
0709 12:22:14.380314 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:14.558658 (+178344us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:14.585960 (+ 27302us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.443402 (+857442us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:15.833129 (+389727us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:15.833587 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:22:15.833793 (+   206us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:15.834063 (+   270us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.834275 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.834342 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:15.857027 (+ 22685us) raft_consensus.cc:1685] finished
0709 12:22:15.857062 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.857245 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27836,"prepare.run_cpu_time_us":7631,"prepare.run_wall_time_us":84388,"raft.queue_time_us":314751,"raft.run_cpu_time_us":3344,"raft.run_wall_time_us":132707,"spinlock_wait_cycles":39325440,"thread_start_us":758672,"threads_started":3,"wal-append.queue_time_us":615922}
I20260709 12:22:15.866873 23289 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:15.781100 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 751) took 1225 ms. Trace:
I20260709 12:22:15.880560 20875 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:15.882045 23277 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.882469 23248 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.856545 23280 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.883483 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.847904 23268 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:15.881480 23287 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.915937 20808 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:15.894555 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.015s	sys 0.028s
W20260709 12:22:15.916468 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.016s	sys 0.028s
W20260709 12:22:15.916798 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 688107us
I20260709 12:22:15.917546 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:15.887601 20881 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:22:15.906143 23179 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:22:15.847080 23267 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.919682 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.920130 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.920647 20803 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:15.904476 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.921489 20810 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:22:15.921772 23268 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.905579 20882 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:15.923537 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:15.923849 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.924288 20884 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:15.924501 20882 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:15.893651 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.925770 20880 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:15.925196 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.927428 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.928812 23038 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.882771 23125 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.869047 23135 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.942278 23135 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.917065 23280 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.944237 23287 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.944509 23135 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.945091 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:15.945349 23280 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.897838 23258 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.946201 23251 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.946321 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:15.906905 23248 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.947889 23258 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.948503 23167 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:15.916273 23277 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.949191 23167 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:15.949260 23248 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.950217 23174 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.950793 23277 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:15.907348 23185 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.915808 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.925822 20882 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:15.952034 20876 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:15.935315 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:15.952854 20878 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:15.953725 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:15.958746 20700 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.769s (SENT)
I20260709 12:22:15.931329 23038 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.925235 23285 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:15.946844 23251 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:15.970887 21435 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:15.975175 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:15.976745 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.972885 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:15.901422 21428 rpcz_store.cc:276] 0709 12:22:14.554077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.556777 (+  2700us) service_pool.cc:224] Handling call
0709 12:22:15.578843 (+1022066us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:15.779565 (+200722us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42856192}
I20260709 12:22:15.966387 23287 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:15.885208 23255 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:15.972245 23163 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.990149 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:15.990078 20810 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:22:15.993144 20810 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 3
I20260709 12:22:15.993420 20810 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:22:15.975737 23189 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:15.993685 20810 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:15.991887 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.997254 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:15.997510 20809 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:15.997918 20727 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:15.998615 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.001781 20806 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:16.002395 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.002573 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.003100 20804 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:15.982429 23293 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.008086 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.014212 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:16.015010 20876 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:16.015246 21434 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:22:16.016520 20700 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.840s (SENT)
I20260709 12:22:16.023365 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.024704 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.025219 20806 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.026095 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.028213 20729 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:15.976480 23243 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:22:16.029071 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:16.030793 21434 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:16.032193 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.032945 21435 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:16.033957 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.034650 21432 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:16.036914 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.040812 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.041550 20726 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:16.044581 22814 raft_consensus.cc:2955] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:16.047238 21427 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:16.047788 20632 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 290) took 9730 ms. Trace:
I20260709 12:22:16.047986 23185 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:16.048445 23024 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:16.024296 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:15.951436 23289 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.023329 23163 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:16.051227 23163 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:16.051446 23289 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:16.052665 23179 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:16.055811 20731 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:16.056372 23167 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:16.057088 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.057713 20731 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:16.058007 23251 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:16.059432 20808 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:16.049496 20803 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:16.047465 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.062909 20882 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:16.058749 23251 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.075498 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.028190 23293 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:16.319736 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.215822 20884 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:16.215379 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.214311 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.335242 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:16.341485 21432 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:16.265998 21430 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.214109 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:16.083129 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66649us
I20260709 12:22:16.083742 20727 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
W20260709 12:22:16.264897 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:16.344185 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.209794 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.170377 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:16.252293 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45130.8us
I20260709 12:22:16.115149 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.345767 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:16.346776 20878 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
W20260709 12:22:16.344832 20700 leader_election.cc:336] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.872s (SENT)
I20260709 12:22:16.221172 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.091981 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:16.348876 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:16.083215 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:16.348217 20733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.009s	sys 0.010s
W20260709 12:22:16.338404 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 119768us
W20260709 12:22:16.346879 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 132862us
W20260709 12:22:16.350666 20733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.009s	sys 0.010s
I20260709 12:22:16.345932 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.356096 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:16.354820 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:16.313827 23298 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:16.136976 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:16.359236 21433 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:16.361747 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 224898us
W20260709 12:22:16.212030 23182 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.000s
I20260709 12:22:16.349999 21434 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.208240 20726 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:16.210914 23163 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.367753 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.214787 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.223913 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.384824 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.388971 20808 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.359774 23277 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:16.358222 23253 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:16.208591 20630 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 4 to 5, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 5 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:16.228986 23295 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:16.216274 23297 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.208069 23287 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.163s
I20260709 12:22:16.362200 23189 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.163970 23167 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.217s
W20260709 12:22:16.390148 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.386456 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.403930 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:16.411516 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 494002us
I20260709 12:22:16.097493 23251 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.224078 23294 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.166s
I20260709 12:22:16.366106 23280 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:16.401659 20879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.023s
I20260709 12:22:16.352610 23024 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:22:16.352833 23199 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:22:16.408321 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63478.6us
I20260709 12:22:16.409370 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.389091 20809 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:16.412678 21432 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:16.415349 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.366875 23038 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:22:16.362080 23125 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:22:16.420001 20879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.008s	sys 0.023s
I20260709 12:22:16.402338 21426 raft_consensus.cc:2376] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5. Re-sending same reply.
W20260709 12:22:16.434033 20700 leader_election.cc:336] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.967s (SENT)
I20260709 12:22:16.444612 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:16.414666 20806 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:16.371124 23182 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.450008 23182 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:16.401839 23277 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.450825 23277 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:16.451414 20802 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:16.404418 21427 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.415488 21428 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:22:16.409431 23189 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:16.370139 23298 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.442831 21435 raft_consensus.cc:2410] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:16.401158 23163 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.419569 23280 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.445449 20875 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:16.448125 20700 leader_election.cc:336] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.972s (SENT)
I20260709 12:22:16.461246 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.404704 23268 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:22:16.406464 20630 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 6, now has a pending config: VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193), VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194), VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195), VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 6 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:16.474102 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.475430 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.408684 23297 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.485725 20806 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.048149 20632 rpcz_store.cc:276] 0709 12:22:06.231048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.231307 (+   259us) service_pool.cc:224] Handling call
0709 12:22:07.226736 (+995429us) spinlock_profiling.cc:243] Waited 92.8 ms on lock 0x7b540007dd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec09090 00007f9e1eca7f3a 00007f9e24a5de9c 00007f9e24b5c04e 00007f9e2301d2a4 00007f9e2301d232 00007f9e2301d161 00007f9e2301d0dc 00007f9e2301c392 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:15.954307 (+8727571us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:12.373542 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:12.381360 (+  7818us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:12.381409 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:22:12.381431 (+    22us) tablet.cc:662] Decoding operations
0709 12:22:12.383072 (+  1641us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:12.383163 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:12.385346 (+  2183us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:22:12.422545 (+ 37199us) tablet.cc:701] Row locks acquired
0709 12:22:12.422570 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:22:12.422722 (+   152us) write_op.cc:270] Start()
0709 12:22:12.422838 (+   116us) write_op.cc:276] Timestamp: P: 1783599732422672 usec, L: 0
0709 12:22:12.422858 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:22:12.423710 (+   852us) log.cc:844] Serialized 7083 byte log entry
0709 12:22:13.245031 (+821321us) op_driver.cc:464] REPLICATION: finished
0709 12:22:13.335304 (+ 90273us) write_op.cc:301] APPLY: starting
0709 12:22:13.336324 (+  1020us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:13.336844 (+   520us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:13.338069 (+  1225us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:13.679094 (+341025us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:13.889169 (+210075us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 12:22:13.923926 (+ 34757us) write_op.cc:312] APPLY: finished
0709 12:22:13.954439 (+ 30513us) log.cc:844] Serialized 125 byte log entry
0709 12:22:13.954632 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:13.954958 (+   326us) write_op.cc:454] Released schema lock
0709 12:22:13.955052 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:13.955678 (+   626us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:13.955712 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1814669,"reported_tablets":29,"spinlock_wait_cycles":213384576,"child_traces":[["op",{"apply.queue_time_us":74845,"apply.run_cpu_time_us":12420,"apply.run_wall_time_us":624397,"num_ops":18,"prepare.queue_time_us":666961,"prepare.run_cpu_time_us":24891,"prepare.run_wall_time_us":657309,"raft.queue_time_us":177371,"raft.run_cpu_time_us":5694,"raft.run_wall_time_us":56407,"replication_time_us":821967,"thread_start_us":877057,"threads_started":4,"wal-append.queue_time_us":604973,"wal-append.run_cpu_time_us":42734,"wal-append.run_wall_time_us":2166857}]]}
I20260709 12:22:16.468478 23189 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.488722 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:16.489464 20881 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.491988 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.493666 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.494266 20808 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
W20260709 12:22:16.483719 23285 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 12:22:16.475315 23280 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:16.497422 20801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.000s	sys 0.035s
I20260709 12:22:16.497129 23304 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:16.516258 21432 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:22:16.500761 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52659us
I20260709 12:22:16.501914 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.520385 20806 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.502514 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.521127 20808 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:16.509265 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.478096 23185 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:16.523155 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.488349 23268 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:16.531352 20805 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:16.477836 23299 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 12:22:16.513221 20801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.000s	sys 0.035s
I20260709 12:22:16.532265 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.534910 20805 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:16.535828 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.536202 20881 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
W20260709 12:22:16.555290 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 143144us
I20260709 12:22:16.537488 20630 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 6, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 6 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:16.561216 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.562283 20805 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:16.564697 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.565722 20875 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:16.529770 23268 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:16.567862 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 201548us
I20260709 12:22:16.574093 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.534880 23295 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:16.589334 20808 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:16.583269 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.577456 20805 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:16.592057 20875 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:16.596407 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.596858 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.597152 20808 raft_consensus.cc:2376] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4. Re-sending same reply.
I20260709 12:22:16.597818 20805 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:16.599152 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.584795 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:16.528348 23185 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:16.601291 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 242456us
I20260709 12:22:16.612581 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:16.575527 23291 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:16.630221 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.586138 23293 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:16.633755 20875 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:16.639643 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.607170 23309 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:16.646734 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90427.9us
I20260709 12:22:16.648984 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.650207 20810 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:16.653671 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.654789 20726 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:16.657035 21400 connection.cc:256] RPC call timeout handler was delayed by -2.9523235409997142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 12:22:16.658551 23167 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:16.659698 23167 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:16.660202 23167 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.660637 23167 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.661676 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.662535 20735 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.664000 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.664754 20727 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:16.665457 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.666039 20728 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:16.666297 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.666751 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.668367 20726 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:16.676360 20630 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 3, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:16.628633 23251 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:16.682641 20735 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:16.688609 20875 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:16.690155 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.646806 23293 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.690853 20808 raft_consensus.cc:2376] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5. Re-sending same reply.
I20260709 12:22:16.646347 23311 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.632020 22986 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 12:22:16.692394 21400 connection.cc:256] RPC call timeout handler was delayed by -2.6472235409996756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260709 12:22:16.697111 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.703490 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:16.693281 23307 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:22:16.706743 20875 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:16.707216 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.707816 20808 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:16.687122 23321 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:16.710444 23321 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:16.711126 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.711498 23321 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.712785 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.713129 23321 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.714332 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.715693 23313 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:16.716127 23313 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:16.704831 23295 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.716997 23313 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.718214 23313 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.679337 23318 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:16.692116 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.701056 23293 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.724733 20808 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:16.723552 23308 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 12:22:16.741444 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 138505us
I20260709 12:22:16.744077 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:16.745908 23305 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 12:22:16.746978 23297 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:16.747577 23300 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:22:16.748916 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1858) took 2864 ms. Trace:
I20260709 12:22:16.749298 20801 rpcz_store.cc:276] 0709 12:22:13.884804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.964437 (+ 79633us) service_pool.cc:224] Handling call
0709 12:22:15.494668 (+1530231us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:15.494884 (+   216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:15.495210 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:15.495239 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:15.816073 (+320834us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:15.816634 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:22:16.559505 (+742871us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:16.568021 (+  8516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:16.574862 (+  6841us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.748651 (+173789us) raft_consensus.cc:1685] finished
0709 12:22:16.748677 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.748830 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40845440,"thread_start_us":717620,"threads_started":1,"wal-append.queue_time_us":712584}
I20260709 12:22:16.750711 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.678354 23309 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.688093 23277 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:16.751637 23302 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:22:16.753413 23277 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:16.751456 20808 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
W20260709 12:22:16.755668 21400 connection.cc:256] RPC call timeout handler was delayed by -2.5284235409999383s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 12:22:16.762290 20630 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 2, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:16.763080 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.763949 21433 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:16.721022 23306 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.724627 23318 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.774024 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.774700 20735 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 2.
I20260709 12:22:16.776957 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.778492 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.779443 20727 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:16.782173 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.782982 20731 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.791827 20731 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
W20260709 12:22:16.793285 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49301.2us
I20260709 12:22:16.793655 20728 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:16.761512 23303 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
W20260709 12:22:16.802189 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 189686us
W20260709 12:22:16.809741 21400 connection.cc:256] RPC call timeout handler was delayed by -1.6865527759996439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 12:22:16.810031 21433 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:16.702395 23311 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:16.811040 21400 connection.cc:256] RPC call timeout handler was delayed by -1.5030759639998905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 12:22:16.813035 21400 connection.cc:256] RPC call timeout handler was delayed by -1.3261986100001195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
I20260709 12:22:16.815512 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.695385 23268 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.814347 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:16.766397 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:16.830538 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.831517 20732 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:16.837158 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.773773 23314 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:16.813961 23301 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:22:16.766587 21426 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:16.836915 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.838405 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.808132 23297 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.793103 23318 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.842808 20735 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:16.830206 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.833837 22986 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:16.880414 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.851351 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:16.846118 20775 connection.cc:256] RPC call timeout handler was delayed by -1.0723531640001056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 12:22:16.881747 20732 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:16.881768 20775 connection.cc:256] RPC call timeout handler was delayed by -1.0210531640000227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
I20260709 12:22:16.884097 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.885478 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.886169 20732 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.822506 23280 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:16.841223 20801 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.891127 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:16.891829 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.894876 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 754) took 1148 ms. Trace:
I20260709 12:22:16.892117 23320 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.895275 21429 rpcz_store.cc:276] 0709 12:22:15.746668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.746923 (+   255us) service_pool.cc:224] Handling call
0709 12:22:15.747475 (+   552us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:16.023749 (+276274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:16.023786 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:16.024513 (+   727us) log.cc:844] Serialized 52 byte log entry
0709 12:22:16.477464 (+452951us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:16.477513 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:16.477586 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.894618 (+417032us) raft_consensus.cc:1685] finished
0709 12:22:16.894640 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.894813 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":310585,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":116,"thread_start_us":665004,"threads_started":2,"wal-append.queue_time_us":504726}
I20260709 12:22:16.900455 21429 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:16.836614 23310 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:16.840794 23163 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:16.902163 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.899529 20630 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:16.842562 23251 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:22:16.846055 23060 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:16.808804 23306 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.913944 20726 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:16.905016 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:16.907254 23325 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:22:16.887734 23268 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.918322 21433 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:16.919209 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.919770 21433 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:16.920555 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.921926 21433 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:16.836437 23326 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:22:16.846405 23024 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:16.912185 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.912763 20775 leader_election.cc:400] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:16.929204 20775 leader_election.cc:403] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:16.916188 20732 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:16.910482 23310 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.847597 23298 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:16.893138 22986 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:16.917953 23060 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.923691 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.936721 20735 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:16.915953 20630 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:16.926878 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:16.939685 20735 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:16.940191 23310 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:16.941293 23289 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:16.945292 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.948526 23280 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:16.935396 23024 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.955025 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.958295 23167 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:16.959823 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47279.8us
I20260709 12:22:16.960162 23309 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:16.960602 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.930138 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:16.911490 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.950879 22986 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:16.930974 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.918390 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:16.898892 23320 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:16.966878 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:16.967531 21434 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:16.965687 23320 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.969013 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.969728 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.970458 20878 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:16.970392 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.932986 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1600) took 1071 ms. Trace:
I20260709 12:22:16.971432 20877 rpcz_store.cc:276] 0709 12:22:15.861970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.875205 (+ 13235us) service_pool.cc:224] Handling call
0709 12:22:15.875499 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:15.875845 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:15.875871 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:15.876385 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:22:16.338630 (+462245us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:16.338682 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:16.338768 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.932785 (+594017us) raft_consensus.cc:1685] finished
0709 12:22:16.932806 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.932925 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":461860,"threads_started":1,"wal-append.queue_time_us":607507}
I20260709 12:22:16.973062 23267 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.973886 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:16.960669 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:16.964372 23293 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:16.866946 23297 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:16.946892 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:16.985873 20809 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:16.999080 21431 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:16.990584 20879 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:16.990482 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:16.989876 20728 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:16.845691 23314 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:17.001509 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 92518.3us
I20260709 12:22:17.005030 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:16.965612 23311 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:17.010867 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:17.011839 20882 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:16.932313 23287 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:22:17.017077 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:17.020845 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.021620 20809 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:17.001757 23182 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:17.029085 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1672) took 2649 ms. Trace:
I20260709 12:22:16.934607 23189 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.043725 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.036178 20733 rpcz_store.cc:276] 0709 12:22:14.379539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.461999 (+1082460us) service_pool.cc:224] Handling call
0709 12:22:15.479731 (+ 17732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:15.491058 (+ 11327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:15.491102 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:15.502319 (+ 11217us) log.cc:844] Serialized 26 byte log entry
0709 12:22:16.351203 (+848884us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:16.362881 (+ 11678us) log.cc:844] Serialized 14 byte log entry
0709 12:22:17.028367 (+665486us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b1800451300. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:17.028572 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:17.028675 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:17.028733 (+    58us) raft_consensus.cc:1685] finished
0709 12:22:17.028836 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:17.028997 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3560,"prepare.queue_time_us":14394,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":5227,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":7196928,"thread_start_us":687413,"threads_started":1,"wal-append.queue_time_us":921123}
I20260709 12:22:17.046978 20809 raft_consensus.cc:2376] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5. Re-sending same reply.
I20260709 12:22:17.045281 21429 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:17.022940 23268 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:17.045244 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.044739 20810 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
W20260709 12:22:17.049935 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 661824us
I20260709 12:22:17.014118 23309 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.062682 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.026237 23287 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:22:17.037994 23248 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:17.014348 23317 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:22:17.046330 23311 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:17.075876 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:17.079918 20700 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.104s (SENT)
I20260709 12:22:17.080757 20877 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.081197 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.082008 23311 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:17.082985 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:17.043457 23288 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.086849 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:17.095883 20728 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:17.068132 23060 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:17.098590 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.099298 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:17.098937 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.103572 20733 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:17.061965 23266 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:22:17.084748 23277 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.091951 23280 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:17.110656 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.116106 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:17.106150 23285 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.119758 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:17.112931 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:17.123565 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:17.083074 23311 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.130070 20732 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:17.131459 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 170866us
I20260709 12:22:17.120795 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.121048 23258 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:17.112526 23277 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.137293 20775 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:17.137898 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:17.136255 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:17.141486 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:17.143101 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:17.140511 20883 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:17.115578 23280 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.128093 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:17.130904 23167 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:17.137732 23321 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:17.152755 20775 leader_election.cc:400] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:17.124327 23285 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.162879 21432 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:17.140995 23297 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:17.143232 23272 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:17.150267 23258 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:17.168521 23189 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:17.177053 23280 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:17.177990 23280 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:17.171079 23272 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:17.152367 23338 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:17.180824 23336 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:22:17.181041 23321 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:17.182224 23272 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:17.183167 23258 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:17.192314 23258 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.195291 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.174060 23285 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:17.153293 23179 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:17.195696 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51819.5us
I20260709 12:22:17.167603 20803 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:17.201108 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.150823 23314 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:17.191309 20726 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:17.178591 23280 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:17.177130 20735 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:22:17.195043 23288 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:17.190639 23320 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.236429 23320 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:17.222585 21434 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:17.241348 21434 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:17.244768 21434 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:17.195590 23268 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.268447 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.271575 20805 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:17.204229 23179 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.204108 23297 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:17.270418 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69433.3us
I20260709 12:22:17.208613 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:17.273743 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:17.276401 20775 leader_election.cc:400] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:17.208115 23289 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:17.276888 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:17.286249 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166539us
I20260709 12:22:17.286769 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:17.287812 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 177221us
I20260709 12:22:17.287914 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.266989 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1885) took 1206 ms. Trace:
I20260709 12:22:17.274837 23179 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:17.205291 23338 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
W20260709 12:22:17.300848 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 99513.4us
I20260709 12:22:17.277561 23297 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.289214 23251 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:17.289064 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:17.297263 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:17.289680 23277 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:17.312105 23349 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:17.313077 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70035.2us
W20260709 12:22:17.316777 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 241209us
I20260709 12:22:17.295739 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:17.329138 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52865.7us
I20260709 12:22:17.331007 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:17.332381 20877 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:17.300617 23319 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.296948 23341 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:17.300446 23321 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:17.334317 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:17.319161 21427 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:17.358889 20730 raft_consensus.cc:2955] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:17.369028 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:17.373787 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.306803 23297 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:17.310858 23251 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:17.368626 20726 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.381958 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:17.303133 23295 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:22:17.383005 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:17.381273 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50378.9us
I20260709 12:22:17.389900 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:17.391642 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:17.392302 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:17.359948 23321 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:17.382356 22814 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.170s	user 0.014s	sys 0.000s
I20260709 12:22:17.359624 23341 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:17.405952 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:17.387631 23266 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:22:17.414567 20884 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:17.299932 20803 rpcz_store.cc:276] 0709 12:22:16.013178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.110366 (+1097188us) service_pool.cc:224] Handling call
0709 12:22:17.219704 (+109338us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:17.450639 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 90631.7us
W20260709 12:22:17.450999 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 116758us
I20260709 12:22:17.338904 23320 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:17.385315 23251 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:17.385816 22742 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:17.436414 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46594.8us
I20260709 12:22:17.390973 23349 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:17.475322 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 170218us
I20260709 12:22:17.453311 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:17.506040 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.421793 23341 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:17.412949 23321 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:17.518150 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.455577 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.383283 23280 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:17.516768 23253 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.097s	user 0.030s	sys 0.013s
W20260709 12:22:17.539100 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85915.3us
I20260709 12:22:17.537704 20807 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:17.540140 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:17.516160 23285 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:17.547086 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 213574us
W20260709 12:22:17.539737 23253 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.030s	sys 0.013s
I20260709 12:22:17.526678 23266 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.513880 22742 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.542253 23285 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:17.586746 23285 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:17.586632 23320 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:17.590392 23320 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.551112 23253 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:17.712826 20630 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.003s	sys 0.025s
I20260709 12:22:17.592571 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:17.744684 20630 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.006s	sys 0.032s
W20260709 12:22:17.608243 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92792.8us
I20260709 12:22:17.749359 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:17.587918 23258 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.751930 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:17.717424 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:17.587066 23318 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.000s
I20260709 12:22:17.578639 23357 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.181s
W20260709 12:22:17.702107 23343 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.342s	user 0.001s	sys 0.000s
W20260709 12:22:17.739923 23345 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:22:17.737505 23285 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.766264 23366 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:17.767057 23352 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 12:22:17.767679 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:17.728333 23362 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 12:22:17.706295 23364 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:22:17.658072 23354 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:17.666802 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:17.738965 23267 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.003s
W20260709 12:22:17.750325 23272 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 12:22:17.678329 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:17.775418 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 437) took 1772 ms. Trace:
I20260709 12:22:17.777194 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.793236 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.794366 20806 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:17.777385 20726 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
W20260709 12:22:17.778122 20801 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.864s	user 0.007s	sys 0.005s
I20260709 12:22:17.591928 23359 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:17.780207 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:17.727351 23365 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.681560 23347 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:17.579123 23266 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.801292 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.809092 23371 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.812098 23255 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:17.779950 23318 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.816717 22742 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:17.801575 23277 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:17.824864 23266 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:17.826731 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1874) took 1835 ms (client timeout 1796 ms). Trace:
I20260709 12:22:17.681702 23358 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.382s
I20260709 12:22:17.828089 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:17.828981 20727 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:17.771402 23258 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.832334 23258 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:17.832597 21431 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:17.834543 23306 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:17.835403 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:17.837841 20807 rpcz_store.cc:269] 0709 12:22:15.990896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.991079 (+   183us) service_pool.cc:224] Handling call
0709 12:22:17.826536 (+1835457us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003c6c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:17.826654 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540389632}
W20260709 12:22:17.851497 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 311426us
I20260709 12:22:17.796769 23366 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:17.852624 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:17.852838 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 187546us
I20260709 12:22:17.792315 23354 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:17.628993 23349 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.860204 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.803463 20801 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:17.862625 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:17.798317 23272 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:17.853706 23277 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:17.879014 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 129737us
I20260709 12:22:17.883889 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:17.855531 23366 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:17.854862 23347 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:17.863822 23359 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:17.890084 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:17.891048 20733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.015s	sys 0.023s
W20260709 12:22:17.891574 20733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.015s	sys 0.023s
I20260709 12:22:17.868247 20726 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:17.885710 23179 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
I20260709 12:22:17.916306 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:17.866149 23365 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:17.892036 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:17.918848 20734 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:17.922062 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:17.919129 20775 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5.
I20260709 12:22:17.797858 21428 rpcz_store.cc:276] 0709 12:22:16.003067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.013998 (+ 10931us) service_pool.cc:224] Handling call
0709 12:22:16.048820 (+ 34822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:16.465780 (+416960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:17.775085 (+1309305us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54006333d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:17.775350 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731770624}
I20260709 12:22:17.920697 20879 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:17.878175 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 356) took 1215 ms. Trace:
I20260709 12:22:17.858074 23374 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:22:17.925114 20730 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:17.882346 23267 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.890673 23248 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.889161 23277 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.889895 23359 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.887571 23266 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:22:17.866580 23253 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:17.957392 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68061.3us
I20260709 12:22:17.926165 23179 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:17.930162 20801 rpcz_store.cc:276] 0709 12:22:16.654593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.751022 (+ 96429us) service_pool.cc:224] Handling call
0709 12:22:17.869584 (+1118562us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:17.965282 23373 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:22:17.966359 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.967347 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 360) took 1309 ms. Trace:
W20260709 12:22:17.967484 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74925.7us
I20260709 12:22:17.967797 20801 rpcz_store.cc:276] 0709 12:22:16.658055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.966024 (+1307969us) service_pool.cc:224] Handling call
0709 12:22:17.967287 (+  1263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:17.968909 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:17.970989 23376 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:22:17.971901 20806 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:17.973371 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.963816 23354 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:17.955348 23359 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.996048 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:17.972023 23297 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.004s
I20260709 12:22:17.971609 23179 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:17.971444 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.002194 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:18.010480 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:17.985702 23382 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:18.016134 23375 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:18.022302 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:17.956389 23366 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.025252 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.020839 23251 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.006634 23361 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.020860 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:18.032029 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:18.032908 23382 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:18.033383 23382 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:18.033763 23382 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:18.035033 23321 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 12:22:18.035404 23320 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:18.035543 23321 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.036008 23320 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.038115 23321 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.038197 23320 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:17.972035 23347 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.040697 23251 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:18.028008 23179 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:18.042459 23179 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:18.043569 23318 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:18.043368 23179 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.002063 23297 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.047267 21434 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:18.048305 23297 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.050082 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 631) took 1169 ms. Trace:
I20260709 12:22:18.052335 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:18.054003 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:17.997812 23365 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:22:18.050733 21434 rpcz_store.cc:276] 0709 12:22:16.880490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.250606 (+370116us) service_pool.cc:224] Handling call
0709 12:22:17.300525 (+ 49919us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.047648 (+747123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.049809 (+  2161us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400690fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:18.050008 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716653824}
I20260709 12:22:18.062326 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:18.031108 23361 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:18.059551 23267 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:18.060385 23366 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.076007 20804 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:22:18.079756 20804 pending_rounds.cc:85] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 1
I20260709 12:22:18.072476 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:18.063329 23375 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:18.063087 23358 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:18.080061 20804 pending_rounds.cc:107] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:22:18.082294 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1685) took 2379 ms. Trace:
I20260709 12:22:18.071247 23135 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:18.082566 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:18.084204 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 635) took 1073 ms. Trace:
I20260709 12:22:18.080509 23255 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:18.082717 20802 rpcz_store.cc:276] 0709 12:22:15.702570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.702746 (+   176us) service_pool.cc:224] Handling call
0709 12:22:15.735379 (+ 32633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:16.451838 (+716459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.082037 (+1630199us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b540006d1d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:18.082236 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646200064}
I20260709 12:22:18.066072 23272 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.082604 20804 raft_consensus.cc:2889] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:18.071058 23341 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.000s
W20260709 12:22:18.078832 23387 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.052s	user 0.000s	sys 0.003s
I20260709 12:22:18.079409 23361 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.076097 23365 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:18.079308 23267 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:18.100515 23377 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:22:18.078944 23277 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.090996 23135 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:18.092938 23060 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.093705 23338 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:18.079867 23179 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:22:18.092429 21431 rpcz_store.cc:276] 0709 12:22:17.010557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.833397 (+822840us) service_pool.cc:224] Handling call
0709 12:22:17.833730 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:17.856205 (+ 22475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.856250 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:17.856802 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:22:17.962559 (+105757us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:18.083805 (+121246us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.083987 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.084037 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.084062 (+    25us) raft_consensus.cc:1685] finished
0709 12:22:18.084069 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.084162 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34223,"prepare.queue_time_us":40575,"prepare.run_wall_time_us":51,"raft.queue_time_us":24452,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":13054,"spinlock_wait_cycles":2305408,"thread_start_us":170306,"threads_started":2,"wal-append.queue_time_us":151578}
I20260709 12:22:18.110106 23060 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.102006 23341 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:18.115357 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:18.115804 23135 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.118875 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:18.146816 23135 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.102813 23267 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:18.150141 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.103282 23382 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.132339 20805 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:18.101861 23358 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:22:18.144659 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:18.145810 23341 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.108589 23365 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:18.152316 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:18.151654 21428 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:18.143980 23179 consensus_queue.cc:1243] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:22:18.154312 23297 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:18.166487 23297 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:18.170277 23297 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.172008 23185 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.156067 23251 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:22:18.155943 23320 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:22:18.182806 23382 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.158902 23359 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.194648 23365 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:18.198350 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.200614 23359 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.201128 23267 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:18.216837 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 1742) took 2002 ms (client timeout 1769 ms). Trace:
W20260709 12:22:18.217317 20881 rpcz_store.cc:269] 0709 12:22:16.214727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.214884 (+   157us) service_pool.cc:224] Handling call
0709 12:22:18.216667 (+2001783us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b540022ecd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:18.216769 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338376192}
I20260709 12:22:18.220523 23024 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.225457 21428 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:18.227316 20729 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:18.213979 23361 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:18.230636 23389 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.207s	user 0.001s	sys 0.001s
I20260709 12:22:18.231798 23357 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:18.231671 23297 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.232188 23357 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:18.233021 23357 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.233815 23297 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:18.234195 23357 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:18.236269 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 143745us
I20260709 12:22:18.242199 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1882) took 2224 ms. Trace:
I20260709 12:22:18.248127 23267 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:18.028549 23199 consensus_queue.cc:1175] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:18.256338 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 445) took 2044 ms. Trace:
I20260709 12:22:18.242749 20802 rpcz_store.cc:276] 0709 12:22:16.010955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.097836 (+2086881us) service_pool.cc:224] Handling call
0709 12:22:18.104189 (+  6353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.104589 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:18.104615 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:18.105024 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 12:22:18.163350 (+ 58326us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:18.168778 (+  5428us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.168996 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.169072 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.235356 (+ 66284us) raft_consensus.cc:1685] finished
0709 12:22:18.235380 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.235551 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4734,"prepare.run_wall_time_us":39,"raft.queue_time_us":188,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":5008,"spinlock_wait_cycles":30685568,"thread_start_us":57941,"threads_started":1,"wal-append.queue_time_us":86227}
I20260709 12:22:18.256765 21435 rpcz_store.cc:276] 0709 12:22:16.211489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.214035 (+  2546us) service_pool.cc:224] Handling call
0709 12:22:18.256066 (+2042031us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:18.256257 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230776704}
I20260709 12:22:18.231812 23361 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.266122 20807 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:18.266884 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1884) took 2254 ms. Trace:
I20260709 12:22:18.267298 20807 rpcz_store.cc:276] 0709 12:22:16.012362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.265334 (+2252972us) service_pool.cc:224] Handling call
0709 12:22:18.265710 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.266571 (+   861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.266798 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.268198 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 626) took 1449 ms. Trace:
I20260709 12:22:18.268934 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1886) took 2241 ms. Trace:
I20260709 12:22:18.271299 20807 rpcz_store.cc:276] 0709 12:22:16.027638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.268097 (+2240459us) service_pool.cc:224] Handling call
0709 12:22:18.268404 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.268574 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:18.268603 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:18.268644 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:18.268680 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.268730 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.268849 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.268579 21430 rpcz_store.cc:276] 0709 12:22:16.819137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.961726 (+142589us) service_pool.cc:224] Handling call
0709 12:22:16.962056 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:17.035704 (+ 73648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.035742 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:17.049182 (+ 13440us) log.cc:844] Serialized 26 byte log entry
0709 12:22:17.127693 (+ 78511us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:17.150085 (+ 22392us) log.cc:844] Serialized 14 byte log entry
0709 12:22:17.150348 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.267828 (+1117480us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:18.267884 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.267928 (+    44us) raft_consensus.cc:1685] finished
0709 12:22:18.268018 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.268144 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1104,"prepare.queue_time_us":41048,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":3783,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":22884,"spinlock_wait_cycles":125610368,"thread_start_us":59850,"threads_started":2,"wal-append.queue_time_us":83136,"wal-append.run_cpu_time_us":2401,"wal-append.run_wall_time_us":1030132}
W20260709 12:22:18.328184 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 625) took 1509 ms (client timeout 1634 ms). Trace:
W20260709 12:22:18.328680 21432 rpcz_store.cc:269] 0709 12:22:16.818846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.868761 (+ 49915us) service_pool.cc:224] Handling call
0709 12:22:18.327951 (+1459190us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540049c854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:18.328102 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532003584}
I20260709 12:22:18.407050 23323 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:18.407879 23323 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:18.409322 23323 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.410383 23323 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:18.415467 23355 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:18.415978 23355 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:18.450896 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.467860 23291 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:18.468497 23291 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.470072 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1893) took 1362 ms. Trace:
I20260709 12:22:18.470482 20805 rpcz_store.cc:276] 0709 12:22:17.107859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.276649 (+168790us) service_pool.cc:224] Handling call
0709 12:22:18.469846 (+1193197us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400126e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:18.470002 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1935328512}
I20260709 12:22:18.479713 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.480399 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:18.480844 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 355) took 1826 ms. Trace:
I20260709 12:22:18.481117 20809 rpcz_store.cc:276] 0709 12:22:16.654318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.108364 (+454046us) service_pool.cc:224] Handling call
0709 12:22:18.479546 (+1371182us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:18.480791 (+  1245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4447616}
I20260709 12:22:18.482295 20809 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.482638 20809 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:18.483695 20809 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.484920 20809 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.496879 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:18.499775 23295 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:18.500634 23060 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.500443 23295 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:18.503414 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.504304 23295 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.507272 20809 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:22:18.507987 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:18.509234 20809 pending_rounds.cc:85] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 4
I20260709 12:22:18.509498 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1878) took 2502 ms. Trace:
I20260709 12:22:18.509886 20804 rpcz_store.cc:276] 0709 12:22:16.006861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.007043 (+   182us) service_pool.cc:224] Handling call
0709 12:22:16.007440 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.092302 (+2084862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.509267 (+416965us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b540006d454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:18.509433 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4746381696}
I20260709 12:22:18.512219 20804 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:18.512918 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1716) took 1873 ms. Trace:
I20260709 12:22:18.513294 20804 rpcz_store.cc:276] 0709 12:22:16.639854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.510718 (+1870864us) service_pool.cc:224] Handling call
0709 12:22:18.511033 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.512652 (+  1619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.512861 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.514891 20804 raft_consensus.cc:2955] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:18.518795 23291 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.509550 20809 pending_rounds.cc:107] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:22:18.526615 20809 raft_consensus.cc:2889] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:18.527406 23182 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:22:18.527477 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1703) took 2109 ms. Trace:
I20260709 12:22:18.527948 20809 rpcz_store.cc:276] 0709 12:22:16.418349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.481836 (+2063487us) service_pool.cc:224] Handling call
0709 12:22:18.482170 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.527130 (+ 44960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.527384 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.533675 23291 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.534775 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:18.535579 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1889) took 2496 ms. Trace:
I20260709 12:22:18.535605 21432 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:18.536164 20805 rpcz_store.cc:276] 0709 12:22:16.039254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.471480 (+2432226us) service_pool.cc:224] Handling call
0709 12:22:18.471825 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.472045 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:18.472392 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.535151 (+ 62759us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:18.535216 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:18.535269 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.535333 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.535487 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"thread_start_us":62428,"threads_started":1,"wal-append.queue_time_us":62537}
I20260709 12:22:18.538296 20805 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:18.538843 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1894) took 1430 ms. Trace:
I20260709 12:22:18.541707 20805 rpcz_store.cc:276] 0709 12:22:17.108168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.537833 (+1429665us) service_pool.cc:224] Handling call
0709 12:22:18.538130 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.538633 (+   503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.538792 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.543179 20805 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:18.543890 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1897) took 1416 ms. Trace:
I20260709 12:22:18.544409 20805 rpcz_store.cc:276] 0709 12:22:17.127928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.542615 (+1414687us) service_pool.cc:224] Handling call
0709 12:22:18.542940 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.543652 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.543841 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.546213 20805 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:22:18.547822 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1898) took 1416 ms. Trace:
I20260709 12:22:18.548825 20805 rpcz_store.cc:276] 0709 12:22:17.131902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.545689 (+1413787us) service_pool.cc:224] Handling call
0709 12:22:18.545993 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.547574 (+  1581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.547770 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.550971 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1899) took 1408 ms. Trace:
I20260709 12:22:18.554814 20805 rpcz_store.cc:276] 0709 12:22:17.142320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.550145 (+1407825us) service_pool.cc:224] Handling call
0709 12:22:18.550454 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.550637 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:18.550670 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:18.550717 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:18.550754 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.550807 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.550917 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.554384 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1890) took 1450 ms. Trace:
I20260709 12:22:18.556964 23285 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.555871 20804 rpcz_store.cc:276] 0709 12:22:17.104250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.514175 (+1409925us) service_pool.cc:224] Handling call
0709 12:22:18.514540 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.514775 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:18.527502 (+ 12727us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.553047 (+ 25545us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.553653 (+   606us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.554018 (+   365us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:18.554069 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:18.554119 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.554185 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.554308 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1010,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":247,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":216,"spinlock_wait_cycles":41728,"thread_start_us":23714,"threads_started":1,"wal-append.queue_time_us":23786}
I20260709 12:22:18.560937 23291 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.561540 23291 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.563647 23291 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.564940 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:18.565658 21432 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:18.567164 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:18.567359 20802 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.568022 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1883) took 2556 ms. Trace:
I20260709 12:22:18.568329 20802 rpcz_store.cc:276] 0709 12:22:16.011683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.259142 (+2247459us) service_pool.cc:224] Handling call
0709 12:22:18.567165 (+308023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:18.567790 (+   625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.567969 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:18.569315 23291 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:18.569802 23291 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:18.570194 23291 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:18.575750 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1958) took 1481 ms. Trace:
I20260709 12:22:18.576207 20735 rpcz_store.cc:276] 0709 12:22:17.094673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.094822 (+   149us) service_pool.cc:224] Handling call
0709 12:22:17.106883 (+ 12061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:17.217825 (+110942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.575490 (+1357665us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:18.575682 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160729856}
I20260709 12:22:18.578658 23291 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.581928 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:18.582598 21432 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:18.587462 23312 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.588330 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 1969) took 1402 ms. Trace:
I20260709 12:22:18.588495 23291 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.589107 23317 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:18.589670 23317 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.591907 23317 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:18.592404 23317 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:18.592451 20850 connection.cc:256] RPC call timeout handler was delayed by -1.2742487109999274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:22:18.592950 23317 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:18.593439 20850 connection.cc:256] RPC call timeout handler was delayed by -1.1977487109998037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 12:22:18.593828 20850 connection.cc:256] RPC call timeout handler was delayed by -1.1959487109998008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 12:22:18.594305 20850 connection.cc:256] RPC call timeout handler was delayed by -1.039432668000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:22:18.594621 20850 connection.cc:256] RPC call timeout handler was delayed by -1.0033487109999442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
I20260709 12:22:18.588119 23312 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.571513 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1892) took 1466 ms. Trace:
I20260709 12:22:18.595152 23317 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:18.613226 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1967) took 1446 ms. Trace:
I20260709 12:22:18.597553 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:18.615072 23391 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.591s	user 0.003s	sys 0.000s
I20260709 12:22:18.615499 23384 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:18.615617 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:18.615952 23384 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:18.616365 23384 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:18.617475 23258 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:18.617786 23295 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:18.588752 20729 rpcz_store.cc:276] 0709 12:22:17.186380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.330343 (+143963us) service_pool.cc:224] Handling call
0709 12:22:18.588137 (+1257794us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b540028cb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:18.588276 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957281152}
I20260709 12:22:18.600291 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:18.625334 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 638) took 1518 ms. Trace:
I20260709 12:22:18.625689 21426 rpcz_store.cc:276] 0709 12:22:17.106874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.202850 (+ 95976us) service_pool.cc:224] Handling call
0709 12:22:18.600078 (+1397228us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:18.625267 (+ 25189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15527296}
I20260709 12:22:18.609279 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1972) took 1421 ms. Trace:
I20260709 12:22:18.627534 20727 rpcz_store.cc:276] 0709 12:22:17.187412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.829570 (+642158us) service_pool.cc:224] Handling call
0709 12:22:17.829961 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:17.868341 (+ 38380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.868377 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:17.877221 (+  8844us) log.cc:844] Serialized 52 byte log entry
0709 12:22:17.943728 (+ 66507us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:17.943767 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.230053 (+286286us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:18.608874 (+378821us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b54003b4554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:18.609008 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.609057 (+    49us) raft_consensus.cc:1685] finished
0709 12:22:18.609070 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.609218 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1748,"raft.queue_time_us":2021,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":7249,"spinlock_wait_cycles":27563008,"thread_start_us":19686,"threads_started":1,"wal-append.queue_time_us":115546}
I20260709 12:22:18.613947 23317 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.613683 20734 rpcz_store.cc:276] 0709 12:22:17.166730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.166967 (+   237us) service_pool.cc:224] Handling call
0709 12:22:17.196359 (+ 29392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:17.929210 (+732851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.612980 (+683770us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54003b5454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:18.613164 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661097344}
I20260709 12:22:18.630515 23317 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.610814 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.631538 20808 raft_consensus.cc:2376] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5. Re-sending same reply.
I20260709 12:22:18.631482 23330 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:18.632179 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1720) took 1878 ms (client timeout 1563 ms). Trace:
I20260709 12:22:18.632670 23317 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.606216 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1891) took 1501 ms. Trace:
I20260709 12:22:18.633205 20801 rpcz_store.cc:276] 0709 12:22:17.104642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.024861 (+920219us) service_pool.cc:224] Handling call
0709 12:22:18.605966 (+581105us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54003a1454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:18.606114 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262293760}
I20260709 12:22:18.599920 23312 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:18.608090 23125 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:18.635258 23125 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:18.632525 20808 rpcz_store.cc:269] 0709 12:22:16.754124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.756395 (+  2271us) service_pool.cc:224] Handling call
0709 12:22:18.610627 (+1854232us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:18.632110 (+ 21483us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4546304}
I20260709 12:22:18.636549 23125 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:18.637933 23125 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:18.603405 20809 rpcz_store.cc:276] 0709 12:22:17.104959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.528799 (+1423840us) service_pool.cc:224] Handling call
0709 12:22:18.529164 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.529608 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:18.529643 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:18.530239 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:22:18.536163 (+  5924us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:18.536200 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.536280 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.571270 (+ 34990us) raft_consensus.cc:1685] finished
0709 12:22:18.571293 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.571452 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":5499,"threads_started":1,"wal-append.queue_time_us":3976}
I20260709 12:22:18.595346 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:18.643436 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48191.5us
I20260709 12:22:18.644289 23291 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:18.646026 23291 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.648185 23291 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:18.649535 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:18.652750 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:18.668865 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.669596 20808 raft_consensus.cc:2376] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8. Re-sending same reply.
I20260709 12:22:18.669590 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:18.657423 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:18.657691 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:18.679702 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:18.680235 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.658274 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.681897 20802 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:18.659731 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.661062 23287 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:18.663555 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:18.688547 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.652822 23312 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:18.627066 23295 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:18.683946 21432 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:18.685094 23412 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:18.685427 23370 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.687426 23304 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:18.689407 20809 raft_consensus.cc:2368] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5.
I20260709 12:22:18.694264 23370 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.694747 23304 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:18.695710 23304 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.696520 23370 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:18.696667 23304 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:18.689438 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 89275.2us
I20260709 12:22:18.691468 23287 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:18.692808 20803 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:18.690953 20801 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:22:18.693408 23312 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:18.703423 20803 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:22:18.704368 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.705057 20809 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:18.708387 20804 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:18.709167 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.708396 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.709990 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.711796 20804 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:18.705439 23287 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:18.714928 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.713526 20809 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.713764 20804 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:18.714514 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:18.727056 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:18.722566 20805 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:18.716290 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:18.721716 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.735642 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:18.736388 21433 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:18.740188 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1887) took 2704 ms. Trace:
I20260709 12:22:18.740638 20807 rpcz_store.cc:276] 0709 12:22:16.035611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.272571 (+2236960us) service_pool.cc:224] Handling call
0709 12:22:18.642248 (+369677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.647637 (+  5389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:18.647672 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:18.648155 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:22:18.691466 (+ 43311us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:18.691512 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.691582 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.739969 (+ 48387us) raft_consensus.cc:1685] finished
0709 12:22:18.739987 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.740123 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4976,"raft.queue_time_us":6169,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":42851,"threads_started":1,"wal-append.queue_time_us":79374}
I20260709 12:22:18.704382 23384 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.749589 20802 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:18.749816 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:18.752908 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:18.754575 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71372.3us
I20260709 12:22:18.755673 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:18.757838 20805 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:18.759377 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.760468 20809 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:18.768517 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:18.747449 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.780391 20809 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:18.773453 20803 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:18.752271 23299 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:18.776726 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 761) took 1879 ms. Trace:
W20260709 12:22:18.782317 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68272.3us
W20260709 12:22:18.779021 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.779736 21433 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:18.751613 23304 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:18.781647 21429 rpcz_store.cc:276] 0709 12:22:16.896933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.806868 (+909935us) service_pool.cc:224] Handling call
0709 12:22:17.807216 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:17.852021 (+ 44805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.852050 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:17.857542 (+  5492us) log.cc:844] Serialized 26 byte log entry
0709 12:22:18.602628 (+745086us) spinlock_profiling.cc:243] Waited 958 us on lock 0x7b1800451d20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:22:18.717924 (+115296us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:18.717967 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.718014 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.776346 (+ 58332us) raft_consensus.cc:1685] finished
0709 12:22:18.776374 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.776649 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32377,"raft.queue_time_us":48543,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2864,"spinlock_wait_cycles":2202496,"thread_start_us":115100,"threads_started":1,"wal-append.queue_time_us":916673}
I20260709 12:22:18.758041 23370 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:18.795655 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:18.769686 23295 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:18.802000 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46475.3us
I20260709 12:22:18.803231 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:18.801906 23323 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:22:18.805378 20801 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:18.798404 23295 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:18.800916 23299 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:22:18.801189 23304 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:18.830484 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:18.839252 23420 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.827576 23295 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:18.841905 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58726.3us
I20260709 12:22:18.844306 20881 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:18.848356 20882 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:18.850809 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:18.849782 23417 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:18.833302 23304 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:18.840583 23314 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.878756 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:18.920612 20878 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:18.917407 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:18.922353 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75657.8us
I20260709 12:22:18.868257 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:18.924216 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:18.924954 20875 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7.
I20260709 12:22:18.926239 20882 raft_consensus.cc:2368] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:18.926625 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:18.927253 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:18.919554 23316 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.930166 23316 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:18.930814 23383 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:18.931236 23383 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:18.946842 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:18.951848 20774 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7.
I20260709 12:22:18.952381 20774 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:22:18.955405 20878 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:18.957582 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:18.958287 20878 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:18.960067 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:18.919287 23358 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.964289 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:18.965258 20774 leader_election.cc:400] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:18.967005 20774 leader_election.cc:403] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:22:18.926040 23412 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:18.971742 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:18.969944 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:18.975982 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.967015 23038 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:18.982219 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 151822us
I20260709 12:22:18.990684 20850 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:18.997422 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:18.978786 23426 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:18.966627 23384 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:18.967826 23182 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7.
I20260709 12:22:18.931813 23314 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.039726 23199 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:19.040383 23199 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:19.040686 23314 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:19.041424 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50885.7us
I20260709 12:22:19.041220 23199 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.042063 23416 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:19.042613 23416 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.044745 23416 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:19.045322 23416 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:19.045439 23422 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:19.045779 23416 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.047395 23422 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:19.004663 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:18.972157 23291 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:18.972568 23358 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.035120 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:19.018704 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:19.003685 23268 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:19.067684 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:19.065847 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142655us
I20260709 12:22:19.066097 23182 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:19.080263 20879 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:19.066324 23426 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:19.080756 20879 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:19.071790 23316 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:19.006573 23383 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.010665 23304 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.083220 20879 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.079301 23314 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:19.090161 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47287.4us
I20260709 12:22:19.081825 23182 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:19.079483 23423 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:19.091758 23426 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.093118 23358 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:19.106216 23323 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.106465 23423 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:19.108297 23420 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:19.108731 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.109192 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.109880 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:19.111788 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:19.089540 23416 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:19.113791 21429 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:19.103963 23314 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.114735 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:19.113207 23383 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.103801 23182 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:19.135668 20805 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:19.138293 20878 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:19.138393 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:19.145468 20805 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:19.145748 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.124189 23323 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:19.150786 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:19.151468 20805 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:19.133113 23425 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:19.136603 23427 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.138018 23421 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:19.158931 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:19.161152 21426 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:19.162505 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:19.155356 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:19.162649 23423 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.208125 23423 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:19.208922 20810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 358) took 2551 ms (client timeout 1661 ms). Trace:
W20260709 12:22:19.209352 20810 rpcz_store.cc:269] 0709 12:22:16.657438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.698386 (+ 40948us) service_pool.cc:224] Handling call
0709 12:22:19.208696 (+2510310us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b540019a0d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:19.208820 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704261376}
I20260709 12:22:19.211647 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 1896) took 2093 ms. Trace:
I20260709 12:22:19.212108 20806 rpcz_store.cc:276] 0709 12:22:17.118297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.800984 (+682687us) service_pool.cc:224] Handling call
0709 12:22:19.211438 (+1410454us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002b6bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:19.211589 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381992192}
W20260709 12:22:19.211826 23438 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:22:19.222383 21429 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:19.223091 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:19.223744 21433 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:19.227018 20807 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:22:19.227717 20807 pending_rounds.cc:85] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 3
I20260709 12:22:19.228175 20807 pending_rounds.cc:107] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:22:19.228642 20807 raft_consensus.cc:2889] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:19.224978 21435 raft_consensus.cc:2955] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:22:19.238232 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88611.1us
I20260709 12:22:19.238487 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:19.257130 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.258080 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:19.158074 23425 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:19.264875 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:19.268723 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.269773 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.272914 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:19.160709 23295 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.277101 20878 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:19.280103 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:19.252108 23421 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:19.288925 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:19.297545 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.305097 20878 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:19.279022 23419 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:19.217780 23323 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:19.293869 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800064f20 after lost signal to thread 19606
I20260709 12:22:19.319746 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.296096 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:19.300899 23384 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:19.313197 23372 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:19.331470 23372 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:19.332086 23372 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.335824 23323 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.335294 23425 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.336212 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.337034 20875 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:22:19.338661 20882 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.340992 20878 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:22:19.341044 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.341509 20878 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 6
I20260709 12:22:19.341750 20878 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:22:19.341995 20878 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:19.162995 23427 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.325987 23295 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:19.326684 23314 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:19.327294 23433 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:22:19.327841 23421 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.316359 23419 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:19.355818 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:19.352020 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82357.5us
I20260709 12:22:19.356930 20807 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:19.371245 23291 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.354606 23293 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:19.407542 23293 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.406118 20884 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:19.363343 23295 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:22:19.354543 23419 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.361655 23319 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:19.371471 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:19.406661 23291 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.369089 23316 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.435207 23319 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.437024 23293 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:19.437465 23319 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:19.439816 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:19.441278 23446 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:22:19.446159 23450 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:22:19.456909 23427 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:19.470894 23316 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.473897 23316 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:19.475342 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.480243 20881 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:22:19.528203 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 169267us
I20260709 12:22:19.474726 23330 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:22:19.538304 23421 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.539093 23330 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.539592 23330 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.524963 23436 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:19.525341 23316 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.561918 23330 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:19.564049 23291 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:19.549641 23417 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:19.565069 23417 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:19.565335 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.570678 23372 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.573753 23372 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:19.574529 23419 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.577189 23419 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:19.579958 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50641.9us
I20260709 12:22:19.590052 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:19.593969 23372 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:19.594408 23372 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.594725 23372 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:19.601586 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 769) took 2552 ms. Trace:
I20260709 12:22:19.547129 23316 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:19.609360 19608 debug-util.cc:398] Leaking SignalData structure 0x7b080036d260 after lost signal to thread 20700
W20260709 12:22:19.609871 19608 kernel_stack_watchdog.cc:198] Thread 20700 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 646ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:22:19.605621 21427 rpcz_store.cc:276] 0709 12:22:17.049224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.884113 (+834889us) service_pool.cc:224] Handling call
0709 12:22:17.927401 (+ 43288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:17.953704 (+ 26303us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:17.993745 (+ 40041us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.197054 (+203309us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b1800379f00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:22:19.443372 (+1246318us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b1800379f00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:19.541672 (+ 98300us) log.cc:844] Serialized 14 byte log entry
0709 12:22:19.551438 (+  9766us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:19.592431 (+ 40993us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:19.601271 (+  8840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:19.601356 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:19.601518 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23049,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":4681,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":28348,"spinlock_wait_cycles":2577510144,"thread_start_us":211074,"threads_started":2,"wal-append.queue_time_us":299162}
W20260709 12:22:19.598325 23447 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.198s	user 0.000s	sys 0.002s
W20260709 12:22:19.624317 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.565887 23417 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.628444 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.602164 23384 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:19.635028 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08001c4d80 after lost signal to thread 20667
W20260709 12:22:19.635802 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800033e00 after lost signal to thread 20700
I20260709 12:22:19.632762 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:19.637671 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:19.637871 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.638525 21432 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:19.656526 20805 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:19.669726 21430 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:19.656220 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.670540 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 772) took 1162 ms. Trace:
I20260709 12:22:19.671339 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:19.657060 20878 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:22:19.672346 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.670962 21430 rpcz_store.cc:276] 0709 12:22:18.507921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:19.635249 (+1127328us) service_pool.cc:224] Handling call
0709 12:22:19.649020 (+ 13771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:19.670254 (+ 21234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:19.670477 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:19.636926 23299 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:19.678843 23310 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:19.656703 23457 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:22:19.627759 23323 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:19.638842 23453 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.101s	user 0.005s	sys 0.000s
I20260709 12:22:19.606503 23419 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:22:19.686102 23457 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:19.691903 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:19.692998 23429 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:19.695261 23426 consensus_queue.cc:1175] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 5 status { last_received { term: 5 index: 4 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:19.656481 23436 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:19.639894 22814 raft_consensus.cc:2764] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:19.635918 23330 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
W20260709 12:22:19.743228 23298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.086s	user 0.031s	sys 0.014s
W20260709 12:22:19.751843 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:19.761816 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.762475 23294 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.634424 23427 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
W20260709 12:22:19.754478 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62646.8us
I20260709 12:22:19.682047 23423 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:19.630597 23439 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
W20260709 12:22:19.742775 20664 debug-util.cc:398] Leaking SignalData structure 0x7b080030f900 after lost signal to thread 20759
W20260709 12:22:19.766006 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800034680 after lost signal to thread 20760
W20260709 12:22:19.767346 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800392460 after lost signal to thread 20834
W20260709 12:22:19.757185 23182 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.608s	user 0.013s	sys 0.000s
I20260709 12:22:19.765244 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:19.773617 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08002c9820 after lost signal to thread 20908
W20260709 12:22:19.776203 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08002ba500 after lost signal to thread 20909
W20260709 12:22:19.778043 20664 debug-util.cc:398] Leaking SignalData structure 0x7b080012c0c0 after lost signal to thread 21459
W20260709 12:22:19.778420 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800396e80 after lost signal to thread 21460
W20260709 12:22:19.778995 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08003b3e20 after lost signal to thread 23248
W20260709 12:22:19.779368 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800257360 after lost signal to thread 23266
W20260709 12:22:19.779654 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800314860 after lost signal to thread 23272
W20260709 12:22:19.779949 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08003bcc60 after lost signal to thread 23273
W20260709 12:22:19.780226 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800102b00 after lost signal to thread 23277
W20260709 12:22:19.780471 20664 debug-util.cc:398] Leaking SignalData structure 0x7b080037cc20 after lost signal to thread 23280
W20260709 12:22:19.780761 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08003f9e40 after lost signal to thread 23288
W20260709 12:22:19.761508 23298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.105s	user 0.032s	sys 0.014s
I20260709 12:22:19.739027 23323 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.770951 23291 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:19.783241 23182 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.798133 23182 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:19.799016 20802 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:19.799731 23182 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.800678 23182 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.802317 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:19.803756 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 1.84154e+06us
I20260709 12:22:19.804862 23285 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:19.805096 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.806950 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.807153 20875 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:19.807523 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:19.807765 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.762964 23294 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:19.815016 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:19.807829 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.818038 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:19.818895 20732 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:22:19.808876 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:19.821262 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.820732 20735 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:19.823585 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:19.824218 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:19.805311 23285 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.831648 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:19.845495 23449 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.440s	user 0.004s	sys 0.005s
I20260709 12:22:19.856475 23355 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.859472 23355 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.868592 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:19.869513 20808 raft_consensus.cc:2376] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6. Re-sending same reply.
I20260709 12:22:19.870185 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 377) took 1152 ms. Trace:
I20260709 12:22:19.787653 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:19.903385 20729 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:19.904641 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:19.871881 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:19.870581 20808 rpcz_store.cc:276] 0709 12:22:18.717434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.746156 (+ 28722us) service_pool.cc:224] Handling call
0709 12:22:19.868372 (+1122216us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:19.870112 (+  1740us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2777472}
I20260709 12:22:19.760712 23436 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:19.808657 23316 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:19.918227 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:19.925225 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:19.874135 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51641.1us
I20260709 12:22:19.903486 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:19.932509 20729 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:19.932889 20734 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:19.903069 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77431.3us
I20260709 12:22:19.923983 20728 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:19.934337 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:19.775650 23372 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:19.917136 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 85576.8us
I20260709 12:22:19.937196 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:19.940026 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:19.795439 23457 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:19.807384 23258 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:19.809048 23288 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:19.964267 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.840222 23341 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.807684 23125 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:20.000963 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.872092 23285 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.008342 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:20.009207 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.010032 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:20.029772 20881 raft_consensus.cc:2376] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5. Re-sending same reply.
I20260709 12:22:20.010881 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:19.872344 23356 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:20.025188 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86638.3us
I20260709 12:22:20.034307 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:20.037606 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.808075 23423 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:20.007336 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.929199 23199 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.924149 23248 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:19.925738 23370 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.987151 23316 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.985230 23383 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:19.943732 21427 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:19.984638 23298 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:19.984819 23293 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.050971 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:19.920403 23384 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:22:19.927489 23421 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:20.052608 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:19.988387 23258 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:19.927798 23419 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.916303 23273 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
W20260709 12:22:20.065058 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.074225 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.032362 23358 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.042884 23277 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:19.819980 23291 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.130738 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:20.054224 23280 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:20.047101 23423 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:20.134191 20882 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:20.062626 23383 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.139458 23383 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.056675 23199 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.143721 23199 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.144253 23199 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.146307 23291 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.146875 23291 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:20.105660 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53169.1us
I20260709 12:22:20.137068 23457 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.072836 23125 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:20.073251 23421 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.147876 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1615) took 1307 ms. Trace:
I20260709 12:22:20.047771 23356 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.165813 20877 rpcz_store.cc:276] 0709 12:22:18.840415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.840543 (+   128us) service_pool.cc:224] Handling call
0709 12:22:18.847360 (+  6817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:19.138614 (+291254us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:19.156052 (+ 17438us) log.cc:844] Serialized 14 byte log entry
0709 12:22:19.279566 (+123514us) log.cc:844] Serialized 14 byte log entry
0709 12:22:19.474404 (+194838us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b1800374e60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:19.848773 (+374369us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b1800374e60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:20.106477 (+257704us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b1800374e60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:22:20.106599 (+   122us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:20.107233 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:22:20.107426 (+   193us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:20.107474 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:20.107564 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:20.147657 (+ 40093us) raft_consensus.cc:1685] finished
0709 12:22:20.147685 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:20.147820 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":222887,"prepare.queue_time_us":19403,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":111,"raft.queue_time_us":389120,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":10591,"spinlock_wait_cycles":15700352,"thread_start_us":377627,"threads_started":3,"wal-append.queue_time_us":123577}
I20260709 12:22:20.070412 23272 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:20.040169 23341 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.133468 23304 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:20.180109 23125 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:20.133096 23319 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:20.182724 23430 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.074s	user 0.004s	sys 0.011s
W20260709 12:22:20.188694 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 423522us
I20260709 12:22:20.149031 23277 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:20.170210 23421 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:20.098531 23258 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:20.060920 23370 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:20.218943 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.531s (SENT) [suppressed 25 similar messages]
I20260709 12:22:20.098806 23419 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:20.149372 23423 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:20.200758 23466 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.201581 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:20.202836 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 165322us
I20260709 12:22:20.229753 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:20.203874 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 263922us
I20260709 12:22:20.231253 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:20.060748 23316 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:20.193084 23463 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:22:20.246770 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:20.159365 23432 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.053s	user 0.017s	sys 0.031s
I20260709 12:22:20.144712 23199 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.198640 23273 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.103942 23293 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.288986 23436 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:20.236083 23285 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.293162 23285 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.258347 23466 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.294018 23285 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:20.294489 23285 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.295648 21435 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:20.296186 23466 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.241892 23419 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.254189 20734 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:20.297824 23419 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:20.261389 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49566.9us
I20260709 12:22:20.294947 23285 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:20.262452 23356 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.263213 23372 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:20.264025 23277 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.219594 23319 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.266182 23182 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.147398 23291 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.267716 23323 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:20.268671 23125 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.276940 23316 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.278344 21427 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:20.281065 23272 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:20.249107 23478 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:20.287606 20732 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:20.287904 23294 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.288125 23273 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.257861 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1614) took 1418 ms. Trace:
I20260709 12:22:20.249614 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.290824 23293 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.262008 23287 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:20.317260 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.318058 20734 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:20.319429 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:20.444463 20805 raft_consensus.cc:2376] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6. Re-sending same reply.
I20260709 12:22:20.445881 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.321141 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.447156 20808 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:20.323325 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.326001 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.330081 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.451880 20805 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:20.451107 20809 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
W20260709 12:22:20.453689 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:20.260790 23432 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.017s	sys 0.032s
W20260709 12:22:20.391521 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 457255us
I20260709 12:22:20.392562 20807 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:20.459048 23489 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.393814 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.459636 23489 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.196204 23358 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.404449 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.405282 20876 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:20.407801 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 185479us
I20260709 12:22:20.408093 20806 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:22:20.409689 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.324782 23466 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:20.465410 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:20.466017 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.466434 23356 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:20.466709 20731 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:20.466888 23489 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.449709 23272 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:20.410225 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 180591us
I20260709 12:22:20.423533 23316 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.338505 23298 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:20.424952 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:20.424815 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.431267 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.440594 20883 rpcz_store.cc:276] 0709 12:22:18.839943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.840083 (+   140us) service_pool.cc:224] Handling call
0709 12:22:18.844995 (+  4912us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:18.845346 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:18.845391 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:18.867873 (+ 22482us) log.cc:844] Serialized 26 byte log entry
0709 12:22:19.075335 (+207462us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:19.075374 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:19.075438 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:20.182088 (+1106650us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:20.243617 (+ 61529us) raft_consensus.cc:1685] finished
0709 12:22:20.243647 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:20.257749 (+ 14102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44334,"raft.queue_time_us":129410,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":8615,"spinlock_wait_cycles":34030848,"thread_start_us":135007,"threads_started":1,"wal-append.queue_time_us":169031}
I20260709 12:22:20.449846 23469 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:20.456367 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.471745 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:20.412232 20810 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:22:20.488930 20882 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:20.473687 20731 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:20.501088 20883 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:20.471079 23272 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.454857 23457 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:20.503932 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 102088us
W20260709 12:22:20.510622 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 168908us
I20260709 12:22:20.534866 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:20.536451 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.536494 20884 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:20.536168 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:20.476601 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 257690us
W20260709 12:22:20.542989 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71317us
I20260709 12:22:20.414489 23468 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:22:20.545251 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.474602 23466 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:20.545274 20880 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:20.548779 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:20.550122 20810 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:20.552925 20806 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:20.558288 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.544075 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:20.560354 20805 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:20.563629 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 63975.2us
I20260709 12:22:20.564456 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:20.565766 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:20.506312 23469 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.568709 23319 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.569456 23478 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:20.569269 23466 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.569680 23477 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:20.569903 23478 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.568277 23272 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:22:20.578436 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.580232 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.570286 23356 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:22:20.580968 20807 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:20.581607 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:20.582228 20876 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:20.570497 23469 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.584259 23478 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:20.590237 23491 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:20.591500 23491 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:20.591854 23341 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.592754 23291 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.593305 23291 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:20.593427 20879 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:20.593652 23291 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:20.595413 23483 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:20.596705 23248 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 12:22:20.597126 23353 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:20.598011 23353 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:20.600113 23258 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.551352 23457 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.601321 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.602147 20734 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:20.602391 23258 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.603258 23457 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.584602 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.485838 23316 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.586153 20731 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:22:20.616268 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.594524 23291 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.503345 23489 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 12:22:20.613592 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 59877.6us
I20260709 12:22:20.613314 23439 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:20.624123 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:20.610538 20806 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:20.620828 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:20.631306 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.626266 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:20.597143 23248 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:20.655084 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 119258us
W20260709 12:22:20.655118 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89546.8us
I20260709 12:22:20.658766 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.659229 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:20.659621 20729 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.662911 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:20.663856 21435 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:20.665627 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:20.702623 23273 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.702962 21427 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:20.705389 23416 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:20.705864 23416 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:20.706224 23416 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.706388 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.707360 20726 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:20.708244 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.709347 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.710427 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.665275 23469 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:20.711802 23487 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:22:20.714427 23298 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.715499 23427 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:20.715909 23427 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:20.716192 23316 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:20.717685 23370 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:20.711833 23469 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:20.718457 23280 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.000s
I20260709 12:22:20.718613 23469 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:20.719545 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60405us
I20260709 12:22:20.720785 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:20.735023 21435 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:22:20.736488 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65503us
I20260709 12:22:20.741231 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.718884 23280 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:20.620014 23182 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:20.748826 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.748950 20700 leader_election.cc:400] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5.
I20260709 12:22:20.749478 20804 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:20.710963 20726 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.749456 20700 leader_election.cc:403] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:22:20.614032 23467 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.126s
I20260709 12:22:20.657590 23451 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:22:20.754925 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:20.756711 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:20.757901 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.760764 20731 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:20.743788 23304 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.766430 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.767511 20806 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 3: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:20.739233 23370 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.654382 23330 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 12:22:20.745270 23248 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:20.777550 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.778630 20810 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:20.765035 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.771832 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.773676 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.755967 23280 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:20.764798 23477 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:20.785199 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.786969 23499 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:20.787541 23498 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:22:20.789145 23417 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:20.792248 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.792536 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.752223 23273 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:22:20.775173 23304 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.902012 23304 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:20.774958 23421 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.001s
W20260709 12:22:20.783367 23383 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.000s
I20260709 12:22:20.935415 23383 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.758200 23291 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:20.888656 23419 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.003s
I20260709 12:22:20.805532 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.806212 23288 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.944620 23288 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.945477 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.814633 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.948151 20809 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:20.949139 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.950807 20809 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:20.818456 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:20.892259 23423 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.004s
W20260709 12:22:20.951838 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 133456us
I20260709 12:22:20.952164 23423 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.824206 23432 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:20.828819 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 102669us
I20260709 12:22:20.829347 20884 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:20.829450 23488 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:20.830436 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.826910 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.972185 20727 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:20.831008 23297 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:20.831143 23460 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.054s	user 0.004s	sys 0.000s
I20260709 12:22:20.831806 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:20.834852 23452 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:22:20.835297 23485 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:22:20.835517 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.996059 20807 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:20.838634 23427 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.840497 23330 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:20.759601 23358 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.754127 23182 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.878683 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:21.008415 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:21.008652 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 145673us
W20260709 12:22:20.879521 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 96330.5us
I20260709 12:22:21.004302 23330 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:21.011924 20807 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:21.012562 23330 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.013407 20807 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:22:21.013783 20807 pending_rounds.cc:85] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:22:21.013882 23330 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.014086 20807 pending_rounds.cc:107] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:22:21.014401 20807 raft_consensus.cc:2889] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:21.015553 23427 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:21.024142 23358 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.024868 23481 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.998859 23419 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:20.995512 23452 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:21.033955 23452 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.899178 23273 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:20.794940 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.893117 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:20.912323 23199 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:20.916728 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 452) took 1783 ms (client timeout 1827 ms). Trace:
W20260709 12:22:21.043367 20876 rpcz_store.cc:269] 0709 12:22:19.133559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:19.151724 (+ 18165us) service_pool.cc:224] Handling call
0709 12:22:20.916464 (+1764740us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:20.916653 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2862034688}
I20260709 12:22:20.918249 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:20.928158 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:21.045426 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 127292us
I20260709 12:22:21.046743 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:20.903038 23421 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:20.784240 23370 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:20.816422 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:20.946978 23288 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:20.952670 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:20.819211 23482 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:20.915092 23341 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:20.953372 23304 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:20.905112 23497 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:22:20.802188 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:20.969329 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.058904 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.059933 20730 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:20.977088 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:20.992362 20809 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:22:20.993829 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 162105us
I20260709 12:22:21.071352 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.024786 23358 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:21.036082 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.894642 20802 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:21.072552 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:20.895807 20732 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:20.970914 23432 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:21.042690 23273 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:21.055214 23452 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:21.048863 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.123313 20876 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:20.975690 23297 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:21.051005 20805 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:21.127655 20729 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:21.054333 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.050983 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.003139 23467 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:20.973107 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.064002 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:20.973027 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.067798 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:21.069347 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.071424 20809 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:21.078145 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101132us
I20260709 12:22:21.147298 20728 raft_consensus.cc:2368] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:21.078238 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:21.075079 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:20.766430 23468 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.055119 23266 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 12:22:21.107924 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.114897 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:21.163285 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85139.7us
W20260709 12:22:21.120092 20879 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.004s
W20260709 12:22:21.163314 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.153723 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:21.134073 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.062211 23304 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.056144 23341 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:21.048607 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:21.159806 23432 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:21.061656 23298 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:20.976074 23383 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:21.165350 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.175341 20727 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
W20260709 12:22:21.166249 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.166787 20810 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.143411 23482 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:21.150022 23467 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.194073 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.194640 20727 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.190909 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:21.168540 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 121867us
I20260709 12:22:21.169353 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.061028 23469 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:21.121429 23330 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.165212 23468 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:21.173491 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.170890 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:21.188428 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.189273 20808 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:22:21.198926 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.190876 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:21.193264 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:21.163362 23319 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:21.172657 20879 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:21.129487 23419 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:22:21.166898 23304 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:21.214824 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:21.229645 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.166306 20729 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:21.230523 20730 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:21.174810 23471 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.215327 23287 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.249930 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:21.447598 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 213705us
W20260709 12:22:21.250259 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 297654us
I20260709 12:22:21.260490 20810 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:21.424917 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:21.208700 23428 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.234s
I20260709 12:22:21.432030 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1636) took 2488 ms. Trace:
W20260709 12:22:21.197539 23294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.009s	sys 0.025s
I20260709 12:22:21.287096 20727 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:21.472126 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.472993 20727 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.468263 20879 rpcz_store.cc:276] 0709 12:22:18.943782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:19.065555 (+121773us) service_pool.cc:224] Handling call
0709 12:22:19.073741 (+  8186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:19.691436 (+617695us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b180014c0a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ecda646 00007f9e1ec1458e 00007f9e1ec95f9e 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:22:20.593068 (+901632us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b58005cf9d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:22:21.431707 (+838639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:21.431940 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2532384256}
I20260709 12:22:21.263646 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:21.302552 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.434453 21433 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:21.435321 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1919) took 1108 ms. Trace:
I20260709 12:22:21.164613 23416 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.244515 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:21.482182 20808 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:21.163753 23451 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:22:21.198026 23432 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.451082 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:21.448623 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:21.452235 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:21.449579 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:21.193674 23472 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:21.460119 20807 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:22:21.439678 23463 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:22:21.236177 23496 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.462553 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.463838 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:21.442312 23317 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:22:21.508090 20877 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:21.171156 23429 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:21.424458 23412 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:21.478639 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:21.200639 23478 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.002s
I20260709 12:22:21.441658 23489 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.198848 23467 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:21.266877 23491 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.038s	sys 0.013s
I20260709 12:22:21.489184 20727 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:21.220095 23330 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:21.445998 23287 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:21.556085 23287 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:21.471151 23294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.010s	sys 0.029s
I20260709 12:22:21.519968 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.235397 23297 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:21.519690 23285 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.122s	user 0.013s	sys 0.001s
I20260709 12:22:21.237062 23468 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.563434 23297 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.531669 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.564450 20730 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:21.234568 23477 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.287s
I20260709 12:22:21.499195 23503 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:21.561766 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.570677 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.458550 23471 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.489638 23423 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:21.218266 23273 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.522328 21426 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:21.440665 23341 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:21.559527 23491 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.038s	sys 0.015s
I20260709 12:22:21.597074 20802 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:21.613816 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.612412 20727 raft_consensus.cc:2368] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
W20260709 12:22:21.615150 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166619us
I20260709 12:22:21.522660 23429 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:21.548197 23480 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:21.621807 20734 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:21.622648 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.544533 23370 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:21.561419 23330 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.638255 20730 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:21.587563 23412 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.590088 23421 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:21.647512 20807 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:21.628026 23429 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:21.620230 23508 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:21.616271 23524 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:21.655151 20802 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:21.570271 23489 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:22:21.647218 23291 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.001s
I20260709 12:22:21.646929 23341 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.616276 23482 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:21.598443 23285 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:21.641469 23358 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:21.681092 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.690619 20734 raft_consensus.cc:2376] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8. Re-sending same reply.
I20260709 12:22:21.681691 20807 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:21.569979 23478 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:21.658375 23353 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:22:21.686648 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 234454us
W20260709 12:22:21.661067 23509 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:22:21.669975 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.523976 23496 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:21.677786 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 228291us
I20260709 12:22:21.663579 23421 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:21.620007 23503 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:21.698205 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:21.661599 23370 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:21.685884 23319 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:21.707967 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.724969 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 356) took 1019 ms. Trace:
I20260709 12:22:21.725390 20734 rpcz_store.cc:276] 0709 12:22:20.705158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.691479 (+986321us) service_pool.cc:224] Handling call
0709 12:22:21.724893 (+ 33414us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:21.710417 23507 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:22:21.687575 23508 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.710875 23531 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:21.711650 23412 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:21.712177 23416 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:21.759512 23482 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:21.760403 20726 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.002s	user 0.011s	sys 0.000s
I20260709 12:22:21.760918 20726 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:21.761482 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 345) took 1199 ms. Trace:
W20260709 12:22:21.712643 23510 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:22:21.761857 20726 rpcz_store.cc:276] 0709 12:22:20.562088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.710199 (+148111us) service_pool.cc:224] Handling call
0709 12:22:21.761425 (+1051226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:21.476939 20803 rpcz_store.cc:276] 0709 12:22:20.327065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.327223 (+   158us) service_pool.cc:224] Handling call
0709 12:22:20.327605 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:20.327885 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:20.743569 (+415684us) log.cc:844] Serialized 14 byte log entry
0709 12:22:21.434953 (+691384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:21.435011 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:21.435065 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:21.435130 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:21.435255 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59146,"prepare.queue_time_us":448112,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":139006,"raft.run_cpu_time_us":8987,"raft.run_wall_time_us":188592,"thread_start_us":975142,"threads_started":3,"wal-append.queue_time_us":451989}
I20260709 12:22:21.758858 23508 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:21.764017 23522 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:22:21.766182 23514 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:22:21.768803 23478 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:21.769629 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.725863 23294 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:21.687065 23469 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:21.773257 23248 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:21.773841 23248 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:21.717556 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:21.775655 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58195.3us
I20260709 12:22:21.681571 23525 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:21.766734 23287 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:21.641129 23298 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:21.721645 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.776604 23488 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:21.701390 23280 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.827s	user 0.012s	sys 0.001s
I20260709 12:22:21.777439 23280 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5.
I20260709 12:22:21.777218 23298 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.777912 20775 leader_election.cc:400] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:21.778252 20731 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:21.778625 23503 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:21.709437 20735 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:21.780200 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1709) took 1063 ms. Trace:
I20260709 12:22:21.780408 23489 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:21.780614 20731 rpcz_store.cc:276] 0709 12:22:20.716504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.784853 (+ 68349us) service_pool.cc:224] Handling call
0709 12:22:21.780028 (+995175us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54003ad4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:21.780144 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068927360}
I20260709 12:22:21.781471 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:21.781805 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.775750 23358 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:21.782445 23358 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.783347 23358 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:21.785483 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:21.786046 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.786688 23285 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:21.777034 23488 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:21.788683 23471 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:21.695421 23353 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:21.790427 23353 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:21.791882 23481 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.766s	user 0.006s	sys 0.006s
I20260709 12:22:21.792397 23481 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:21.710911 23457 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:21.793205 23512 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:22:21.793078 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.719797 23421 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.783162 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.781827 23248 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:21.778816 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:21.795511 20731 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:21.811383 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.812225 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 351) took 1108 ms. Trace:
I20260709 12:22:21.812625 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:21.812604 20731 rpcz_store.cc:276] 0709 12:22:20.703408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.811104 (+1107696us) service_pool.cc:224] Handling call
0709 12:22:21.812164 (+  1060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:21.813773 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.795651 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1916) took 1475 ms. Trace:
W20260709 12:22:21.746527 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 120015us
I20260709 12:22:21.814574 20801 rpcz_store.cc:276] 0709 12:22:20.320734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.324555 (+  3821us) service_pool.cc:224] Handling call
0709 12:22:20.454886 (+130331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:20.975344 (+520458us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:20.975390 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:21.010583 (+ 35193us) log.cc:844] Serialized 26 byte log entry
0709 12:22:21.239976 (+229393us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:21.240011 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:21.240281 (+   270us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:21.795396 (+555115us) raft_consensus.cc:1685] finished
0709 12:22:21.795420 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:21.795587 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90730,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":661920,"threads_started":2,"wal-append.queue_time_us":556148}
I20260709 12:22:21.801014 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:21.816558 20734 raft_consensus.cc:2376] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7. Re-sending same reply.
I20260709 12:22:21.816829 20732 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:21.810011 23457 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.818147 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.818816 20884 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:21.819943 23457 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:21.732477 23370 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.820585 23421 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:21.751519 23531 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.823419 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1770) took 1218 ms. Trace:
W20260709 12:22:21.823707 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44978.3us
W20260709 12:22:21.824015 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46845.2us
I20260709 12:22:21.814510 20726 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:21.824851 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:21.823835 20880 rpcz_store.cc:276] 0709 12:22:20.605468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.606008 (+   540us) service_pool.cc:224] Handling call
0709 12:22:20.624277 (+ 18269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:20.624627 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:20.624653 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:20.808210 (+183557us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b180035c9e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a
0709 12:22:20.809362 (+  1152us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:20.810107 (+   745us) log.cc:844] Serialized 26 byte log entry
0709 12:22:20.821020 (+ 10913us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b180035c9e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e22b7358f 00007f9e1bbb2571 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd
0709 12:22:21.065665 (+244645us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:21.065752 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:21.065851 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:21.823208 (+757357us) raft_consensus.cc:1685] finished
0709 12:22:21.823226 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:21.823357 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41220,"raft.queue_time_us":262,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":32757,"spinlock_wait_cycles":45980416,"thread_start_us":254972,"threads_started":1,"wal-append.queue_time_us":614543}
I20260709 12:22:21.800323 23478 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:21.828964 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:21.829108 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:21.839335 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.830135 20884 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:22.008636 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.993224 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.049260 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:21.894521 20880 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:22.009945 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:21.866133 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:21.830771 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:21.986989 21432 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:21.832402 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.082463 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.081923 20730 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:21.905544 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:21.995056 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:22.083657 20807 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:21.834518 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:22.017988 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.084646 20734 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:21.994439 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:22.017271 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.085413 20801 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:21.960001 21429 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:22.085343 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1719) took 1316 ms. Trace:
W20260709 12:22:21.822551 23452 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.002s
W20260709 12:22:22.100016 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82156.4us
I20260709 12:22:22.108522 20727 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:21.828751 23503 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:21.892642 23417 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.939s	user 0.011s	sys 0.000s
W20260709 12:22:22.088321 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 222310us
I20260709 12:22:22.092113 23489 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:22.111919 23294 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:22.094219 23477 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.112421 23520 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:22.060496 23330 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:22.079804 23499 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.845911 23478 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.890579 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:22.115319 20884 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:22.092088 23297 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.890391 23319 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.958881 23517 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:21.958958 23298 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:22.074088 23481 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.057458 23488 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:21.993849 23427 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.978s	user 0.007s	sys 0.004s
I20260709 12:22:22.118008 23182 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.121290 20878 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:22.061012 23358 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.086864 20734 rpcz_store.cc:276] 0709 12:22:20.769222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.817188 (+1047966us) service_pool.cc:224] Handling call
0709 12:22:22.085286 (+268098us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.108990 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:22.125989 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 43617.7us
W20260709 12:22:22.170205 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61308.8us
I20260709 12:22:21.823115 23531 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.127977 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:21.893603 23370 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.186671 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:22.065330 23304 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.001s
I20260709 12:22:21.899379 23527 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:21.892091 23293 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.861380 23491 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:21.891508 23526 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.218981 23526 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.220175 23478 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.194607 23511 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:22.221237 23511 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.221342 23526 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.221673 23511 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.136021 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:22.223855 23330 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:22.224426 23527 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:22.225345 23527 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.901476 23287 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:22.145241 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.141016 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:22.227644 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:22.228169 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1723) took 1280 ms. Trace:
I20260709 12:22:22.228474 20730 rpcz_store.cc:276] 0709 12:22:20.948122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.125647 (+1177525us) service_pool.cc:224] Handling call
0709 12:22:22.228108 (+102461us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:22.229182 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.229473 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:22.229739 23539 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.404s	user 0.000s	sys 0.003s
W20260709 12:22:22.230298 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.230994 23545 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:22.231590 23545 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:21.892509 23534 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:22.234774 23534 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.235977 23358 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.183790 23291 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:22:22.236938 23534 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.238696 23439 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:22.125214 23503 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:21.828434 23525 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.309s
I20260709 12:22:22.129314 23341 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:22.128615 23489 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.148821 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.131356 23294 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.146392 23538 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.132946 22814 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.060072 23467 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.062878 23468 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.189076 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:22.191800 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:22.192835 20801 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:22.193337 20878 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:22.188457 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.140512 23477 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:22.171551 23481 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.202168 23488 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.163650 20726 raft_consensus.cc:2376] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4. Re-sending same reply.
I20260709 12:22:22.275939 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:22.275035 23481 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.276594 20726 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:22.277356 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46798 (request call id 20) took 1507 ms. Trace:
I20260709 12:22:22.277724 20726 rpcz_store.cc:276] 0709 12:22:20.769624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.275802 (+1506178us) service_pool.cc:224] Handling call
0709 12:22:22.277269 (+  1467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.180892 23524 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:22.204124 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1994) took 1803 ms. Trace:
I20260709 12:22:22.279590 20735 rpcz_store.cc:276] 0709 12:22:20.401072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.457258 (+ 56186us) service_pool.cc:224] Handling call
0709 12:22:20.457626 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.779419 (+1321793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.203825 (+424406us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54001d8154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:22.204051 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2878327168}
W20260709 12:22:22.281301 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.172286 23298 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.261530 23489 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:22.207451 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:22.287324 20883 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:22.208590 20732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1704) took 1493 ms (client timeout 1925 ms). Trace:
I20260709 12:22:22.210893 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:22.167309 23319 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.211278 23293 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.296921 20734 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:22.286187 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.298115 20878 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:22.195005 23304 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.213346 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 465) took 1047 ms. Trace:
W20260709 12:22:22.213647 23429 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.001s
I20260709 12:22:22.149827 23499 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:22.164522 23297 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.215152 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.164784 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.234241 23545 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.230309 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1720) took 1353 ms. Trace:
I20260709 12:22:22.308673 20730 rpcz_store.cc:276] 0709 12:22:20.877485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.229164 (+1351679us) service_pool.cc:224] Handling call
0709 12:22:22.230253 (+  1089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.247788 21435 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:22.281159 23467 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.247151 23491 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:22.254951 23544 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:22:22.255892 23543 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:22:22.255795 23273 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:22.256837 23542 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:22:22.239120 23439 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:22.267071 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 78078.1us
I20260709 12:22:22.268443 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:22.268803 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 436447us
I20260709 12:22:22.344030 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:22.344367 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.248478 23503 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.205914 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:22.348850 20803 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:22.262837 23294 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:22.288308 20732 rpcz_store.cc:269] 0709 12:22:20.715182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.074748 (+359566us) service_pool.cc:224] Handling call
0709 12:22:22.208360 (+1133612us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54003b4554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:22.208530 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536228608}
W20260709 12:22:22.295440 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84641.2us
I20260709 12:22:22.352252 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:22.271255 23341 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:22.309561 21433 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:22:22.270334 22814 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:22.249430 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.355896 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:22.311228 23515 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.246s	user 0.001s	sys 0.001s
I20260709 12:22:22.313972 23499 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.315541 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:22.317577 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.317904 23297 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.315495 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.310689 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.308524 23429 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.284508 23531 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.319077 20730 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:22.375722 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 344) took 1813 ms. Trace:
I20260709 12:22:22.376201 20730 rpcz_store.cc:276] 0709 12:22:20.561824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.309729 (+1747905us) service_pool.cc:224] Handling call
0709 12:22:22.318844 (+  9115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.375366 (+ 56522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.375650 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.323524 20883 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:22.323804 23353 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.325519 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.331472 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:22.382373 20880 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:22.333338 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:22.281627 21428 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:22.302362 21432 rpcz_store.cc:276] 0709 12:22:21.165608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.165731 (+   123us) service_pool.cc:224] Handling call
0709 12:22:21.166052 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.082935 (+916883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.213113 (+130178us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54005f3a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:22.213285 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887184256}
I20260709 12:22:22.349748 23481 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:22:22.339404 21430 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:22.341537 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:22.345050 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.352499 20732 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:22.357437 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:22.355921 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:22.392927 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 347) took 1830 ms. Trace:
I20260709 12:22:22.362237 23280 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.362950 20884 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:22.400754 21429 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:22.401152 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:22.345687 23258 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:22.371309 20803 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:22.409201 20880 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:22.350994 23199 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.358801 23341 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:22.382843 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51440.5us
I20260709 12:22:22.385738 20883 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:22.304893 23488 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:22.362574 23547 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:22.418506 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:22.389634 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.352483 23478 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 12:22:22.393401 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60133.8us
I20260709 12:22:22.393283 20732 rpcz_store.cc:276] 0709 12:22:20.562557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.352022 (+1789465us) service_pool.cc:224] Handling call
0709 12:22:22.352337 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.392643 (+ 40306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.392861 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.424305 20732 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:22:22.424574 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.352144 23266 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:22:22.424911 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 354) took 1720 ms. Trace:
I20260709 12:22:22.425319 20732 rpcz_store.cc:276] 0709 12:22:20.704487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.423675 (+1719188us) service_pool.cc:224] Handling call
0709 12:22:22.423971 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.424677 (+   706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.424855 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.426095 23266 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:22.426678 20732 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:22:22.409225 23258 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:22.396605 20882 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:22.398082 20730 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:22.427624 23266 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:22.354832 23356 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:22:22.428416 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 350) took 1802 ms. Trace:
I20260709 12:22:22.428264 23199 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:22.369634 23466 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.778s	user 0.030s	sys 0.050s
I20260709 12:22:22.403164 23285 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:22.429201 23466 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.838s	user 0.030s	sys 0.050s
W20260709 12:22:22.402182 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60707us
I20260709 12:22:22.403349 23538 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:22.430794 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:22.431553 23285 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:22.431897 23491 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.432077 23285 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:22.432981 23285 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:22.428850 20730 rpcz_store.cc:276] 0709 12:22:20.626294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.377282 (+1750988us) service_pool.cc:224] Handling call
0709 12:22:22.397647 (+ 20365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.428169 (+ 30522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.428363 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.433733 23496 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.430819 23199 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.436097 21426 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:22.436338 23511 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:22.438535 23511 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.440161 23298 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:22.417054 23297 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:22.447500 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1655) took 1004 ms. Trace:
I20260709 12:22:22.414868 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:22.393230 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:22.427335 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1707) took 1711 ms. Trace:
I20260709 12:22:22.450404 21435 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:22:22.450490 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57329.5us
I20260709 12:22:22.372474 23439 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.451493 20880 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.387696 23524 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:22.412464 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.450125 20732 rpcz_store.cc:276] 0709 12:22:20.715960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.426102 (+1710142us) service_pool.cc:224] Handling call
0709 12:22:22.426424 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:22.427081 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.427284 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.456317 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1712) took 1739 ms. Trace:
I20260709 12:22:22.447908 20876 rpcz_store.cc:276] 0709 12:22:21.443155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.048636 (+605481us) service_pool.cc:224] Handling call
0709 12:22:22.110011 (+ 61375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:22.126783 (+ 16772us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:22.148924 (+ 22141us) log.cc:844] Serialized 14 byte log entry
0709 12:22:22.355841 (+206917us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:22.370384 (+ 14543us) log.cc:844] Serialized 52 byte log entry
0709 12:22:22.370604 (+   220us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:22.370657 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.370738 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:22.447253 (+ 76515us) raft_consensus.cc:1685] finished
0709 12:22:22.447277 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:22.447439 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24821,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":3037,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"spinlock_wait_cycles":8758528,"thread_start_us":165749,"threads_started":1,"wal-append.queue_time_us":191715}
I20260709 12:22:22.456739 20732 rpcz_store.cc:276] 0709 12:22:20.717275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.455327 (+1738052us) service_pool.cc:224] Handling call
0709 12:22:22.455677 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.455892 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.455930 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:22.455987 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:22.456051 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.456122 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:22.456252 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.458714 20732 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:22.460495 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1713) took 1743 ms. Trace:
I20260709 12:22:22.461113 20732 rpcz_store.cc:276] 0709 12:22:20.717518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.457969 (+1740451us) service_pool.cc:224] Handling call
0709 12:22:22.458317 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.460159 (+  1842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.460429 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.410257 23248 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:22.462894 20732 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:22.458557 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:22.466746 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:22.415462 23341 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:22.466854 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.469161 20878 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:22.453567 23478 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.474295 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.463564 23298 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:22.467648 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:22.466921 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1714) took 1749 ms. Trace:
I20260709 12:22:22.484495 20732 rpcz_store.cc:276] 0709 12:22:20.717736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.462149 (+1744413us) service_pool.cc:224] Handling call
0709 12:22:22.462680 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.466489 (+  3809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.466868 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.460979 23524 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:22.464428 23480 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:22.470304 23538 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:22.469216 23297 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:22.496868 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:22.617462 20878 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.512185 20732 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:22.619135 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.620290 20876 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 12:22:22.625558 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 269085us
I20260709 12:22:22.626495 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1718) took 1868 ms. Trace:
I20260709 12:22:22.626858 20732 rpcz_store.cc:276] 0709 12:22:20.757714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.485488 (+1727774us) service_pool.cc:224] Handling call
0709 12:22:22.485842 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.618544 (+132702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.626424 (+  7880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.481313 23439 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:22.727149 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 43626.2us
I20260709 12:22:22.481900 23477 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.496162 23524 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:22.515911 23526 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:22.658759 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:22.662025 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:22.701052 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.480455 23417 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.493598 23298 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.645601 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1725) took 1696 ms. Trace:
I20260709 12:22:22.774010 20734 rpcz_store.cc:276] 0709 12:22:20.948848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.169602 (+1220754us) service_pool.cc:224] Handling call
0709 12:22:22.645263 (+475661us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b5400298bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:22.645503 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73546496}
I20260709 12:22:22.647149 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1652) took 1586 ms. Trace:
I20260709 12:22:22.762391 20732 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:22.775631 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1727) took 1826 ms. Trace:
I20260709 12:22:22.776108 20732 rpcz_store.cc:276] 0709 12:22:20.949556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.732853 (+1783297us) service_pool.cc:224] Handling call
0709 12:22:22.733371 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.775355 (+ 41984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.775576 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.513123 23468 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.770081 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:22.616401 23319 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.798197 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.798986 20802 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
W20260709 12:22:22.764636 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.778410 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1998) took 1545 ms. Trace:
I20260709 12:22:22.800572 20732 rpcz_store.cc:276] 0709 12:22:21.232597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.777161 (+1544564us) service_pool.cc:224] Handling call
0709 12:22:22.777780 (+   619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.778010 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.778048 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:22.778104 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:22.778148 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.778218 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:22.778344 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.802731 20732 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:22.775331 20875 rpcz_store.cc:276] 0709 12:22:21.060771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.062724 (+  1953us) service_pool.cc:224] Handling call
0709 12:22:21.079944 (+ 17220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.214226 (+134282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:21.214257 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:21.214299 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:21.214337 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.646793 (+1432456us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400305a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:22.646863 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:22.647071 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308301568}
I20260709 12:22:22.622797 23538 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:22.513899 23537 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:22.787262 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.791235 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.793569 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.806578 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2000) took 1549 ms. Trace:
I20260709 12:22:22.807233 20732 rpcz_store.cc:276] 0709 12:22:21.257174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.801907 (+1544733us) service_pool.cc:224] Handling call
0709 12:22:22.802302 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.806194 (+  3892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.806502 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.726794 23511 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.628328 23293 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.795425 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:22.808768 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 407708us
I20260709 12:22:22.831210 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:22.793569 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.810942 20732 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:22.830804 20809 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:22.832515 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2001) took 1575 ms. Trace:
I20260709 12:22:22.832864 20732 rpcz_store.cc:276] 0709 12:22:21.257484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.810336 (+1552852us) service_pool.cc:224] Handling call
0709 12:22:22.810708 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.832254 (+ 21546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.832457 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:22.819609 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 395046us
I20260709 12:22:22.820636 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1999) took 1572 ms. Trace:
I20260709 12:22:22.622377 23297 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.629132 23304 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:22.824106 20878 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:22.828073 20804 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:22.834827 20727 rpcz_store.cc:276] 0709 12:22:21.247984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.781132 (+1533148us) service_pool.cc:224] Handling call
0709 12:22:22.820157 (+ 39025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.820341 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.820364 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:22.820404 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:22.820442 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.820494 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:22.820587 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.840370 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.840629 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.789948 23298 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:22.840891 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.860497 20802 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:22.622117 23480 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.842137 20734 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:22.861920 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1997) took 1653 ms. Trace:
I20260709 12:22:22.862368 20734 rpcz_store.cc:276] 0709 12:22:21.208534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.774971 (+1566437us) service_pool.cc:224] Handling call
0709 12:22:22.828984 (+ 54013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.861619 (+ 32635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.861856 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.842491 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:22.863891 20803 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:22.840370 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:22.864779 20806 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:22.848181 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.876124 20806 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:22.921864 21430 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:22.798089 23496 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:22.810218 23526 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:22.815253 23538 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:22.924147 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 220838us
I20260709 12:22:22.840131 23297 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.936399 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:22.797461 23417 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:22.853511 23285 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:22.943279 21430 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:22.948760 23199 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:22.949358 23551 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
W20260709 12:22:22.950654 23550 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:22:22.951349 23417 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:22.943979 20727 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:22.954268 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 160758us
I20260709 12:22:22.956568 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:22.956789 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.842684 23511 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:22.933490 23429 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:22.920204 23468 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:22.919694 23491 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:22:22.859436 23481 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:22:22.933156 23496 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:22.978956 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:22.958637 23531 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:22.967257 23248 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:23.026543 23422 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:23.027251 23422 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:23.027783 23422 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.035625 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:23.037926 23540 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.147s	user 0.001s	sys 0.001s
W20260709 12:22:23.040781 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 192655us
I20260709 12:22:23.042814 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:22.989933 23429 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:23.015882 23503 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.656s	user 0.007s	sys 0.003s
I20260709 12:22:23.017642 23481 consensus_queue.cc:1243] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:23.018183 23248 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.983817 23285 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:22:22.960448 23556 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:22:23.038379 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:23.051074 20880 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 12:22:22.998636 23555 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:22:23.088455 20880 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:23.056478 23248 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.055438 23199 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:22.931474 23537 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.049463 23466 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
W20260709 12:22:23.111092 23557 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:22:23.114004 23439 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:23.114300 23273 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.772s	user 0.005s	sys 0.004s
I20260709 12:22:23.054780 23563 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:22:23.051605 23429 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.090384 23491 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:23.115026 23353 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.730s	user 0.008s	sys 0.000s
I20260709 12:22:23.054165 23503 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.021718 23526 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.095690 23499 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:23.125622 20729 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:23.138293 20731 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
W20260709 12:22:23.138892 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 169056us
W20260709 12:22:23.131444 23294 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.769s	user 0.007s	sys 0.003s
I20260709 12:22:23.148617 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.149617 20880 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:23.140539 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 339) took 2577 ms. Trace:
I20260709 12:22:23.160076 20726 rpcz_store.cc:276] 0709 12:22:20.560106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.278517 (+1718411us) service_pool.cc:224] Handling call
0709 12:22:22.278857 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.279080 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:22.315610 (+ 36530us) log.cc:844] Serialized 14 byte log entry
0709 12:22:22.770730 (+455120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:23.073723 (+302993us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:23.089233 (+ 15510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.092848 (+  3615us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.137327 (+ 44479us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37616,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":763,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":16627,"thread_start_us":307466,"threads_started":1,"wal-append.queue_time_us":614299}
I20260709 12:22:23.130772 23511 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.131057 23517 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:23.150790 23294 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.172079 23248 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.173411 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1654) took 2109 ms. Trace:
I20260709 12:22:23.173909 23564 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:23.174777 23517 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:23.175984 23517 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.177003 23517 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.176786 20882 rpcz_store.cc:276] 0709 12:22:21.064366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.714932 (+650566us) service_pool.cc:224] Handling call
0709 12:22:21.715325 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.427909 (+712584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.173071 (+745162us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540005ad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:23.173240 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566306688}
W20260709 12:22:23.145653 23552 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:22:23.180492 20727 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:23.129092 23491 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.183892 23537 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.187958 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1779) took 1492 ms. Trace:
I20260709 12:22:23.188417 20884 rpcz_store.cc:276] 0709 12:22:21.695631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.149470 (+453839us) service_pool.cc:224] Handling call
0709 12:22:22.163320 (+ 13850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.400049 (+236729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.187703 (+787654us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540026cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:23.187890 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458354944}
I20260709 12:22:23.176256 23564 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:23.126224 23439 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.194098 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.210295 20728 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.198979 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 786) took 1071 ms. Trace:
I20260709 12:22:23.211737 21433 rpcz_store.cc:276] 0709 12:22:22.127536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.127714 (+   178us) service_pool.cc:224] Handling call
0709 12:22:23.198796 (+1071082us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54006333d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:23.198924 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294088448}
I20260709 12:22:23.212924 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.213482 21433 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:23.214603 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:23.202359 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:23.199465 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.217491 21433 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:23.218591 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:23.219640 21433 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:23.191671 23489 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.891s	user 0.011s	sys 0.000s
W20260709 12:22:23.192327 23280 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.782s	user 0.007s	sys 0.003s
W20260709 12:22:23.207727 23558 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:22:23.225113 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.225425 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.233690 20875 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:23.222325 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:23.237195 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 274936us
W20260709 12:22:23.264689 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 154702us
I20260709 12:22:23.263279 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:23.163008 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.260402 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.234549 20733 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:23.196653 23422 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.215636 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 791) took 1068 ms. Trace:
I20260709 12:22:23.231899 23280 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:23.269083 20880 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:23.249739 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.191278 23248 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.305827 20882 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:23.304193 20735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.098s	sys 0.143s
W20260709 12:22:23.306531 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:23.310084 20735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.098s	sys 0.143s
I20260709 12:22:23.296077 21433 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
W20260709 12:22:23.310634 23524 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.001s
I20260709 12:22:23.332348 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.337574 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:23.327441 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:23.355763 23530 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.530s	user 0.005s	sys 0.018s
I20260709 12:22:23.284848 23537 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:23.357694 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.359494 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.227931 23489 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.349711 23466 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.375108 21433 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:23.349103 23569 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:23.358737 23417 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:23.361114 23562 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:23.369623 23516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.459s	user 0.000s	sys 0.027s
W20260709 12:22:23.372777 23478 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.895s	user 0.009s	sys 0.003s
I20260709 12:22:23.352591 23521 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.352177 23524 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:23.429718 23516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.530s	user 0.000s	sys 0.028s
W20260709 12:22:23.431972 23341 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.952s	user 0.009s	sys 0.005s
W20260709 12:22:23.528564 20731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 363) took 1822 ms (client timeout 1930 ms). Trace:
I20260709 12:22:23.529227 23422 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:23.193970 20633 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.492321 20726 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:23.493139 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:23.498603 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:23.499863 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 331018us
I20260709 12:22:23.500819 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:23.502979 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:23.507872 20729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 355) took 2802 ms (client timeout 2165 ms). Trace:
W20260709 12:22:23.510365 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.537098 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:23.539352 23297 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.002s
I20260709 12:22:23.540122 23297 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.541114 23563 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:23.541850 21431 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:23.434160 23478 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:23.437687 23568 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:22:23.580792 23571 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.430215 23569 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.581717 23571 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.434022 23521 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.427830 23466 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.442476 23503 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.526438 23489 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.431221 23429 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:23.528160 23439 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.531436 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.526542 21433 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:22:23.618120 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 658) took 1132 ms. Trace:
I20260709 12:22:23.618628 21433 rpcz_store.cc:276] 0709 12:22:22.485189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.426358 (+941169us) service_pool.cc:224] Handling call
0709 12:22:23.525747 (+ 99389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.617688 (+ 91941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.618017 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:23.541433 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48835us
I20260709 12:22:23.621443 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:23.622440 23304 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.530046 23341 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:23.431710 23562 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.623309 23304 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.623803 23481 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:23.624521 23248 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:23.625038 23248 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:23.625469 23248 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.625568 23562 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:23.626000 23304 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.627068 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.630553 21433 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:23.630977 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.631747 20727 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.632407 23524 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.632997 23521 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.305402 21428 rpcz_store.cc:276] 0709 12:22:22.147407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.417218 (+269811us) service_pool.cc:224] Handling call
0709 12:22:22.441244 (+ 24026us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:22.441783 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.441832 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:22.442612 (+   780us) log.cc:844] Serialized 52 byte log entry
0709 12:22:22.774836 (+332224us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:23.128178 (+353342us) log.cc:844] Serialized 14 byte log entry
0709 12:22:23.143480 (+ 15302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.159118 (+ 15638us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:23.200472 (+ 41354us) raft_consensus.cc:1685] finished
0709 12:22:23.205618 (+  5146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.211752 (+  6134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41759,"prepare.queue_time_us":361102,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":9278,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":5209,"thread_start_us":438678,"threads_started":2,"wal-append.queue_time_us":479566}
I20260709 12:22:23.635398 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:23.636830 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.640048 23478 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:23.640571 23478 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:23.641002 23478 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.641863 23481 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:23.645143 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:23.541476 23576 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:23.539095 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.647553 20734 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:23.647750 23280 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:23.648183 23280 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:23.648437 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:23.649448 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.649513 23280 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.650259 20734 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.556468 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:23.651122 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 1993) took 3255 ms. Trace:
I20260709 12:22:23.651396 23280 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.651811 20807 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:22:23.651654 20733 rpcz_store.cc:276] 0709 12:22:20.395251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.405474 (+ 10223us) service_pool.cc:224] Handling call
0709 12:22:20.423603 (+ 18129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:23.308849 (+2885246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.650593 (+341744us) spinlock_profiling.cc:243] Waited 2.81 s on lock 0x7b54002490d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:23.650921 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6464055552}
W20260709 12:22:23.529247 20731 rpcz_store.cc:269] 0709 12:22:21.705648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.813524 (+107876us) service_pool.cc:224] Handling call
0709 12:22:23.528069 (+1714545us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400226fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:23.528306 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3044530048}
I20260709 12:22:23.572454 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:23.576004 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 1775) took 2510 ms (client timeout 2162 ms). Trace:
W20260709 12:22:23.534618 20729 rpcz_store.cc:269] 0709 12:22:20.704845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.438746 (+733901us) service_pool.cc:224] Handling call
0709 12:22:23.507275 (+2068529us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54003acfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:23.507582 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3611355392}
I20260709 12:22:23.586160 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2002) took 2326 ms. Trace:
W20260709 12:22:23.653762 20877 rpcz_store.cc:269] 0709 12:22:21.065018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.436864 (+371846us) service_pool.cc:224] Handling call
0709 12:22:23.575572 (+2138708us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b54003f6e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:23.575783 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12739968}
I20260709 12:22:23.654394 20732 rpcz_store.cc:276] 0709 12:22:21.259489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.833622 (+1574133us) service_pool.cc:224] Handling call
0709 12:22:22.918695 (+ 85073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:22.919064 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.919092 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:22.925693 (+  6601us) log.cc:844] Serialized 26 byte log entry
0709 12:22:23.073408 (+147715us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:23.073473 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.073574 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:23.585648 (+512074us) raft_consensus.cc:1685] finished
0709 12:22:23.585687 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.585925 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":690,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":6424,"thread_start_us":111692,"threads_started":1,"wal-append.queue_time_us":206167}
I20260709 12:22:23.579146 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 796) took 1412 ms. Trace:
I20260709 12:22:23.589704 23297 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:23.597848 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.657078 21435 rpcz_store.cc:276] 0709 12:22:22.166796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.454443 (+287647us) service_pool.cc:224] Handling call
0709 12:22:22.467929 (+ 13486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.616147 (+148218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.616175 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:22.616224 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:22.616268 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.578528 (+962260us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:23.578595 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.578895 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":959,"raft.queue_time_us":406,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10694,"spinlock_wait_cycles":339948288}
I20260709 12:22:23.595824 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 343) took 3034 ms. Trace:
I20260709 12:22:23.657215 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.612236 23491 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.612731 23569 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.613739 23496 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.615823 23480 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.627537 23571 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.627796 20878 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:23.629792 23341 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.599659 20884 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:23.605751 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 790) took 1464 ms. Trace:
W20260709 12:22:23.718154 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.720043 23556 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.720788 23556 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.649327 20882 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:23.639060 20883 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:23.653595 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.725136 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:23.726016 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.727015 20733 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:23.727566 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.727953 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.727371 23272 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:23.728856 20733 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:23.729849 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.730610 23556 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.735642 21432 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:23.654937 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.673089 23258 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.659003 20735 rpcz_store.cc:276] 0709 12:22:20.561158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.281047 (+1719889us) service_pool.cc:224] Handling call
0709 12:22:22.281441 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:22.295120 (+ 13679us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:22.491453 (+196333us) log.cc:844] Serialized 14 byte log entry
0709 12:22:23.494173 (+1002720us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:23.495633 (+  1460us) log.cc:844] Serialized 26 byte log entry
0709 12:22:23.495973 (+   340us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:23.496035 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.496221 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:23.595316 (+ 99095us) raft_consensus.cc:1685] finished
0709 12:22:23.595337 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.595565 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10908,"prepare.queue_time_us":21781,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":31719,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":27542,"thread_start_us":723711,"threads_started":2,"wal-append.queue_time_us":827568}
I20260709 12:22:23.662889 23576 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:22:23.653630 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.653609 20726 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
W20260709 12:22:23.674230 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.691820 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:23.695858 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:23.700210 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1792) took 1266 ms. Trace:
I20260709 12:22:23.707906 23481 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:23.670313 20633 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 4 to 5, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 5 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.715760 23297 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:23.718210 23555 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.718997 23294 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.672330 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:23.718225 21431 rpcz_store.cc:276] 0709 12:22:22.141346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.141548 (+   202us) service_pool.cc:224] Handling call
0709 12:22:22.783379 (+641831us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:22.783670 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.548615 (+764945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.605270 (+ 56655us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:23.605658 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752355968}
I20260709 12:22:23.731159 23272 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:23.671638 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.731113 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:23.727897 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66694.8us
I20260709 12:22:23.746129 20727 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:23.749219 23556 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.751394 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 797) took 1577 ms. Trace:
I20260709 12:22:23.752228 20731 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 3.
I20260709 12:22:23.743672 20876 rpcz_store.cc:276] 0709 12:22:22.433633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.807930 (+374297us) service_pool.cc:224] Handling call
0709 12:22:22.808314 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.267892 (+459578us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:23.267941 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:23.291422 (+ 23481us) log.cc:844] Serialized 26 byte log entry
0709 12:22:23.699484 (+408062us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:23.699571 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.699710 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:23.699769 (+    59us) raft_consensus.cc:1685] finished
0709 12:22:23.699785 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.699984 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135199,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":10692,"thread_start_us":506092,"threads_started":2,"wal-append.queue_time_us":222415}
I20260709 12:22:23.771447 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.759362 20882 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:23.751442 20806 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:23.762032 20734 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:23.779484 20729 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 3.
I20260709 12:22:23.763465 20732 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:23.722970 23417 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:23.791349 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.792308 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.794880 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:23.796558 20728 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.574s	user 0.013s	sys 0.004s
I20260709 12:22:23.796998 20728 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:23.796792 23555 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.798053 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.799156 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:23.801834 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84169.6us
I20260709 12:22:23.801517 20733 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:23.804783 20731 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:22:23.799294 21434 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:23.803364 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:23.804659 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.801323 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1791) took 1308 ms. Trace:
I20260709 12:22:23.809690 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:23.812011 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.814977 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.815315 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.816486 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.816556 23341 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:23.817147 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:23.817458 20726 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:23.817903 23341 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:23.818388 23341 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:23.818768 23491 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:23.819432 23491 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.819890 23491 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.819615 23341 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.821295 23341 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:23.823390 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:23.824561 23272 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:23.819635 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 124019us
I20260709 12:22:23.818711 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:23.802634 23298 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.877s	user 0.012s	sys 0.008s
I20260709 12:22:23.792343 23417 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.828037 20735 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:23.762430 23563 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:23.833348 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:23.839617 20728 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.841480 23466 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.842052 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:23.843688 23499 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:23.843900 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:23.844074 20729 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:23.810350 20879 rpcz_store.cc:276] 0709 12:22:22.433371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.986634 (+553263us) service_pool.cc:224] Handling call
0709 12:22:23.019597 (+ 32963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.020132 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:23.350167 (+330035us) log.cc:844] Serialized 14 byte log entry
0709 12:22:23.586661 (+236494us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:23.587518 (+   857us) log.cc:844] Serialized 26 byte log entry
0709 12:22:23.587717 (+   199us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:23.587758 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.587882 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:23.733288 (+145406us) raft_consensus.cc:1685] finished
0709 12:22:23.733307 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.741756 (+  8449us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2402,"prepare.queue_time_us":410680,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":121329,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":12227,"thread_start_us":299541,"threads_started":2,"wal-append.queue_time_us":60495}
I20260709 12:22:23.851233 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.852767 23587 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:23.778448 21426 rpcz_store.cc:276] 0709 12:22:22.173641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.466515 (+292874us) service_pool.cc:224] Handling call
0709 12:22:22.512729 (+ 46214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.728321 (+215592us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:23.155650 (+427329us) log.cc:844] Serialized 14 byte log entry
0709 12:22:23.750579 (+594929us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:23.750680 (+   101us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:23.750809 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.750941 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.751160 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74983,"prepare.queue_time_us":369443,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":81993,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":35101,"thread_start_us":747010,"threads_started":2,"wal-append.queue_time_us":388485}
I20260709 12:22:23.854396 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1750) took 1009 ms. Trace:
I20260709 12:22:23.824918 23496 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:23.859877 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.784991 23481 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:23.864060 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.799176 23555 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:23.869454 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.870903 20806 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
W20260709 12:22:23.873492 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55197.8us
I20260709 12:22:23.884315 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2003) took 2540 ms. Trace:
I20260709 12:22:23.783042 23304 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:22:23.887518 20732 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:23.890096 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.894714 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.892192 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 470) took 1577 ms. Trace:
I20260709 12:22:23.895849 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:23.897491 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64298.7us
I20260709 12:22:23.882627 23499 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:23.898646 20729 rpcz_store.cc:276] 0709 12:22:21.336362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.843273 (+2506911us) service_pool.cc:224] Handling call
0709 12:22:23.843769 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.876750 (+ 32981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.876968 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:23.883530 23587 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:23.901767 20733 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:23.831648 23556 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:23.906447 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64525.6us
I20260709 12:22:23.908892 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.907549 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.911175 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:23.876386 23298 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.886600 20726 rpcz_store.cc:276] 0709 12:22:22.844517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.818269 (+973752us) service_pool.cc:224] Handling call
0709 12:22:23.854318 (+ 36049us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:23.903532 23576 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:23.916373 23538 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.964s	user 0.017s	sys 0.004s
I20260709 12:22:23.916905 23531 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:23.934630 23531 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.935077 23531 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.936563 22814 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.917282 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:23.939440 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:23.939715 23538 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:23.940647 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52370us
W20260709 12:22:23.929205 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 85461.4us
I20260709 12:22:23.940943 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:23.941620 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:23.941717 20802 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:23.941772 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:23.919689 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:23.943042 20810 raft_consensus.cc:2368] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5.
I20260709 12:22:23.943836 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:23.945963 20804 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:22:23.947364 20726 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:23.947898 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:23.903792 23304 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:22:23.900537 21429 rpcz_store.cc:276] 0709 12:22:22.291761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.785572 (+493811us) service_pool.cc:224] Handling call
0709 12:22:23.202432 (+416860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:23.202658 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:23.206162 (+  3504us) log.cc:844] Serialized 14 byte log entry
0709 12:22:23.616483 (+410321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:23.616622 (+   139us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:23.616781 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.616889 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.867844 (+250955us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54006333d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:23.868935 (+  1091us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5003,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":2208,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":157,"spinlock_wait_cycles":957861632,"thread_start_us":409689,"threads_started":1,"wal-append.queue_time_us":244063}
I20260709 12:22:23.948971 23516 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:23.949553 23297 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:23.950039 23516 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:23.950474 23516 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:23.950807 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:23.916436 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.927690 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:23.953212 23298 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.954129 21427 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:23.948621 23280 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:23.954609 23556 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:23.955308 23556 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:23.955154 23280 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.955879 23556 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:23.958145 23280 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:23.960075 23481 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.961222 23545 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:23.961820 23545 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.928305 20728 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:23.932772 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.933940 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1711) took 3216 ms. Trace:
I20260709 12:22:23.923491 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:23.948063 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 367) took 2227 ms. Trace:
I20260709 12:22:23.951520 23516 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.929972 23248 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:23.963892 20732 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.951269 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:23.927989 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:23.964325 20726 rpcz_store.cc:276] 0709 12:22:21.720808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.946818 (+2226010us) service_pool.cc:224] Handling call
0709 12:22:23.947163 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:23.947817 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.947986 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:23.953243 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.971728 20729 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
W20260709 12:22:23.966542 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59139.7us
I20260709 12:22:23.920909 23478 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.934926 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:23.908054 23294 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:23.967890 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:23.975090 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62730us
I20260709 12:22:23.972856 21432 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:23.972036 21434 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:23.955042 23466 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.899572 23563 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.984694 20809 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:23.990731 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.992223 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:23.963625 20730 rpcz_store.cc:276] 0709 12:22:20.717042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.434372 (+1717330us) service_pool.cc:224] Handling call
0709 12:22:22.434728 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:22.435055 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.435081 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:23.656245 (+1221164us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:23.677058 (+ 20813us) log.cc:844] Serialized 26 byte log entry
0709 12:22:23.878362 (+201304us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:23.878399 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.884449 (+  6050us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:23.933326 (+ 48877us) raft_consensus.cc:1685] finished
0709 12:22:23.933359 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.933730 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5457,"raft.run_cpu_time_us":45,"raft.run_wall_time_us":44,"spinlock_wait_cycles":28582144,"thread_start_us":200896,"threads_started":1,"wal-append.queue_time_us":228495}
I20260709 12:22:23.968942 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:23.994577 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.973035 23489 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.973562 23516 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.023739 20882 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:23.972802 23496 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.997268 20801 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:23.997694 23467 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:23.998724 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:23.999619 23563 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:23.999778 21429 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEARNER]: Advancing to term 7
I20260709 12:22:23.996012 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.002552 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.006696 20732 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:23.990670 23480 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:23.992978 23297 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.007959 21432 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:22:24.010149 23298 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:24.024524 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
W20260709 12:22:24.039052 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 97390.8us
W20260709 12:22:24.044345 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95000.7us
I20260709 12:22:24.044559 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:23.993855 23294 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:24.045837 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:24.035521 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45074.1us
I20260709 12:22:24.041049 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:24.007845 23517 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:24.040706 20879 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:24.043764 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:24.048396 20807 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:24.040655 23467 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.049371 23545 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.047854 23294 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.049409 23480 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.073513 23480 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.074141 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:24.075845 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:24.077538 20809 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:24.061097 20810 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:24.065729 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1663) took 1719 ms. Trace:
I20260709 12:22:24.071167 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:23.956926 23556 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:23.981348 23531 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.087947 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:24.060925 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:24.097747 20802 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:24.103423 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.059355 23258 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:24.048120 23298 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:24.057432 23498 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.105998 20733 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:24.061486 23248 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.059111 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 656) took 1574 ms. Trace:
I20260709 12:22:24.089646 23524 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:24.081077 20881 rpcz_store.cc:276] 0709 12:22:22.346595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.421458 (+ 74863us) service_pool.cc:224] Handling call
0709 12:22:23.643339 (+1221881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.644154 (+   815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:23.644209 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:23.644926 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 12:22:23.872829 (+227903us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:23.872881 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.880712 (+  7831us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:24.064744 (+184032us) raft_consensus.cc:1685] finished
0709 12:22:24.064762 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:24.065363 (+   601us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003f6e50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:24.065547 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1006,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":129,"spinlock_wait_cycles":2672984448,"thread_start_us":227357,"threads_started":1,"wal-append.queue_time_us":331512}
I20260709 12:22:24.092280 23576 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.124785 23561 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.125042 20801 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:24.106323 23480 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:24.126576 23571 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:24.071125 23417 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:24.129161 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.052178 21426 raft_consensus.cc:2376] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6. Re-sending same reply.
I20260709 12:22:24.117811 23298 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.059356 23297 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.144791 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:24.131243 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1748) took 1273 ms. Trace:
W20260709 12:22:24.162104 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.129747 23545 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:24.090665 23517 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.173202 23517 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.070916 23587 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.096060 23496 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.151553 20726 rpcz_store.cc:276] 0709 12:22:22.843972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.983137 (+1139165us) service_pool.cc:224] Handling call
0709 12:22:24.039697 (+ 56560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.093325 (+ 53628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:24.093373 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:24.100819 (+  7446us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:24.116750 (+ 15931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:24.116851 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:24.117018 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2540,"raft.queue_time_us":45576,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":10310,"spinlock_wait_cycles":1953536}
I20260709 12:22:24.176435 21429 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:22:24.181978 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 104968us
I20260709 12:22:24.132045 23422 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 12:22:24.187923 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 116872us
I20260709 12:22:24.189914 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:24.192893 20728 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:22:24.192366 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104408us
I20260709 12:22:24.196157 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:24.141749 23562 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:22:24.200152 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a48bec319402467881543ac686fb8f0c"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:24.200264 23531 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.200480 23478 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.199908 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:24.218475 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.213891 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:24.218695 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44329.4us
I20260709 12:22:24.220031 20729 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:24.217116 23383 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.165678 23564 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
W20260709 12:22:24.235989 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 263374us
I20260709 12:22:24.129462 21427 rpcz_store.cc:276] 0709 12:22:22.484685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.130125 (+645440us) service_pool.cc:224] Handling call
0709 12:22:23.130473 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:23.954569 (+824096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:24.058785 (+104216us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b540049c854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:24.059029 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893382912}
I20260709 12:22:24.240542 20849 leader_election.cc:400] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:22:24.244326 20849 leader_election.cc:403] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:22:24.150285 23498 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:24.246255 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.170552 23576 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:24.247668 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57865.5us
I20260709 12:22:24.251452 23480 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:24.252925 23480 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.242197 23531 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:24.259979 20735 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:24.227705 23383 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.224094 23294 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.186425 23561 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.268836 23561 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.269459 23561 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:22:24.263085 23599 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:24.263163 23503 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.264669 20729 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:22:24.265419 23467 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:24.265527 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44706us
W20260709 12:22:24.278093 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 217227us
I20260709 12:22:24.232967 23491 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.248510 23417 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:24.244109 23517 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:22:24.296674 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:24.295295 20732 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:24.299383 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85581.2us
W20260709 12:22:24.301337 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 156373us
W20260709 12:22:24.304636 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46189.9us
I20260709 12:22:24.306213 23603 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:24.306686 23603 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:24.307647 23603 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.308929 23603 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:24.311764 23607 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:24.312750 23607 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:24.313223 23607 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:24.314193 23607 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.316800 21430 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:24.290915 23531 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:24.322702 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.323583 20808 raft_consensus.cc:2368] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9.
I20260709 12:22:24.324033 20807 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:24.316071 23545 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:24.332175 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 136694us
I20260709 12:22:24.333508 23602 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:24.334100 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.346886 20884 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:22:24.320403 23297 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:24.347500 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.327019 23599 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 12:22:24.335965 23341 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.514s	user 0.013s	sys 0.000s
I20260709 12:22:24.336757 23531 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.337558 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:24.301441 23499 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.338569 23496 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.339062 23576 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.341565 23489 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:24.349606 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.360183 23467 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:24.365293 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:24.349035 23602 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:24.367858 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.368299 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:24.350934 23545 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.367882 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:24.396912 20734 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:24.374428 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.393168 20879 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:24.407501 20731 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:24.330998 23417 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:22:24.412720 23555 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:24.413637 23555 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.413822 23298 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.414193 23555 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.416082 23298 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:24.417018 23417 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:24.417443 23417 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:24.418344 23417 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.420060 23417 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.399463 23545 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.422492 20878 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:24.396368 23602 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:24.398056 23524 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:24.431923 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:24.447212 20884 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:24.411051 23496 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:24.453508 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45263.9us
W20260709 12:22:24.486810 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 119035us
I20260709 12:22:24.487933 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:24.489063 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:24.492347 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 127179us
I20260709 12:22:24.452517 23496 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:24.428009 23467 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:24.503480 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44172.8us
I20260709 12:22:24.503674 23422 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:24.508147 23422 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:24.509289 23422 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.510608 23383 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.492092 23564 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:24.511569 23564 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.511957 23564 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:24.504091 23556 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:24.504889 23617 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:24.458163 23489 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:24.515327 23489 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.515815 23489 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:24.516044 23610 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:22:24.516326 20633 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:24.517539 23294 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.518199 20727 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.518152 20633 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:24.457226 23466 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:24.457706 23561 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:24.505541 23499 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.519435 23491 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.492599 23498 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.505626 23496 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.527155 23516 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.527594 20881 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:22:24.528818 20884 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:24.529531 23467 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.529949 20803 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:24.531630 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.544905 20727 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:24.545704 20726 raft_consensus.cc:606] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:24.546244 20726 raft_consensus.cc:614] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:24.547708 20806 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:24.542186 23561 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:24.553818 22814 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:24.554916 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.554204 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.556512 23498 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:24.556809 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.557682 23499 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:24.563339 23499 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:24.558401 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.558171 23516 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.559350 23467 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:24.559976 23439 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:24.562575 23489 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.565876 20884 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:24.565733 20877 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
W20260709 12:22:24.566807 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 229301us
I20260709 12:22:24.554435 23561 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.570592 23556 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:24.570915 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:24.564523 23499 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.572260 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:24.572351 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.573421 20880 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:24.569555 23503 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:24.574139 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:24.574954 23503 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:24.575166 23608 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:24.575966 23608 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:24.576970 23608 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.546833 23538 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:24.585165 23617 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:24.572314 23499 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:24.565228 23524 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.576992 23516 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.588420 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.575536 23503 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:24.595067 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.578243 23439 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:24.599596 23439 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:24.607771 20633 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 4, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 4 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:24.618391 23439 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.615521 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:24.625804 23489 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:24.631768 23503 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.635601 23555 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.635836 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.636732 20877 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:24.637866 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.639465 23439 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.643489 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:24.643916 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:24.644165 21434 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.644589 21431 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 3.
I20260709 12:22:24.646736 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:24.647595 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:24.610708 23429 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:24.648334 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:24.651675 21430 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:24.620040 20880 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:24.603292 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:24.657496 21434 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:24.648761 23429 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:24.678763 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.679813 20701 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:24.690819 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:24.683511 23607 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.693053 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:24.693233 23607 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.693820 20808 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:24.696877 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68913.1us
I20260709 12:22:24.698382 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:24.700748 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.701534 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.703423 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.704002 20732 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
W20260709 12:22:24.705811 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104453us
I20260709 12:22:24.714231 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:24.715023 21429 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:24.722771 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:24.702941 23383 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:24.611219 23602 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.740128 20728 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:24.746256 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.735256 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:24.750025 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51694.6us
I20260709 12:22:24.751662 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:24.753454 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:24.754563 23524 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:24.756141 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:24.756600 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.760196 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:24.761304 20805 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:22:24.763640 23616 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.237s	user 0.002s	sys 0.002s
W20260709 12:22:24.763453 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 83734.8us
I20260709 12:22:24.764436 23429 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:24.764946 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:24.765825 23429 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.767565 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.768401 20733 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:24.769634 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:24.771446 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.772686 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.773247 20735 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:24.736860 23422 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:24.775595 23422 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:24.741873 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:24.755035 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:24.731642 23604 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:24.775970 23422 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.789242 20732 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:24.790750 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.793913 20849 leader_election.cc:400] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6.
I20260709 12:22:24.799561 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:24.798972 20807 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:22:24.720868 23480 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:22:24.815431 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:24.799853 20883 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:24.744586 23383 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:24.801091 20801 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:24.797727 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:24.803802 23623 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:24.781265 23538 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.803932 23571 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:24.804206 20810 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:24.804426 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:24.804762 20802 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:24.792708 23561 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:22:24.806310 23602 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:24.813143 23526 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:24.876304 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:24.746202 23607 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:24.832041 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.851974 23480 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:24.852756 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51336.6us
I20260709 12:22:24.853519 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:24.788224 23556 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:24.854130 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:24.814440 23496 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.870082 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:24.851398 22814 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:24.853153 23383 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.852411 23623 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.939595 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.928529 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:24.893085 20732 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:24.937533 23608 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:24.938038 20805 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:22:24.823159 23576 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:22:24.876173 23526 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:24.851485 23266 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 12:22:24.877956 23538 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:24.865543 23498 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 12:22:25.234757 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2016) took 1337 ms. Trace:
I20260709 12:22:24.940341 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:25.240087 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 446290us
W20260709 12:22:25.236310 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 445204us
I20260709 12:22:25.127978 20803 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:25.051167 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:25.244822 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:25.246573 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.879480 23563 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.242106 20730 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:25.241752 20732 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:25.164875 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:24.982807 20802 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:25.240032 20727 rpcz_store.cc:276] 0709 12:22:23.897542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.906076 (+  8534us) service_pool.cc:224] Handling call
0709 12:22:25.234582 (+1328506us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54000e79d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:25.234699 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234305408}
I20260709 12:22:25.248773 20727 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:22:25.249078 20727 pending_rounds.cc:85] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 6
I20260709 12:22:25.249315 20727 pending_rounds.cc:107] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:22:25.249593 20727 raft_consensus.cc:2889] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:25.251101 20727 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:24.877779 23521 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:24.882695 23556 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.880419 23480 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:24.956519 23623 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.883836 23571 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:25.261083 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:25.260891 23526 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.261361 20727 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:24.938788 23562 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:24.937589 22814 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.262120 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.260779 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:25.221376 23564 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.003s
W20260709 12:22:25.195739 23439 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.001s
W20260709 12:22:25.235882 23630 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:22:25.275898 20733 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:25.260694 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:25.234212 23587 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:25.299024 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44186.8us
I20260709 12:22:25.300689 20849 leader_election.cc:400] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5.
I20260709 12:22:25.301008 20849 leader_election.cc:403] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:22:25.303879 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:25.305176 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.271741 23576 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:22:25.185552 23595 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.229s	user 0.004s	sys 0.003s
I20260709 12:22:25.235824 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.286703 20732 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:25.296288 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.291064 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:25.308216 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:25.310719 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 382631us
W20260709 12:22:25.311498 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 457398us
I20260709 12:22:25.312647 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:25.315372 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:25.287853 23608 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:25.317525 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:25.291335 23562 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:25.319785 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:25.270424 23563 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.321004 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2004) took 3866 ms. Trace:
I20260709 12:22:25.317520 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:25.278340 23556 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:25.345820 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:25.326645 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1956) took 1265 ms. Trace:
I20260709 12:22:25.347692 20807 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:25.311533 23272 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.325709 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:25.320252 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.326465 21435 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:25.329454 20731 rpcz_store.cc:276] 0709 12:22:21.454610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.907256 (+2452646us) service_pool.cc:224] Handling call
0709 12:22:23.968685 (+ 61429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.409315 (+440630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.320737 (+911422us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54001d8654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:25.320934 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998215424}
W20260709 12:22:25.350631 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50084.5us
I20260709 12:22:25.347182 20806 rpcz_store.cc:276] 0709 12:22:24.051098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.054033 (+  2935us) service_pool.cc:224] Handling call
0709 12:22:24.054330 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.548266 (+493936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.316280 (+768014us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54003c7154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:25.316492 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133978880}
I20260709 12:22:25.347103 23498 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:25.354117 23498 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.354619 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:25.321535 23499 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:25.359402 20727 pending_rounds.cc:85] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 4
I20260709 12:22:25.308177 23576 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:25.357292 21430 raft_consensus.cc:2368] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9.
I20260709 12:22:25.326416 23439 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5.
I20260709 12:22:25.314378 23531 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:25.355736 21434 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:25.360684 20727 pending_rounds.cc:107] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:22:25.363818 23526 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:25.392818 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:25.364501 22814 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:25.364977 23624 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:22:25.376793 20727 raft_consensus.cc:2889] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:25.378554 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:25.381007 23531 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:25.364562 23625 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 12:22:25.387243 20879 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:22:25.411703 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.387868 21428 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:25.388510 23629 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
I20260709 12:22:25.388955 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:25.391062 23626 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:22:25.358424 23272 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:25.387300 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.308627 23564 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.359045 23266 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.354612 23498 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:25.383625 21426 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:25.435570 20733 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
W20260709 12:22:25.441606 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.398981 23499 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.441578 20807 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:25.443631 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:25.448071 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.468534 20806 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:25.469132 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:25.391711 23608 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:25.459275 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58855.4us
W20260709 12:22:25.460211 21431 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.016s	sys 0.040s
I20260709 12:22:25.461076 21434 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:25.434970 23429 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.001s
I20260709 12:22:25.441481 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:25.476706 20881 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:25.496248 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:25.471987 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 159388us
I20260709 12:22:25.472702 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:25.514616 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.002s	sys 0.023s
W20260709 12:22:25.503392 21431 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.016s	sys 0.049s
W20260709 12:22:25.465930 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.703s (SENT) [suppressed 26 similar messages]
W20260709 12:22:25.517086 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.002s	sys 0.023s
W20260709 12:22:25.517333 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51436.1us
I20260709 12:22:25.474762 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:25.519455 20877 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:25.527349 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:25.528255 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:25.533252 20807 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:25.536175 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:25.536988 21434 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:25.538096 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 69014.8us
I20260709 12:22:25.484946 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.492249 23613 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:25.552693 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111312us
I20260709 12:22:25.557910 20801 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:25.458211 23561 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:22:25.565140 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:25.490352 23383 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.463279 22814 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:22:25.566339 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.567279 23524 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:25.441344 23571 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:25.561158 23545 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.111s	user 0.006s	sys 0.006s
I20260709 12:22:25.566910 23496 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:25.465785 23604 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:22:25.473260 23266 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:25.588930 23632 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:22:25.590072 23655 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:25.592641 23383 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:25.573385 23576 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.594837 23576 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:25.473954 23499 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.590448 23564 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:25.590178 23266 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:25.601051 21435 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:25.610394 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.611512 21435 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:25.597982 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:25.634013 20884 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:25.487403 23562 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.575232 21427 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:25.581426 23524 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:25.614301 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98542us
W20260709 12:22:25.610157 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45170.8us
I20260709 12:22:25.582859 23466 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:25.647930 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:25.612406 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:25.597450 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:25.585279 23538 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:25.646131 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:25.651624 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:25.606098 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:25.597421 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.607887 23651 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:25.658263 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 119328us
I20260709 12:22:25.584669 23613 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.649374 23524 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:25.665385 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:25.670291 21435 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:25.587286 23545 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:25.673295 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.665591 23538 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:25.684340 21434 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:25.662766 23563 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:25.690009 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.691408 21435 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:25.692679 21432 raft_consensus.cc:2368] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:25.690640 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46798 (request call id 21) took 1560 ms. Trace:
I20260709 12:22:25.688583 23647 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:25.693225 20726 rpcz_store.cc:276] 0709 12:22:24.121223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.181506 (+ 60283us) service_pool.cc:224] Handling call
0709 12:22:25.681753 (+1500247us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54000e79d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec99853 00007f9e23c6760a 00007f9e1d1ed7d4 00007f9e1d1ed762 00007f9e1d1ed691 00007f9e1d1ed60c 00007f9e1d1ec8c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:25.681909 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783778688}
I20260709 12:22:25.696609 23656 raft_consensus.cc:1064] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:22:25.699059 23656 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.702576 20876 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:25.703346 23648 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:25.704604 23642 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:22:25.706385 23499 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:25.708315 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:25.709272 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:25.709787 20733 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:25.710635 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62828.5us
W20260709 12:22:25.711097 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.711413 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:25.665841 23651 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.731945 20731 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.716267 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:25.751091 20633 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 2 to 4, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 4 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.745239 21429 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:25.716727 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:25.715657 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.760818 20732 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:25.760926 21435 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:25.716704 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:25.747014 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1754) took 2164 ms. Trace:
I20260709 12:22:25.761698 20730 rpcz_store.cc:276] 0709 12:22:23.582593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.993672 (+411079us) service_pool.cc:224] Handling call
0709 12:22:25.237334 (+1243662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:25.256514 (+ 19180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.746788 (+490274us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000e79d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:25.746934 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2754988928}
I20260709 12:22:25.692634 23531 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:25.714402 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:25.763504 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:25.764700 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.765344 20882 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.766072 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.766374 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:25.767217 23545 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:25.767494 20877 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:25.728700 23480 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.769151 23480 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:25.690009 23562 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:25.734426 23555 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:25.771580 23555 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:25.772420 23555 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.773355 23555 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:25.774192 20883 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
W20260709 12:22:25.732300 23638 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.237s	user 0.003s	sys 0.000s
I20260709 12:22:25.691649 23496 raft_consensus.cc:2764] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:22:25.735644 21427 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:25.776789 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1757) took 1071 ms. Trace:
I20260709 12:22:25.776887 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.777690 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:25.740348 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:25.779008 23496 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:25.695345 23538 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:25.779697 23496 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:25.780407 23538 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.780643 23496 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.782177 23498 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.775064 23439 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.778859 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:25.785879 20804 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.002s
W20260709 12:22:25.785974 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 219341us
I20260709 12:22:25.786441 20804 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
W20260709 12:22:25.788322 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.716457 21431 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:22:25.812127 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95927.1us
W20260709 12:22:25.750279 23422 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.927s	user 0.004s	sys 0.004s
W20260709 12:22:25.819437 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 223614us
I20260709 12:22:25.835068 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:25.778836 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.854594 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:25.715312 23499 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:25.714761 23613 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:25.862629 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 145978us
I20260709 12:22:25.724762 23272 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:25.812507 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:25.856545 23489 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:25.856954 23617 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:25.870385 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.868986 20633 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 1 to 2, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 2 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.871061 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:25.777340 20808 rpcz_store.cc:276] 0709 12:22:24.705524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.714047 (+  8523us) service_pool.cc:224] Handling call
0709 12:22:24.745579 (+ 31532us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:24.751003 (+  5424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:24.751052 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:24.792182 (+ 41130us) log.cc:844] Serialized 52 byte log entry
0709 12:22:25.145798 (+353616us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:25.230048 (+ 84250us) log.cc:844] Serialized 14 byte log entry
0709 12:22:25.230294 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.230403 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:25.731691 (+501288us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:25.776498 (+ 44807us) raft_consensus.cc:1685] finished
0709 12:22:25.776593 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.776727 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4972,"prepare.queue_time_us":83628,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":7577,"raft.queue_time_us":5265,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":12102,"spinlock_wait_cycles":9800064,"thread_start_us":277074,"threads_started":1,"wal-append.queue_time_us":602661}
I20260709 12:22:25.770972 23556 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.783612 23439 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.733682 23524 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.858011 23526 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:25.865088 23531 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:25.886868 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 120566us
I20260709 12:22:25.899168 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:25.899477 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 380) took 1035 ms. Trace:
W20260709 12:22:25.888960 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 188499us
I20260709 12:22:25.869416 23655 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:25.893952 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:25.858704 23604 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:22:25.897717 20810 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:22:25.931584 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.931949 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:25.934706 20877 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:25.884563 23489 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:25.858551 23422 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:25.899814 20734 rpcz_store.cc:276] 0709 12:22:24.863662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.262369 (+398707us) service_pool.cc:224] Handling call
0709 12:22:25.262699 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:25.263081 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:25.263106 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:25.263718 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 12:22:25.460578 (+196860us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:25.514436 (+ 53858us) log.cc:844] Serialized 14 byte log entry
0709 12:22:25.889466 (+375030us) log.cc:844] Serialized 14 byte log entry
0709 12:22:25.899120 (+  9654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.899212 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:25.899261 (+    49us) raft_consensus.cc:1685] finished
0709 12:22:25.899272 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.899423 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38891,"prepare.queue_time_us":25434,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":12939,"raft.queue_time_us":86609,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":176,"thread_start_us":534823,"threads_started":3,"wal-append.queue_time_us":208528}
I20260709 12:22:25.938509 23467 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:25.935483 20877 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:25.945844 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:25.940371 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 153466us
W20260709 12:22:25.971490 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71181.2us
I20260709 12:22:25.880863 23613 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 12:22:25.982242 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:25.938338 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79856us
I20260709 12:22:25.891346 23526 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:25.984319 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:25.898658 23531 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.986193 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:25.994122 20878 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:25.900748 23272 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:25.995563 23630 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:22:26.000790 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:26.003113 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1679) took 1246 ms. Trace:
I20260709 12:22:25.969504 23662 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:25.886246 23658 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 12:22:26.005853 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:25.900861 23499 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.972108 23429 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:25.985836 20877 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:25.989560 23526 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.024406 20880 rpcz_store.cc:276] 0709 12:22:24.757177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.758649 (+  1472us) service_pool.cc:224] Handling call
0709 12:22:24.758975 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.759231 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:24.979364 (+220133us) log.cc:844] Serialized 14 byte log entry
0709 12:22:25.463898 (+484534us) log.cc:844] Serialized 14 byte log entry
0709 12:22:25.783448 (+319550us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18003bd9a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:26.002705 (+219257us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:26.002767 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:26.002823 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.002899 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:26.003041 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20247,"prepare.queue_time_us":468751,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3444,"raft.queue_time_us":428230,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":26041,"spinlock_wait_cycles":80021632,"thread_start_us":666473,"threads_started":3,"wal-append.queue_time_us":315676}
I20260709 12:22:26.037074 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:25.984647 23489 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:25.997108 23272 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.001513 23531 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.045917 20875 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 6. Ops: []
I20260709 12:22:25.998864 23655 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:26.052094 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:26.051563 20630 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 299) took 14502 ms (client timeout 15000 ms). Trace:
W20260709 12:22:26.064492 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58708.6us
W20260709 12:22:26.066522 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:26.067312 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.037173 23662 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:26.050917 23272 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:26.029958 20805 raft_consensus.cc:2955] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:26.075282 23657 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:26.050666 23655 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.053907 23498 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.087302 20878 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:26.079061 23623 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.767s	user 0.005s	sys 0.006s
I20260709 12:22:26.076287 22814 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:26.048861 23561 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:22:26.091781 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:26.049053 23499 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.051926 23429 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.079282 23662 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:26.100872 23645 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:22:26.101373 23499 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.102236 23561 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:22:26.103726 23668 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:22:26.104669 23556 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.105140 23677 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:26.107395 22814 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:26.055217 20630 rpcz_store.cc:269] 0709 12:22:11.495647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.496210 (+   563us) service_pool.cc:224] Handling call
0709 12:22:25.986198 (+14489988us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:17.520932 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:17.728715 (+207783us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:17.728779 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:22:17.728807 (+    28us) tablet.cc:662] Decoding operations
0709 12:22:17.729950 (+  1143us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:17.730043 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:17.730096 (+    53us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:22:17.730976 (+   880us) tablet.cc:701] Row locks acquired
0709 12:22:17.731020 (+    44us) write_op.cc:260] PREPARE: finished
0709 12:22:17.754490 (+ 23470us) write_op.cc:270] Start()
0709 12:22:17.774644 (+ 20154us) write_op.cc:276] Timestamp: P: 1783599737731144 usec, L: 0
0709 12:22:17.774675 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:22:17.775397 (+   722us) log.cc:844] Serialized 3232 byte log entry
0709 12:22:18.032807 (+257410us) op_driver.cc:464] REPLICATION: finished
0709 12:22:18.387833 (+355026us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b180030c7c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:18.388024 (+   191us) write_op.cc:301] APPLY: starting
0709 12:22:18.388137 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:18.388385 (+   248us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:18.388415 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:18.389419 (+  1004us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:18.389606 (+   187us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:22:18.389636 (+    30us) write_op.cc:312] APPLY: finished
0709 12:22:18.389965 (+   329us) log.cc:844] Serialized 59 byte log entry
0709 12:22:18.390256 (+   291us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:18.390405 (+   149us) write_op.cc:454] Released schema lock
0709 12:22:18.390457 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:18.392390 (+  1933us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:18.392423 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":286352,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":42655,"apply.run_cpu_time_us":2857,"apply.run_wall_time_us":4317,"num_ops":7,"prepare.queue_time_us":386298,"prepare.run_cpu_time_us":18815,"prepare.run_wall_time_us":389763,"raft.queue_time_us":55570,"raft.run_cpu_time_us":12187,"raft.run_wall_time_us":38457,"replication_time_us":257943,"spinlock_wait_cycles":5569408,"thread_start_us":814200,"threads_started":4,"wal-append.queue_time_us":168867,"wal-append.run_cpu_time_us":38034,"wal-append.run_wall_time_us":1450169}]]}
I20260709 12:22:26.112396 23675 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:26.112656 23429 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.113341 20880 raft_consensus.cc:2410] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:26.119704 23524 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.122197 23422 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.086524 23655 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.134856 23538 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:26.136471 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 323336us
W20260709 12:22:26.136955 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152788us
I20260709 12:22:26.137555 23499 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.139588 23587 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:26.187500 23531 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.219812 23587 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:26.099874 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 208712us
I20260709 12:22:26.143291 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1681) took 1385 ms. Trace:
I20260709 12:22:26.223389 20883 rpcz_store.cc:276] 0709 12:22:24.757782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.308306 (+550524us) service_pool.cc:224] Handling call
0709 12:22:26.143024 (+834718us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:26.143211 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048228224}
I20260709 12:22:26.225016 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:26.144860 23678 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.145707 20878 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:26.151206 20849 leader_election.cc:400] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9.
I20260709 12:22:26.228564 20849 leader_election.cc:403] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:22:26.160194 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:26.188382 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 187677us
I20260709 12:22:26.188445 20884 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:26.096712 23623 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.096021 23657 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.204613 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:26.152462 23675 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.206393 22814 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.208341 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:26.147586 23666 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:26.214128 23526 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:26.205157 23647 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 12:22:26.217810 23489 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.151605 23651 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:26.197520 23561 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:26.237751 23672 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:26.222918 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 361115us
W20260709 12:22:26.227061 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:26.231941 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.241207 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:26.241364 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:26.233385 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:26.241993 20884 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:26.232458 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 98642.5us
I20260709 12:22:26.235694 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:26.231787 23675 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:26.229799 23555 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.246037 23670 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.254112 20776 leader_election.cc:400] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6.
I20260709 12:22:26.234505 23524 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:26.231637 23531 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.221092 23587 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:26.256419 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.239102 23422 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9.
I20260709 12:22:26.236050 23429 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.260924 21432 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:22:26.270578 21432 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 7
I20260709 12:22:26.270924 21432 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:22:26.271247 21432 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:26.263447 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:26.238915 23647 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:26.236208 23677 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:26.276394 23677 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.263885 20732 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:26.280114 23429 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.281061 23657 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.282196 23429 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:26.282557 23623 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:26.277984 23662 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.231406 23499 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.244102 23556 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:26.269567 20735 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:26.267568 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:26.268843 23685 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:26.231952 23678 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.272835 23663 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.258028 23670 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.301121 23678 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.302374 23670 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:26.273191 23684 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:22:26.273756 23681 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:22:26.275309 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:26.256263 23651 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:22:26.308632 20726 raft_consensus.cc:606] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:26.309012 20726 raft_consensus.cc:614] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:26.275424 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.264549 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.272039 20729 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:26.309825 20733 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:26.278018 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1802) took 1727 ms. Trace:
I20260709 12:22:26.310592 20875 rpcz_store.cc:276] 0709 12:22:24.550331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.550461 (+   130us) service_pool.cc:224] Handling call
0709 12:22:25.165181 (+614720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:26.048390 (+883209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.277351 (+228961us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:26.277818 (+   467us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54001fe350. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:26.277962 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3339419648}
I20260709 12:22:26.255872 23555 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.255759 23480 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:26.269677 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:26.297108 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:26.298306 21429 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:26.299388 23658 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.294515 23675 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.300117 23663 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.303957 20731 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:22:26.260102 20849 connection.cc:256] RPC call timeout handler was delayed by -1.3473158000001604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 12:22:26.366446 20849 connection.cc:256] RPC call timeout handler was delayed by -1.3178570109998873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:22:26.261032 23496 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:22:26.315671 21435 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:26.373595 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59034.6us
I20260709 12:22:26.317608 20884 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:26.239455 23666 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:26.366827 20849 connection.cc:256] RPC call timeout handler was delayed by -1.164615799999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
I20260709 12:22:26.372648 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:26.377184 20875 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.365459 23587 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.371378 23679 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:26.371367 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:26.367452 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 102365us
I20260709 12:22:26.313091 23685 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:26.318023 23524 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:26.374815 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:26.267874 23561 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.367393 23647 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:26.316330 23526 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:26.379752 23595 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.027s	user 0.004s	sys 0.008s
I20260709 12:22:26.392135 20733 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:26.394431 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:26.405025 23663 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.405326 23675 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.405706 23499 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.406286 23685 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.381995 23679 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.398197 23604 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.408011 23604 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.409281 23679 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.374265 23658 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.410276 23604 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.405706 23695 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:22:26.411118 23647 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.413669 23647 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.384333 23662 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:26.411643 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.314465 23556 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.383869 21428 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:26.416529 23556 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.418648 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:26.419601 22814 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:26.420019 22814 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.419996 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:26.422034 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:26.422214 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:26.433264 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:26.439955 21435 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:26.430207 23429 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:26.421900 23272 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.442999 23693 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:26.444049 20728 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:26.441490 23429 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.444885 20875 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:26.445273 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:26.448163 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 183833us
I20260709 12:22:26.449641 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:26.443368 23272 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.451350 23658 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.451723 23489 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.452044 23662 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:26.452327 23489 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:26.452621 23681 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:26.453640 23655 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.454854 23662 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:26.426661 22814 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.455767 23655 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.458266 23662 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.458796 23662 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.461406 23662 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:26.461817 23666 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.461894 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.451372 23563 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:22:26.463380 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:26.464138 20804 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.463527 23563 consensus_queue.cc:1243] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:26.469174 20733 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:26.474169 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:26.427000 23684 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:26.479892 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.485056 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.450932 23607 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:26.485854 20735 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:26.484517 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:26.466576 23429 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.463455 23524 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.492009 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:26.492254 23654 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:26.492713 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:26.508638 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 196899us
I20260709 12:22:26.508858 20802 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:26.508581 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:26.493587 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.493753 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:26.496209 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.496901 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:26.510406 20803 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.443542 23693 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.501761 20732 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:22:26.476468 23670 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:26.506311 20804 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:26.509665 20735 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:26.511005 20732 pending_rounds.cc:85] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 5
I20260709 12:22:26.519667 20732 pending_rounds.cc:107] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:22:26.520048 20732 raft_consensus.cc:2889] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:26.506771 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 96372us
I20260709 12:22:26.484517 23681 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:26.498597 23651 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:26.499346 23607 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.541483 23496 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:26.542361 23496 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:26.543778 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:26.545008 20809 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:22:26.545567 20809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 410) took 1517 ms (client timeout 1590 ms). Trace:
W20260709 12:22:26.546054 20809 rpcz_store.cc:269] 0709 12:22:25.028392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.543407 (+1515015us) service_pool.cc:224] Handling call
0709 12:22:26.545453 (+  2046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:26.547847 23692 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:26.548684 23692 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:26.549222 23692 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:26.553267 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.554155 20729 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:26.554620 23662 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.555382 23662 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.555872 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 1801) took 2005 ms. Trace:
I20260709 12:22:26.556412 20881 rpcz_store.cc:276] 0709 12:22:24.549967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.550144 (+   177us) service_pool.cc:224] Handling call
0709 12:22:26.555572 (+2005428us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b5400366d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:26.555757 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2117778304}
I20260709 12:22:26.557806 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:26.558322 23662 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.558542 20881 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:26.548072 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1761) took 1750 ms. Trace:
I20260709 12:22:26.563130 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.565104 20809 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:26.574074 20803 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:26.576510 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:26.560787 20810 rpcz_store.cc:276] 0709 12:22:24.797915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.870443 (+ 72528us) service_pool.cc:224] Handling call
0709 12:22:24.924764 (+ 54321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:25.905951 (+981187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.547720 (+641769us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54003d6dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:26.547985 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236768128}
I20260709 12:22:26.578629 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:26.507453 23684 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:26.583739 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55717.3us
I20260709 12:22:26.584353 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.580430 20733 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:26.518357 23693 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.588019 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.589315 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.591665 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:26.560441 23607 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.590169 21429 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:26.507917 23521 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:22:26.594568 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.597019 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.605875 20802 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:26.606014 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:26.601467 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:26.606686 20733 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.592633 20809 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:26.503446 23563 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:22:26.581948 23677 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.616611 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.604136 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:26.610791 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:26.612036 20730 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:26.619084 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.587881 23651 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:26.623112 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.622206 21433 raft_consensus.cc:2368] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8.
I20260709 12:22:26.626734 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:26.627312 20734 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:26.622025 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:26.628477 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:26.629328 20729 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:26.608546 23662 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.608549 23658 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.593031 23266 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:22:26.560212 23654 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:26.656442 20881 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:26.656564 23658 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.657817 20881 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:26.635941 23655 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:26.659698 23655 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:26.660164 23640 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.305s	user 0.003s	sys 0.006s
I20260709 12:22:26.597607 23666 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:22:26.615829 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:26.662325 21427 raft_consensus.cc:2376] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5. Re-sending same reply.
I20260709 12:22:26.663475 23604 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:26.663993 23662 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.664768 23651 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.623853 23677 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.639097 23657 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:26.667737 23657 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:26.668648 23657 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.655867 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.674798 21435 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:26.677289 20807 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:26.660235 23655 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.650884 23684 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.724596 23577 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:26.728432 20729 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:26.728541 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.730206 23429 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.651933 23697 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.658927 23658 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:26.734308 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84208.8us
I20260709 12:22:26.735081 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:26.785012 21429 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 3.
I20260709 12:22:26.786253 21429 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:22:26.632683 23555 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.736737 20809 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:26.787590 21429 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:22:26.789422 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.739990 20733 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:26.790779 23617 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:26.631950 23496 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.653882 23687 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:26.792100 20728 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:22:26.793396 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.760275 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:26.750078 20734 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:26.805424 21435 raft_consensus.cc:2376] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7. Re-sending same reply.
I20260709 12:22:26.735539 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1766) took 1390 ms. Trace:
W20260709 12:22:26.783093 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 181751us
I20260709 12:22:26.788326 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:26.701177 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.808295 20803 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:26.798250 20879 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:26.810086 20879 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:26.788379 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:26.739490 23666 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.811474 21432 raft_consensus.cc:2376] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5. Re-sending same reply.
I20260709 12:22:26.784092 23266 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:26.842433 20879 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.796182 23496 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:26.945875 23692 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.977548 23649 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:26.985917 23649 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.841889 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.820356 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:26.851292 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:26.905270 23644 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.514s	user 0.004s	sys 0.002s
I20260709 12:22:26.990548 20803 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
W20260709 12:22:26.909732 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.857884 23613 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.883929 23429 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:26.806241 20805 rpcz_store.cc:276] 0709 12:22:25.345054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.862139 (+517085us) service_pool.cc:224] Handling call
0709 12:22:25.880652 (+ 18513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:25.969289 (+ 88637us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:26.220716 (+251427us) log.cc:844] Serialized 14 byte log entry
0709 12:22:26.415550 (+194834us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b1800410020. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:26.581958 (+166408us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:26.582556 (+   598us) log.cc:844] Serialized 52 byte log entry
0709 12:22:26.584088 (+  1532us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:26.584127 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.584274 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:26.676488 (+ 92214us) raft_consensus.cc:1685] finished
0709 12:22:26.676597 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:26.735468 (+ 58871us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96300,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":24900,"spinlock_wait_cycles":4046592,"thread_start_us":360778,"threads_started":2,"wal-append.queue_time_us":161628}
I20260709 12:22:26.759905 22814 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:26.797320 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 271438us
I20260709 12:22:26.797932 23670 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:26.855614 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 127472us
I20260709 12:22:26.798516 23654 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:26.994571 20879 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.858338 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.817082 23697 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.997097 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.989289 23666 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:26.797605 23587 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.203s
I20260709 12:22:26.994782 23266 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.005713 23666 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.006615 23266 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.006718 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.006861 23674 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:27.007702 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 670) took 1483 ms. Trace:
I20260709 12:22:26.996716 23613 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.008116 21430 rpcz_store.cc:276] 0709 12:22:25.524569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.006409 (+1481840us) service_pool.cc:224] Handling call
0709 12:22:27.007635 (+  1226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:27.009740 23613 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.009838 23649 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.010721 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.012012 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.013137 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.013837 20810 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 4: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:27.014297 23561 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:27.014600 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.014776 23561 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:26.850351 23655 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.015733 23561 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.017347 23561 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:27.003558 23697 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.017376 23655 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.018527 20879 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:22:27.018915 20879 pending_rounds.cc:85] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 6
I20260709 12:22:27.020270 22814 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:27.021030 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.022210 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.023742 23670 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:26.966503 23693 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.027576 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:27.028512 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 815) took 1478 ms. Trace:
I20260709 12:22:27.029240 23693 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:27.029605 23699 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.429s	user 0.001s	sys 0.000s
I20260709 12:22:27.028882 21426 rpcz_store.cc:276] 0709 12:22:25.549872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.552566 (+  2694us) service_pool.cc:224] Handling call
0709 12:22:27.027408 (+1474842us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:27.028422 (+  1014us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29571456}
I20260709 12:22:27.019152 20879 pending_rounds.cc:107] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:22:27.031642 20879 raft_consensus.cc:2889] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:27.033267 23587 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:27.033710 23587 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:27.032922 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 495) took 1452 ms. Trace:
I20260709 12:22:27.001818 23521 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.035940 23674 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:27.036491 23674 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:27.036079 23521 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:27.037380 23674 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.038655 23674 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.039222 20809 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:27.034662 23587 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.040686 23587 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:26.993757 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 173514us
I20260709 12:22:26.986905 23692 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:26.791165 23617 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:27.046778 23685 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.047456 23617 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.048940 23617 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.995443 23480 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:27.049973 23480 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.050410 23480 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:27.053251 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:27.054275 20732 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:27.055310 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.055763 23649 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.056366 23649 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.061110 23662 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.002880 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.063437 20805 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:27.038749 23521 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.001561 20802 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.068313 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1818) took 1083 ms. Trace:
I20260709 12:22:27.064384 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:27.069221 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.071931 20729 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:27.077093 20810 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.072335 20879 rpcz_store.cc:276] 0709 12:22:25.580729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.589222 (+  8493us) service_pool.cc:224] Handling call
0709 12:22:25.589520 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:26.798154 (+1208634us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54000de6a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca11c0 00007f9e1eca2231 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c
0709 12:22:27.032197 (+234043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.032854 (+   657us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032902912}
W20260709 12:22:27.070493 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57154.2us
I20260709 12:22:27.075114 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:26.998764 20850 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.078151 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:26.968364 23607 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.094038 20807 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.094771 20732 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:27.083058 20878 rpcz_store.cc:276] 0709 12:22:25.985304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.092580 (+107276us) service_pool.cc:224] Handling call
0709 12:22:26.145300 (+ 52720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:26.228079 (+ 82779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.068046 (+839967us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b540026ded0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:27.068240 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104048896}
I20260709 12:22:27.106271 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:27.108309 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 109609us
I20260709 12:22:27.113430 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.113792 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:27.109550 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:27.068682 23266 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.138257 21434 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.139016 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 490) took 1444 ms. Trace:
I20260709 12:22:27.119004 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.140564 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.140967 20802 raft_consensus.cc:2376] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5. Re-sending same reply.
I20260709 12:22:27.113335 23623 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:27.122020 22814 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:27.143813 20808 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.139336 21434 rpcz_store.cc:276] 0709 12:22:25.694729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.694860 (+   131us) service_pool.cc:224] Handling call
0709 12:22:27.137845 (+1442985us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:27.138054 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.138741 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.138956 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17890048}
I20260709 12:22:27.126051 23439 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:27.129774 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.132604 23672 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:27.067860 23685 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.119855 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:27.147596 22814 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:27.148099 23266 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.153679 23672 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:27.160024 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:27.150127 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:27.161302 20732 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:27.154256 20804 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.162737 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.163925 20804 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:27.164950 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.175362 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.176425 20802 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:27.115015 23655 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.161707 22814 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:27.199530 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.200526 20850 leader_election.cc:400] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8.
I20260709 12:22:27.203774 20850 leader_election.cc:403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:27.168118 23439 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:27.203915 20804 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.206290 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:27.217191 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:27.219683 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:27.193223 23655 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.223747 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:27.213814 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.224182 23466 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:27.237694 23685 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.237430 23466 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.214712 23623 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:27.245107 20877 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:27.223243 23545 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:27.225951 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.226622 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.249106 20805 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:27.233549 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.234957 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.247118 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.248826 20804 raft_consensus.cc:2376] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6. Re-sending same reply.
I20260709 12:22:27.235630 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:27.235987 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:27.232342 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:27.253933 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.253909 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:27.254554 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.224653 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:27.257560 20883 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:27.224339 23467 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8.
I20260709 12:22:27.259900 20878 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.262084 20880 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:27.233034 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.234889 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.264849 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.225185 20802 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.248050 23266 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.267473 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.267810 23266 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.267978 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.268620 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.269078 20807 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:27.232992 23649 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.234967 23674 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:27.268635 20802 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:27.271538 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.267043 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.272564 21433 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:27.270198 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.233098 23681 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:27.276317 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.280933 23655 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:27.281821 23696 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.398s	user 0.009s	sys 0.000s
I20260709 12:22:27.284436 23677 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.236783 23607 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.248618 23545 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.285866 23545 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:27.291520 20803 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:27.268798 23666 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:22:27.293047 20877 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:27.293138 23480 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.293790 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1813) took 1389 ms. Trace:
I20260709 12:22:27.294157 20877 rpcz_store.cc:276] 0709 12:22:25.904104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.904238 (+   134us) service_pool.cc:224] Handling call
0709 12:22:25.904536 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.211900 (+1307364us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b5400320620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec14a1d 00007f9e1ebfb1a4 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419
0709 12:22:27.293541 (+ 81641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.293729 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150874368}
I20260709 12:22:27.250926 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.291018 22814 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.295809 21426 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:27.299206 20804 raft_consensus.cc:2368] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6.
I20260709 12:22:27.300189 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:22:27.300913 20802 raft_consensus.cc:606] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.301318 20802 raft_consensus.cc:614] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:27.303658 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:27.308987 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.309687 20881 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:27.313616 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.315917 20877 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:27.317276 23545 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.319588 23545 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:27.320429 23545 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:27.321041 23545 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:27.322166 23545 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.268651 23266 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:27.324440 20805 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:27.324635 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.298177 23654 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:27.275732 23272 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.330593 20877 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:27.297923 23467 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:27.311349 23712 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:22:27.308915 23556 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.336683 20802 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:27.312040 23649 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.344193 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.348055 20805 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:27.348879 20877 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:27.334087 23654 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:27.312322 23587 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:22:27.352260 23654 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:27.323562 23674 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:27.354116 23674 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.356384 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:27.356781 22814 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.357182 20879 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:27.358098 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.359499 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.360152 20878 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:27.360973 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:27.361061 23439 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:27.361656 20878 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:27.362347 23623 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:27.362447 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.363511 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.364126 20883 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:27.364976 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.344651 23272 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.366047 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.366632 20883 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:27.366775 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.367638 23272 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.368036 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.368388 20884 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:27.368637 21434 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:27.369825 20727 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:27.332054 23429 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.371150 23429 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.363564 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:27.323330 23666 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:22:27.373214 23429 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:27.375048 20810 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:27.376646 23266 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:27.364986 23672 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.377987 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:27.379333 23649 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.380229 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:27.381603 23649 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.382220 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:27.382702 23466 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.370601 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.382917 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.383460 23480 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.383579 20884 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:27.384877 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:27.384965 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:27.379642 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 292817us
I20260709 12:22:27.385836 20884 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:27.386797 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.387537 20806 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:22:27.389446 23417 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:22:27.391350 20878 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:27.390476 20803 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:27.385114 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.400583 20879 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:22:27.397816 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.402372 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:27.405375 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:27.412567 20879 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.406852 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:27.418205 20877 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:27.406936 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.421087 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:27.379679 23672 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:27.401964 23666 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:27.404250 23556 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:27.430734 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.444844 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.449296 20877 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:27.449194 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.449904 20878 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:27.450210 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.410279 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:27.427268 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:27.452231 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:27.453047 23657 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:27.437178 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46204.1us
W20260709 12:22:27.428977 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:27.455622 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:27.456063 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 186584us
I20260709 12:22:27.456264 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1691) took 1694 ms. Trace:
I20260709 12:22:27.443280 23658 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:27.405586 23521 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:27.410866 23655 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:22:27.406327 23617 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:22:27.447451 23272 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:27.405548 23417 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:22:27.448486 23556 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:27.407511 23429 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:22:27.436565 23383 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:27.456741 20876 rpcz_store.cc:276] 0709 12:22:25.761608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.761767 (+   159us) service_pool.cc:224] Handling call
0709 12:22:25.853688 (+ 91921us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:25.899423 (+ 45735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:25.899467 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:25.987830 (+ 88363us) log.cc:844] Serialized 26 byte log entry
0709 12:22:27.054314 (+1066484us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b18003bd640. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:22:27.316843 (+262529us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:27.316894 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.316957 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:27.434242 (+117285us) raft_consensus.cc:1685] finished
0709 12:22:27.434271 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:27.456204 (+ 21933us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44873,"raft.queue_time_us":79473,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":93249,"spinlock_wait_cycles":3448576,"thread_start_us":262351,"threads_started":1,"wal-append.queue_time_us":1367331}
I20260709 12:22:27.465597 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:27.478173 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 253575us
I20260709 12:22:27.485131 20879 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:27.424556 23685 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:27.493191 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83007.3us
I20260709 12:22:27.438302 23666 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.436753 23571 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:27.517284 23684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.021s	sys 0.014s
I20260709 12:22:27.486692 23649 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:27.520925 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:27.492609 23561 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:27.498590 23383 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:27.534747 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.526464 23666 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:27.542161 23684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.021s	sys 0.014s
I20260709 12:22:27.526717 23571 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.573870 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.541277 23649 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.579558 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.552218 23439 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:27.547528 23383 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:27.591827 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:27.593256 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.541095 23672 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:22:27.598577 20877 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:27.602660 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.599896 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.603587 20876 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:27.539638 23722 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:22:27.581394 23571 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.552407 23467 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:22:27.617671 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:27.580013 23674 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:27.578466 23561 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:27.581667 23649 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.622025 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:27.604650 23587 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:27.584249 23439 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.590142 23562 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:27.605232 21428 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:27.613134 23670 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:27.640365 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.608350 23657 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:27.624789 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:27.610379 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:27.627501 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:27.642889 20728 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:27.641067 23684 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:27.613566 23429 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.583493 23556 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.648830 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.643522 23727 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:27.624217 23587 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:27.650429 23727 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:27.650732 23587 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:27.651391 23727 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.651705 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.652681 23727 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:27.653636 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.655853 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.656795 20810 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:27.658063 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.658217 23657 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.650444 20875 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:27.660149 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.661774 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:27.638298 23702 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.690s	user 0.008s	sys 0.004s
I20260709 12:22:27.663156 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.612919 23685 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:27.664805 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.668794 23623 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:27.669350 20804 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.669308 23623 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:27.670423 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.611655 23617 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.671479 23617 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.665683 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.673835 20802 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.668789 23685 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.674676 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.675959 20806 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:27.670447 23623 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:27.680639 23704 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.825s	user 0.006s	sys 0.006s
I20260709 12:22:27.681185 23623 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.682318 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1778) took 1215 ms. Trace:
I20260709 12:22:27.683704 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.682773 20801 rpcz_store.cc:276] 0709 12:22:26.467078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.467175 (+    97us) service_pool.cc:224] Handling call
0709 12:22:26.484076 (+ 16901us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:26.492122 (+  8046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:26.492154 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:26.492809 (+   655us) log.cc:844] Serialized 52 byte log entry
0709 12:22:26.576436 (+ 83627us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:26.967943 (+391507us) log.cc:844] Serialized 14 byte log entry
0709 12:22:26.968590 (+   647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.968693 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:27.679916 (+711223us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:27.681975 (+  2059us) raft_consensus.cc:1685] finished
0709 12:22:27.682086 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:27.682249 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":590,"prepare.queue_time_us":445839,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":529,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":239,"spinlock_wait_cycles":13756544,"thread_start_us":448971,"threads_started":2,"wal-append.queue_time_us":308221}
I20260709 12:22:27.684513 20802 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.685246 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:27.685894 23729 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:22:27.686295 20801 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:27.683825 23561 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.685305 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.666242 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.666004 20810 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:27.639218 23439 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:27.702469 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:27.701799 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 92063.5us
I20260709 12:22:27.701799 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.704445 20804 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:27.706123 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:27.706895 20801 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:27.707129 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:27.708221 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:27.708058 20875 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:27.712626 20881 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:27.688274 23684 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:27.718005 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.713315 20807 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:27.714133 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.715298 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.623804 23674 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.716738 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1775) took 1248 ms. Trace:
I20260709 12:22:27.719432 20809 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:27.719636 20808 rpcz_store.cc:276] 0709 12:22:26.461484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.461608 (+   124us) service_pool.cc:224] Handling call
0709 12:22:27.709287 (+1247679us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540025d5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:27.709489 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565880576}
I20260709 12:22:27.716990 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.721684 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:27.724313 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:27.642120 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 176601us
I20260709 12:22:27.727104 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:27.693279 23670 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:27.731405 20804 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:27.730497 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.704257 23480 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:27.746147 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:27.724876 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 269546us
I20260709 12:22:27.736977 21434 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:22:27.739773 20809 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.740321 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:27.740958 20878 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:27.698128 23671 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:22:27.772328 21431 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:27.772482 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:27.775293 20808 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:27.770032 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:27.702824 23722 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:22:27.787164 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:27.735093 23466 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:27.705324 23672 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:22:27.712133 23607 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 12:22:27.711840 23524 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:22:27.811678 20729 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:27.745697 23480 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:27.813583 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.793821 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:27.821012 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:27.826962 20806 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:27.833776 20804 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.840297 23736 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.841102 23736 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:27.835331 23480 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:27.850621 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.815510 23617 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:27.858456 20806 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:27.863683 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.866194 21434 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:27.793331 23654 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.862632 23383 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:27.887269 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 272125us
I20260709 12:22:27.877460 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:27.889789 21431 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:27.890434 20734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.007s	sys 0.028s
I20260709 12:22:27.888099 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.895736 20884 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.890580 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.892632 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:27.897293 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:27.876648 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 422243us
I20260709 12:22:27.893778 21432 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:27.903077 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.863114 23738 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:27.883831 23681 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.895680 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.906958 21434 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:27.907567 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:27.924360 21431 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:27.908569 20735 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:27.875658 23524 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:27.887662 23685 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:27.918322 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 148400us
I20260709 12:22:27.919162 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:27.899212 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.914134 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.927604 21433 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 4: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:27.937350 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 213081us
I20260709 12:22:27.914144 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:27.951887 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.953249 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:27.953661 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54598.2us
I20260709 12:22:27.908375 23467 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:27.956022 21435 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:27.957412 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:27.965096 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.940702 23657 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:27.975456 23657 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.975965 23657 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:27.979017 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 826) took 1417 ms. Trace:
I20260709 12:22:27.979516 21428 rpcz_store.cc:276] 0709 12:22:26.562064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.575935 (+ 13871us) service_pool.cc:224] Handling call
0709 12:22:26.608572 (+ 32637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.647747 (+1039175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.978768 (+331021us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b5400750154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:27.978949 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2240647936}
I20260709 12:22:27.981245 23417 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:27.981835 23417 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:27.917915 23623 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:27.918619 23734 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:27.967291 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53272.9us
I20260709 12:22:27.917934 23738 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:27.972424 21434 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:22:27.944372 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 123174us
I20260709 12:22:27.926262 23524 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:27.857609 23670 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.957841 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:27.992916 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:27.958596 23722 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:27.958010 23562 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:27.998728 21427 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:27.972911 23655 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:27.983400 23545 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:22:27.990592 23734 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.960121 23736 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.008140 23587 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.009248 23657 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.956665 23681 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:27.999614 23266 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:28.010586 23738 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.011544 23672 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:27.967104 23556 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:28.018718 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:28.016367 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58575.5us
W20260709 12:22:28.053725 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60925.4us
I20260709 12:22:27.967892 23429 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:22:28.012205 23524 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.020955 21431 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:28.013223 23732 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:22:28.066462 23746 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:28.025394 23709 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.040s	user 0.000s	sys 0.007s
I20260709 12:22:27.983060 23417 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.071587 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1780) took 1601 ms. Trace:
W20260709 12:22:28.071098 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52476.7us
I20260709 12:22:28.072113 23417 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.073156 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:28.075847 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:28.076012 23752 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:28.076644 23752 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:28.077004 23752 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.077315 23752 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:28.072093 20730 rpcz_store.cc:276] 0709 12:22:26.470403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.622976 (+152573us) service_pool.cc:224] Handling call
0709 12:22:26.653059 (+ 30083us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:26.653606 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:26.653656 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:26.738067 (+ 84411us) log.cc:844] Serialized 26 byte log entry
0709 12:22:26.988287 (+250220us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:26.988328 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.988418 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:27.983629 (+995211us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:28.071268 (+ 87639us) raft_consensus.cc:1685] finished
0709 12:22:28.071295 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:28.071518 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1957,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":84,"spinlock_wait_cycles":10920192,"thread_start_us":197421,"threads_started":1,"wal-append.queue_time_us":194264}
W20260709 12:22:28.079550 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.028272 23655 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:28.049619 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97101.6us
I20260709 12:22:28.088601 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 415) took 1551 ms. Trace:
W20260709 12:22:28.091063 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.093298 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:28.093667 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:28.054535 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.020323 23467 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:28.058961 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 93989.5us
I20260709 12:22:28.094576 23467 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.095053 23467 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:28.032236 23562 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.095628 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.094367 20881 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:27.993894 23466 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:28.020749 23743 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.095788 23562 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:28.044032 23734 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.055575 23736 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.089947 20803 rpcz_store.cc:276] 0709 12:22:26.537111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.292397 (+755286us) service_pool.cc:224] Handling call
0709 12:22:27.390235 (+ 97838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.399775 (+  9540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:28.088307 (+688532us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b540027bd50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:28.088522 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224223616}
I20260709 12:22:28.119658 20733 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:28.066883 23681 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.103303 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.027698 23685 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:22:28.105031 20730 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:28.051306 23758 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:28.107293 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:28.110175 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55693.8us
I20260709 12:22:28.104957 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:28.127249 20734 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:28.097386 23655 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:28.131790 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:28.133652 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:28.135411 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:28.087666 23722 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:22:28.121307 23739 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.101099 23556 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.145257 23429 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:28.050822 23754 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:22:28.147518 23741 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.148087 23741 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:28.150583 23649 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.520s	user 0.006s	sys 0.004s
I20260709 12:22:28.152437 23750 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.136004 20730 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:28.159837 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.168022 23722 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:28.168861 23722 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:28.106026 23654 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:28.208436 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:28.210846 20730 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:28.125017 23524 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.130479 23758 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.215606 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:28.218406 23587 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:28.171252 23383 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.135260 23748 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.220093 23748 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:28.221262 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.222236 23748 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:28.222720 23748 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.225035 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:28.225885 21431 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:28.229323 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:28.151028 23649 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:28.212258 23648 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.311s	user 0.020s	sys 0.015s
I20260709 12:22:28.234593 21434 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:22:28.235883 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76149.1us
I20260709 12:22:28.230885 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:28.223579 23685 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:28.241708 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.217211 23758 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:28.251420 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 148159us
I20260709 12:22:28.253140 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.253582 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:28.256925 20726 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:28.258376 20734 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
W20260709 12:22:28.240603 23648 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.020s	sys 0.015s
I20260709 12:22:28.230237 23765 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:28.243510 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:28.223121 23748 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.254230 23685 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.147086 23429 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.301712 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:28.315521 20730 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:28.259531 23521 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:28.324533 23649 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.326020 20880 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:28.264055 23607 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:22:28.232465 23754 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:22:28.304690 23762 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:28.308024 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.310904 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.330140 20730 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:28.247534 23466 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.304450 23763 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:28.348446 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.352607 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:28.345417 23439 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.640s	user 0.016s	sys 0.000s
I20260709 12:22:28.318642 23734 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:28.358546 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47734.3us
W20260709 12:22:28.360214 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 130958us
I20260709 12:22:28.364053 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:28.365123 23671 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:28.365581 23671 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:28.365499 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.366278 23671 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.343361 23755 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:28.332353 23649 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.343779 23657 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:28.371726 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.372110 20734 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:22:28.347558 23681 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.402904 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:28.403271 21429 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:28.403714 21431 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:28.404327 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:28.378473 23727 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.005s
I20260709 12:22:28.409299 20727 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:28.368961 23545 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:28.369469 23755 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.375862 23670 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.411942 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:28.328912 23429 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.466955 21400 leader_election.cc:400] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6.
I20260709 12:22:28.467458 21400 leader_election.cc:403] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:28.460299 21430 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:22:28.465422 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46041.4us
I20260709 12:22:28.472894 23545 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:28.473480 23771 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:22:28.469952 21428 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.474700 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1791) took 1044 ms. Trace:
I20260709 12:22:28.475158 20810 rpcz_store.cc:276] 0709 12:22:27.430545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.776354 (+345809us) service_pool.cc:224] Handling call
0709 12:22:27.795595 (+ 19241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.048156 (+252561us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:28.048731 (+   575us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:28.055077 (+  6346us) log.cc:844] Serialized 26 byte log entry
0709 12:22:28.255609 (+200532us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:28.255661 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:28.255756 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:28.474410 (+218654us) raft_consensus.cc:1685] finished
0709 12:22:28.474438 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:28.474629 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28850,"raft.queue_time_us":121087,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7547,"thread_start_us":307500,"threads_started":2,"wal-append.queue_time_us":195389}
I20260709 12:22:28.368957 23439 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.411373 23687 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.496383 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:28.416626 23762 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:28.467218 23666 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.886s	user 0.012s	sys 0.011s
I20260709 12:22:28.400918 23681 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.367548 23763 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.461334 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:28.555680 20876 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:28.410928 23768 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:28.462857 23755 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:28.514000 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 413) took 1977 ms. Trace:
I20260709 12:22:28.548249 20734 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:28.549055 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:28.552845 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:28.565789 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:28.494653 23545 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6.
W20260709 12:22:28.521390 23571 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.899s	user 0.010s	sys 0.001s
W20260709 12:22:28.513801 23774 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:22:28.484637 23740 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.513077 23674 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:28.551888 23623 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.000s
I20260709 12:22:28.471457 23727 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 12:22:28.513072 23748 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.566478 20807 rpcz_store.cc:276] 0709 12:22:26.536473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.297011 (+760538us) service_pool.cc:224] Handling call
0709 12:22:27.709634 (+412623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.718779 (+  9145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:28.513776 (+794997us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b540025d5d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:28.513925 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948148608}
W20260709 12:22:28.552718 23480 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.011s
I20260709 12:22:28.471020 23654 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:22:28.482733 23496 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 12:22:28.480731 23775 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.186s
I20260709 12:22:28.480777 23739 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 12:22:28.542263 23687 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:28.554117 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57838.7us
I20260709 12:22:28.541677 23753 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 12:22:28.516681 23737 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:28.570405 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:28.596706 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:28.704896 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:28.648000 20804 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:22:28.722813 20804 pending_rounds.cc:85] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 6
I20260709 12:22:28.617576 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:28.712989 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:28.570508 20733 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:22:28.597038 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48074.4us
I20260709 12:22:28.707983 23623 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:28.618171 23545 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:28.727233 23783 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:22:28.727250 23545 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.729063 21428 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:28.564531 23746 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:28.494061 23764 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:28.541474 23383 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.703768 23685 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.724519 20801 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:28.648883 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:28.559401 22814 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:28.731349 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.730808 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:28.555858 23762 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:28.568704 21427 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:28.731698 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 161356us
I20260709 12:22:28.723964 20804 pending_rounds.cc:107] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:22:28.646366 23779 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.569859 23743 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:28.610599 23417 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:28.702378 23429 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:28.618474 23439 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:28.595765 23750 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.599992 20803 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:28.551110 23666 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.749585 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:28.750690 20732 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:28.618304 23521 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:28.752110 20804 raft_consensus.cc:2889] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:28.753453 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:28.708813 23737 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:28.562528 23562 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.732007 23734 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:28.751272 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.008s	sys 0.009s
I20260709 12:22:28.556977 23738 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.595438 23772 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.153s
I20260709 12:22:28.728869 23684 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.744745 23727 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:28.735916 23775 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:28.709456 23480 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.878229 20808 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:22:28.879032 23758 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.003s
I20260709 12:22:28.757448 20806 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:22:28.761053 23655 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.016s
I20260709 12:22:28.794023 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:28.892149 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98220.4us
I20260709 12:22:28.818913 23545 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.802413 23670 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:22:28.760910 23777 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:28.801797 23521 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.820082 23417 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:22:28.847795 23778 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:22:28.847599 23761 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 12:22:28.814978 20734 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:22:28.755447 23383 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.729672 23770 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:22:28.817668 23681 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.756033 23623 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:28.818194 23467 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.719s	user 0.018s	sys 0.000s
I20260709 12:22:28.770882 23674 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:28.731389 23740 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.802402 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:28.859766 23687 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.859588 23762 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:28.959664 23789 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 12:22:28.860766 23788 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:22:28.792722 22814 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:28.817904 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 249890us
I20260709 12:22:28.901227 23429 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:22:28.864126 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:28.979504 20730 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:28.979898 23775 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:28.980629 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:28.981257 20730 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:28.980958 23775 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:28.866500 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.008s	sys 0.009s
W20260709 12:22:28.983110 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 741419us
W20260709 12:22:28.866010 23752 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.788s	user 0.000s	sys 0.024s
I20260709 12:22:28.792094 23779 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.879108 23480 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.571401 23571 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.880556 23617 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:28.881114 23562 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:28.882169 20728 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:28.883304 23738 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:29.001379 23779 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:28.884312 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:29.002096 20807 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:28.885968 20801 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:28.887624 23684 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:28.871369 23727 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:28.887907 23666 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:28.905110 23266 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.849s	user 0.008s	sys 0.012s
I20260709 12:22:28.904816 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:28.888427 23763 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:29.010630 23571 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:29.011402 23266 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:29.012564 23808 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:28.901325 23764 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:29.013002 23808 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:29.013223 23764 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:29.013718 23764 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:29.013948 23808 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.015352 23808 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:29.016021 23727 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.017287 23727 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.954401 20733 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:29.005410 23687 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:28.941146 23649 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.562s	user 0.004s	sys 0.006s
I20260709 12:22:29.020843 23649 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.896134 23655 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.022078 20731 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:29.023166 23655 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:28.908529 23777 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:28.976361 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 82999.8us
W20260709 12:22:28.951460 23721 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.429s	user 0.005s	sys 0.000s
W20260709 12:22:28.862847 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109472us
I20260709 12:22:29.024888 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:28.985085 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:29.024519 23777 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.025713 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1814) took 3090 ms. Trace:
I20260709 12:22:29.026818 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:29.026121 20882 rpcz_store.cc:276] 0709 12:22:25.934817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.934961 (+   144us) service_pool.cc:224] Handling call
0709 12:22:25.969476 (+ 34515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:26.000083 (+ 30607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:26.000132 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:26.000749 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:22:27.160501 (+1159752us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b18003be000. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:22:27.303300 (+142799us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:27.303337 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.303398 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:28.839240 (+1535842us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:29.025303 (+186063us) raft_consensus.cc:1685] finished
0709 12:22:29.025324 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:29.025448 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30837,"raft.queue_time_us":45656,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":16675,"spinlock_wait_cycles":15600256,"thread_start_us":142592,"threads_started":1,"wal-append.queue_time_us":1404378}
I20260709 12:22:28.986212 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:28.989990 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:29.028023 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:29.028484 20732 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:29.029157 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:29.030082 23266 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:29.030313 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:28.999845 23383 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:28.863619 23739 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:22:29.002166 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 117933us
I20260709 12:22:28.887372 23782 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:29.006587 20802 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:28.903549 23737 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.993603 23617 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:28.955865 23623 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:28.984270 23674 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:28.897406 23758 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.020036 23687 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:29.027379 23790 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:22:29.027776 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2052) took 1402 ms. Trace:
I20260709 12:22:28.987104 23417 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.995810 22814 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.038304 20730 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
W20260709 12:22:29.076570 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90547.3us
I20260709 12:22:29.077013 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:29.053833 23649 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.008647 23738 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:29.091694 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:29.062806 20733 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:29.055020 23737 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:29.015182 23671 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.644s	user 0.007s	sys 0.002s
I20260709 12:22:29.067926 20806 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:29.071540 20735 rpcz_store.cc:276] 0709 12:22:27.625868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.626050 (+   182us) service_pool.cc:224] Handling call
0709 12:22:27.626439 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.926571 (+300132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.027547 (+1100976us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:29.027715 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648280448}
I20260709 12:22:29.078284 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:28.956089 23467 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:28.986425 23752 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.032016 23740 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:29.085532 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:29.053645 23765 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:29.087558 23802 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:29.088197 23800 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:22:29.089413 23804 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:22:29.084971 23417 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.065119 23739 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:29.094301 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:29.135416 20733 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:22:29.104480 20735 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:29.107714 23623 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:29.107966 23687 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:29.135983 20735 raft_consensus.cc:606] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:29.136404 20735 raft_consensus.cc:614] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:29.113698 23521 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:29.141875 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63672.7us
W20260709 12:22:29.108719 20760 heartbeater.cc:646] Failed to heartbeat to 127.19.36.254:46039 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.36.254:46039 timed out after 15.000s (SENT)
W20260709 12:22:29.055521 23734 consensus_queue.cc:1175] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 7 status { last_received { term: 7 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 7 index: 8 } }
I20260709 12:22:29.109040 23671 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.111366 20880 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:29.112483 23467 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:29.068450 23674 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.148360 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:29.138727 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:29.121605 23802 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:29.113080 23764 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.123627 23666 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:29.153765 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:29.153995 23766 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:29.150362 23740 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:29.064936 23782 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.147368 23738 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:29.165624 20730 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:29.114662 23617 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.151464 23739 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:29.159662 23429 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:29.153028 23802 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:29.162590 23467 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:29.150642 23761 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:22:29.148017 23762 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:22:29.169900 23521 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:29.185349 23429 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:29.184944 23740 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:29.225378 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 79406.9us
I20260709 12:22:29.207285 23807 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:29.231317 20732 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:29.232535 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:29.168895 23803 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:22:29.207933 23617 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:29.280597 21428 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:29.209759 23467 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:29.283607 21429 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:29.285542 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:29.284968 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:29.286193 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:29.286783 21426 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:29.288590 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:29.289287 21434 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:29.291419 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 1972) took 2286 ms. Trace:
I20260709 12:22:29.227514 23429 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:29.229395 23658 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:29.556969 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:29.585011 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 288724us
I20260709 12:22:29.291868 20805 rpcz_store.cc:276] 0709 12:22:27.076383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.404082 (+327699us) service_pool.cc:224] Handling call
0709 12:22:27.410765 (+  6683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.497048 (+ 86283us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:27.526829 (+ 29781us) log.cc:844] Serialized 14 byte log entry
0709 12:22:27.686686 (+159857us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b1800301540. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:22:27.871736 (+185050us) log.cc:844] Serialized 14 byte log entry
0709 12:22:27.875589 (+  3853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:27.876015 (+   426us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:27.876089 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.291160 (+1415071us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:29.291240 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:29.291360 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31687,"prepare.queue_time_us":33640,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":4733,"raft.queue_time_us":265458,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":17189,"spinlock_wait_cycles":208655360,"thread_start_us":176559,"threads_started":1,"wal-append.queue_time_us":443825,"wal-append.run_cpu_time_us":1673,"wal-append.run_wall_time_us":1033250}
I20260709 12:22:29.586701 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:29.493163 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:29.584584 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:29.277391 23649 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:29.295130 23738 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.488389 23803 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.601004 23561 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:29.233199 23674 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:29.608536 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 115500us
I20260709 12:22:29.345639 23760 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:29.615862 23556 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:29.347955 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:29.608222 23672 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:29.619259 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1784) took 1513 ms. Trace:
I20260709 12:22:29.656592 20731 rpcz_store.cc:276] 0709 12:22:28.105760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.105879 (+   119us) service_pool.cc:224] Handling call
0709 12:22:28.131060 (+ 25181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:29.022431 (+891371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.618957 (+596526us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b540028cb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:29.619164 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048603904}
I20260709 12:22:29.278074 23819 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.410s
W20260709 12:22:29.626375 23755 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.030s	user 0.010s	sys 0.000s
I20260709 12:22:29.531929 23266 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:29.282915 23758 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:29.323048 23814 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:22:29.633622 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:29.633937 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:29.520285 23727 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.003s
W20260709 12:22:29.343674 23816 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:22:29.307246 23562 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.350s
I20260709 12:22:29.642812 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:29.278816 23807 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:29.371564 23804 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.585284 23671 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:29.653800 23674 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:29.748129 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:29.748842 20803 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:29.749747 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:29.750396 20803 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.750238 21426 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:29.344478 23764 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:29.652853 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:29.659363 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:29.751686 20809 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
W20260709 12:22:29.751833 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92774.8us
W20260709 12:22:29.625342 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1712) took 2026 ms (client timeout 2428 ms). Trace:
W20260709 12:22:29.700508 23623 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.019s	sys 0.051s
I20260709 12:22:29.639402 23757 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.638911 23521 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.719887 20805 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:29.638620 23687 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.638286 23571 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:29.672762 23672 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:29.657516 23684 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.000s
I20260709 12:22:29.647469 23760 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:29.612082 23658 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.742111 23417 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:29.760636 23417 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:29.761106 23417 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.761865 23658 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:29.741911 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 99207.6us
I20260709 12:22:29.288637 23739 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.742787 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:29.763540 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:29.647251 23561 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.764106 20809 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:29.764667 23834 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.765208 23834 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.641891 23803 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.717988 23617 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:22:29.371351 23740 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.741773 23777 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:29.758538 23804 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.752512 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:29.752465 20876 rpcz_store.cc:269] 0709 12:22:27.598789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.622316 (+ 23527us) service_pool.cc:224] Handling call
0709 12:22:29.625003 (+2002687us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54000a32d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:29.625234 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103134976}
I20260709 12:22:29.592183 23467 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.769941 23383 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:29.647665 23649 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:29.771032 23649 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:29.724431 23755 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.764042 20731 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:29.773437 23755 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:29.773641 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:29.774139 23755 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:29.774566 23755 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:29.774631 23617 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:29.745731 23562 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:30.042088 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:30.007351 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.028164 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 258491us
I20260709 12:22:30.031625 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.047463 20882 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.838650 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1792) took 2407 ms. Trace:
I20260709 12:22:30.027035 23804 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:30.051784 23762 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.837675 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.048355 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:30.054455 20884 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:29.839599 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:29.755224 23623 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.019s	sys 0.051s
I20260709 12:22:30.051153 20808 rpcz_store.cc:276] 0709 12:22:27.430853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.815402 (+384549us) service_pool.cc:224] Handling call
0709 12:22:27.839569 (+ 24167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.878592 (+1039023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.838325 (+959733us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002b70d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:29.838577 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2388103936}
W20260709 12:22:30.023572 23812 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.790s	user 0.001s	sys 0.003s
W20260709 12:22:29.836154 23737 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.730s	user 0.008s	sys 0.003s
W20260709 12:22:30.039373 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 96895.9us
I20260709 12:22:29.928922 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:30.055977 20730 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:29.867178 20803 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:29.809665 20809 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:29.995925 23800 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:30.008607 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:29.785789 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1722) took 1322 ms. Trace:
I20260709 12:22:30.060425 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:30.077808 20878 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.078275 20880 rpcz_store.cc:276] 0709 12:22:28.463428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.463581 (+   153us) service_pool.cc:224] Handling call
0709 12:22:28.474806 (+ 11225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:29.150949 (+676143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.785510 (+634561us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:29.785712 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463489408}
I20260709 12:22:29.823653 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:30.061527 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:30.060905 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.080431 20729 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
W20260709 12:22:29.842523 23802 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.002s
I20260709 12:22:29.774863 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:29.839944 23808 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.824s	user 0.012s	sys 0.000s
I20260709 12:22:29.755270 23524 raft_consensus.cc:2764] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:22:29.952471 23752 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:30.072005 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:30.087927 20875 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:29.785032 20802 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1801) took 2194 ms (client timeout 2272 ms). Trace:
W20260709 12:22:30.072762 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.074337 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:30.075567 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:30.076030 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:30.090613 20801 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:30.082015 20882 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:29.836202 23735 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:29.999989 23666 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:30.082852 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.083549 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:30.057579 23833 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:22:30.083547 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:30.084383 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:30.051857 23755 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.085711 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:30.099892 20876 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.100869 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:30.101919 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:29.998606 23741 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.092237 20882 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:30.092232 20803 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: [7.7-7.8]
I20260709 12:22:30.105468 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.106076 20729 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:29.995564 23429 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.094923 20884 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:29.784456 23521 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:30.089046 20802 rpcz_store.cc:269] 0709 12:22:27.590751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.702133 (+111382us) service_pool.cc:224] Handling call
0709 12:22:29.784774 (+2082641us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003ca354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:29.784963 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2995414912}
I20260709 12:22:30.071655 23777 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:30.097622 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.117250 20880 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:30.108590 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.077190 23800 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:30.107287 23266 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:30.076658 23845 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.783309 23571 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.118129 20802 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:30.118278 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:30.119086 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.119621 20875 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:29.779513 23757 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.052997 23762 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.988747 23672 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.103107 23839 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.096664 23735 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:30.106822 23666 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:30.064455 23562 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.103785 23808 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.134773 20876 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:29.819231 23687 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:29.758340 23674 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:30.139523 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 91288.1us
I20260709 12:22:30.141188 23545 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:30.112694 23524 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:30.094209 23737 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:22:30.200261 23808 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.151741 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:30.218938 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67257.6us
I20260709 12:22:30.219772 23808 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:30.092712 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.220814 20730 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:30.221628 23752 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:30.222173 23752 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:30.091102 23802 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.224332 23571 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:30.224855 23562 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.225472 23802 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.159659 23658 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:22:30.204589 23545 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:30.227833 23800 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.228098 23545 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.228907 23800 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:30.229465 23545 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.157805 20808 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:30.157876 20806 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:29.840780 23760 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.232439 23760 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.232544 23429 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.127233 23777 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.233572 20881 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:30.234745 23777 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.202491 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:30.240475 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:30.241181 20733 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:30.243364 23524 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:30.210918 23674 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:30.203293 23849 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:22:30.244165 23524 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:30.214620 23757 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.153326 23845 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.155376 23758 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:22:30.252341 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:30.216409 23672 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:30.244938 23524 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:30.152263 23617 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:30.247709 20878 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:30.248497 20730 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:22:30.268472 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:30.268896 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.269755 20730 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:30.269465 20878 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:30.270956 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.271776 20730 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:30.272017 20776 leader_election.cc:336] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.985s (SENT)
I20260709 12:22:30.272619 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:30.273355 20730 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:22:30.211009 23687 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.250099 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.276435 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:30.277158 20875 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.216405 23521 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.253080 20803 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:30.259305 20733 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:30.280308 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.260005 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:30.157115 23266 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:30.265020 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:30.246917 23799 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.342s	user 0.002s	sys 0.000s
I20260709 12:22:30.240833 23417 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:30.282910 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:30.288830 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.295692 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.298738 20875 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:30.298552 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:30.299305 20884 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:30.299968 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.262130 23845 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:30.261472 23524 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:30.266649 23571 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:22:30.223074 23752 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.159868 23839 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.665027 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.666429 20878 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:30.666898 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:30.624889 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:30.547549 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 245644us
W20260709 12:22:30.602058 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1709) took 3004 ms (client timeout 1880 ms). Trace:
W20260709 12:22:30.682538 20877 rpcz_store.cc:269] 0709 12:22:27.598097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.602432 (+  4335us) service_pool.cc:224] Handling call
0709 12:22:30.601810 (+2999378us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54000965d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:30.601979 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4580739840}
I20260709 12:22:30.688710 20877 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:30.611810 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:30.301337 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.603020 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2056) took 1896 ms. Trace:
I20260709 12:22:30.722254 20881 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:30.722254 20734 rpcz_store.cc:276] 0709 12:22:28.706371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.706536 (+   165us) service_pool.cc:224] Handling call
0709 12:22:28.706836 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.908172 (+201336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.602739 (+1694567us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54002977d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:30.602954 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247901568}
I20260709 12:22:30.784159 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:30.598346 20726 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:30.789070 20802 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:30.784304 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:30.723558 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.790349 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:30.790669 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.790946 20733 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:30.791292 20734 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
I20260709 12:22:30.723454 20881 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:30.802599 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.801469 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1814) took 1157 ms. Trace:
I20260709 12:22:30.805176 20801 rpcz_store.cc:276] 0709 12:22:29.643941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.101978 (+458037us) service_pool.cc:224] Handling call
0709 12:22:30.102339 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.102542 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:30.204917 (+102375us) log.cc:844] Serialized 14 byte log entry
0709 12:22:30.801025 (+596108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:30.801090 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:30.801156 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.801240 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:30.801391 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":99952,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"thread_start_us":687448,"threads_started":3}
I20260709 12:22:30.807687 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:30.809626 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:30.812412 20878 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:30.816277 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1783) took 2710 ms. Trace:
I20260709 12:22:30.821853 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:30.332504 23849 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:30.520398 23850 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:22:30.859864 23849 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:30.463842 23847 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:22:30.832710 20733 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:30.860443 23849 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:30.861470 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.862329 20733 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
W20260709 12:22:30.822582 23753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.717s	user 0.002s	sys 0.031s
I20260709 12:22:30.840755 22814 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.824012 20728 rpcz_store.cc:276] 0709 12:22:28.105460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.125164 (+ 19704us) service_pool.cc:224] Handling call
0709 12:22:28.151636 (+ 26472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:28.997157 (+845521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.816047 (+1818890us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54002e4fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:30.816223 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679615872}
W20260709 12:22:30.864192 23753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.762s	user 0.002s	sys 0.032s
I20260709 12:22:30.864370 22814 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.865067 20728 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:30.865218 20734 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [4.3-7.5]
I20260709 12:22:30.866427 20728 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:30.866621 22814 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.841223 23763 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:30.835107 23740 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.843266 23383 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:30.867375 23763 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.867715 23383 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.868470 20733 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:30.869448 23763 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.802174 23658 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:30.841740 23417 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.826046 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:30.847536 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.861613 23849 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.869832 23383 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:30.869995 23658 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.867669 23740 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.845144 23807 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:30.288477 23617 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.331359 23521 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.493s
I20260709 12:22:30.870272 23417 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.870554 20801 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:30.962107 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:31.026455 23849 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:31.027034 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:30.954704 20809 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.048s	sys 0.093s
W20260709 12:22:31.027930 20809 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.051s	sys 0.093s
I20260709 12:22:31.028079 23863 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:30.918932 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.867388 20734 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:30.944427 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:31.029969 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:30.610110 23756 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:30.864372 23839 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:31.039762 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:31.040381 20728 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:30.297775 23758 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:22:30.801746 23467 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.007s	sys 0.013s
I20260709 12:22:30.978396 23524 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.063287 20728 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:30.456645 23561 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.029104 23782 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:30.654058 23666 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.542111 23755 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:30.655663 23770 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:30.331944 23757 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:31.033195 23756 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:31.076280 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:31.042768 23807 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:31.071898 23775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.015s	sys 0.019s
W20260709 12:22:31.080178 23467 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.012s	sys 0.017s
I20260709 12:22:30.488724 23266 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:31.032784 23863 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:31.081000 23521 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 12:22:30.326121 23735 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.222847 23757 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:31.228066 20882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.008s	sys 0.045s
W20260709 12:22:31.240880 20882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.008s	sys 0.046s
I20260709 12:22:31.220534 23770 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:31.228319 23775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.016s	sys 0.020s
I20260709 12:22:31.243878 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1860) took 1181 ms. Trace:
I20260709 12:22:31.244513 20882 rpcz_store.cc:276] 0709 12:22:30.062616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.121591 (+ 58975us) service_pool.cc:224] Handling call
0709 12:22:30.137632 (+ 16041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.137876 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:30.337554 (+199678us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.243366 (+905812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:31.243449 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:31.243519 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.243620 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.243797 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21807,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":3310,"thread_start_us":1091492,"threads_started":2}
W20260709 12:22:31.266961 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 426522us
I20260709 12:22:31.241958 23757 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:31.288125 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1785) took 3182 ms. Trace:
I20260709 12:22:31.288708 20727 rpcz_store.cc:276] 0709 12:22:28.106124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.106249 (+   125us) service_pool.cc:224] Handling call
0709 12:22:28.131163 (+ 24914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.415335 (+284172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.287833 (+2872498us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54002e54d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:31.288050 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587850624}
W20260709 12:22:31.293262 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.294776 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:31.278574 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 467618us
I20260709 12:22:31.295763 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:31.296662 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.301103 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:31.279335 23868 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:22:31.301535 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 447) took 1484 ms. Trace:
I20260709 12:22:31.267987 23735 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.309259 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:31.310801 20775 leader_election.cc:400] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:31.314697 20775 leader_election.cc:403] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:22:31.302958 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.011s	sys 0.011s
W20260709 12:22:31.320104 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.012s	sys 0.011s
W20260709 12:22:31.321499 20776 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.523s (SENT)
I20260709 12:22:31.328447 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:31.328558 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:31.301008 23521 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.304554 20805 rpcz_store.cc:276] 0709 12:22:29.817247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.107577 (+290330us) service_pool.cc:224] Handling call
0709 12:22:30.124068 (+ 16491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.124302 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:22:30.276668 (+152366us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.301145 (+1024477us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:31.301209 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:31.301264 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.301350 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.301476 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":354847,"prepare.queue_time_us":78641,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":601046,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":14608,"thread_start_us":801131,"threads_started":3,"wal-append.queue_time_us":751973}
I20260709 12:22:31.303711 23687 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:31.303158 23467 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:31.337085 23417 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:31.359303 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:31.368902 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 403) took 2818 ms. Trace:
I20260709 12:22:31.370071 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:31.369673 20726 rpcz_store.cc:276] 0709 12:22:28.550409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.550608 (+   199us) service_pool.cc:224] Handling call
0709 12:22:28.571367 (+ 20759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.789734 (+2218367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.368529 (+578795us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54002990d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:31.368810 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4660885760}
I20260709 12:22:31.359087 23768 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:31.337450 23763 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:31.382488 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:31.383905 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:31.267760 23756 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.359829 23658 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.365835 23740 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:31.375378 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:31.398066 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:31.337815 22814 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:22:31.310359 23864 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 12:22:31.378124 23763 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.402216 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:31.379653 23862 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:22:31.308976 23770 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:31.359920 23521 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.365732 23758 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:31.385955 23768 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:22:31.395084 23859 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:22:31.388368 23873 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:31.422161 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.392421 23417 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:31.417904 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:31.400420 23756 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:31.392920 23467 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:31.408948 23763 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:31.404446 23658 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.408653 23666 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:31.427520 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 116862us
I20260709 12:22:31.446100 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:31.432333 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1809) took 1798 ms (client timeout 1908 ms). Trace:
I20260709 12:22:31.411818 23617 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:31.423416 23860 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:22:31.424173 23873 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:31.437601 23866 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:22:31.439059 23834 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.440937 20802 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 12:22:31.442219 23870 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:22:31.447106 20806 rpcz_store.cc:269] 0709 12:22:29.633508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.633630 (+   122us) service_pool.cc:224] Handling call
0709 12:22:31.432132 (+1798502us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:31.432263 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":991550848}
I20260709 12:22:31.420454 23770 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:31.431840 23467 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.426137 23876 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:31.457430 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 440) took 2850 ms. Trace:
I20260709 12:22:31.484577 20804 rpcz_store.cc:276] 0709 12:22:28.601784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.602111 (+   327us) service_pool.cc:224] Handling call
0709 12:22:28.602509 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:28.804389 (+201880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.451972 (+2647583us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b54003c9954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:31.452160 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103430400}
I20260709 12:22:31.416363 23740 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 12:22:31.468175 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 146697us
I20260709 12:22:31.445669 23417 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:31.463568 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152830us
W20260709 12:22:31.483453 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68860.2us
I20260709 12:22:31.436082 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:31.492951 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46921.6us
I20260709 12:22:31.496544 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:31.462623 23875 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:31.433606 23768 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:22:31.462987 23617 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:31.502725 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:31.500767 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 448) took 1455 ms (client timeout 1687 ms). Trace:
I20260709 12:22:31.501241 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1812) took 1857 ms. Trace:
I20260709 12:22:31.483907 23467 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:31.490775 23876 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:31.503789 20808 rpcz_store.cc:269] 0709 12:22:30.045842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.063178 (+ 17336us) service_pool.cc:224] Handling call
0709 12:22:31.500601 (+1437423us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b5400127854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:31.500710 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132936064}
I20260709 12:22:31.451704 23658 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:31.489030 23765 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:31.463181 23873 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:31.501801 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 413) took 1682 ms. Trace:
I20260709 12:22:31.520300 20731 rpcz_store.cc:276] 0709 12:22:29.819224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.153762 (+334538us) service_pool.cc:224] Handling call
0709 12:22:30.180460 (+ 26698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.234254 (+ 53794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.234287 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:30.234329 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:30.234365 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.501524 (+1267159us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b54000e74d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:31.501573 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.501726 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123257728}
I20260709 12:22:31.504197 20809 rpcz_store.cc:276] 0709 12:22:29.643368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.090941 (+447573us) service_pool.cc:224] Handling call
0709 12:22:30.109114 (+ 18173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.109260 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:30.202030 (+ 92770us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.416222 (+1214192us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.500923 (+ 84701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:31.500981 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:31.501027 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.501087 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.501189 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7831,"prepare.queue_time_us":553980,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":498555,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":64,"thread_start_us":1368506,"threads_started":5,"wal-append.queue_time_us":831443}
I20260709 12:22:31.507886 22814 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:31.493963 23863 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:31.523002 20776 leader_election.cc:336] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.879s (SENT)
I20260709 12:22:31.525940 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 409) took 2458 ms. Trace:
I20260709 12:22:31.526417 23882 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:31.538859 20735 rpcz_store.cc:276] 0709 12:22:29.067573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.149215 (+ 81642us) service_pool.cc:224] Handling call
0709 12:22:29.149631 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:29.150891 (+  1260us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:31.443461 (+2292570us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b180000fac0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:22:31.443727 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.462817 (+ 19090us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:31.482853 (+ 20036us) log.cc:844] Serialized 26 byte log entry
0709 12:22:31.483038 (+   185us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:31.483073 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.483181 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.525546 (+ 42365us) raft_consensus.cc:1685] finished
0709 12:22:31.525675 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.525859 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9035,"prepare.queue_time_us":2288582,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":19594,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":15382,"spinlock_wait_cycles":3817728,"thread_start_us":15928,"threads_started":1,"wal-append.queue_time_us":51514}
I20260709 12:22:31.528470 23617 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:31.529829 23887 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:31.530885 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1865) took 1246 ms. Trace:
I20260709 12:22:31.551159 20879 rpcz_store.cc:276] 0709 12:22:30.284398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.726752 (+442354us) service_pool.cc:224] Handling call
0709 12:22:30.816397 (+ 89645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:30.836225 (+ 19828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.836255 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:30.836719 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:22:31.034927 (+198208us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:31.034964 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.035037 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.530646 (+495609us) raft_consensus.cc:1685] finished
0709 12:22:31.530664 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.530815 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19413,"raft.queue_time_us":19498,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":93,"thread_start_us":197845,"threads_started":1,"wal-append.queue_time_us":453517}
I20260709 12:22:31.493218 23834 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:22:31.519663 23521 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.563395 23417 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:31.575526 23874 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.180s	user 0.001s	sys 0.001s
W20260709 12:22:31.550231 23872 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:22:31.516412 23758 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:31.543802 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:31.548643 23871 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:31.517195 23876 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:31.550973 23887 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.552846 23885 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:31.743039 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 220039us
I20260709 12:22:31.658752 23871 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:31.669637 23849 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.640s	user 0.000s	sys 0.013s
W20260709 12:22:31.708652 23838 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:22:31.668110 23887 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:31.687660 23666 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.768208 22814 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:31.793140 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 299064us
W20260709 12:22:31.792207 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 289570us
I20260709 12:22:31.796137 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:31.802088 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2005) took 1122 ms. Trace:
W20260709 12:22:31.786403 23807 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.004s
I20260709 12:22:31.804587 20729 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:31.806908 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 704) took 2525 ms. Trace:
I20260709 12:22:31.785562 23658 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:31.809952 23746 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.801075 22814 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.807395 21431 rpcz_store.cc:276] 0709 12:22:29.277809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.277943 (+   134us) service_pool.cc:224] Handling call
0709 12:22:29.278294 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:29.279018 (+   724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:29.279046 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:29.289781 (+ 10735us) log.cc:844] Serialized 52 byte log entry
0709 12:22:29.640057 (+350276us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:29.714332 (+ 74275us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.795752 (+2081420us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b180013de00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:31.796085 (+   333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.796157 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.796201 (+    44us) raft_consensus.cc:1685] finished
0709 12:22:31.802364 (+  6163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.802645 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9392,"prepare.queue_time_us":78183,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":427,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3107,"spinlock_wait_cycles":5013760,"thread_start_us":297213,"threads_started":2,"wal-append.queue_time_us":448349,"wal-append.run_cpu_time_us":2079,"wal-append.run_wall_time_us":1002106}
I20260709 12:22:31.794095 23666 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.836966 23871 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.867317 23746 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.868014 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:31.869300 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:31.872443 23806 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.872690 23768 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.872992 23806 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.873449 22814 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.873270 23768 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.875206 23806 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.876226 23768 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.877115 23768 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.877635 23768 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.878571 23901 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:31.879122 23901 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:31.880158 23768 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.844532 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:31.832000 23737 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:31.880973 23737 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:31.881184 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 446) took 2650 ms. Trace:
I20260709 12:22:31.881350 23901 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:31.881855 23737 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:31.831398 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.834563 23834 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:31.830511 23649 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:31.847141 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:31.787557 23888 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:31.850672 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 412) took 2558 ms. Trace:
I20260709 12:22:31.852501 20881 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:31.852716 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:31.854203 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:31.878088 23757 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.000s
W20260709 12:22:31.916168 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71784us
W20260709 12:22:31.914384 20850 connection.cc:256] RPC call timeout handler was delayed by -1.1249755460003144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 12:22:31.801004 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 304404us
I20260709 12:22:31.881593 20810 rpcz_store.cc:276] 0709 12:22:29.231051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.231231 (+   180us) service_pool.cc:224] Handling call
0709 12:22:29.231545 (+   314us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:29.233385 (+  1840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:29.233420 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:29.282493 (+ 49073us) log.cc:844] Serialized 52 byte log entry
0709 12:22:29.744199 (+461706us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:30.260108 (+515909us) log.cc:844] Serialized 14 byte log entry
0709 12:22:30.262517 (+  2409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.262626 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.546556 (+1283930us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b180028bee0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:31.880899 (+334343us) raft_consensus.cc:1685] finished
0709 12:22:31.881000 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.881127 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9455,"prepare.queue_time_us":457940,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":55,"raft.queue_time_us":50874,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":114889,"spinlock_wait_cycles":4381056,"thread_start_us":583510,"threads_started":2,"wal-append.queue_time_us":487643}
W20260709 12:22:31.928260 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 132147us
W20260709 12:22:31.928954 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76075.6us
I20260709 12:22:31.930461 20878 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:31.848415 23895 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.190s
W20260709 12:22:31.969159 21400 connection.cc:256] RPC call timeout handler was delayed by -1.8158055700000659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:22:32.023986 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 442) took 3420 ms (client timeout 1886 ms). Trace:
W20260709 12:22:32.027691 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.121246 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:31.999557 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.009860 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:31.982923 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:32.009472 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:32.120626 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.021474 21429 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:32.123731 21430 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:22:32.121615 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.123898 21400 connection.cc:256] RPC call timeout handler was delayed by -1.767735021000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 12:22:32.011933 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:32.124739 21432 raft_consensus.cc:2376] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6. Re-sending same reply.
I20260709 12:22:32.026268 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1815) took 2382 ms. Trace:
I20260709 12:22:32.126045 20803 rpcz_store.cc:276] 0709 12:22:29.644262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.104564 (+460302us) service_pool.cc:224] Handling call
0709 12:22:30.104887 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.278190 (+173303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.026003 (+1747813us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54002b61d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:32.026203 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326817536}
I20260709 12:22:32.009018 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:32.070878 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.127781 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.128129 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.128341 20735 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:22:32.128466 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.010763 21434 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 12:22:32.120713 20807 rpcz_store.cc:269] 0709 12:22:28.603857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.704628 (+100771us) service_pool.cc:224] Handling call
0709 12:22:32.023765 (+3319137us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54002aadd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:32.023883 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641817472}
I20260709 12:22:32.122515 21428 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:32.130425 20735 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:32.123126 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:32.131042 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:32.124361 20732 rpcz_store.cc:276] 0709 12:22:29.292569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.292698 (+   129us) service_pool.cc:224] Handling call
0709 12:22:29.323677 (+ 30979us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:29.324110 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:29.324669 (+   559us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:29.325425 (+   756us) log.cc:844] Serialized 52 byte log entry
0709 12:22:29.815560 (+490135us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:31.667902 (+1852342us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.850257 (+182355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.850419 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.850480 (+    61us) raft_consensus.cc:1685] finished
0709 12:22:31.850492 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.850614 (+   122us) inbound_call.cc:177] Queueing success response
0709 12:22:32.007489 (+156875us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b1800263da0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e22b7358f 00007f9e1bbb2571 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd
Metrics: {"mutex_wait_us":1759455,"prepare.queue_time_us":1628775,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":46818,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":19455,"spinlock_wait_cycles":31702400,"thread_start_us":801061,"threads_started":3,"wal-append.queue_time_us":856644,"wal-append.run_cpu_time_us":2191,"wal-append.run_wall_time_us":1029262}
I20260709 12:22:32.347834 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2004) took 1680 ms. Trace:
I20260709 12:22:32.348299 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:32.349326 20735 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:32.349858 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:32.350695 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.348453 20806 rpcz_store.cc:276] 0709 12:22:30.667400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.478457 (+811057us) service_pool.cc:224] Handling call
0709 12:22:31.478831 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.479070 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:31.792907 (+313837us) log.cc:844] Serialized 14 byte log entry
0709 12:22:32.347472 (+554565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:32.347534 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:32.347592 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.347661 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.347774 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":215723,"prepare.queue_time_us":12840,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":658742,"threads_started":2}
I20260709 12:22:31.883177 23737 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:32.238955 21399 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.687s (SENT) [suppressed 36 similar messages]
I20260709 12:22:32.120859 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:32.354897 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 234107us
I20260709 12:22:32.356045 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:32.308172 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 184179us
I20260709 12:22:32.242273 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1806) took 2523 ms. Trace:
I20260709 12:22:32.199518 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:32.361454 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:32.361348 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:32.208588 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.005548 21426 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:32.364468 20734 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:22:32.151173 20879 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:32.366416 20849 connection.cc:256] RPC call timeout handler was delayed by -1.235386538000057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
I20260709 12:22:32.361235 20729 rpcz_store.cc:276] 0709 12:22:29.718869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.147578 (+428709us) service_pool.cc:224] Handling call
0709 12:22:30.147947 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.806742 (+1658795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.241944 (+435202us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54001d8154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:32.242189 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3809419520}
W20260709 12:22:32.367015 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1966865379999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
I20260709 12:22:32.357753 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:32.369796 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1723865379999552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 12:22:31.927112 23849 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:31.987293 23763 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.535s	user 0.004s	sys 0.009s
I20260709 12:22:32.372611 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:31.930898 23761 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:32.449081 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.375037 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 251949us
W20260709 12:22:32.451110 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.010164 23804 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.183706 23800 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:32.130745 23834 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:31.943907 23807 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:32.416329 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:32.416843 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:31.985316 23832 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.965818 23818 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:31.965202 23841 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.068674 23746 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.013818 23666 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.004230 23803 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:32.008062 23898 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:32.414974 23896 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:31.985584 23681 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.176400 23739 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:32.425755 23899 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.123s	user 0.000s	sys 0.006s
W20260709 12:22:32.426246 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.067422 23762 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:32.309134 23836 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.133466 23663 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.133297 23772 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.020804 23765 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:32.009567 23819 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:32.427022 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.499995 21426 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:32.426811 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:32.469741 23684 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:22:32.428718 20879 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:32.231263 23815 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:31.986503 23871 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.453644 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.470132 23800 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.417680 23734 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.458190 20883 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:32.463928 20730 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:32.464449 20726 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:32.467689 20876 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:32.467932 20882 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:32.501895 23770 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.010s	user 0.012s	sys 0.004s
I20260709 12:22:32.513525 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:32.499658 23765 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.514775 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:32.149655 23757 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.427747 23895 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:32.327050 23882 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.134s
I20260709 12:22:32.458069 23888 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.455994 23648 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.465356 23753 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:32.494042 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 136392us
I20260709 12:22:32.469599 23649 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:32.544727 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.525048 20801 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:32.494954 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 139027us
I20260709 12:22:32.558189 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:32.005035 21427 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:32.559837 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:32.561372 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:32.458735 23761 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.503475 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:32.565145 20879 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
I20260709 12:22:32.465312 23804 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.455497 23763 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.506646 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:32.498379 23739 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:32.501055 23905 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 12:22:32.507989 21430 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 4.
W20260709 12:22:32.503247 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132723us
I20260709 12:22:32.489091 23898 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:32.515728 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 276797us
I20260709 12:22:32.517073 20883 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:32.485883 23818 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.491950 23879 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:22:32.556829 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:32.590947 20876 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.563884 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:32.502418 23772 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.514693 23734 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:32.566372 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:32.494568 23681 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:32.606029 20700 connection.cc:256] RPC call timeout handler was delayed by -2.1626740970000355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 12:22:32.578737 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.501154 23663 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.579131 20883 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:32.488682 23841 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.487480 23832 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.572718 23770 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:32.611521 20879 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:22:32.583420 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.583678 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.526216 23757 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.523869 23735 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:22:32.613432 21430 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:32.615777 21431 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.594563 20807 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:32.616312 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.616752 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:32.617385 21430 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:32.595419 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:32.618170 20806 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:32.598521 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:32.619524 20805 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:32.508069 23836 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.501288 23834 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:32.507648 23819 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.567932 23782 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:32.598812 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:32.635840 20804 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:32.593457 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:32.636387 20876 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:22:32.571333 23882 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:22:32.608165 20881 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:32.608355 20700 connection.cc:256] RPC call timeout handler was delayed by -2.0933740969999235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 12:22:32.521803 23815 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.632248 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.643071 21427 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:32.644944 21431 raft_consensus.cc:2410] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:32.556056 23648 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.609607 23912 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:22:32.565285 23649 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.635703 21426 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.650488 20805 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:32.652073 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:32.652712 20805 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:32.604738 20803 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:32.449084 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:32.656424 21428 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.565021 23753 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.661299 23753 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.662034 23753 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:32.604024 23734 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.665235 23758 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.665766 23758 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.570878 23896 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:22:32.667905 23896 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:32.668349 23896 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:32.668852 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.669400 23896 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:32.605934 23750 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:32.639526 20700 connection.cc:256] RPC call timeout handler was delayed by -1.9976287049998973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260709 12:22:32.671661 20700 connection.cc:256] RPC call timeout handler was delayed by -1.939035237999633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 12:22:32.569480 23849 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:32.637163 23757 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:32.641399 21433 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:32.609483 23655 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:32.610255 23803 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:32.633814 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.702543 21430 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:32.609683 23666 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:32.639752 23897 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
W20260709 12:22:32.672158 20700 connection.cc:256] RPC call timeout handler was delayed by -1.859374097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 12:22:32.711977 20700 connection.cc:256] RPC call timeout handler was delayed by -1.8341740969999591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 12:22:32.712502 20700 connection.cc:256] RPC call timeout handler was delayed by -1.7887352379998447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 12:22:32.712836 20700 connection.cc:256] RPC call timeout handler was delayed by -1.7288740969997889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
I20260709 12:22:32.640349 21435 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:32.674764 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:32.713692 20806 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:32.679936 21426 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
W20260709 12:22:32.699282 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.706755 23807 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:32.708519 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:32.715376 20700 connection.cc:256] RPC call timeout handler was delayed by -1.660474097000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
I20260709 12:22:32.715736 21430 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:32.716966 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.718089 21430 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:32.674937 23896 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:32.715778 20700 connection.cc:256] RPC call timeout handler was delayed by -1.5704740969999875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
W20260709 12:22:32.721261 20700 connection.cc:256] RPC call timeout handler was delayed by -1.1963465639996684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22719999999999999s for a response
I20260709 12:22:32.721873 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:32.706689 23757 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:32.719650 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:32.684031 23467 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.163s	user 0.011s	sys 0.001s
I20260709 12:22:32.722775 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.674374 23849 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:32.727577 23841 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:32.729116 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2012) took 1904 ms (client timeout 1603 ms). Trace:
W20260709 12:22:32.729686 20801 rpcz_store.cc:269] 0709 12:22:30.824852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.026746 (+201894us) service_pool.cc:224] Handling call
0709 12:22:32.728821 (+1702075us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:32.729017 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3442826624}
I20260709 12:22:32.684142 23910 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:32.731384 23910 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:32.725960 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 124206us
I20260709 12:22:32.667891 23758 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.754727 21428 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:32.735529 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:32.671689 23750 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.760294 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:32.716689 23666 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.764969 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.769297 21427 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
W20260709 12:22:32.765574 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 172214us
I20260709 12:22:32.772552 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.773423 21428 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:32.736970 23467 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:32.709048 23655 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.719012 23879 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:22:32.759338 23758 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:22:32.781301 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.782047 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 223923us
I20260709 12:22:32.781910 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:32.732245 23910 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:32.791332 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:32.774286 23674 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:32.779213 23681 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:32.792683 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:32.778236 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.771332 23684 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.772019 23658 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:32.770088 23841 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:32.798480 21430 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:32.799834 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:32.811688 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:32.822710 23666 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:32.826383 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.799002 23674 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.783365 23735 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:22:32.789768 23417 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:22:32.812189 23882 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:22:32.882413 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 219020us
I20260709 12:22:32.806143 23658 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:32.813287 23758 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:32.828649 23910 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:32.870421 23876 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.203s	user 0.009s	sys 0.000s
I20260709 12:22:32.814098 23804 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:32.790776 23926 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:22:32.882896 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.809088 23912 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:32.884481 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:32.860359 23909 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:22:32.868525 23919 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:22:32.867808 23914 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:22:32.836117 23674 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.790023 23849 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:32.827446 23916 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.809283 23681 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:32.888429 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 127808us
I20260709 12:22:32.892185 23898 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:32.892704 23649 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.893030 23750 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.894168 23655 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.894657 23758 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:32.887174 23772 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:32.895280 23887 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.102s	user 0.009s	sys 0.003s
I20260709 12:22:32.858922 23734 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.900877 23692 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:22:32.862962 23927 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.899658 23876 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:32.850368 23763 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:32.885703 23873 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.349s	user 0.012s	sys 0.000s
I20260709 12:22:32.835652 23923 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:32.812005 23841 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.809384 23684 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.880077 23800 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:32.816334 23677 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:22:32.896132 23467 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:32.884900 23658 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.815866 23757 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:32.892211 23915 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 12:22:32.900092 23737 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.002s
I20260709 12:22:32.811182 23807 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.896462 23882 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:32.827162 23765 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.956734 23765 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.927116 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:32.957679 23765 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.958223 23765 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.959344 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:32.960202 23750 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:32.960843 23750 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:32.893244 23648 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.962567 23765 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:32.935587 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 119304us
I20260709 12:22:32.933739 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:32.936707 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.967453 23467 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:32.967734 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.967859 23467 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.968326 23467 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:32.907802 23912 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.968791 23912 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:32.975498 23913 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:22:32.976081 23763 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.976650 23763 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.977792 23876 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:32.934967 23845 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:22:32.979439 23763 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.979249 23758 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.981714 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:32.983769 23734 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:32.984788 23684 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:32.937482 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129546us
I20260709 12:22:32.859477 23925 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:22:32.912047 23836 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:32.832691 23761 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:32.931758 23735 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.986501 23841 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:32.942700 23927 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:32.987308 23927 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:32.899361 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 139121us
I20260709 12:22:32.988672 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:32.990384 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:32.943251 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.940194 23849 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.947754 20732 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:32.948033 23873 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.004268 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:32.936764 23916 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:32.937685 23681 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.004935 21432 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:33.017563 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:32.985956 23912 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.018384 23916 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.023788 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:33.027652 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.028502 20883 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:33.028590 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.032927 20877 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:22:33.029222 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.012452 23734 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:33.037139 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.038471 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.038689 23681 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.039361 21430 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:33.041070 23658 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.044137 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.044927 20728 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 12:22:33.045459 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.045565 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.046680 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:33.047470 20728 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:33.048303 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:33.049471 20728 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:33.050482 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.050434 23649 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:33.051587 20727 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:33.052482 23649 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.052785 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.052995 23649 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:33.053501 23649 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.055249 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.053603 20727 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:33.055588 23932 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:33.056748 23932 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:32.990087 23888 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.058015 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.059026 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.046705 21432 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:33.006227 23873 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.063468 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:33.064584 20774 connection.cc:256] RPC call timeout handler was delayed by -1.1951531089998753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
I20260709 12:22:33.065559 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.065682 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.074137 20877 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:32.943113 23887 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.021279 23655 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:33.015712 23763 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:33.074064 20803 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:33.074229 20806 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:33.067771 23762 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:33.082650 20879 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:33.065568 23936 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:22:33.075901 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.085741 20805 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:33.057816 23916 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:33.085333 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:33.073032 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:33.074708 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.063092 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.088203 20801 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:33.045086 23658 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.087797 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:33.061054 23734 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:33.071581 23925 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:33.094383 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:33.090847 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.092813 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:33.095878 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.060770 23896 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:22:33.088629 23916 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:33.116300 23916 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.096741 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.089423 23933 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.201s
I20260709 12:22:33.103513 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.103778 23734 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.082760 23762 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:33.094391 23925 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.106263 23888 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:33.115316 23912 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.127833 20806 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:33.106302 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:33.105126 23832 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:33.134377 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:33.135866 23757 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.158793 23757 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.162572 23757 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.156378 23740 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.163700 23888 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:33.163704 23740 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.167174 23927 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.167992 23762 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.169847 23762 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.170246 20880 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 12:22:33.169896 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63667.5us
I20260709 12:22:33.173883 23740 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:33.135605 23735 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:33.136492 23896 consensus_queue.cc:1243] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:33.088469 23763 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.160173 23739 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.186183 23888 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:33.160023 23876 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:33.193171 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.157945 23937 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.137615 23925 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:33.200541 23927 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:33.202739 23906 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.694s	user 0.003s	sys 0.003s
I20260709 12:22:33.157482 23467 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.212464 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.210922 23937 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.215826 21432 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:33.193756 23887 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.196674 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:33.222301 20806 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.206514 23882 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.196764 23875 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.235847 20809 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:33.236156 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.236861 21432 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:33.255436 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.256438 20809 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:33.256726 20806 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:33.258956 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:33.264225 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 132249us
I20260709 12:22:33.265090 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.265091 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:33.266340 21432 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:33.234931 23916 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.253698 23927 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.273159 23927 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.274497 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.275122 20727 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.275609 23927 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:33.255769 23931 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.255410 23740 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.268221 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.287846 20735 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.288003 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:33.268961 23467 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.268926 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.269300 23937 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:33.269541 23865 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:33.270170 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.296139 20732 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.270524 23746 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:33.239492 23882 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.272217 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.272670 23916 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.236225 23875 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.277697 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.282426 23649 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.282832 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.283818 23834 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:33.296626 23782 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:33.281448 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.279496 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.310317 20809 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:33.299959 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98632.8us
I20260709 12:22:33.304242 20728 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:33.305383 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:33.305979 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1726) took 3057 ms (client timeout 1769 ms). Trace:
I20260709 12:22:33.306939 20729 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:33.311968 20875 rpcz_store.cc:269] 0709 12:22:30.248312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.300039 (+ 51727us) service_pool.cc:224] Handling call
0709 12:22:33.305766 (+3005727us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54003f98d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:33.305906 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358219648}
I20260709 12:22:33.307263 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:33.286036 21435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 854) took 1279 ms (client timeout 1574 ms). Trace:
W20260709 12:22:33.312973 21435 rpcz_store.cc:269] 0709 12:22:32.006896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.008709 (+  1813us) service_pool.cc:224] Handling call
0709 12:22:33.285859 (+1277150us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:33.285975 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179397376}
I20260709 12:22:33.313362 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:33.298470 23746 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:33.313011 21432 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:33.299655 23865 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.316495 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 521) took 1290 ms. Trace:
I20260709 12:22:33.296254 23740 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.317322 21432 rpcz_store.cc:276] 0709 12:22:32.026033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.298015 (+1271982us) service_pool.cc:224] Handling call
0709 12:22:33.316437 (+ 18422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:33.328068 20735 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:33.328671 23815 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.329555 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:33.329952 23904 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.903s	user 0.001s	sys 0.000s
I20260709 12:22:33.330587 23687 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:33.331157 23687 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.332403 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:33.333019 23649 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.333254 21429 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:33.333750 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 713) took 1157 ms. Trace:
I20260709 12:22:33.334038 21429 rpcz_store.cc:276] 0709 12:22:32.176057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.332130 (+1156073us) service_pool.cc:224] Handling call
0709 12:22:33.333694 (+  1564us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:33.335129 20727 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:33.332052 23687 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.336158 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.336866 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.340372 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.339416 20879 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:33.341063 21435 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:33.341455 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.342065 20879 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:33.342981 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.344242 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:33.346544 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:33.347553 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.318209 23896 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:33.348444 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.349335 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:33.352121 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.353256 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:33.353426 20876 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:33.352634 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.354931 21429 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:33.352702 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:33.349862 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.356153 21432 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:33.356849 23739 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:33.357017 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.357295 23739 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.357649 23739 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:33.357764 20883 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:33.358349 23740 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:33.361207 23772 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:22:33.362224 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.362831 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.367642 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.370573 21435 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:33.369650 23772 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:33.373106 20729 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:33.374151 20806 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:33.374436 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:33.376515 23875 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.352389 23871 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.351095 23671 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:33.380035 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.380659 21429 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:33.381064 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.381352 20732 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
W20260709 12:22:33.347908 23835 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.007s	sys 0.003s
I20260709 12:22:33.382656 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:33.352623 23896 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.384286 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.385892 20876 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:33.361660 23815 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:33.387115 23896 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:33.382530 23571 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:33.389717 23571 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.390020 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.390831 20876 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:22:33.390627 23571 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.391664 23845 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:33.296404 23931 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:33.382447 23835 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.008s	sys 0.003s
I20260709 12:22:33.392068 23845 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.395521 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.394141 23772 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.397670 23931 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:33.388666 23815 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:33.398727 23755 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.398355 23674 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.400008 23763 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.400767 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 513) took 1040 ms. Trace:
I20260709 12:22:33.400842 23882 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.402207 23663 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:33.405663 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 518) took 1380 ms. Trace:
I20260709 12:22:33.410606 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:33.384434 23875 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:33.416018 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:33.418702 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.469458 20732 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:33.470397 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.470784 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:33.362126 23746 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.472127 20875 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:33.471993 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.473043 21435 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:33.415671 23815 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:33.473392 21432 rpcz_store.cc:276] 0709 12:22:32.025282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.385885 (+1360603us) service_pool.cc:224] Handling call
0709 12:22:33.405155 (+ 19270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:33.405337 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:33.405363 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:33.405419 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:33.405451 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.405506 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:33.405605 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:33.474414 20881 rpcz_store.cc:276] 0709 12:22:32.360885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.361047 (+   162us) service_pool.cc:224] Handling call
0709 12:22:33.400578 (+1039531us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b540005b254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:33.400705 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564143488}
I20260709 12:22:33.418329 21429 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:22:33.479737 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 520) took 1454 ms. Trace:
I20260709 12:22:33.480129 21429 rpcz_store.cc:276] 0709 12:22:32.025781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.410690 (+1384909us) service_pool.cc:224] Handling call
0709 12:22:33.418005 (+  7315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:33.479495 (+ 61490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.479684 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:33.468724 23617 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:33.480942 23617 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.478050 20807 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:33.482723 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:33.484108 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.491633 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:31.802729 20802 rpcz_store.cc:276] 0709 12:22:30.680131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.480810 (+800679us) service_pool.cc:224] Handling call
0709 12:22:31.481175 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.481427 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:31.506502 (+ 25075us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.801500 (+294998us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:31.801573 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:31.801645 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.801766 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.801999 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28807,"prepare.queue_time_us":24964,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":17958,"thread_start_us":279493,"threads_started":1}
I20260709 12:22:33.496858 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.483932 23755 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:33.500260 23755 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.500561 21432 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:33.500687 23896 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.501273 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 715) took 1127 ms. Trace:
I20260709 12:22:33.501609 21432 rpcz_store.cc:276] 0709 12:22:32.374167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.478838 (+1104671us) service_pool.cc:224] Handling call
0709 12:22:33.500245 (+ 21407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:33.501032 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.501220 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:33.502837 23896 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.500617 23755 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:33.505648 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.498505 20802 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:33.507761 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:33.508857 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.511619 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:33.502616 23617 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.498898 23439 raft_consensus.cc:2764] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:22:33.517767 23746 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:33.527786 23945 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 12:22:33.437872 23920 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:33.532555 23920 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.459882 23845 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.533754 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.534335 20732 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:33.534255 23845 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.510353 21435 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:22:33.543627 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 710) took 1418 ms. Trace:
I20260709 12:22:33.544070 21435 rpcz_store.cc:276] 0709 12:22:32.125019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.478204 (+1353185us) service_pool.cc:224] Handling call
0709 12:22:33.510144 (+ 31940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:33.543250 (+ 33106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.543552 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:33.542065 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:33.550630 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:33.548413 20732 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:33.557572 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:33.505648 23875 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.570783 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.571915 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.572410 20804 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:33.573096 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:33.573407 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.573292 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:33.553710 23439 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.499899 23819 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.584761 23886 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:33.585247 23886 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.585580 23886 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.580580 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1876) took 1128 ms. Trace:
I20260709 12:22:33.411913 23865 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.569379 23876 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:33.595023 23875 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.549891 23925 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.591032 20884 rpcz_store.cc:276] 0709 12:22:32.451880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.452353 (+   473us) service_pool.cc:224] Handling call
0709 12:22:32.452642 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:32.452862 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:33.387143 (+934281us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b18003bea80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf
0709 12:22:33.387324 (+   181us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.565287 (+177963us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.565560 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.565746 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:33.565802 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:33.565866 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.565954 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:33.580489 (+ 14535us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196163,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":175,"spinlock_wait_cycles":70958592,"thread_start_us":370878,"threads_started":2}
I20260709 12:22:33.592926 23883 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:33.620435 20805 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:33.626056 20804 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:33.628183 23674 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.628727 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:33.630563 23834 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:33.630761 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.631524 20804 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:33.606482 23876 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.614825 23883 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.612950 23865 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.631107 23834 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:33.645462 23834 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:33.649343 20776 connection.cc:256] RPC call timeout handler was delayed by -1.8481945770004131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 12:22:33.651515 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:33.652243 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:33.652748 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:33.653429 20776 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.574s (SENT)
I20260709 12:22:33.661067 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.662087 20804 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:33.622802 23755 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:33.665073 23871 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:33.679904 23920 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.684002 23876 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.719084 23925 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.717551 23886 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:33.722636 23921 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.901s	user 0.000s	sys 0.005s
I20260709 12:22:33.723165 23886 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:33.754463 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.759298 20732 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:33.759724 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.761271 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
W20260709 12:22:33.655048 20776 connection.cc:256] RPC call timeout handler was delayed by -1.6924476359999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32810000000000006s for a response
I20260709 12:22:33.763470 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:33.763825 20776 connection.cc:256] RPC call timeout handler was delayed by -1.0255945769999926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2646s for a response
I20260709 12:22:33.765050 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.765947 20804 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:33.769584 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.762907 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.772058 20729 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:33.767140 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:33.776863 20804 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:33.770777 20803 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:33.783317 20803 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:33.780997 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 515) took 1341 ms. Trace:
I20260709 12:22:33.792280 20878 rpcz_store.cc:276] 0709 12:22:32.439498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.454241 (+ 14743us) service_pool.cc:224] Handling call
0709 12:22:32.454560 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:32.461447 (+  6887us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:32.461474 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:32.467361 (+  5887us) log.cc:844] Serialized 26 byte log entry
0709 12:22:32.753693 (+286332us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:32.753741 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.753815 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:33.722054 (+968239us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:33.780283 (+ 58229us) raft_consensus.cc:1685] finished
0709 12:22:33.780325 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:33.780770 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8277,"raft.queue_time_us":7109,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8374,"spinlock_wait_cycles":17720064,"thread_start_us":254056,"threads_started":1,"wal-append.queue_time_us":332928}
I20260709 12:22:33.800693 23571 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:33.801774 23571 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.804769 23687 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:33.806468 23687 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:33.806877 23687 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:33.808291 23687 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.810066 23687 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.684801 23739 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.818306 23739 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.821749 23755 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.842029 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.843386 20727 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:33.843353 23571 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.877492 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:33.880379 20803 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:33.880021 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 89027.6us
I20260709 12:22:33.904172 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:33.908335 20806 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:22:33.921304 23925 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:33.926220 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:33.927524 23925 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:33.927985 23925 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.928375 23925 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:33.937860 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.940415 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 850) took 1934 ms. Trace:
I20260709 12:22:33.941286 23922 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:33.942167 23922 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:33.942970 21426 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 853) took 1936 ms (client timeout 1975 ms). Trace:
W20260709 12:22:33.943503 21426 rpcz_store.cc:269] 0709 12:22:32.006641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.506315 (+499674us) service_pool.cc:224] Handling call
0709 12:22:33.942564 (+1436249us) spinlock_profiling.cc:243] Waited 64.8 ms on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:33.942774 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149132032}
I20260709 12:22:33.947057 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 722) took 1174 ms. Trace:
I20260709 12:22:33.947589 21430 rpcz_store.cc:276] 0709 12:22:32.772877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.835096 (+ 62219us) service_pool.cc:224] Handling call
0709 12:22:33.946604 (+1111508us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:33.946841 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320028032}
W20260709 12:22:33.949698 21431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 515) took 2016 ms (client timeout 1685 ms). Trace:
W20260709 12:22:33.950217 21431 rpcz_store.cc:269] 0709 12:22:31.932770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.982550 (+ 49780us) service_pool.cc:224] Handling call
0709 12:22:33.949171 (+1966621us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400477a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:33.949448 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132912384}
W20260709 12:22:33.951248 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:33.952212 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.952292 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 851) took 1946 ms. Trace:
I20260709 12:22:33.952862 21433 rpcz_store.cc:276] 0709 12:22:32.006083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.010964 (+  4881us) service_pool.cc:224] Handling call
0709 12:22:32.011433 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:32.683730 (+672297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.951640 (+1267910us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b540029ca54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:33.952065 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442898304}
I20260709 12:22:33.955329 23925 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:33.944783 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 249575us
I20260709 12:22:33.960498 23925 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:33.962239 23815 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:33.963599 20803 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:33.940956 21434 rpcz_store.cc:276] 0709 12:22:32.005785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.010002 (+  4217us) service_pool.cc:224] Handling call
0709 12:22:32.010385 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:32.175785 (+165400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.939548 (+1763763us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b54004da420. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca2a02 00007f9e1ec9edd6 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:33.940155 (+   607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33006464}
I20260709 12:22:33.983793 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:33.984678 20729 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.985527 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.966833 23815 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:33.989236 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:33.990387 20884 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:33.994901 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.999657 20727 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
W20260709 12:22:34.005578 23961 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:22:34.005733 20884 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:34.007395 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.008152 21430 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:34.008674 23883 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.003751 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.009332 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.012519 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.015482 21433 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.020313 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.022050 23818 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.023432 21426 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.025144 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.026319 20876 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:34.029297 23746 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.030093 23746 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.034009 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:34.036026 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.036988 23922 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.038069 23865 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:34.038772 23865 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.039510 23752 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.040063 23865 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.042152 20808 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:34.042460 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:34.043582 23875 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:34.038926 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:34.045739 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 518) took 1578 ms. Trace:
I20260709 12:22:34.046221 20882 rpcz_store.cc:276] 0709 12:22:32.467152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.514535 (+ 47383us) service_pool.cc:224] Handling call
0709 12:22:34.045235 (+1530700us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540022f6d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:34.045499 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414616960}
I20260709 12:22:34.047642 23834 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:34.048059 23834 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.048417 23834 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.048468 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1739) took 1018 ms. Trace:
I20260709 12:22:34.049028 20880 rpcz_store.cc:276] 0709 12:22:33.029947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.030078 (+   131us) service_pool.cc:224] Handling call
0709 12:22:33.031066 (+   988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:33.170829 (+139763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.047880 (+877051us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400059e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:34.048247 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319263616}
I20260709 12:22:34.040244 23752 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.063246 20803 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:34.064177 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.063097 20808 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:34.077162 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:34.078116 21428 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:34.078915 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.079541 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.080470 21435 raft_consensus.cc:2376] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7. Re-sending same reply.
I20260709 12:22:34.080950 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.081756 21432 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.083410 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.083586 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.083801 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:34.086241 21430 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.088640 23266 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.089696 23878 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.090370 23878 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.091423 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:34.091511 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.092238 21429 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.092458 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:34.092535 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.093489 20879 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 4: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:34.095029 23739 raft_consensus.cc:2764] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:22:34.099243 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.103253 23886 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:34.103677 23886 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:34.104100 23886 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.106189 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:34.107509 23896 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:34.107676 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60377.5us
I20260709 12:22:34.113306 23772 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:34.116927 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.117636 23896 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.119375 20883 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:34.116642 20881 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:22:34.116623 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.122205 20729 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:34.127614 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.128928 20880 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:34.130177 23617 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.130653 20879 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:34.137384 21433 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:34.139734 21432 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:34.150154 23752 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.155417 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.156209 20883 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:34.157226 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.158773 23524 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.159556 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.160144 21431 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:22:34.160508 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.161401 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.164106 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.190981 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:34.187067 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144879us
I20260709 12:22:34.164814 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.167522 20884 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:34.181377 20883 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:34.164726 20803 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:22:34.150987 23752 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:34.184635 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.206351 21426 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:34.206956 21432 raft_consensus.cc:2410] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:34.205613 21430 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:34.198305 20803 pending_rounds.cc:85] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 3
I20260709 12:22:34.212141 20881 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:34.179128 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:34.132190 23746 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.172355 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.217185 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.207665 20803 pending_rounds.cc:107] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:22:34.223915 21429 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:34.224252 20803 raft_consensus.cc:2889] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:34.186473 23896 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.234670 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.205950 23752 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:34.254626 23895 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.671s	user 0.010s	sys 0.043s
W20260709 12:22:34.255230 23895 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.681s	user 0.010s	sys 0.043s
I20260709 12:22:34.255119 23896 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:34.255926 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:34.256119 23895 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.257544 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1841) took 1171 ms. Trace:
I20260709 12:22:34.257377 23895 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.257933 20805 rpcz_store.cc:276] 0709 12:22:33.085846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.095610 (+  9764us) service_pool.cc:224] Handling call
0709 12:22:34.257159 (+1161549us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54002aadd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:34.257355 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158973312}
I20260709 12:22:34.245088 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:34.240551 23752 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.280051 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.261067 20876 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:34.268419 20727 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
W20260709 12:22:34.277453 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 105426us
I20260709 12:22:34.260716 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:34.271229 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.264035 20875 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:34.259161 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:34.285677 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.286639 20803 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 4.
W20260709 12:22:34.267396 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 134861us
I20260709 12:22:34.267624 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:34.267411 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:34.293560 20731 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.266889 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.294871 20732 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:34.294821 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 145306us
I20260709 12:22:34.296293 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:34.297048 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.277122 23885 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:34.278031 23266 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:34.255952 23782 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:34.297111 23752 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.310194 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.260176 23967 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:34.259774 23763 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.294538 23925 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:34.313016 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52420.1us
I20260709 12:22:34.302117 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.264891 23687 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:22:34.306923 23885 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.322700 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.313774 23925 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.340921 20809 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.323755 23782 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.343290 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:34.343796 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45807.9us
I20260709 12:22:34.344599 23782 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:34.345181 23782 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.345620 23782 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.324072 23967 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:34.348008 20728 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:34.348116 23967 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.335952 23925 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.350009 20734 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:22:34.324620 23885 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.354720 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.363274 20727 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.357668 20805 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 4.
I20260709 12:22:34.384222 23879 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.384956 23879 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.386389 23752 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.387220 23971 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:34.387619 20802 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:22:34.387766 23971 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.387918 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.388211 23971 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.387831 23865 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:34.388590 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60866.9us
I20260709 12:22:34.389139 23865 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:34.389497 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.389549 23865 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.391069 23766 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:34.391600 23766 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.391285 23865 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.392009 23766 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.387627 23873 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:22:34.392779 23865 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.392967 23873 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:34.393471 23873 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.395490 23755 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:34.396020 23755 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.396490 23782 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:34.396685 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:34.397028 23746 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:34.397473 23746 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.397634 23687 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:34.397850 23746 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:34.399358 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95756.6us
I20260709 12:22:34.400213 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.322554 23266 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.374770 20803 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:34.402063 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.394464 23873 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.404708 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.406867 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.407699 20809 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.408542 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.410259 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2036) took 1139 ms. Trace:
I20260709 12:22:34.410667 20802 rpcz_store.cc:276] 0709 12:22:33.270916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.550292 (+279376us) service_pool.cc:224] Handling call
0709 12:22:34.410037 (+859745us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b54003c6c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:34.410198 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843237632}
I20260709 12:22:34.411718 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.412744 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.413534 20802 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.413448 20809 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:34.414090 23971 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.414719 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.415354 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.415504 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:34.415670 23782 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:34.415944 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:34.416221 20734 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.417580 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:34.416157 23782 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.419606 23782 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.420832 23849 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:34.421901 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.422842 20882 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:34.424186 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.426054 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.426651 20882 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.416641 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.427788 20809 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:34.428746 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.429356 20810 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:22:34.429729 20810 pending_rounds.cc:85] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:22:34.429946 20810 pending_rounds.cc:107] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:22:34.430164 20810 raft_consensus.cc:2889] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:34.353525 23763 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.414672 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.434185 20731 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
W20260709 12:22:34.442061 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50975.8us
I20260709 12:22:34.440340 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.447747 20884 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:22:34.449273 20727 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:34.452006 20809 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:34.455615 20805 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.451689 20882 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
W20260709 12:22:34.457211 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 161161us
I20260709 12:22:34.456326 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.466960 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.448585 23752 raft_consensus.cc:2764] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:22:34.469417 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1827) took 1870 ms. Trace:
I20260709 12:22:34.472743 20810 rpcz_store.cc:276] 0709 12:22:32.594571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.594746 (+   175us) service_pool.cc:224] Handling call
0709 12:22:32.595103 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:34.430633 (+1835530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.431511 (+   878us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54003d6dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:34.464957 (+ 33446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4217009920}
W20260709 12:22:34.464421 23770 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.660s	user 0.014s	sys 0.033s
I20260709 12:22:34.482362 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.484369 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.486506 20802 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
W20260709 12:22:34.478578 23770 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.840s	user 0.015s	sys 0.033s
I20260709 12:22:34.451314 23834 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.486653 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:34.500525 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:34.472127 23973 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.455173 23266 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.506640 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.511405 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.497510 23770 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:34.516803 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.517335 20879 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:34.519368 23971 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.521785 23834 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:34.521999 23765 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:34.470932 23886 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.523465 23770 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.527835 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 453) took 1243 ms. Trace:
I20260709 12:22:34.528287 20806 rpcz_store.cc:276] 0709 12:22:33.284207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:34.044246 (+760039us) service_pool.cc:224] Handling call
0709 12:22:34.201405 (+157159us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:34.214672 (+ 13267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:34.242988 (+ 28316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:34.243030 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:34.275483 (+ 32453us) log.cc:844] Serialized 26 byte log entry
0709 12:22:34.431736 (+156253us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:34.431774 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.431842 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.527493 (+ 95651us) raft_consensus.cc:1685] finished
0709 12:22:34.527511 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.527748 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":151,"raft.queue_time_us":7759,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":14664,"spinlock_wait_cycles":25227008,"thread_start_us":155861,"threads_started":1,"wal-append.queue_time_us":236738}
I20260709 12:22:34.505677 23524 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.533214 20731 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:34.533850 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.549676 23524 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:34.551106 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.535609 23383 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:22:34.552543 23524 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.554781 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.555900 20875 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:22:34.556772 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.557886 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:34.536260 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:34.538300 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 74476.6us
I20260709 12:22:34.539753 23266 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.540326 20730 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.540867 23886 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.541867 20732 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:34.542564 20882 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:34.497408 23876 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:22:34.543349 23765 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:34.547222 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.512141 23912 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.568336 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.513684 23873 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.577564 20734 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:34.514899 23879 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:34.580943 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:34.582794 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.583040 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.550414 23755 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.570034 23524 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:34.589223 23978 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:34.589751 23978 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:34.590207 23978 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.590939 23818 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.577682 23782 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.591572 23818 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.592878 20884 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
I20260709 12:22:34.593684 23849 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:34.594390 23849 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:34.591421 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 173962us
I20260709 12:22:34.596511 23763 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:34.597126 23763 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:34.598371 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.570590 23973 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:34.594457 23818 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.604159 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.579480 23765 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.604019 23979 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:34.605221 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.608410 20732 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:34.615093 20731 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:34.617141 23834 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.584237 23912 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.618395 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.619123 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:34.620594 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:34.604462 23922 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:22:34.608006 23973 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:34.623556 23973 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.611776 23979 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:34.628990 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:34.630867 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 81920.6us
W20260709 12:22:34.631544 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73840.5us
I20260709 12:22:34.613492 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:34.612429 23524 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.640228 23524 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.641968 23819 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:22:34.611860 23765 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.643489 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:34.644101 20727 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.647787 20728 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:34.647986 23922 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:34.648361 23922 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:34.649400 23922 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.634097 23818 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:22:34.650691 23922 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.651047 23818 consensus_queue.cc:1243] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:22:34.601720 23849 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.640259 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:34.658907 20727 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:34.665725 23978 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.672497 20731 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:34.632643 23879 raft_consensus.cc:2764] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:34.628826 23865 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:34.680783 20877 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:34.680737 23895 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:34.681643 23895 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:34.682354 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53438us
I20260709 12:22:34.684392 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.685274 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:34.686084 20875 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:34.688557 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:34.689682 23876 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:34.690146 23876 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:34.690585 23876 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:34.687717 23895 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.696283 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:34.680145 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 57565.8us
I20260709 12:22:34.675782 23755 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:34.704010 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 200192us
W20260709 12:22:34.706146 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 92776.9us
I20260709 12:22:34.695374 23849 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:34.711097 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:34.709883 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.652309 23746 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.676726 23971 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:22:34.703244 23818 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:34.395341 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:34.706188 23770 raft_consensus.cc:2764] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:22:34.671756 23885 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.698591 23879 raft_consensus.cc:2764] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:34.723825 23849 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.734091 23849 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.733194 20726 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:34.741466 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57262.2us
I20260709 12:22:34.746413 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:34.724452 23818 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.753856 23978 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.754802 23746 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.755004 23765 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:34.759303 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 119130us
I20260709 12:22:34.761430 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:34.760535 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:34.765905 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45145.7us
I20260709 12:22:34.769153 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.769809 20727 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:34.770586 23865 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:34.771265 23865 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:34.771559 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.771883 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:34.772222 23865 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.772969 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.773545 20733 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:34.775692 20728 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:34.776276 23885 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:34.776942 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:34.777891 23878 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:34.747224 23971 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:34.778988 23971 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:34.779884 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:34.780057 23971 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.781401 23772 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:34.761015 23818 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.784746 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1845) took 1651 ms. Trace:
I20260709 12:22:34.785145 20804 rpcz_store.cc:276] 0709 12:22:33.133740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.777608 (+643868us) service_pool.cc:224] Handling call
0709 12:22:33.838264 (+ 60656us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:33.843768 (+  5504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:33.843816 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:33.899262 (+ 55446us) log.cc:844] Serialized 52 byte log entry
0709 12:22:33.997993 (+ 98731us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:33.999315 (+  1322us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.001224 (+  1909us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.001463 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.001524 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.176635 (+175111us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b18003f8c80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:34.269264 (+ 92629us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b18003f8c80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:34.340077 (+ 70813us) raft_consensus.cc:1685] finished
0709 12:22:34.340112 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.784573 (+444461us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b54002b61d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:34.784682 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1061,"prepare.queue_time_us":5684,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":68,"raft.queue_time_us":6956,"raft.run_cpu_time_us":531,"raft.run_wall_time_us":13197,"spinlock_wait_cycles":244042624,"thread_start_us":76322,"threads_started":1,"wal-append.queue_time_us":108242}
I20260709 12:22:34.702983 23979 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.776948 23782 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.788122 23979 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.789678 20802 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:34.738797 23766 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.792254 20729 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:34.792549 23766 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.793370 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.793766 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.794291 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.781515 23971 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:34.763025 23982 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:22:34.777778 20732 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.795680 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.798786 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.781949 23772 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.737753 23973 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:34.768699 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:34.782070 23981 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:22:34.815883 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.770644 23873 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:34.835279 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.837291 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.843715 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:34.844141 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.843425 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:34.832589 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:34.848579 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:34.851150 20727 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:34.816210 23772 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.848340 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:34.857287 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 111006us
I20260709 12:22:34.856868 20729 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:34.853787 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 144025us
I20260709 12:22:34.843400 23687 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.830056 23782 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:34.836607 23383 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:34.777817 23925 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.830044 23895 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.843441 23766 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:34.873951 20809 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:34.874758 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.849448 23878 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:22:34.876928 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 217407us
I20260709 12:22:34.877072 20805 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:34.882341 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.884975 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:34.887171 21432 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:34.891994 20732 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:34.893172 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:34.893935 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
W20260709 12:22:34.898082 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 118327us
I20260709 12:22:34.890206 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.901217 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.902762 20806 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
I20260709 12:22:34.864140 23687 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:34.933184 20730 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:34.934239 20727 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:34.911864 20731 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:22:34.923579 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50733.8us
I20260709 12:22:34.941361 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.887975 23885 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.941265 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.949049 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:34.958868 20733 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:34.961694 20701 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:34.925292 23765 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:34.966041 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:34.972155 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 210785us
I20260709 12:22:34.905246 23996 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:34.932361 23524 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:34.994297 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:34.984419 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:34.966293 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.997778 20805 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:34.965677 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:34.965298 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 155111us
I20260709 12:22:34.999132 21435 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.001082 20809 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:34.972024 20732 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:22:34.993908 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61943.4us
I20260709 12:22:35.004243 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.004706 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:34.896929 23895 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.007169 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:35.008524 20809 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.010145 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:35.008580 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:34.967082 23931 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:35.015434 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:34.995586 23617 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.965677 23993 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.023219 20802 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:35.024812 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:35.025594 20805 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.027400 23996 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:35.028729 23996 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:35.029505 23996 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.035245 23996 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.036693 23996 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:35.037760 23996 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:35.041711 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:35.046514 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.994460 23885 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:34.971872 23383 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:22:35.015304 23931 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:35.049669 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:35.055038 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:35.055716 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:35.056087 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60575us
I20260709 12:22:35.063380 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:35.066748 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:35.068394 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 84184.9us
I20260709 12:22:35.074115 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:35.012110 23978 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:22:35.088748 20884 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:35.087980 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134964us
I20260709 12:22:35.097281 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:35.048403 23882 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:22:35.099725 20875 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:35.100764 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.100625 23925 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.089067 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.107904 20882 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:35.108311 20805 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:35.112190 21428 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:35.112609 21428 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:35.112875 23865 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:35.113452 23865 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.113427 21428 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.114552 21428 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.115679 20879 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:35.019376 23524 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.116417 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.118669 21433 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:35.122653 23987 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:35.088032 23864 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:22:35.123145 23987 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.123551 23987 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.124651 24003 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.125947 20883 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:22:35.126413 20883 pending_rounds.cc:85] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 4
I20260709 12:22:35.126739 20883 pending_rounds.cc:107] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:22:35.127002 20883 raft_consensus.cc:2889] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:35.127399 24004 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.006s
I20260709 12:22:35.127637 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.044142 23617 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.090497 23973 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:35.135118 21435 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:35.134761 20809 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:35.130141 21427 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:35.134547 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 122021us
I20260709 12:22:35.039870 23996 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.137836 20809 pending_rounds.cc:85] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 3
I20260709 12:22:35.138441 20809 pending_rounds.cc:107] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:22:35.138756 20809 raft_consensus.cc:2889] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:35.151423 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 191945us
I20260709 12:22:35.051632 23876 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.079645 23999 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:35.169965 21431 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:35.127458 23383 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:22:35.171638 21399 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
I20260709 12:22:35.175532 21399 leader_election.cc:403] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:35.174316 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:35.128219 23931 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.188709 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.152351 23766 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.178613 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:35.152103 23895 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.196744 24015 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.196990 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1743) took 2118 ms. Trace:
W20260709 12:22:35.185629 24016 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:22:35.194973 20810 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:35.195145 23978 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.195005 23982 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:35.196017 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53649.2us
I20260709 12:22:35.196393 23383 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:35.194783 23873 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:35.194815 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2045) took 1888 ms. Trace:
I20260709 12:22:35.188097 23878 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.185735 23649 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:35.186275 24001 raft_consensus.cc:2764] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:22:35.223908 23383 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:35.224861 23383 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.228786 23982 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.219255 23895 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:22:35.231648 23878 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.210659 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:35.186786 24007 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.197428 20877 rpcz_store.cc:276] 0709 12:22:33.078193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.078372 (+   179us) service_pool.cc:224] Handling call
0709 12:22:33.080248 (+  1876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:34.681477 (+1601229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:35.196703 (+515226us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:35.196921 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3673425792}
I20260709 12:22:35.187419 23687 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.222479 23266 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:35.243489 23266 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:35.222935 24014 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.244479 24014 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.244596 23266 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.246169 23266 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.198215 23999 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:35.263200 24007 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.265806 23617 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.270417 23999 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:35.280701 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1842) took 2191 ms (client timeout 1837 ms). Trace:
W20260709 12:22:35.281299 20801 rpcz_store.cc:269] 0709 12:22:33.088723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.090609 (+  1886us) service_pool.cc:224] Handling call
0709 12:22:35.280226 (+2189617us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54002a99d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:35.280450 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3626712320}
I20260709 12:22:35.297092 23865 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.321880 23931 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.326642 23864 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.327303 23617 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.328094 23617 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.329618 23687 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.330780 23865 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:35.345242 21431 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:22:35.396409 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60783.5us
W20260709 12:22:35.398351 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 224075us
I20260709 12:22:35.413686 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:35.414705 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:35.397259 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 222783us
I20260709 12:22:35.424077 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:35.425297 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 297783us
W20260709 12:22:35.426944 23755 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.717s	user 0.005s	sys 0.020s
W20260709 12:22:35.429349 24006 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.198s	user 0.000s	sys 0.003s
I20260709 12:22:35.430330 20884 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:35.431170 23524 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.377591 20632 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:35.431694 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 427499us
I20260709 12:22:35.432518 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:35.433230 24018 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.187s	user 0.002s	sys 0.000s
I20260709 12:22:35.432773 20632 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 5, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:35.434124 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:35.434989 21429 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:35.435352 23765 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.435441 21429 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:35.435989 23765 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.436483 21429 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.437625 21429 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.438192 23765 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.434350 20632 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:35.217497 20807 rpcz_store.cc:276] 0709 12:22:33.285725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:34.052773 (+767048us) service_pool.cc:224] Handling call
0709 12:22:34.165625 (+112852us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:34.415051 (+249426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:34.446694 (+ 31643us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:34.474040 (+ 27346us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.514146 (+ 40106us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.514461 (+   315us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:34.514520 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:34.514575 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.514649 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.949948 (+435299us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b18003fe200. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:35.174201 (+224253us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54003c6c50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:35.174296 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24049,"prepare.queue_time_us":9197,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":97,"raft.queue_time_us":31780,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":23795,"spinlock_wait_cycles":582086016,"thread_start_us":19794,"threads_started":1,"wal-append.queue_time_us":139886}
I20260709 12:22:35.443609 23912 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.442386 20632 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 2 to 3, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:22:35.445405 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45937.8us
W20260709 12:22:35.395390 23849 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.002s	sys 0.051s
I20260709 12:22:35.448694 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.450865 20804 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:35.330283 23687 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.458443 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.462951 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:35.445201 23883 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.464565 23738 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.457509 23895 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:22:35.465219 23738 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:35.449465 23849 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.002s	sys 0.052s
I20260709 12:22:35.467332 23752 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:35.469221 20633 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 270) took 22121 ms (client timeout 15000 ms). Trace:
I20260709 12:22:35.469478 23687 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.470013 24015 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
W20260709 12:22:35.473187 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.473402 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.473985 21432 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:35.475363 23832 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:35.475529 23763 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:35.475953 23832 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:35.476480 23832 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.476692 23763 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.477451 23763 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:35.477798 23763 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:35.479485 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:35.480712 24026 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:22:35.483816 23885 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.486141 24007 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:35.467324 23922 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.816s	user 0.017s	sys 0.004s
I20260709 12:22:35.487426 23865 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.422111 23992 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.487346 24014 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:35.449445 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 163418us
I20260709 12:22:35.488507 23992 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.475370 23978 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.491634 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:35.493827 23524 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:35.492089 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:35.460357 20632 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 4, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 4 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:35.494624 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47766.8us
I20260709 12:22:35.501251 21428 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:35.502370 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.502615 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:35.411085 23864 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:22:35.503043 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:35.538189 20876 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:35.538618 21432 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
W20260709 12:22:35.513814 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.478541 23763 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.490003 24012 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:35.525699 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50530.9us
I20260709 12:22:35.432538 23982 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.526016 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:35.532353 20875 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:35.464144 23818 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.682s	user 0.015s	sys 0.003s
I20260709 12:22:35.537514 21429 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
W20260709 12:22:35.537884 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43487.2us
I20260709 12:22:35.503620 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.572151 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.573915 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.574882 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:35.575497 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 86959.9us
W20260709 12:22:35.548633 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56615.8us
I20260709 12:22:35.575783 20878 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:22:35.577363 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.571761 21429 pending_rounds.cc:85] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 6
I20260709 12:22:35.579067 21429 pending_rounds.cc:107] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:22:35.579380 21429 raft_consensus.cc:2889] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:35.574548 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.580222 20881 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:35.582226 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.503522 23978 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:35.585064 20632 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 3, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:35.593417 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:35.593895 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:35.570482 23815 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:22:35.494277 23878 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:35.593086 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 101518us
W20260709 12:22:35.594684 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 97203.4us
I20260709 12:22:35.601166 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:35.594723 21428 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:35.603634 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:35.603760 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.604374 21432 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:35.571087 24012 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.573622 23766 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:35.502424 23873 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:22:35.607014 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.611244 20881 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:35.570760 23993 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:35.609042 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.494591 23755 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 12:22:35.634763 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:35.609254 20632 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 4, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 4 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:35.630460 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:35.635017 20879 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:35.616386 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.590782 23738 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:35.675457 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44683.9us
I20260709 12:22:35.573784 23819 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 12:22:35.674037 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.539014 23524 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.606936 23882 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:35.604074 23925 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.084s
W20260709 12:22:35.605677 23971 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.811s	user 0.011s	sys 0.006s
I20260709 12:22:35.599172 23865 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:35.622936 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:35.694690 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83761.9us
I20260709 12:22:35.695865 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:35.701372 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.702358 20632 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:35.583266 23818 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:35.692344 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62447.2us
I20260709 12:22:35.713280 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:35.530311 23987 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.771111 20881 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:35.780906 23617 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.781684 23876 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.603374 23383 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 12:22:35.782688 23524 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.603257 23978 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:35.783849 23772 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.913s	user 0.016s	sys 0.009s
I20260709 12:22:35.784324 23772 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.785267 20875 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:35.634423 23766 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:35.469627 20633 rpcz_store.cc:269] 0709 12:22:13.347884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.348165 (+   281us) service_pool.cc:224] Handling call
0709 12:22:35.468963 (+22120798us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:26.283259 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:26.283951 (+   692us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:26.283999 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:22:26.284017 (+    18us) tablet.cc:662] Decoding operations
0709 12:22:26.284934 (+   917us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:26.285022 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:26.285054 (+    32us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:22:26.285726 (+   672us) tablet.cc:701] Row locks acquired
0709 12:22:26.285774 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:22:26.285955 (+   181us) write_op.cc:270] Start()
0709 12:22:26.286101 (+   146us) write_op.cc:276] Timestamp: P: 1783599746285887 usec, L: 0
0709 12:22:26.286122 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:22:26.373398 (+ 87276us) log.cc:844] Serialized 2898 byte log entry
0709 12:22:27.621339 (+1247941us) spinlock_profiling.cc:243] Waited 8.46 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:28.133099 (+511760us) op_driver.cc:464] REPLICATION: finished
0709 12:22:28.595048 (+461949us) write_op.cc:301] APPLY: starting
0709 12:22:28.636264 (+ 41216us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:28.723279 (+ 87015us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:28.728296 (+  5017us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:28.814295 (+ 85999us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:28.869337 (+ 55042us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:22:28.874542 (+  5205us) write_op.cc:312] APPLY: finished
0709 12:22:28.972459 (+ 97917us) log.cc:844] Serialized 59 byte log entry
0709 12:22:28.997332 (+ 24873us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:29.008812 (+ 11480us) write_op.cc:454] Released schema lock
0709 12:22:29.051799 (+ 42987us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:29.121410 (+ 69611us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:29.124432 (+  3022us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2982624,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":317477,"apply.run_cpu_time_us":5572,"apply.run_wall_time_us":530580,"mutex_wait_us":4305,"num_ops":7,"prepare.queue_time_us":178512,"prepare.run_cpu_time_us":37592,"prepare.run_wall_time_us":401663,"raft.queue_time_us":385877,"raft.run_cpu_time_us":9144,"raft.run_wall_time_us":202631,"replication_time_us":1812584,"spinlock_wait_cycles":19447168,"thread_start_us":617725,"threads_started":4,"wal-append.queue_time_us":439125,"wal-append.run_cpu_time_us":40000,"wal-append.run_wall_time_us":3135917}]]}
W20260709 12:22:35.785897 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49113.7us
W20260709 12:22:35.736699 20850 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.565s (SENT)
I20260709 12:22:35.788653 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:35.737417 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.607205 24012 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.789455 20879 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:35.629971 23922 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:35.672294 23763 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:35.795866 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100065us
I20260709 12:22:35.737021 23925 consensus_queue.cc:1243] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:22:35.710510 23765 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:35.736672 23770 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:22:35.818619 20700 connection.cc:256] RPC call timeout handler was delayed by -1.0719536470001003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 12:22:35.805619 21430 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:35.787606 23266 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.010s
I20260709 12:22:35.774232 23912 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:35.819512 20877 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:35.821034 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 248953us
I20260709 12:22:35.784950 23978 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:35.737804 23992 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.629434 23982 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 12:22:35.823729 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:35.820482 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:35.830937 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 117739us
I20260709 12:22:35.689218 23895 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.835148 20879 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:35.836498 23880 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:35.836949 23880 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:35.837359 21429 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:35.837790 23832 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.837404 23880 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.840176 23832 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.840020 23766 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.841379 23818 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.843422 23864 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.843792 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:35.818251 23765 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.844614 23765 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.845494 21400 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
I20260709 12:22:35.846055 23772 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:35.846297 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:35.846683 23772 raft_consensus.cc:2764] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:22:35.805737 23752 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:35.777956 23993 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.847594 23752 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.848204 23752 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.848308 24005 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.218s
I20260709 12:22:35.846529 23880 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.849407 23993 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.715054 23882 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:35.864374 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66679.3us
I20260709 12:22:35.868762 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:35.849851 23979 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:35.864058 20632 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 5, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 5 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:35.874148 23880 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.857570 23912 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.853783 23617 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:35.875615 23617 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:35.877146 23912 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:35.836841 23770 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:35.884006 23770 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.885718 23752 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.887660 23772 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:35.887908 23752 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:35.888411 23973 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:35.888937 23973 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.889362 23973 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:35.891893 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.893270 23864 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:35.874229 20632 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 5, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 5 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:35.876664 23617 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:35.904222 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:35.795029 23865 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:35.911760 21426 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 7
I20260709 12:22:35.912189 21426 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:22:35.912496 21426 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:35.916548 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:35.917188 20728 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.694875 24001 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.169s
I20260709 12:22:35.928097 20632 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:35.915027 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:35.930438 21434 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.931092 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:35.905365 23882 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:35.929922 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 224927us
I20260709 12:22:35.933315 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:35.958804 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:35.934017 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:35.930156 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 329072us
I20260709 12:22:35.934894 20729 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:35.937278 20727 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:35.937562 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:35.963994 20809 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:35.965898 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44856us
I20260709 12:22:35.966441 21432 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:35.907477 23772 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:35.913420 23992 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:35.915232 23878 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:35.984184 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:35.986544 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.988042 20810 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:35.988224 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:35.992439 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:35.993513 20879 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:35.994961 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:35.995813 20876 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 5: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:35.997341 20732 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:22:35.967324 24029 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:22:35.994370 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:35.999953 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.004396 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:36.006834 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:35.907333 23978 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 12:22:36.005165 20804 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:36.019951 20733 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:35.936319 24007 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:36.022547 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:36.023595 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 216786us
I20260709 12:22:36.035568 20809 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:36.038077 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:35.987313 24015 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:36.032676 20726 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:35.977962 23992 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:35.997884 23878 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.043485 21432 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:22:36.044734 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45112.4us
I20260709 12:22:36.043907 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.052935 20810 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:36.019407 23687 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:36.057536 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:36.059553 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:36.039463 24018 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:36.062747 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.067356 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:36.059415 23266 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:36.071256 24023 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
W20260709 12:22:36.075316 24020 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.436s	user 0.001s	sys 0.000s
W20260709 12:22:36.075829 24011 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.590s	user 0.002s	sys 0.002s
W20260709 12:22:36.077319 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89174.6us
I20260709 12:22:36.014810 23993 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:22:36.055792 23878 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.086372 20733 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:36.046950 24015 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.050504 23992 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:36.101262 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.106669 21434 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:36.110682 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.120877 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.122047 20809 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:36.123253 20728 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:36.059329 23687 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:36.084188 23925 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:36.060532 24035 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:36.127539 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.130574 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:36.126400 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.111985 23979 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.141134 20728 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:36.152972 20726 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:36.141758 20804 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:36.144690 21431 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:36.146561 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.104085 24015 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.047132 24007 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.130797 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.156225 20807 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:36.144384 23687 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:36.159041 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.160200 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.160265 20803 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:36.124569 24022 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:36.147167 23832 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.229354 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.156751 23922 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.159778 23770 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:36.242087 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
W20260709 12:22:36.245999 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 85838us
I20260709 12:22:36.169785 24034 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.248977 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:36.251260 20810 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:36.251608 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:36.252208 20804 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:36.149560 23925 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.254837 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:36.251863 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:36.262588 20803 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:36.263281 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:36.265883 20801 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:22:36.266340 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 214411us
I20260709 12:22:36.156528 23979 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.148743 23772 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.249583 23988 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.278172 20801 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:22:36.270023 23782 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:36.252331 23978 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:36.265770 23971 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:22:36.290956 20810 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:36.159466 24035 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.302542 20804 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:36.303884 20802 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:36.315822 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.253669 23765 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.290784 23770 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:36.317878 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.289708 23782 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.284997 23987 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.273054 23766 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 12:22:36.335708 24042 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:22:36.336676 23925 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:36.346079 23979 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:36.347638 24039 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:22:36.353312 20734 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:36.361085 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.270751 24031 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:22:36.311163 24022 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:36.322237 23971 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:36.373152 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 271940us
W20260709 12:22:36.334978 23883 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.870s	user 0.009s	sys 0.000s
I20260709 12:22:36.261771 24034 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.326116 23999 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.354477 23978 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:36.381990 24022 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:36.328275 23782 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:36.394706 20801 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:36.306382 24012 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.352752 23865 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:36.306881 23988 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:36.441098 24036 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 12:22:36.441514 24008 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:36.442173 24008 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:36.442948 24043 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:22:36.381155 23979 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:22:36.451141 24049 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:36.451841 24049 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.454887 24049 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:36.397517 23999 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.455322 23765 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.455961 24007 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:36.457079 24046 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:22:36.458678 24056 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:22:36.464643 24035 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:36.465201 24044 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:22:36.467329 23383 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:36.467760 23383 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:36.467468 24022 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.468830 24022 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:36.468567 23383 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.469758 23383 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.470350 23882 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.398797 24055 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:36.471650 24055 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:36.472644 24055 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.473115 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:36.473735 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
W20260709 12:22:36.477288 24048 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 12:22:36.479159 24000 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:36.480182 24000 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:36.481076 24000 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.482267 24000 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:36.484418 24012 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:36.485997 23988 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.487121 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.403796 23978 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:36.488018 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:36.458570 24034 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.487986 20801 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:36.498518 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:36.462985 23973 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.000s
I20260709 12:22:36.500537 20803 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:36.504086 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:36.506145 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 142005us
I20260709 12:22:36.507540 20804 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:36.509734 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:36.511827 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71542.2us
I20260709 12:22:36.457130 23999 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:36.513458 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:36.514487 23763 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.002s
I20260709 12:22:36.517429 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 544) took 1629 ms. Trace:
I20260709 12:22:36.521324 21430 rpcz_store.cc:276] 0709 12:22:34.888789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:34.897180 (+  8391us) service_pool.cc:224] Handling call
0709 12:22:36.517241 (+1620061us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b54005f3a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:36.517368 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1982878080}
I20260709 12:22:36.522675 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.593252 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:36.593910 20804 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:36.593559 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69738.1us
W20260709 12:22:36.601153 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88585.4us
I20260709 12:22:36.604218 20803 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:36.504603 23878 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.586862 23988 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:36.661119 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.508427 23687 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.506592 23973 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.589648 23999 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:36.598762 23772 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:36.669109 24052 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.669950 24057 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:36.670424 24057 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.670792 24057 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.676066 23882 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.678196 23878 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.679569 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.680507 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.619731 23770 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:36.678279 23992 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.000s
I20260709 12:22:36.698262 23971 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:36.699517 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.700223 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:36.746059 20804 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:36.746546 20801 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:36.746677 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 118839us
I20260709 12:22:36.702196 24064 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:22:36.703032 23979 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:36.700868 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.703311 23763 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:36.703850 24032 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.601s	user 0.006s	sys 0.009s
I20260709 12:22:36.709146 23978 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.713934 23687 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.715459 23999 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.704766 24015 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.784710 24015 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:36.747557 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:36.786825 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.747933 20803 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:36.745976 20807 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:36.689965 24039 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:22:36.750684 20806 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:36.754895 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:36.764169 23266 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:36.764802 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:36.764907 20735 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:36.766517 20729 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:36.660202 23782 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:36.773424 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:36.791596 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.791718 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.782250 23999 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:36.793036 20732 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.793323 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.796151 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:36.796702 20880 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.768932 23763 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:36.800279 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.803431 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:36.750006 23992 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.805528 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.809692 23880 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:36.810611 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.811259 23880 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:36.811928 23880 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:36.789098 23819 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.823426 24051 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
W20260709 12:22:36.832988 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.798254 24001 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:36.799328 24064 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:36.804922 23763 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:36.835681 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 555) took 1044 ms. Trace:
I20260709 12:22:36.802439 23999 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:36.835872 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88429.4us
I20260709 12:22:36.807250 24015 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:36.797159 24066 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:22:36.839854 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.838764 21428 rpcz_store.cc:276] 0709 12:22:35.791419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.791582 (+   163us) service_pool.cc:224] Handling call
0709 12:22:35.802565 (+ 10983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:35.803023 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:35.803063 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:35.803616 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:22:36.518319 (+714703us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18003d1da0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:22:36.801287 (+282968us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:36.801345 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:36.801437 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:36.831249 (+ 29812us) raft_consensus.cc:1685] finished
0709 12:22:36.831269 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:36.835615 (+  4346us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13805,"raft.queue_time_us":105,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10921,"spinlock_wait_cycles":23077120,"thread_start_us":282520,"threads_started":1,"wal-append.queue_time_us":984271}
I20260709 12:22:36.848802 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:36.813235 23880 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.851329 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:36.835465 24064 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.808786 23383 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:22:36.854570 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1904) took 1354 ms. Trace:
I20260709 12:22:36.855084 20882 rpcz_store.cc:276] 0709 12:22:35.499823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.501090 (+  1267us) service_pool.cc:224] Handling call
0709 12:22:35.534051 (+ 32961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:35.534285 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:35.534325 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:35.554623 (+ 20298us) log.cc:844] Serialized 26 byte log entry
0709 12:22:35.794805 (+240182us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:35.794846 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:35.794907 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:36.703213 (+908306us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:36.848860 (+145647us) raft_consensus.cc:1685] finished
0709 12:22:36.848890 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:36.854486 (+  5596us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15521536,"thread_start_us":188879,"threads_started":1,"wal-append.queue_time_us":425814}
I20260709 12:22:36.838621 24051 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.814078 23266 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.867868 23383 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:36.868331 23383 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:36.844694 24052 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.872509 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:36.833742 23819 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.842665 23772 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:22:36.882967 23772 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:36.883348 23772 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:36.884172 23772 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:36.885409 23772 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:36.877282 23999 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.859030 24064 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.840619 23770 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.878737 23973 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:36.879700 23266 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:36.880245 23782 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:36.880565 23766 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:36.881145 20810 pending_rounds.cc:85] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 4
W20260709 12:22:36.954845 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 150707us
I20260709 12:22:36.957783 20810 pending_rounds.cc:107] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:22:36.958101 20810 raft_consensus.cc:2889] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:36.854178 24040 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 12:22:36.964673 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 115823us
I20260709 12:22:36.965485 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:36.965564 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:36.966328 20727 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:36.966588 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.871063 24051 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.869163 23383 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:36.948415 23971 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:36.970876 23979 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:36.980355 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:36.986572 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:36.972371 23766 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:36.974406 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:36.977085 24052 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:37.005264 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:37.006696 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:36.993410 23978 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:37.007617 20735 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:36.975287 23266 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:22:36.998494 24040 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:37.001313 23383 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:36.978513 23782 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:22:36.998145 23687 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:36.999517 24012 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:37.009673 23978 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.037283 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:37.037981 20802 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:36.981647 23979 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.010593 24001 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:22:37.018227 23992 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:37.049255 23766 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.051945 24057 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.054304 24057 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:37.047132 23999 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.057683 23999 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.043325 24077 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:37.059981 23999 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:37.060736 23819 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:37.062879 24015 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:37.064704 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.065606 20882 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:37.067870 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:37.034122 24012 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:37.041777 23978 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:37.089955 20880 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:37.091918 24012 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:37.150100 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 103034us
W20260709 12:22:37.088239 24063 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.574s	user 0.008s	sys 0.000s
I20260709 12:22:37.037402 23687 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.136886 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.088371 24081 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.093179 23992 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.087808 23988 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:37.091526 23999 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:37.059994 24077 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.144817 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:37.147854 20735 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:37.035427 23971 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.156358 20732 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:37.173993 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.186476 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.187114 21430 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:37.176450 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:37.181030 21427 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:37.173744 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:37.190474 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:37.193563 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:37.169843 24000 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.003s
I20260709 12:22:37.149600 24040 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:37.209872 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:37.205050 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:37.213708 21427 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:37.166240 23925 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.169762 24035 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:37.179333 24022 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.000s
W20260709 12:22:37.225492 24078 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 12:22:37.226924 24077 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:37.227597 24077 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:37.228384 24091 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.228991 24091 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.199756 24015 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:37.231729 24015 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.232095 24015 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:37.188493 24049 raft_consensus.cc:2764] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:22:37.244325 24079 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:22:37.189373 23999 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:37.261790 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.178951 24064 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.262621 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.266263 20732 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:37.203435 23819 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:37.228114 24077 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.277020 21434 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:37.280318 21430 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:22:37.293112 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50614.4us
I20260709 12:22:37.219739 23883 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:37.298022 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:37.298761 21435 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:22:37.294024 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.156s (SENT) [suppressed 29 similar messages]
I20260709 12:22:37.197747 23992 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.231088 24091 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:37.196990 24081 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.314396 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.245607 24051 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:37.265694 23999 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:37.245631 23988 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:37.300431 23883 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.340018 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:37.347687 21430 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:37.221776 23925 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.237282 24035 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.280871 23832 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.306675 24052 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:37.307092 24089 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.310570 23873 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.349921 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.341486 24074 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:37.308132 23978 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.368192 23819 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:37.347133 23883 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:37.379043 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.379877 21430 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:37.384157 23895 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.520s	user 0.013s	sys 0.001s
I20260709 12:22:37.347977 23988 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.341704 24090 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:22:37.425199 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.425844 20733 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:22:37.426980 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.371129 24052 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.428192 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.429234 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.429909 20733 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:37.383164 24074 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:37.350600 24040 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:37.413601 24087 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:22:37.418645 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1846) took 3381 ms. Trace:
I20260709 12:22:37.419716 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.420507 23978 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:37.420858 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.421955 20728 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:37.465979 20884 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:37.466157 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.420533 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.425961 20727 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:37.466881 21434 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:37.412654 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.381165 24041 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:37.362011 23832 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:37.412276 23763 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.004s
I20260709 12:22:37.483156 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.482903 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.376818 23873 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.441300 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.441450 20880 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:22:37.433045 23772 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.002s
I20260709 12:22:37.441802 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.491091 21434 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:22:37.441857 23383 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:37.494717 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.499356 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.499987 20880 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:37.413972 23895 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:37.441947 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.443215 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.446218 24084 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:37.470767 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108563us
I20260709 12:22:37.465248 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:37.537186 21435 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:37.474821 20850 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.516s (SENT)
I20260709 12:22:37.375985 24089 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.474717 20728 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:37.544484 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.470183 24101 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:37.545303 20879 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:37.480887 24077 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:37.454823 20808 rpcz_store.cc:276] 0709 12:22:34.062052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:34.099941 (+ 37889us) service_pool.cc:224] Handling call
0709 12:22:34.100304 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:36.398082 (+2297778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:36.398110 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:36.398145 (+    35us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:36.398180 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.418395 (+1020215us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54002b6bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:37.418439 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:37.418566 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":251,"raft.queue_time_us":2440,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":33249,"spinlock_wait_cycles":5283637632}
I20260709 12:22:37.475673 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.487123 20878 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:37.488965 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:37.438583 24049 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.488909 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.567029 20733 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:37.486434 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57093.5us
I20260709 12:22:37.569003 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:37.493474 24064 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:37.493587 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:37.494715 24093 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:22:37.495376 24085 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:37.496678 23832 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:37.497349 24052 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:37.599663 23832 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.599941 24052 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.600271 23832 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.600499 24052 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.497720 23687 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:37.490531 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.501227 20877 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:37.535341 20726 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:37.474448 20883 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:22:37.539047 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 893) took 1067 ms. Trace:
I20260709 12:22:37.540683 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.550479 21430 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:37.609890 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:37.556254 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 81450us
I20260709 12:22:37.572053 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.536736 24091 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:37.596017 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.618700 20884 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:37.596903 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.602295 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:37.602640 20875 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:37.620091 24106 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:22:37.571810 24101 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.620666 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.621385 20877 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:37.623557 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.624217 20883 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:37.624524 24052 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:37.625149 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.625214 23999 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.626251 20883 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:22:37.607740 24074 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.627895 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.629156 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:37.630270 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.491641 23763 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.609205 20876 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:37.610702 21434 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:37.608086 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:37.631110 20880 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:37.626246 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.633343 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.534162 23883 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.537027 23383 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:37.495019 23819 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.639477 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.829784 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.831122 20883 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:37.567956 24084 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:37.832540 23999 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:37.834735 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:37.759752 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 312861us
I20260709 12:22:37.856369 20880 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:37.856945 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:37.748992 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:37.838742 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.622937 24101 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:37.808557 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:37.859066 21426 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:37.749449 21432 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:37.719990 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.721903 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:37.609711 21429 rpcz_store.cc:276] 0709 12:22:36.471813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.479438 (+  7625us) service_pool.cc:224] Handling call
0709 12:22:37.235827 (+756389us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:22:37.236040 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.236236 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:37.236520 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 12:22:37.536066 (+299546us) log.cc:844] Serialized 14 byte log entry
0709 12:22:37.538402 (+  2336us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:37.538533 (+   131us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:37.538660 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.538777 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:37.538953 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1633,"prepare.queue_time_us":45610,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":13307,"raft.queue_time_us":41060,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":15312,"spinlock_wait_cycles":62729856,"thread_start_us":264983,"threads_started":2,"wal-append.queue_time_us":235519}
I20260709 12:22:37.749825 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.869278 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.870460 21430 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:37.868611 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:37.878577 21434 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:37.784976 20850 leader_election.cc:336] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.711s (SENT)
I20260709 12:22:37.951649 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:37.953269 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 168325us
I20260709 12:22:37.950815 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.955874 21434 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:37.956694 20726 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:37.957563 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:37.959414 21429 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:37.959242 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.960968 21434 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:37.962639 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.679714 24090 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:37.956884 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:37.617312 24091 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:37.966869 20877 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:37.987444 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:37.988549 21426 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 5: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:37.720064 24106 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.868547 24014 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:37.982195 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:38.004423 21430 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:37.631817 23895 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:37.680267 23992 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:37.766907 24081 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:37.866181 23925 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:37.680806 24064 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:37.996275 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:37.996654 20882 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:37.647042 24097 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:22:37.997442 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:38.015038 21432 raft_consensus.cc:2376] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7. Re-sending same reply.
I20260709 12:22:38.001051 20884 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:37.583540 24049 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:38.005766 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:38.022679 21429 raft_consensus.cc:2376] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6. Re-sending same reply.
I20260709 12:22:38.008167 20883 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:37.919266 24035 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:38.018329 21434 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:37.972574 24090 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.019950 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:38.027467 21426 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:38.027904 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:38.019367 23992 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:37.993721 23383 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:38.032519 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:38.033690 21434 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:38.002113 23999 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:38.045876 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:38.024958 23925 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:38.017395 23880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.006s	sys 0.018s
W20260709 12:22:38.018851 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 483621us
I20260709 12:22:38.051565 20882 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:38.037269 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:38.005375 24106 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:38.013017 24014 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:38.025594 24064 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:38.371459 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:38.358982 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:38.909075 20809 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:38.234800 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:38.910034 21432 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:38.025420 24101 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:38.031473 24090 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:38.052937 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:38.265961 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:38.267016 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:38.052069 21426 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
W20260709 12:22:38.153539 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 295311us
I20260709 12:22:38.964003 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:38.358564 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:38.965822 20802 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:38.379748 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.043s
W20260709 12:22:38.966461 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.015s	sys 0.044s
I20260709 12:22:38.912245 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1794) took 1413 ms. Trace:
I20260709 12:22:38.052002 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:38.541982 20803 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:38.966958 20882 rpcz_store.cc:276] 0709 12:22:37.498939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.014377 (+515438us) service_pool.cc:224] Handling call
0709 12:22:38.042398 (+ 28021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:38.911129 (+868731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.912160 (+  1031us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:38.559329 20881 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.004s	sys 0.007s
W20260709 12:22:38.967909 20881 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.005s	sys 0.010s
W20260709 12:22:38.381922 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:38.907104 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:38.917397 20879 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.277s	user 0.006s	sys 0.012s
I20260709 12:22:38.969623 20879 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
W20260709 12:22:38.970281 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1779) took 1475 ms (client timeout 1568 ms). Trace:
I20260709 12:22:38.032255 24035 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.922219 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:39.131048 20847 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.008s	sys 0.015s
W20260709 12:22:39.131496 20847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.009s	sys 0.015s
W20260709 12:22:38.513027 20850 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.000s	sys 0.008s
W20260709 12:22:39.131911 20850 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.146s	user 0.003s	sys 0.010s
W20260709 12:22:39.132345 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.17186e+06us
W20260709 12:22:38.968839 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.36071e+06us
I20260709 12:22:39.133903 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:39.134490 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:38.918609 23880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.420s	user 0.022s	sys 0.048s
I20260709 12:22:38.976627 20806 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
W20260709 12:22:38.991880 23383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.012s
W20260709 12:22:38.844206 24117 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:22:38.831791 24015 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.599s	user 0.002s	sys 0.010s
I20260709 12:22:39.162173 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.162765 20806 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:38.845017 24119 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 12:22:39.113432 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 563) took 1140 ms. Trace:
W20260709 12:22:38.153100 24108 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:22:39.168991 20809 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:22:38.569109 23971 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.005s	sys 0.010s
W20260709 12:22:39.130398 20877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.070s	user 0.033s	sys 0.071s
W20260709 12:22:38.146122 24012 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.973s	user 0.008s	sys 0.003s
W20260709 12:22:38.371472 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.050s	sys 0.038s
W20260709 12:22:39.173473 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.611s	user 0.050s	sys 0.045s
I20260709 12:22:38.080299 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:38.367064 24114 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.314s	user 0.000s	sys 0.000s
W20260709 12:22:38.970692 20879 rpcz_store.cc:269] 0709 12:22:37.494909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.546248 (+ 51339us) service_pool.cc:224] Handling call
0709 12:22:38.970211 (+1423963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.137588 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:39.137987 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:39.140527 21427 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:38.971496 20878 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:39.176084 21426 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:38.045786 23832 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:38.063215 23988 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:39.176862 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 434) took 1123 ms. Trace:
I20260709 12:22:38.227121 23982 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.915691 24049 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:38.438462 23763 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:38.918435 24101 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.227264 24034 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:38.922271 23883 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:38.955633 24084 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:39.151531 21431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 895) took 2662 ms (client timeout 2168 ms). Trace:
I20260709 12:22:39.161047 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:39.213364 20883 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:39.162185 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:39.163631 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:39.153730 20732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.093s	user 0.059s	sys 0.142s
W20260709 12:22:39.215782 20732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.059s	sys 0.142s
I20260709 12:22:39.209971 24101 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:39.166145 21429 rpcz_store.cc:276] 0709 12:22:37.972649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.038826 (+ 66177us) service_pool.cc:224] Handling call
0709 12:22:39.113370 (+1074544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.152096 24035 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:38.088099 23999 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:39.169863 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.019s	sys 0.024s
W20260709 12:22:39.171077 20877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.111s	user 0.033s	sys 0.072s
W20260709 12:22:39.225129 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.019s	sys 0.024s
I20260709 12:22:39.171010 21432 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
W20260709 12:22:39.173846 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.68479e+06us
I20260709 12:22:39.192106 24122 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.175236 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.235926 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:39.236032 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1865) took 1807 ms. Trace:
I20260709 12:22:39.147485 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.236939 20804 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:39.178520 24123 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:38.045146 24091 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.185348 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:39.185851 21426 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:39.177623 20726 rpcz_store.cc:276] 0709 12:22:38.051145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.051654 (+   509us) service_pool.cc:224] Handling call
0709 12:22:39.174192 (+1122538us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:39.173056 23971 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.214s	user 0.006s	sys 0.014s
I20260709 12:22:39.165591 23880 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:39.249419 24038 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 3.132s	user 0.012s	sys 0.037s
I20260709 12:22:39.249388 23880 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.250244 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1858) took 1827 ms. Trace:
I20260709 12:22:39.239974 24091 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:39.251600 23880 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.219596 24034 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.166393 23925 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:39.174584 23383 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.219056 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:39.252748 23925 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:39.252992 20809 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:39.253333 21429 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 5: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:39.253708 23925 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.157819 20802 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:39.192183 24049 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:39.255107 24034 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.221140 23993 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.169368 24015 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.252796 23383 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.258965 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.259404 23993 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.260232 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:39.212460 21431 rpcz_store.cc:269] 0709 12:22:36.472448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.475282 (+  2834us) service_pool.cc:224] Handling call
0709 12:22:39.133551 (+2658269us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aeee 000000000036eb39 00007f9e23bee1e0 00007f9e23cb70c4 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:39.134833 (+  1282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28293376}
I20260709 12:22:39.260664 21430 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:39.060351 24111 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:22:39.231629 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 759) took 1189 ms. Trace:
I20260709 12:22:39.233393 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.112653 24109 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:22:39.236704 20727 rpcz_store.cc:276] 0709 12:22:37.427348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.467393 (+ 40045us) service_pool.cc:224] Handling call
0709 12:22:37.467805 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:37.565841 (+ 98036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:37.565895 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:37.583381 (+ 17486us) log.cc:844] Serialized 52 byte log entry
0709 12:22:38.017149 (+433768us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:38.017200 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.017287 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.235043 (+1217756us) raft_consensus.cc:1685] finished
0709 12:22:39.235056 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.235169 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23855,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":215,"thread_start_us":493743,"threads_started":2,"wal-append.queue_time_us":908860}
I20260709 12:22:39.240238 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 561) took 1289 ms. Trace:
I20260709 12:22:39.263536 20728 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:39.263718 21426 rpcz_store.cc:276] 0709 12:22:37.950685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.137205 (+1186520us) service_pool.cc:224] Handling call
0709 12:22:39.240177 (+102972us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.250648 20732 rpcz_store.cc:276] 0709 12:22:37.423328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.469384 (+ 46056us) service_pool.cc:224] Handling call
0709 12:22:37.490052 (+ 20668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:37.996446 (+506394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:37.996481 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:38.025582 (+ 29101us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.216206 (+1190624us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:39.216249 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.218187 (+  1938us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54002495d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:39.218237 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.243749 (+ 25512us) raft_consensus.cc:1685] finished
0709 12:22:39.243770 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.250178 (+  6408us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28874,"raft.queue_time_us":36056,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4959,"spinlock_wait_cycles":1097077632,"thread_start_us":1155740,"threads_started":1,"wal-append.queue_time_us":1176891}
I20260709 12:22:39.254172 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:39.174901 24040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.004s	sys 0.031s
I20260709 12:22:39.189155 23832 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:39.176899 24012 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.216293 24014 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.190788 23982 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:39.222785 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59226.1us
I20260709 12:22:39.266069 21429 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:39.243010 24131 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.262708 21432 rpcz_store.cc:276] 0709 12:22:38.042022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.128308 (+ 86286us) service_pool.cc:224] Handling call
0709 12:22:39.231562 (+1103254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.255410 24125 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:39.257745 24034 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:39.270924 23832 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.285319 24125 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.285903 23832 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.286114 24012 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.286747 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.310431 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.276835 24111 consensus_queue.cc:1243] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:22:39.288898 24035 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:39.314177 24049 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:39.289865 23883 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:39.314700 24049 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:39.290539 24124 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:22:39.291447 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1791) took 1793 ms. Trace:
I20260709 12:22:39.316720 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.317736 20726 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:39.316676 20877 rpcz_store.cc:276] 0709 12:22:37.498169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.001908 (+503739us) service_pool.cc:224] Handling call
0709 12:22:38.002266 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:38.002637 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:38.002676 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:38.005461 (+  2785us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.225464 (+1220003us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:39.225505 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.225576 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.289559 (+ 63983us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:39.291117 (+  1558us) raft_consensus.cc:1685] finished
0709 12:22:39.291145 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.291386 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41033,"raft.queue_time_us":27523,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":16375,"spinlock_wait_cycles":3459328,"thread_start_us":1165269,"threads_started":1,"wal-append.queue_time_us":1159613}
I20260709 12:22:39.291548 24106 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.295354 24131 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.288085 23993 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.298409 24015 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.299141 21433 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:39.318992 20730 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:39.320580 24131 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:39.322485 23873 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.323050 24008 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:39.325776 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.326500 21433 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:39.299808 24125 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.297008 21431 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:39.339592 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 756) took 1587 ms. Trace:
I20260709 12:22:39.340106 21431 rpcz_store.cc:276] 0709 12:22:37.752239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.264164 (+1511925us) service_pool.cc:224] Handling call
0709 12:22:39.274399 (+ 10235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.339181 (+ 64782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.339508 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.301254 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:39.301685 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:39.261850 23383 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.240043 24122 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.275552 24136 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:22:39.311836 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1881) took 2244 ms. Trace:
I20260709 12:22:39.411301 21429 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:39.416724 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 758) took 1405 ms. Trace:
I20260709 12:22:39.417446 21429 rpcz_store.cc:276] 0709 12:22:38.011429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.253922 (+1242493us) service_pool.cc:224] Handling call
0709 12:22:39.416634 (+162712us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.416389 20810 rpcz_store.cc:276] 0709 12:22:37.067405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.233078 (+165673us) service_pool.cc:224] Handling call
0709 12:22:37.233423 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:37.389233 (+155810us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:37.389271 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:37.411940 (+ 22669us) log.cc:844] Serialized 26 byte log entry
0709 12:22:37.567517 (+155577us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:37.567582 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.311401 (+1743819us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54003d5ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:39.311453 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.311506 (+    53us) raft_consensus.cc:1685] finished
0709 12:22:39.311520 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.311774 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9499,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":5190,"spinlock_wait_cycles":330213760,"thread_start_us":80376,"threads_started":1,"wal-append.queue_time_us":255529}
I20260709 12:22:39.425076 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.426084 20810 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:39.428377 20727 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:39.431914 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 559) took 1481 ms. Trace:
I20260709 12:22:39.432524 21431 rpcz_store.cc:276] 0709 12:22:37.950138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.341124 (+1390986us) service_pool.cc:224] Handling call
0709 12:22:39.431283 (+ 90159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.431505 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.431542 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:39.431587 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:39.431621 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.431680 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.431836 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.437995 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:39.436012 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159168us
I20260709 12:22:39.450407 20809 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:39.452199 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:39.452678 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 314298us
I20260709 12:22:39.457610 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.458745 20804 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6.
I20260709 12:22:39.455966 20802 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:39.409655 23993 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:39.409271 24015 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.470338 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:39.460608 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 899) took 1592 ms. Trace:
I20260709 12:22:39.410552 23832 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:39.471571 21426 rpcz_store.cc:276] 0709 12:22:37.868147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.264383 (+1396236us) service_pool.cc:224] Handling call
0709 12:22:39.264693 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:39.265124 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.265148 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:39.265699 (+   551us) log.cc:844] Serialized 52 byte log entry
0709 12:22:39.458344 (+192645us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:39.458404 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.458500 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.458566 (+    66us) raft_consensus.cc:1685] finished
0709 12:22:39.458586 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.460518 (+  1932us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1573,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"thread_start_us":166279,"threads_started":1,"wal-append.queue_time_us":130983}
I20260709 12:22:39.474103 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 565) took 1467 ms. Trace:
I20260709 12:22:39.475554 20735 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
W20260709 12:22:39.476717 20735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 1852) took 2059 ms (client timeout 1568 ms). Trace:
I20260709 12:22:39.474788 24146 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:22:39.477279 23873 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:39.477222 24134 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:39.474511 21426 rpcz_store.cc:276] 0709 12:22:38.006242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.473170 (+1466928us) service_pool.cc:224] Handling call
0709 12:22:39.473518 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.473721 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.473752 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:39.473800 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:39.473838 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.473899 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.474020 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.478140 23819 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:39.478760 24014 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.478646 24106 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:39.479425 24000 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:39.479660 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:39.482019 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 558) took 1562 ms. Trace:
I20260709 12:22:39.482165 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.504567 20809 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:39.486222 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.504745 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:39.486814 21401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.016s	sys 0.020s
W20260709 12:22:39.505460 21401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.017s	sys 0.021s
I20260709 12:22:39.503582 21432 rpcz_store.cc:276] 0709 12:22:37.919848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.275351 (+1355503us) service_pool.cc:224] Handling call
0709 12:22:39.275588 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.301880 (+ 26292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.301925 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:39.301983 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:39.302052 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.481768 (+179716us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54005df7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:39.481818 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.481961 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59799424}
I20260709 12:22:39.506085 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:39.506745 20727 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:39.468679 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 546505us
I20260709 12:22:39.491374 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.507385 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:39.507934 20804 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:39.491550 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.508687 20810 raft_consensus.cc:2368] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:39.492216 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.492653 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.481353 24124 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:39.509600 20802 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:39.509785 20808 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:39.509963 24124 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:22:39.483757 20735 rpcz_store.cc:269] 0709 12:22:37.417504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.424992 (+  7488us) service_pool.cc:224] Handling call
0709 12:22:39.437828 (+2012836us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:22:39.476643 (+ 38815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12375296}
I20260709 12:22:39.494560 21426 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:39.472785 24148 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:39.482543 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:39.512691 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.511475 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:39.485098 23763 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:39.512727 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 760) took 1462 ms. Trace:
I20260709 12:22:39.435810 23999 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.519393 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.510380 24124 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:39.520979 21426 rpcz_store.cc:276] 0709 12:22:38.050339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.490234 (+1439895us) service_pool.cc:224] Handling call
0709 12:22:39.490623 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.512463 (+ 21840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.512664 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:39.526182 20809 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:39.475934 23993 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.527595 20810 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:39.528620 20735 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:22:39.483158 24015 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.530380 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:39.508705 24140 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.535318 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:39.513738 23383 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:39.520594 23772 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.521653 23922 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:39.537284 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.485734 23832 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.531307 20735 pending_rounds.cc:85] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 4
I20260709 12:22:39.518980 23925 raft_consensus.cc:2764] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:22:39.529646 24111 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:39.502993 24014 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.541816 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.566892 20810 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:39.568723 24144 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:39.569411 23832 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.570371 24064 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:39.571326 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:39.512264 24055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.909s	user 0.017s	sys 0.019s
I20260709 12:22:39.544942 24040 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:39.545821 24015 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.533826 23993 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.547638 23383 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:39.503737 24134 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.548928 23982 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:39.503253 23819 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:39.552302 21427 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 744) took 1803 ms (client timeout 1981 ms). Trace:
I20260709 12:22:39.504213 24101 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.553656 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 903) took 1558 ms. Trace:
I20260709 12:22:39.591614 21431 rpcz_store.cc:276] 0709 12:22:37.995507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.433970 (+1438463us) service_pool.cc:224] Handling call
0709 12:22:39.552235 (+118265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.552481 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:39.552509 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:39.552571 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:39.552609 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.552680 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.553480 (+   800us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54007b2ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:39.553591 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271044992}
I20260709 12:22:39.511016 24000 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.596733 24014 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.581532 24090 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.598160 24000 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.599742 20810 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:39.600309 24090 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.601226 24154 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.601922 24154 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.607000 24124 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.608318 24124 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:39.608383 23819 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.573103 24091 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.609368 24091 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.611738 24134 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.612263 24091 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.541121 24109 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:39.615119 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.616151 20879 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:22:39.556155 23772 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.557117 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:39.560186 24031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.884s	user 0.011s	sys 0.021s
I20260709 12:22:39.562721 20735 pending_rounds.cc:107] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:22:39.580175 24131 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:39.540716 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 59271us
I20260709 12:22:39.564942 20727 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:39.491365 24049 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.650315 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:39.584185 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 425108us
I20260709 12:22:39.586648 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.672499 20810 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:39.673214 20732 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:39.589607 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:39.578928 24055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.976s	user 0.017s	sys 0.019s
W20260709 12:22:39.585968 21427 rpcz_store.cc:269] 0709 12:22:37.748453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.748618 (+   165us) service_pool.cc:224] Handling call
0709 12:22:39.552091 (+1803473us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54007b2ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:39.552240 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2949015936}
I20260709 12:22:39.680944 21432 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:39.531286 23763 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.593173 23987 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:39.593142 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1931) took 2122 ms. Trace:
I20260709 12:22:39.566102 24111 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:39.559306 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:39.620301 20735 raft_consensus.cc:2889] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:39.562592 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.580600 23992 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:22:39.639006 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:39.651919 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 746) took 1902 ms (client timeout 1825 ms). Trace:
I20260709 12:22:39.653455 24131 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.660125 23772 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.651767 24122 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:39.661458 24141 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:22:39.661322 24152 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:39.659034 24101 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.543697 24140 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.663498 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:39.647332 24064 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.650730 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1920) took 2212 ms. Trace:
I20260709 12:22:39.729717 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.665329 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.659385 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.681969 20726 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
W20260709 12:22:39.687939 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.730451 20732 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:39.730588 20727 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:39.730811 20804 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
W20260709 12:22:39.620210 24031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.944s	user 0.011s	sys 0.021s
I20260709 12:22:39.731635 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.732216 20727 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:39.732923 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:39.695726 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 179059us
W20260709 12:22:39.708078 21433 rpcz_store.cc:269] 0709 12:22:37.749130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.749606 (+   476us) service_pool.cc:224] Handling call
0709 12:22:39.651736 (+1902130us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:39.651845 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3306096384}
W20260709 12:22:39.708668 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:39.695968 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 136796us
W20260709 12:22:39.710079 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71221.1us
I20260709 12:22:39.711833 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.734637 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:39.693470 20881 rpcz_store.cc:276] 0709 12:22:37.470773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.881481 (+410708us) service_pool.cc:224] Handling call
0709 12:22:37.972518 (+ 91037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.972719 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:39.064836 (+1092117us) log.cc:844] Serialized 14 byte log entry
0709 12:22:39.592815 (+527979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:39.592865 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:39.592908 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.592969 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.593087 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34131,"prepare.queue_time_us":1215709,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1675,"raft.queue_time_us":79266,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":28202,"thread_start_us":1475182,"threads_started":3,"wal-append.queue_time_us":478660}
I20260709 12:22:39.729656 20875 rpcz_store.cc:276] 0709 12:22:37.437894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.438020 (+   126us) service_pool.cc:224] Handling call
0709 12:22:37.442308 (+  4288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.619902 (+177594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.650440 (+2030538us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400305a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:39.650663 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367978880}
I20260709 12:22:39.711033 24040 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:39.713357 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.713495 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:39.736130 20731 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
W20260709 12:22:39.692505 24057 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.293s	user 0.037s	sys 0.053s
I20260709 12:22:39.697960 24111 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:39.710119 24049 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:39.742440 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.743259 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.712342 23765 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:39.652382 23383 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.707718 23987 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.740392 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.740073 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:39.744045 20732 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:39.746582 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:39.751556 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 88131.8us
I20260709 12:22:39.752242 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:39.738946 24161 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:39.754007 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.754510 20804 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
W20260709 12:22:39.754770 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.737262 24154 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:39.743228 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:39.745608 24123 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:39.749945 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:39.739929 23999 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.781688 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.747785 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:39.739737 24064 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.756736 20727 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:39.757532 24066 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:39.760079 24162 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:39.791970 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.792409 21400 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6.
I20260709 12:22:39.760594 24142 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.739532 20810 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:39.761519 24159 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:39.743494 24150 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.747650 23992 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:39.745656 23880 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.747514 24090 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:39.774971 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.837607 20804 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
W20260709 12:22:39.778121 24096 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 2.231s	user 0.007s	sys 0.016s
I20260709 12:22:39.757025 23925 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:39.779671 24076 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:39.842787 24076 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:39.843305 23763 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:39.843820 24076 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.845144 24076 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:39.781261 23782 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:39.781311 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.783048 20731 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:39.779285 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:39.745707 24085 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:39.741276 24057 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.355s	user 0.037s	sys 0.053s
I20260709 12:22:39.756029 24161 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:39.790385 20775 connection.cc:256] RPC call timeout handler was delayed by -1.6669768440001462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:22:39.859830 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80652.6us
I20260709 12:22:39.860594 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:39.791024 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.792857 21400 leader_election.cc:403] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:22:39.865110 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72874.8us
I20260709 12:22:39.794225 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:39.790234 24146 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:39.795238 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.796974 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.829938 20801 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:39.755839 24136 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:22:39.755853 24040 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:22:39.784773 24106 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.838963 23880 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.792021 24008 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.792474 24162 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:39.850159 20809 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:39.851828 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:39.837281 24090 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:39.822741 24142 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.791324 23873 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:22:39.868160 23925 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:39.787828 24123 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.834198 24159 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.835362 24150 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.869020 20808 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:39.872862 20806 raft_consensus.cc:1217] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Deduplicated request from leader. Original: 5.7->[8.8-8.8]   Dedup: 8.8->[]
I20260709 12:22:39.866446 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:39.872371 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:39.868423 24085 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.884150 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 562) took 1918 ms. Trace:
W20260709 12:22:39.899221 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 157332us
I20260709 12:22:39.959575 20804 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:39.881598 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 748) took 2131 ms. Trace:
I20260709 12:22:39.886464 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1876) took 2857 ms. Trace:
I20260709 12:22:39.790882 23999 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.892073 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1793) took 2393 ms. Trace:
W20260709 12:22:39.960786 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 94413us
I20260709 12:22:39.981401 20805 rpcz_store.cc:276] 0709 12:22:37.028762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.028925 (+   163us) service_pool.cc:224] Handling call
0709 12:22:37.029373 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:37.038644 (+  9271us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:37.038664 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:37.044347 (+  5683us) log.cc:844] Serialized 26 byte log entry
0709 12:22:37.374495 (+330148us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:37.374558 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.374655 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.777433 (+2402778us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:39.886154 (+108721us) raft_consensus.cc:1685] finished
0709 12:22:39.886176 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.886405 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93409,"raft.queue_time_us":5376,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":36630,"spinlock_wait_cycles":10411392,"thread_start_us":230991,"threads_started":1,"wal-append.queue_time_us":324373}
I20260709 12:22:39.985579 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:39.871423 24151 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:39.992229 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.992566 20726 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
W20260709 12:22:39.870779 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 135746us
W20260709 12:22:39.958822 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 164687us
I20260709 12:22:39.871649 24134 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 5. Current term is 6.
I20260709 12:22:39.882140 24131 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:40.000717 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.001331 20881 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:39.849834 23992 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:40.003901 23992 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.004431 23880 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.005329 23987 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:40.006275 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.006971 20808 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:40.007442 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.007748 24123 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:40.007977 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.008111 20883 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:40.008639 20808 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:40.009606 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.960530 20806 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:22:39.981307 21430 rpcz_store.cc:276] 0709 12:22:37.749710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.137740 (+1388030us) service_pool.cc:224] Handling call
0709 12:22:39.881411 (+743671us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b540042ed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:39.881542 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193923456}
I20260709 12:22:39.980481 21429 rpcz_store.cc:276] 0709 12:22:37.966586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.418560 (+1451974us) service_pool.cc:224] Handling call
0709 12:22:39.418982 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.571630 (+152648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.571663 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:39.571708 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:39.571749 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.883920 (+312171us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54007602d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:39.883966 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.884093 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350512000}
W20260709 12:22:39.981871 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85269.2us
I20260709 12:22:39.987080 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:39.879961 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:39.892817 24067 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 3.224s	user 0.009s	sys 0.027s
I20260709 12:22:39.873111 23763 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:39.867353 24161 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:39.983752 24157 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:39.972463 24090 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:39.877269 24064 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:39.896149 24152 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:39.891145 24041 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:39.972398 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 100102us
I20260709 12:22:39.995378 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:39.997243 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.026153 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:39.991647 24151 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:39.998445 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:39.999001 24053 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:39.972050 24140 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.978715 20810 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:40.031315 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.032020 20810 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:39.887084 23765 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:40.012187 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:40.015102 20732 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:22:40.016577 20804 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:39.981244 24077 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:40.017555 21400 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:40.017102 20808 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:39.990135 20876 rpcz_store.cc:276] 0709 12:22:37.498686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.003511 (+504825us) service_pool.cc:224] Handling call
0709 12:22:38.003836 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.134404 (+1130568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.134434 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.153431 (+ 18997us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.509450 (+356019us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:39.509493 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.891688 (+382195us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540005b254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:39.891737 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.891784 (+    47us) raft_consensus.cc:1685] finished
0709 12:22:39.891797 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.892013 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2599411584,"thread_start_us":320916,"threads_started":1,"wal-append.queue_time_us":357530}
I20260709 12:22:40.006217 24085 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:39.895922 23782 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 12:22:40.024842 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:40.024896 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:39.997788 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:40.027475 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.004434 24159 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.028815 20877 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:39.875334 24146 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:39.982131 24008 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:39.983078 24106 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.023537 24134 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
W20260709 12:22:40.130666 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118575us
I20260709 12:22:40.131496 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:40.132011 21400 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 12:22:40.105475 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.132607 20805 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:40.134397 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.132582 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:40.119318 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:40.135860 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118448us
I20260709 12:22:40.144965 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.145942 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:40.028163 24041 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:40.025359 23763 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:40.127866 24067 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:22:40.143149 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:40.031009 24151 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.129814 24135 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:40.141775 24049 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:40.136245 24053 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:40.073030 23987 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:40.151868 20801 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:40.033299 24154 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 12:22:40.153508 24008 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.154728 24085 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:40.168694 24135 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:40.174754 24085 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.155516 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.174873 24135 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.174979 24161 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.156481 24173 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:40.156229 23763 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:40.157671 23999 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.130695 24150 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:40.189518 24089 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:40.129575 24137 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.146s
I20260709 12:22:40.144101 24131 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:40.183179 23987 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:40.193610 24131 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:40.194096 24131 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:40.194355 23987 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.184736 24161 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.202782 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.019467 24111 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.161230 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:40.163025 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:40.167301 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.067408 24064 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:40.136375 24077 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:40.205960 24077 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.206650 24111 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.128320 24091 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:22:40.207553 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:40.208165 21432 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:40.208503 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:40.185194 20808 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:22:40.210199 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.027513 24162 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.169114 24049 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.211427 24049 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.211527 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:40.184176 24092 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 2.976s	user 0.007s	sys 0.005s
I20260709 12:22:40.158159 24106 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.176878 24085 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:40.204123 24142 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.172348 24134 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:40.192914 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.210142 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.132310 23765 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:40.212502 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:40.214859 24162 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:40.214756 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:40.215327 23818 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:22:40.215951 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54804.3us
I20260709 12:22:40.147151 23925 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:40.226171 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.272703 20808 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.273366 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:40.224082 20805 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:40.277050 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.277952 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.271224 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.278805 20810 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:40.281602 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.282024 21433 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:40.282899 20727 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
W20260709 12:22:40.289891 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.290742 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:40.272492 23818 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:40.293881 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.294776 20727 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:40.298331 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.222831 23772 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:40.218472 24064 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:40.303826 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.301808 23772 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.295815 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.310525 20805 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:40.311748 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.318938 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:40.437036 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:40.438241 20731 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.296473 24076 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:40.434616 23772 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.457455 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.458374 20810 raft_consensus.cc:2368] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:40.459216 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.459831 20805 raft_consensus.cc:2376] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7. Re-sending same reply.
I20260709 12:22:40.464526 24090 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.467684 24090 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:40.472949 24176 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:22:40.483326 24146 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:40.486471 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.487684 20810 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:40.488992 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1925) took 3040 ms. Trace:
I20260709 12:22:40.489432 20878 rpcz_store.cc:276] 0709 12:22:37.448132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.881492 (+433360us) service_pool.cc:224] Handling call
0709 12:22:38.971259 (+1089767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.184046 (+212787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.488791 (+1304745us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400267850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:40.488934 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2505276672}
W20260709 12:22:40.502183 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 209520us
I20260709 12:22:40.525146 23770 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:40.580246 20808 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:40.581058 20877 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:40.581519 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.585387 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 443) took 1350 ms. Trace:
I20260709 12:22:40.585994 20730 rpcz_store.cc:276] 0709 12:22:39.234542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.310055 (+ 75513us) service_pool.cc:224] Handling call
0709 12:22:40.585140 (+1275085us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b54000c53d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:40.585301 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17701632}
I20260709 12:22:40.591215 20730 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:40.591253 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:40.593590 20810 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:40.276865 24134 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:40.606196 20730 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:22:40.622520 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:40.625126 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:40.624308 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57601.4us
I20260709 12:22:40.641157 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:40.591768 24137 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.645686 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.297966 23818 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.651010 20810 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:40.632977 24134 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:40.650517 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:40.659909 20850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.019s	sys 0.019s
W20260709 12:22:40.661283 20850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.019s	sys 0.019s
I20260709 12:22:40.660816 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:40.640172 24076 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:22:40.665074 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.658282 23973 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:40.665607 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.035s	sys 0.080s
W20260709 12:22:40.681457 21428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.011s	sys 0.011s
W20260709 12:22:40.683602 21428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.012s	sys 0.012s
W20260709 12:22:40.683211 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.036s	sys 0.080s
W20260709 12:22:40.686139 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 553567us
I20260709 12:22:40.651146 24180 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 12:22:40.661660 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 634762us
I20260709 12:22:40.694120 20804 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:22:40.701493 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 457) took 1008 ms. Trace:
I20260709 12:22:40.702025 20726 rpcz_store.cc:276] 0709 12:22:39.692616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.741423 (+ 48807us) service_pool.cc:224] Handling call
0709 12:22:40.701273 (+959850us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54002990d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:40.701409 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482626432}
I20260709 12:22:40.704566 20804 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:40.699960 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:40.707355 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:40.708316 20808 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:40.708671 20804 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:40.707898 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:40.722328 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:40.722565 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:40.723205 20809 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:22:40.696408 23383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.046s	sys 0.079s
I20260709 12:22:40.655821 24137 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.723309 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:40.733865 21426 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:22:40.730711 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:40.664666 23818 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:40.732990 20731 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:40.736471 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:40.682964 23973 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:40.734239 21426 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 6
I20260709 12:22:40.747124 20733 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:22:40.748502 21426 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:22:40.748905 21426 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:40.749300 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:40.730319 23383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.046s	sys 0.080s
I20260709 12:22:40.666347 24161 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:40.735401 23763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.019s	sys 0.005s
W20260709 12:22:40.750941 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.003s	sys 0.020s
I20260709 12:22:40.751299 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:40.760955 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.003s	sys 0.020s
I20260709 12:22:40.761148 21434 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:40.721637 23782 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.755589 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:40.722018 24111 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:40.779907 24111 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:40.768680 23763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.019s	sys 0.006s
I20260709 12:22:40.780354 24111 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:40.759899 20730 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:22:40.731896 24140 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.006s	sys 0.024s
W20260709 12:22:40.761246 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 614050us
I20260709 12:22:40.729955 23967 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:40.768301 21432 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:40.658885 24064 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.737522 23770 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:22:40.747552 24091 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:40.771606 24142 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:40.753226 23973 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.773350 23782 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:40.791508 24140 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.007s	sys 0.024s
I20260709 12:22:40.760471 24161 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.784475 20878 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:40.786031 20735 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:22:40.808732 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:40.789903 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:40.747593 24076 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.794428 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.790627 23967 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.813192 20877 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
W20260709 12:22:40.775336 24151 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.006s
I20260709 12:22:40.810426 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.815161 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:40.820622 23782 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:40.806231 24051 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:40.805990 24142 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:40.835573 23967 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.838455 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.846213 20878 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:40.842365 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:40.847258 20877 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:40.811269 24161 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:40.873719 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.894488 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:40.841260 23987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.020s	sys 0.037s
I20260709 12:22:40.900453 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.910348 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:40.911320 20877 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
W20260709 12:22:40.949007 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:40.850466 23765 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 12:22:41.009009 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 447) took 1445 ms. Trace:
I20260709 12:22:41.009594 20732 rpcz_store.cc:276] 0709 12:22:39.563576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.775914 (+212338us) service_pool.cc:224] Handling call
0709 12:22:39.776247 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:40.134142 (+357895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.008709 (+874567us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54003ad9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:41.008938 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548751744}
W20260709 12:22:40.903961 23987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.020s	sys 0.037s
I20260709 12:22:40.836694 24076 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:40.839743 24151 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:41.032519 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.705840 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:41.043694 20733 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:22:41.055742 24179 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 12:22:40.845320 24051 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:40.993295 24089 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:41.060765 24131 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.000s
W20260709 12:22:41.079861 20774 connection.cc:256] RPC call timeout handler was delayed by -3.2525221010000678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
W20260709 12:22:41.072721 24186 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
W20260709 12:22:41.073565 24077 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.864s	user 0.010s	sys 0.000s
I20260709 12:22:41.017540 23765 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.096223 21427 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:22:41.097064 21435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 734) took 3768 ms (client timeout 1565 ms). Trace:
W20260709 12:22:41.097517 21435 rpcz_store.cc:269] 0709 12:22:37.329039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.329391 (+   352us) service_pool.cc:224] Handling call
0709 12:22:41.095739 (+3766348us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:41.096989 (+  1250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292543232}
I20260709 12:22:41.084761 23383 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:41.093067 24137 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:41.122577 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.132s	sys 0.230s
I20260709 12:22:41.078302 24089 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.079116 24131 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:41.117655 21400 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.024s	sys 0.060s
I20260709 12:22:41.129627 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:41.153394 21400 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.025s	sys 0.060s
W20260709 12:22:41.153858 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 661351us
W20260709 12:22:41.139074 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.132s	sys 0.230s
I20260709 12:22:41.158032 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.158632 21431 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:41.159502 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.094043 24077 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.081772 24142 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.178493 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.127601 24185 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.180255 21433 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:22:41.181618 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.182265 20730 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:22:41.180148 20877 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:41.193259 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.194092 21435 raft_consensus.cc:2376] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9. Re-sending same reply.
I20260709 12:22:41.200564 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.202008 24131 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:41.203006 24187 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:22:41.204131 24134 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.003s
I20260709 12:22:41.204542 24134 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.205363 24161 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.206875 24134 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:41.202111 21431 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:41.207340 24161 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:41.207777 24183 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:22:41.207880 24051 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.180580 24137 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:41.212457 24089 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:41.218537 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1897) took 1205 ms. Trace:
I20260709 12:22:41.226011 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:41.226338 20801 rpcz_store.cc:276] 0709 12:22:40.013159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.023428 (+ 10269us) service_pool.cc:224] Handling call
0709 12:22:41.218349 (+1194921us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b540019a5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:41.218469 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42847360}
I20260709 12:22:41.219977 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.172081 24188 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:41.222892 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:41.227517 21433 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:41.228040 21435 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:41.228667 24137 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.242357 23765 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:41.250941 21428 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.042s	sys 0.141s
I20260709 12:22:41.228782 24188 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:41.254920 21428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.043s	sys 0.141s
W20260709 12:22:41.245508 23772 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.000s
W20260709 12:22:41.252645 24041 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.090s	user 0.004s	sys 0.008s
W20260709 12:22:41.260696 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.004s
W20260709 12:22:41.285492 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.008s	sys 0.004s
I20260709 12:22:41.272204 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 725) took 4168 ms. Trace:
I20260709 12:22:41.199014 24185 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:41.260111 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.006s	sys 0.007s
W20260709 12:22:41.287330 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.006s	sys 0.007s
W20260709 12:22:41.286556 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 549475us
I20260709 12:22:41.257208 24012 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:41.259725 24188 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:41.290648 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2089) took 1497 ms. Trace:
I20260709 12:22:41.270972 24076 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:41.292634 20803 rpcz_store.cc:276] 0709 12:22:39.793501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.793633 (+   132us) service_pool.cc:224] Handling call
0709 12:22:39.830809 (+ 37176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.988030 (+157221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.988058 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:39.988100 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:39.988127 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.290414 (+1302287us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:41.290457 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.290589 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361098496}
I20260709 12:22:41.286124 24136 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:41.304867 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:41.306339 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.007s	sys 0.008s
W20260709 12:22:41.306730 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.007s	sys 0.008s
I20260709 12:22:41.310717 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.314074 24131 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:41.307420 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 492304us
W20260709 12:22:41.309038 24049 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.097s	user 0.010s	sys 0.000s
I20260709 12:22:41.315937 24049 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.318223 20726 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:22:41.319244 24049 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.321506 24185 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:41.322142 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:41.323748 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1940) took 1311 ms. Trace:
I20260709 12:22:41.324205 20883 rpcz_store.cc:276] 0709 12:22:40.012662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.012808 (+   146us) service_pool.cc:224] Handling call
0709 12:22:40.021396 (+  8588us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:40.021748 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.021778 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:40.031511 (+  9733us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.157340 (+1125829us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:41.157396 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.157497 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:41.323478 (+165981us) raft_consensus.cc:1685] finished
0709 12:22:41.323506 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.323676 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33968,"raft.queue_time_us":2131,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":43753,"thread_start_us":1020031,"threads_started":1,"wal-append.queue_time_us":1194679}
W20260709 12:22:41.326217 24085 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.079s	user 0.007s	sys 0.002s
I20260709 12:22:41.272671 23772 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.328112 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:41.332062 21431 raft_consensus.cc:2376] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9. Re-sending same reply.
I20260709 12:22:41.282899 24041 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.299002 24051 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:41.340616 20774 connection.cc:256] RPC call timeout handler was delayed by -2.5252007660001254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
I20260709 12:22:41.306444 24136 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:41.302685 24151 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:41.326637 24085 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.327811 24137 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:41.294268 24012 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.346060 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:41.367298 20701 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:41.341187 20774 connection.cc:256] RPC call timeout handler was delayed by -2.3713007660003314s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 12:22:41.371567 20732 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:41.371749 20774 connection.cc:256] RPC call timeout handler was delayed by -2.3614007660003153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 12:22:41.291441 21428 rpcz_store.cc:276] 0709 12:22:37.089412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.089536 (+   124us) service_pool.cc:224] Handling call
0709 12:22:37.135404 (+ 45868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.139688 (+  4284us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:37.170159 (+ 30471us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.162040 (+2991881us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b1800306820. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:22:41.256837 (+1094797us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:41.257021 (+   184us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:41.257069 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.257149 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.257289 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8064,"prepare.queue_time_us":30283,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":466125,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":18192,"spinlock_wait_cycles":39925120,"thread_start_us":1094427,"threads_started":2,"wal-append.queue_time_us":3646545}
W20260709 12:22:41.372306 20774 connection.cc:256] RPC call timeout handler was delayed by -2.3353007660002731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 12:22:41.372721 20774 connection.cc:256] RPC call timeout handler was delayed by -2.31331407100015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 12:22:41.371551 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:41.373093 20774 connection.cc:256] RPC call timeout handler was delayed by -2.2237007660000927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
W20260709 12:22:41.373788 20774 connection.cc:256] RPC call timeout handler was delayed by -2.1089908740000283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 12:22:41.374218 20774 connection.cc:256] RPC call timeout handler was delayed by -2.0705429250001544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 12:22:41.374572 20774 connection.cc:256] RPC call timeout handler was delayed by -2.0498429250001209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
W20260709 12:22:41.374953 20774 connection.cc:256] RPC call timeout handler was delayed by -2.0437007660002564s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
W20260709 12:22:41.375324 20774 connection.cc:256] RPC call timeout handler was delayed by -2.0102429250000569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:22:41.375679 20774 connection.cc:256] RPC call timeout handler was delayed by -2.0093429250000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:22:41.376080 20774 connection.cc:256] RPC call timeout handler was delayed by -1.9994429250000394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 12:22:41.376765 20774 connection.cc:256] RPC call timeout handler was delayed by -1.9148429249999026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 12:22:41.377311 20774 connection.cc:256] RPC call timeout handler was delayed by -1.907642924999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
W20260709 12:22:41.377764 20774 connection.cc:256] RPC call timeout handler was delayed by -1.7197926530002405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
W20260709 12:22:41.378345 20774 connection.cc:256] RPC call timeout handler was delayed by -1.6727429249999659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 12:22:41.378806 20774 connection.cc:256] RPC call timeout handler was delayed by -1.5710429250002562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 12:22:41.379263 20774 connection.cc:256] RPC call timeout handler was delayed by -1.4927429250001296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 12:22:41.380277 20774 connection.cc:256] RPC call timeout handler was delayed by -1.1075429249999615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25710000000000005s for a response
I20260709 12:22:41.345155 23876 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:41.384339 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 224006us
I20260709 12:22:41.383262 20727 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:41.381466 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.387240 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.388355 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:41.394438 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.369585 24131 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:41.369302 24185 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:41.367937 24191 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:41.396813 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.376792 23766 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:41.389299 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:41.404268 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.375838 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:41.412020 20732 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:41.394057 23925 raft_consensus.cc:2764] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:41.344955 24051 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.413618 23925 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:41.414995 24051 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.416451 24197 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:41.417004 24197 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:41.417452 24197 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:41.432710 23925 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:41.432713 24193 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:41.435472 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.393862 24196 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.381253 24151 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:22:41.422344 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:41.419739 23973 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.003s
I20260709 12:22:41.458530 23973 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:41.459846 24177 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.807s	user 0.008s	sys 0.000s
W20260709 12:22:41.460592 23967 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.003s
I20260709 12:22:41.460830 23973 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:41.447590 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:41.462742 23818 consensus_meta.cc:287] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 12:22:41.462920 24101 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:41.463685 21430 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:41.463946 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:41.464318 24101 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:41.464694 24101 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:41.465041 24101 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:41.448045 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:41.449640 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 503) took 1151 ms. Trace:
I20260709 12:22:41.467069 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.466400 20810 rpcz_store.cc:276] 0709 12:22:40.298337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.788270 (+489933us) service_pool.cc:224] Handling call
0709 12:22:40.803049 (+ 14779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:40.803520 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.803563 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:40.808049 (+  4486us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.307963 (+499914us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:41.308005 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.308067 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:41.449418 (+141351us) raft_consensus.cc:1685] finished
0709 12:22:41.449438 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.449579 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2248,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3971,"thread_start_us":472152,"threads_started":1,"wal-append.queue_time_us":535160}
I20260709 12:22:41.467679 21426 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:41.468106 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:41.468825 20810 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:41.469580 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:41.470319 20810 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:41.472383 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.451290 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2159) took 1250 ms. Trace:
I20260709 12:22:41.474483 20727 rpcz_store.cc:276] 0709 12:22:40.200620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.318625 (+118005us) service_pool.cc:224] Handling call
0709 12:22:41.451081 (+1132456us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540028d554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:41.451234 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2437607808}
I20260709 12:22:41.475617 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:41.476468 20727 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:41.477334 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.478504 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.452744 20807 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:41.479862 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:41.480633 21428 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
W20260709 12:22:41.482600 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:41.483291 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 61015.1us
I20260709 12:22:41.456975 24012 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.457772 24076 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:41.460983 23967 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.445060 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:41.490630 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45694.5us
I20260709 12:22:41.490674 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:41.389264 23876 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.484236 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1937) took 1486 ms. Trace:
I20260709 12:22:41.453233 24188 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.484987 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:41.501179 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 115257us
I20260709 12:22:41.502225 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:41.441879 24193 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:41.515341 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:41.491437 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.482589 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.493803 20882 rpcz_store.cc:276] 0709 12:22:39.997468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.998671 (+  1203us) service_pool.cc:224] Handling call
0709 12:22:39.999039 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:39.999447 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.999473 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:40.000053 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 12:22:40.091926 (+ 91873us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:40.091967 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.092063 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:41.459280 (+1367217us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:41.483990 (+ 24710us) raft_consensus.cc:1685] finished
0709 12:22:41.484011 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.484184 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7,"raft.queue_time_us":6518,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":60155776,"thread_start_us":91425,"threads_started":1,"wal-append.queue_time_us":478932}
I20260709 12:22:41.497457 20807 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:41.487003 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:41.404989 24131 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.520125 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.523789 21400 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:41.554162 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:41.493743 23849 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:22:41.562875 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:41.555662 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:41.489846 23925 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:41.565200 23818 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:41.565716 23818 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:41.566148 23818 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:41.514879 24111 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.732s	user 0.005s	sys 0.006s
I20260709 12:22:41.419620 24191 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.495718 24161 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.570122 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.578569 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.571041 23973 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:41.575887 20810 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:41.584123 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.404276 24136 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:41.568042 24135 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.392s	user 0.012s	sys 0.001s
I20260709 12:22:41.575379 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:41.587109 20731 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:41.561563 24022 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:41.474166 24196 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.558800 24200 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:41.569326 24012 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:41.580116 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.572991 24089 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:41.512573 24199 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:22:41.610221 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.598912 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.614777 20727 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:41.613871 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:41.593581 24200 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:41.619949 24200 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:41.621749 24085 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.622288 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2147) took 2084 ms. Trace:
I20260709 12:22:41.622676 20735 rpcz_store.cc:276] 0709 12:22:39.537682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.755954 (+218272us) service_pool.cc:224] Handling call
0709 12:22:39.756259 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:40.808870 (+1052611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.622039 (+813169us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000c49d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:41.622228 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2367389568}
I20260709 12:22:41.621847 23967 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:41.624859 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.629978 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:41.574272 24111 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.594993 24194 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:41.628388 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.630867 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.636317 20727 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:41.636672 20735 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:22:41.637140 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:41.614547 24137 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:41.632169 21430 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:41.615998 24136 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.647240 20733 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:22:41.589087 23772 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:41.609586 24012 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:41.655478 24217 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.655472 24012 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:41.656683 24111 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.598560 24041 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.609602 24142 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.601029 24161 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:41.595950 24135 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.615236 24203 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:41.609963 24089 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:41.627391 24022 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:41.611166 24151 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:22:41.688611 21428 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:22:41.651607 23925 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:41.653074 23772 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:41.600389 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:41.614712 23973 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:22:41.626854 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141935us
I20260709 12:22:41.663736 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:41.672873 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.674010 20878 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:22:41.701148 20774 leader_election.cc:336] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.501s (SENT)
I20260709 12:22:41.701720 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.657974 23876 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:41.662428 24137 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:41.702306 20878 raft_consensus.cc:2376] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11. Re-sending same reply.
I20260709 12:22:41.705209 21429 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:41.705493 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.706835 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.665357 24218 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:41.666167 24161 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:41.714061 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.718828 20878 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:41.719689 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.720379 20878 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:41.716099 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.721402 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.722323 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.671075 24089 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:41.726730 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.728315 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.728906 20878 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:41.730075 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.731068 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.732151 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.732833 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.693434 24041 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:41.738447 20882 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:41.741297 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:41.703658 23967 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:22:41.706889 24137 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:41.713332 24081 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:41.667011 24142 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:41.915364 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 315040us
I20260709 12:22:41.918452 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.725178 24161 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:41.628594 24193 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.733276 24203 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:41.916076 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:41.924717 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.925662 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:41.927132 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.926749 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:41.931849 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:41.937672 24208 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
W20260709 12:22:41.939936 24207 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:22:41.941926 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2108) took 1297 ms. Trace:
W20260709 12:22:41.943742 24216 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
I20260709 12:22:41.939384 24022 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:41.942266 20805 rpcz_store.cc:276] 0709 12:22:40.644476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.290809 (+646333us) service_pool.cc:224] Handling call
0709 12:22:41.299381 (+  8572us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:41.299851 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.299891 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:41.302392 (+  2501us) log.cc:844] Serialized 52 byte log entry
0709 12:22:41.495052 (+192660us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:41.640414 (+145362us) log.cc:844] Serialized 14 byte log entry
0709 12:22:41.663650 (+ 23236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.664149 (+   499us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:41.941713 (+277564us) raft_consensus.cc:1685] finished
0709 12:22:41.941732 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.941868 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":137216,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2109,"raft.queue_time_us":18739,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":23006,"spinlock_wait_cycles":8495360,"thread_start_us":259740,"threads_started":3,"wal-append.queue_time_us":262504}
W20260709 12:22:41.950927 24204 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
W20260709 12:22:41.951503 24198 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.379s	user 0.000s	sys 0.001s
I20260709 12:22:41.914997 24137 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:41.921500 24224 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:41.954267 24101 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.954777 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:41.955438 24212 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 12:22:41.914709 24041 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.733471 24218 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:41.956377 24209 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:22:41.957460 24214 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.260s	user 0.000s	sys 0.000s
I20260709 12:22:41.958513 24142 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:41.923110 24203 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:41.960495 24196 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:41.937618 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:41.976692 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51418.3us
I20260709 12:22:41.919687 23971 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:41.993018 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 290677us
W20260709 12:22:42.017800 20774 leader_election.cc:336] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.568s (SENT)
I20260709 12:22:41.991737 24125 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:42.019392 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:41.996626 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 291770us
I20260709 12:22:41.998081 20883 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:41.989355 24193 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.000718 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:42.027969 20884 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:41.944782 24131 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:41.996192 24151 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.010141 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:41.993470 24197 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.575s	user 0.001s	sys 0.011s
I20260709 12:22:42.020953 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:42.022351 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:42.022851 24118 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:42.046200 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:42.047677 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:42.023905 24218 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:42.024449 24228 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:42.025388 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:42.025687 24101 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:42.020412 24035 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.026561 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:42.097934 20808 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:42.093581 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:42.099170 20875 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:41.994519 24191 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:42.032554 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:42.038309 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:42.042953 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:42.126245 20884 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:42.126716 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83841.3us
I20260709 12:22:42.045167 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:42.094655 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:42.091275 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:42.030104 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:42.129845 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 99816.6us
I20260709 12:22:42.128918 20882 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:42.176302 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:42.177206 20882 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:42.178557 21400 leader_election.cc:400] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:42.180004 21400 leader_election.cc:403] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:42.193686 20878 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:42.196053 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:42.214787 20878 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:42.032944 24135 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:42.241815 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.035627 24225 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:42.248778 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118088us
I20260709 12:22:42.174700 24035 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:42.260882 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 228387us
I20260709 12:22:42.267910 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:42.460413 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.311442 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:42.461586 20849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 2.098s (SENT) [suppressed 25 similar messages]
I20260709 12:22:42.026794 23971 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.464864 23971 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:42.390681 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 365331us
I20260709 12:22:42.050436 23973 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.231s
I20260709 12:22:42.050196 24076 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.220s
I20260709 12:22:42.088675 24219 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.984797 23925 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.395020 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:42.050666 24229 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:42.475708 24229 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:42.476302 24229 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:42.246575 24101 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:42.474082 24219 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:42.480165 24101 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:42.481405 24101 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.412649 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2133) took 5645 ms. Trace:
I20260709 12:22:42.035939 24193 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:42.485203 24193 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:42.089920 24197 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.483235 20734 rpcz_store.cc:276] 0709 12:22:36.767253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.767365 (+   112us) service_pool.cc:224] Handling call
0709 12:22:36.786325 (+ 18960us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:36.786711 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:36.786744 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:22:39.697686 (+2910942us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:39.709820 (+ 12134us) log.cc:844] Serialized 78 byte log entry
0709 12:22:39.837158 (+127338us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:39.861308 (+ 24150us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.006023 (+2144715us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b18003f7900. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:22:42.006237 (+   214us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.372650 (+366413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.399559 (+ 26909us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:42.412242 (+ 12683us) raft_consensus.cc:1685] finished
0709 12:22:42.412269 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.412509 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2133086,"prepare.queue_time_us":2173269,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":883,"raft.run_cpu_time_us":439,"raft.run_wall_time_us":49637,"spinlock_wait_cycles":163547776,"thread_start_us":473802,"threads_started":2,"wal-append.queue_time_us":115120,"wal-append.run_cpu_time_us":2817,"wal-append.run_wall_time_us":1192611}
I20260709 12:22:42.487764 24197 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:42.090098 24151 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.490108 24151 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:42.112418 24012 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.253844 20884 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:42.491927 20884 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:42.492420 24012 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:42.493572 20884 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:42.452674 20808 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:42.496004 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:42.497021 20808 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:42.498153 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:42.453248 20880 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:42.454687 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:42.503033 24012 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:42.503217 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48573.8us
I20260709 12:22:42.503542 24012 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:42.504137 24012 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:42.505092 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:42.393987 23819 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:42.393651 24191 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:42.393458 23967 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:42.507982 24197 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:42.512379 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:42.514158 24012 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.514608 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:42.516218 24151 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:42.516760 24151 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:42.517217 24151 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:42.519505 24012 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:42.529495 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:42.531631 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:42.532413 20808 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:42.533489 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:42.535590 20884 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:22:42.540664 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 563) took 3304 ms (client timeout 3494 ms). Trace:
W20260709 12:22:42.542097 20879 rpcz_store.cc:269] 0709 12:22:39.148894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.185084 (+ 36190us) service_pool.cc:224] Handling call
0709 12:22:42.447879 (+3262795us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540022f6d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:42.453106 (+  5227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864019328}
I20260709 12:22:42.434984 23266 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:42.545603 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:42.551568 24237 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:22:42.553298 24234 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:22:42.540582 24225 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:42.558614 24225 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:42.436120 24081 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:42.566743 24193 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:42.541054 24218 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:42.611698 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 349544us
I20260709 12:22:42.613592 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:42.618414 21430 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:42.618847 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:42.620011 21435 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:42.630264 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:42.568181 24225 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:42.635952 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:42.631215 21430 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:42.570794 24089 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.830s	user 0.003s	sys 0.006s
I20260709 12:22:42.270004 24203 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:42.643777 24206 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.627s	user 0.000s	sys 0.001s
I20260709 12:22:42.645102 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2111) took 1952 ms. Trace:
I20260709 12:22:42.645447 20803 rpcz_store.cc:276] 0709 12:22:40.692328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.296655 (+604327us) service_pool.cc:224] Handling call
0709 12:22:41.307228 (+ 10573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.307618 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.307648 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.308108 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.552598 (+244490us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:41.552640 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.552713 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:42.644895 (+1092182us) raft_consensus.cc:1685] finished
0709 12:22:42.644914 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.645042 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1065,"spinlock_wait_cycles":45333248,"thread_start_us":244115,"threads_started":1,"wal-append.queue_time_us":609530}
W20260709 12:22:42.608952 24221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.000s	sys 0.025s
I20260709 12:22:42.571585 24081 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.631908 23967 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:42.647152 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:42.681281 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:42.664367 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 583) took 1051 ms. Trace:
I20260709 12:22:42.687405 21428 rpcz_store.cc:276] 0709 12:22:41.613347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.613551 (+   204us) service_pool.cc:224] Handling call
0709 12:22:41.613914 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.689273 (+ 75359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.664068 (+974795us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b5400776854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:42.664286 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153977728}
I20260709 12:22:42.665688 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2106) took 2210 ms. Trace:
I20260709 12:22:42.689275 20801 rpcz_store.cc:276] 0709 12:22:40.455800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.231731 (+775931us) service_pool.cc:224] Handling call
0709 12:22:41.242029 (+ 10298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.242224 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:41.492484 (+250260us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.665015 (+1172531us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18003bfbc0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a
0709 12:22:42.665262 (+   247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:42.665326 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:42.665386 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.665479 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.665614 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":351673,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":105,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":92,"spinlock_wait_cycles":27580160,"thread_start_us":356587,"threads_started":2,"wal-append.queue_time_us":169444}
I20260709 12:22:42.667382 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:42.668190 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2101) took 2449 ms. Trace:
I20260709 12:22:42.668825 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2166) took 1628 ms. Trace:
I20260709 12:22:42.696133 20730 rpcz_store.cc:276] 0709 12:22:41.040774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.040961 (+   187us) service_pool.cc:224] Handling call
0709 12:22:41.041242 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.183623 (+142381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.668509 (+1484886us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:42.668753 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323063680}
I20260709 12:22:42.669574 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2167) took 1628 ms. Trace:
I20260709 12:22:42.534904 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:42.697959 20726 rpcz_store.cc:276] 0709 12:22:41.041617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.041815 (+   198us) service_pool.cc:224] Handling call
0709 12:22:41.074496 (+ 32681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.318574 (+244078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.669284 (+1350710us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54002e4fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:42.669504 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559981312}
I20260709 12:22:42.670318 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2170) took 1333 ms. Trace:
I20260709 12:22:42.671375 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:42.701311 20804 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:42.672114 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 566) took 2466 ms. Trace:
I20260709 12:22:42.702597 21429 rpcz_store.cc:276] 0709 12:22:40.206023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.207727 (+  1704us) service_pool.cc:224] Handling call
0709 12:22:40.208002 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.705613 (+1497611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.671839 (+966226us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400729a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:42.672043 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3442814848}
I20260709 12:22:42.679993 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 773) took 1520 ms. Trace:
I20260709 12:22:42.704424 21432 rpcz_store.cc:276] 0709 12:22:41.159631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.226946 (+ 67315us) service_pool.cc:224] Handling call
0709 12:22:41.230133 (+  3187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.230507 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.230547 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.231154 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.662807 (+1431653us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b180041a760. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1
0709 12:22:42.663055 (+   248us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:42.663115 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.679504 (+ 16389us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:42.679572 (+    68us) raft_consensus.cc:1685] finished
0709 12:22:42.679604 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.679859 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16257,"raft.queue_time_us":150558,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":14749,"spinlock_wait_cycles":10178304,"thread_start_us":143334,"threads_started":1,"wal-append.queue_time_us":267800}
I20260709 12:22:42.672873 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 473) took 3244 ms. Trace:
I20260709 12:22:42.707567 20806 rpcz_store.cc:276] 0709 12:22:39.428286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.428408 (+   122us) service_pool.cc:224] Handling call
0709 12:22:39.428740 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:40.013538 (+584798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.006627 (+1993089us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5400180720. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca2a02 00007f9e1ec9edd6 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:42.672619 (+665992us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b540019a0d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:42.672805 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025565440}
I20260709 12:22:42.699620 20733 rpcz_store.cc:276] 0709 12:22:41.336586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.336757 (+   171us) service_pool.cc:224] Handling call
0709 12:22:41.346108 (+  9351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.647774 (+301666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.670034 (+1022260us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54003b4554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:42.670252 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691801472}
W20260709 12:22:42.714229 20728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.012s	sys 0.031s
W20260709 12:22:42.727308 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.693684 20802 rpcz_store.cc:276] 0709 12:22:40.218989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.699259 (+480270us) service_pool.cc:224] Handling call
0709 12:22:41.219132 (+519873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.219412 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:41.435521 (+216109us) log.cc:844] Serialized 14 byte log entry
0709 12:22:41.636780 (+201259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:41.637027 (+   247us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:41.637098 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.637349 (+   251us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.667974 (+1030625us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b540019a5d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:42.668118 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":353157,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":2229,"raft.queue_time_us":79321,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"spinlock_wait_cycles":1116190080,"thread_start_us":416464,"threads_started":3,"wal-append.queue_time_us":244268}
W20260709 12:22:42.758314 20728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.017s	sys 0.036s
I20260709 12:22:42.681926 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:42.621999 24118 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:22:42.755988 23818 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.189s	user 0.010s	sys 0.002s
W20260709 12:22:42.756626 24200 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.108s	user 0.008s	sys 0.003s
I20260709 12:22:42.660456 24194 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:42.782047 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:42.630177 24173 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 12:22:42.713940 23819 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:42.675424 23687 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:42.816972 20729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.010s	sys 0.011s
I20260709 12:22:42.788348 20802 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:42.662369 24161 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.002s
W20260709 12:22:42.824839 20729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.010s	sys 0.011s
I20260709 12:22:42.663452 24041 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:42.662650 24081 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:42.676050 23967 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:42.838869 20875 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.660s	user 0.000s	sys 0.008s
I20260709 12:22:42.811753 24240 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.787331 24001 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:42.843389 20875 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
W20260709 12:22:42.843876 20847 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.000s	sys 0.025s
W20260709 12:22:42.846869 20847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.000s	sys 0.025s
W20260709 12:22:42.844216 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 382623us
W20260709 12:22:42.850028 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.640002 24089 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:42.847935 20735 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.015s	sys 0.010s
I20260709 12:22:42.789158 24194 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:42.851749 20875 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:22:42.895434 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.895809 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:42.898149 20875 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:42.900023 20876 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:42.844269 23967 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:42.839591 24161 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:42.788607 23818 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.797016 24200 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:42.847798 24001 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:42.901315 20735 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.015s	sys 0.018s
I20260709 12:22:42.899051 20875 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:42.897565 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.927053 20882 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:42.927050 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:42.929854 20875 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:42.949560 20881 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:42.904448 24035 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:42.849552 24240 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:42.956053 24022 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.062216 24022 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:43.082434 24022 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:43.051560 24245 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 12:22:43.052430 24243 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
W20260709 12:22:43.056028 24252 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:22:43.086339 23818 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:43.059729 24137 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.056s	user 0.008s	sys 0.004s
I20260709 12:22:43.087549 24137 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.090536 24137 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:43.063172 24240 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.067394 24157 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:43.096843 24161 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:43.096618 24157 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.097936 24001 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:43.099593 24001 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:42.936415 24246 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:43.103865 24246 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.105405 24157 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.106571 24246 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:42.953305 23782 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:43.113233 24227 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:22:43.115813 23782 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:43.117138 23782 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:43.050505 24247 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:22:43.071055 24035 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.121047 24035 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:43.121937 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2120) took 1080 ms. Trace:
I20260709 12:22:43.122473 20810 rpcz_store.cc:276] 0709 12:22:42.041797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.041941 (+   144us) service_pool.cc:224] Handling call
0709 12:22:42.042277 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.042514 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.042541 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.042984 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.610681 (+567697us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:42.610800 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.611122 (+   322us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.121639 (+510517us) raft_consensus.cc:1685] finished
0709 12:22:43.121666 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.121857 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":215623,"threads_started":1,"wal-append.queue_time_us":628246}
I20260709 12:22:43.124094 24035 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.124408 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:42.950341 24242 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:43.125777 24242 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.128767 24242 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:43.125223 20810 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.069888 24089 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:42.926766 24041 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.136487 24041 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.078095 24200 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:43.191753 20875 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:43.192569 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1964) took 1157 ms. Trace:
I20260709 12:22:43.192955 20875 rpcz_store.cc:276] 0709 12:22:42.035045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.851111 (+816066us) service_pool.cc:224] Handling call
0709 12:22:42.851589 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.192299 (+340710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.192503 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:43.212566 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1962) took 1178 ms. Trace:
I20260709 12:22:43.213068 20884 rpcz_store.cc:276] 0709 12:22:42.034550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.536933 (+502383us) service_pool.cc:224] Handling call
0709 12:22:42.537318 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:42.538163 (+   845us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:42.894839 (+356676us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.212006 (+317167us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.212169 (+   163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.212224 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:43.212282 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.212369 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.212504 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":529151,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"thread_start_us":665934,"threads_started":2}
I20260709 12:22:43.184242 24157 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.223879 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:43.224673 20810 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:43.221105 20802 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:43.225107 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1965) took 1189 ms. Trace:
I20260709 12:22:43.226468 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:43.227252 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.226028 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.229079 20810 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:43.182726 24041 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:43.240123 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.242585 20802 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
W20260709 12:22:43.253847 24229 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.003s
I20260709 12:22:43.254381 24229 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:43.255726 24200 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:22:43.264068 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:43.260778 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.266981 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.225651 20884 rpcz_store.cc:276] 0709 12:22:42.035306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.224087 (+1188781us) service_pool.cc:224] Handling call
0709 12:22:43.224438 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.224686 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:43.224724 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.224782 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:43.224827 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.224896 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.225038 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:43.268062 24151 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.750s	user 0.011s	sys 0.003s
I20260709 12:22:43.280210 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:43.281823 20733 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:43.281824 20802 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:43.283398 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:43.284029 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.235790 24157 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.272372 24200 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:43.310246 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.311136 20808 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.323297 20810 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
W20260709 12:22:43.344079 24262 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:22:43.347507 24260 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:22:43.356149 24256 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:22:43.358080 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1816) took 1353 ms. Trace:
I20260709 12:22:43.360339 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1963) took 1325 ms. Trace:
I20260709 12:22:43.372289 20880 rpcz_store.cc:276] 0709 12:22:42.004663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.499994 (+495331us) service_pool.cc:224] Handling call
0709 12:22:42.500440 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.500912 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.500954 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.501582 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.937439 (+435857us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:42.937479 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.937546 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.357834 (+420288us) raft_consensus.cc:1685] finished
0709 12:22:43.357857 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.358012 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5244,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":435406,"threads_started":1,"wal-append.queue_time_us":770505}
I20260709 12:22:43.372457 20879 rpcz_store.cc:276] 0709 12:22:42.034814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.567280 (+532466us) service_pool.cc:224] Handling call
0709 12:22:42.620451 (+ 53171us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:42.620951 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.621001 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:42.638906 (+ 17905us) log.cc:844] Serialized 52 byte log entry
0709 12:22:42.924576 (+285670us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:43.225277 (+300701us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.237280 (+ 12003us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.237497 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.237595 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.360041 (+122446us) raft_consensus.cc:1685] finished
0709 12:22:43.360071 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.360263 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1024,"prepare.queue_time_us":147298,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":13181,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":11019,"thread_start_us":420008,"threads_started":2,"wal-append.queue_time_us":502708}
I20260709 12:22:43.382042 20809 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:43.387084 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1881) took 1992 ms. Trace:
I20260709 12:22:43.387607 20733 rpcz_store.cc:276] 0709 12:22:41.394377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.758351 (+1363974us) service_pool.cc:224] Handling call
0709 12:22:42.786221 (+ 27870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.284365 (+498144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.386821 (+102456us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54003aded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:43.387017 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1139258752}
I20260709 12:22:43.359695 24157 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:43.366070 24229 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.303757 24246 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:43.348527 24200 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:43.371531 20726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.006s	sys 0.051s
I20260709 12:22:43.392941 20808 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
W20260709 12:22:43.394531 20726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.007s	sys 0.051s
I20260709 12:22:43.291481 24151 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.394124 24246 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:43.401639 20880 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:43.404534 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1819) took 1359 ms. Trace:
I20260709 12:22:43.405000 20880 rpcz_store.cc:276] 0709 12:22:42.045073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.391714 (+1346641us) service_pool.cc:224] Handling call
0709 12:22:43.401392 (+  9678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.404187 (+  2795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.404459 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:43.408298 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:43.412006 20880 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:43.413354 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2172) took 2050 ms. Trace:
I20260709 12:22:43.413872 20731 rpcz_store.cc:276] 0709 12:22:41.363460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.363688 (+   228us) service_pool.cc:224] Handling call
0709 12:22:41.364100 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.587569 (+223469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.413024 (+1825455us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54000e6fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:43.413265 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512428288}
I20260709 12:22:43.422116 20808 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:43.419093 24194 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.000s
I20260709 12:22:43.425961 24194 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.428119 24194 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:43.430068 24225 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.743s	user 0.010s	sys 0.001s
I20260709 12:22:43.430564 24225 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:43.431238 23967 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.435128 23967 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:43.443059 20803 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:43.433295 24157 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:43.448050 24246 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.449115 24188 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.454288 24151 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:43.452313 24157 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:43.457145 24157 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.467880 24151 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:43.499429 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1817) took 1455 ms. Trace:
I20260709 12:22:43.499961 20884 rpcz_store.cc:276] 0709 12:22:42.044290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.271898 (+1227608us) service_pool.cc:224] Handling call
0709 12:22:43.272352 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:43.272783 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:43.272824 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:43.275862 (+  3038us) log.cc:844] Serialized 26 byte log entry
0709 12:22:43.422832 (+146970us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:43.422876 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.422948 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.499165 (+ 76217us) raft_consensus.cc:1685] finished
0709 12:22:43.499208 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.499362 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":683,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":1931,"thread_start_us":56165,"threads_started":1,"wal-append.queue_time_us":119348}
I20260709 12:22:43.505812 24151 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.510123 24151 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:43.515726 24151 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:43.520944 20810 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:43.524899 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1906) took 1055 ms. Trace:
I20260709 12:22:43.525594 20810 rpcz_store.cc:276] 0709 12:22:42.469576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.496546 (+1026970us) service_pool.cc:224] Handling call
0709 12:22:43.503054 (+  6508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.524502 (+ 21448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.524815 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:43.525981 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1905) took 1056 ms. Trace:
I20260709 12:22:43.527290 20808 rpcz_store.cc:276] 0709 12:22:42.469283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.433630 (+964347us) service_pool.cc:224] Handling call
0709 12:22:43.434009 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.434207 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:43.471037 (+ 36830us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.525594 (+ 54557us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.525650 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:43.525710 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.525784 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.525918 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68237,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":103,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":44928,"thread_start_us":87560,"threads_started":2,"wal-append.queue_time_us":54312}
I20260709 12:22:43.535379 20808 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:22:43.564312 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 913) took 2181 ms. Trace:
I20260709 12:22:43.564919 21431 rpcz_store.cc:276] 0709 12:22:41.382529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.382755 (+   226us) service_pool.cc:224] Handling call
0709 12:22:41.394432 (+ 11677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.394719 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:41.602508 (+207789us) log.cc:844] Serialized 14 byte log entry
0709 12:22:41.651712 (+ 49204us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.007902 (+356190us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18003f8620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:43.563676 (+1555774us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b18003f8620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:22:43.563815 (+   139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.563902 (+    87us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:43.563976 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.564069 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.564233 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":258727,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":101119,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":8261,"spinlock_wait_cycles":33057536,"thread_start_us":307822,"threads_started":3,"wal-append.queue_time_us":133571,"wal-append.run_cpu_time_us":1619,"wal-append.run_wall_time_us":1310330}
I20260709 12:22:43.532645 24151 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.606050 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1902) took 1137 ms. Trace:
I20260709 12:22:43.635807 20809 rpcz_store.cc:276] 0709 12:22:42.468359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.383150 (+914791us) service_pool.cc:224] Handling call
0709 12:22:43.383527 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:43.383816 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:43.383846 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:43.384393 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:22:43.443991 (+ 59598us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:43.548123 (+104132us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.605022 (+ 56899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.605658 (+   636us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.605742 (+    84us) raft_consensus.cc:1685] finished
0709 12:22:43.605761 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.605958 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60483,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":39,"thread_start_us":139696,"threads_started":2,"wal-append.queue_time_us":38578}
W20260709 12:22:43.643126 20776 connection.cc:256] RPC call timeout handler was delayed by -1.0386714759997631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 12:22:43.642104 24272 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:22:43.645519 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:43.648972 23849 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:43.647145 24077 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:43.658597 24215 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 2.001s	user 0.005s	sys 0.000s
W20260709 12:22:43.661237 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:43.664953 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 582) took 2231 ms. Trace:
I20260709 12:22:43.665560 21426 rpcz_store.cc:276] 0709 12:22:41.433729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.468272 (+ 34543us) service_pool.cc:224] Handling call
0709 12:22:41.468663 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.469068 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.469097 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.469534 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.585466 (+115932us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:41.585914 (+   448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.586001 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.573724 (+1987723us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:43.664557 (+ 90833us) raft_consensus.cc:1685] finished
0709 12:22:43.664594 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.664888 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7006464,"thread_start_us":115575,"threads_started":1,"wal-append.queue_time_us":171303}
I20260709 12:22:43.669167 24151 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:43.671118 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.684226 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 506) took 1030 ms. Trace:
I20260709 12:22:43.684583 20806 rpcz_store.cc:276] 0709 12:22:42.654358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.670835 (+1016477us) service_pool.cc:224] Handling call
0709 12:22:43.684161 (+ 13326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:43.685607 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.681810 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 570) took 3474 ms. Trace:
I20260709 12:22:43.693658 21427 rpcz_store.cc:276] 0709 12:22:40.207024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.209554 (+  2530us) service_pool.cc:224] Handling call
0709 12:22:41.095885 (+886331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.096629 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.681581 (+2584952us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b540039d850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:43.681743 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037621632}
I20260709 12:22:43.682803 20808 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:43.695770 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.697448 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.689112 24101 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.689381 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 571) took 2870 ms. Trace:
I20260709 12:22:43.698503 20877 rpcz_store.cc:276] 0709 12:22:40.836819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.912106 (+ 75287us) service_pool.cc:224] Handling call
0709 12:22:41.037654 (+125548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.184302 (+146648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.689134 (+2504832us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400366854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:43.689313 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326668160}
W20260709 12:22:43.689924 24248 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.790s	user 0.000s	sys 0.001s
W20260709 12:22:43.691138 24150 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 12:22:43.691421 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 836247us
I20260709 12:22:43.692145 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2155) took 3495 ms. Trace:
I20260709 12:22:43.705616 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:43.706429 20884 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:43.692308 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1869) took 2318 ms. Trace:
I20260709 12:22:43.692727 24173 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:43.707490 20735 rpcz_store.cc:276] 0709 12:22:41.374044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.637806 (+263762us) service_pool.cc:224] Handling call
0709 12:22:41.638508 (+   702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.639900 (+  1392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.639931 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.644250 (+  4319us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.965948 (+321698us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:42.925369 (+959421us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.691996 (+766627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.692078 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.692126 (+    48us) raft_consensus.cc:1685] finished
0709 12:22:43.692139 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.692254 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":810096,"prepare.queue_time_us":394952,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":1256,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2429,"thread_start_us":1171534,"threads_started":2,"wal-append.queue_time_us":342647}
I20260709 12:22:43.712818 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.714910 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:43.702225 20729 rpcz_store.cc:276] 0709 12:22:40.197025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.486268 (+289243us) service_pool.cc:224] Handling call
0709 12:22:40.486658 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.486901 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:42.009498 (+1522597us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.691790 (+1682292us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.691842 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:43.691886 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.691953 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.692062 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2441778,"prepare.queue_time_us":1612158,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"thread_start_us":819737,"threads_started":1,"wal-append.queue_time_us":939697}
I20260709 12:22:43.725724 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.726244 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1884) took 2312 ms. Trace:
I20260709 12:22:43.726670 20808 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:43.723541 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.728554 20806 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:43.731153 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.731992 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:43.736740 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.738302 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:43.752297 20877 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:43.752737 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.753494 20808 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:43.754309 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:43.726715 20733 rpcz_store.cc:276] 0709 12:22:41.413499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.389869 (+1976370us) service_pool.cc:224] Handling call
0709 12:22:43.393293 (+  3424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.393504 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:43.403424 (+  9920us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.725919 (+322495us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.725974 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:43.726022 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.726085 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.726191 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":272797,"prepare.queue_time_us":12733,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":37105,"threads_started":1,"wal-append.queue_time_us":37439}
I20260709 12:22:43.756927 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:43.757315 21435 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:43.755265 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.760274 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:43.762511 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:43.766381 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:43.766517 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 121080us
I20260709 12:22:43.767112 20735 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:43.768518 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:43.769111 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:43.769536 20884 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:22:43.769709 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:43.770974 20884 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:43.719947 24173 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.771848 20880 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:43.784706 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:43.785494 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:43.785912 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:43.773061 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1873) took 2397 ms. Trace:
I20260709 12:22:43.722690 24219 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.787694 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:43.724172 24191 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:43.733843 23819 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.778790 20877 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:43.788511 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:43.774477 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:43.724216 24101 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:43.788652 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:43.902343 24203 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:43.891104 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 1960) took 2214 ms (client timeout 2364 ms). Trace:
I20260709 12:22:43.903362 24001 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.904116 24001 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:43.906559 24084 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:43.903595 20881 rpcz_store.cc:269] 0709 12:22:41.676207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.741045 (+ 64838us) service_pool.cc:224] Handling call
0709 12:22:43.890825 (+2149780us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54000965d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:43.891021 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2776516480}
I20260709 12:22:43.907146 24084 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.822770 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1875) took 2447 ms. Trace:
I20260709 12:22:43.874756 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1879) took 2493 ms. Trace:
I20260709 12:22:43.894678 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2123) took 1039 ms. Trace:
I20260709 12:22:43.908759 20734 rpcz_store.cc:276] 0709 12:22:41.375809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.486579 (+1110770us) service_pool.cc:224] Handling call
0709 12:22:42.487078 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.570075 (+ 82997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.570185 (+   110us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.574338 (+  4153us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.956779 (+382441us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:43.201614 (+244835us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.822337 (+620723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.822439 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.822502 (+    63us) raft_consensus.cc:1685] finished
0709 12:22:43.822523 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.822693 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":525037,"prepare.queue_time_us":166715,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2470,"raft.queue_time_us":572737,"spinlock_wait_cycles":47605376,"thread_start_us":451680,"threads_started":2,"wal-append.queue_time_us":630860}
I20260709 12:22:43.910224 21432 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:43.909545 20810 rpcz_store.cc:276] 0709 12:22:42.854991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.527586 (+672595us) service_pool.cc:224] Handling call
0709 12:22:43.544468 (+ 16882us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:43.545059 (+   591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:43.545090 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:22:43.567774 (+ 22684us) log.cc:844] Serialized 616 byte log entry
0709 12:22:43.713660 (+145886us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:43.737779 (+ 24119us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.743340 (+  5561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.743441 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.894377 (+150936us) raft_consensus.cc:1685] finished
0709 12:22:43.894402 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.894598 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88778,"prepare.queue_time_us":42366,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":19023,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":27858,"thread_start_us":72105,"threads_started":1,"wal-append.queue_time_us":187880}
I20260709 12:22:43.806272 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:43.801911 21431 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:43.763897 24200 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.802336 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:43.801604 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:43.798906 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:43.824878 24219 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:22:43.842737 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:43.797922 20727 rpcz_store.cc:276] 0709 12:22:41.375309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.638038 (+262729us) service_pool.cc:224] Handling call
0709 12:22:41.638284 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.638463 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:22:42.185267 (+546804us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.619223 (+433956us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.688558 (+1069335us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b180044c2c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:43.772754 (+ 84196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.772799 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:43.772847 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.772909 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.773008 (+    99us) inbound_call.cc:177] Queueing success response
0709 12:22:43.882463 (+109455us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
Metrics: {"mutex_wait_us":505942,"prepare.queue_time_us":909903,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":77,"raft.queue_time_us":1108148,"raft.run_cpu_time_us":37190,"raft.run_wall_time_us":1266912,"spinlock_wait_cycles":39420032,"thread_start_us":474390,"threads_started":2,"wal-append.queue_time_us":655640}
W20260709 12:22:43.800526 24261 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:22:43.792584 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:43.922714 24053 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:43.923552 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1878) took 2542 ms. Trace:
I20260709 12:22:43.907770 24111 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:43.925799 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:43.957602 21426 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:43.958127 20731 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:43.948999 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:43.923123 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 148706us
I20260709 12:22:43.784070 24173 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:43.942929 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:43.926265 24109 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:22:43.960109 23876 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:43.926023 24191 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.961254 21428 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:43.957412 24200 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.784145 24005 raft_consensus.cc:2764] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:43.963603 24225 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:43.924506 23819 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:43.965236 24084 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:43.966490 20731 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:43.966823 24053 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:43.959151 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 510) took 1278 ms (client timeout 1623 ms). Trace:
I20260709 12:22:43.951761 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1913) took 1311 ms. Trace:
I20260709 12:22:43.956965 24101 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.970726 24001 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:43.771778 24157 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.200s
I20260709 12:22:43.924849 23266 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:22:43.972069 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:43.864748 23772 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:43.924933 20735 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:43.973562 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 577) took 2491 ms (client timeout 1537 ms). Trace:
W20260709 12:22:44.041180 20883 rpcz_store.cc:269] 0709 12:22:41.482175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.482359 (+   184us) service_pool.cc:224] Handling call
0709 12:22:43.973382 (+2491023us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:43.973493 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1100409600}
I20260709 12:22:43.957386 20730 rpcz_store.cc:276] 0709 12:22:41.380676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.754816 (+1374140us) service_pool.cc:224] Handling call
0709 12:22:42.755173 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:43.689626 (+934453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:43.689657 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:43.690181 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:22:43.822839 (+132658us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:43.822886 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.823268 (+   382us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.923334 (+100066us) raft_consensus.cc:1685] finished
0709 12:22:43.923353 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.923493 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":933968,"raft.queue_time_us":934133,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":72,"thread_start_us":132296,"threads_started":1,"wal-append.queue_time_us":231498}
I20260709 12:22:44.042675 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.043450 20730 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:44.044492 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.045781 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:43.976040 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1949) took 2302 ms. Trace:
I20260709 12:22:43.976729 20810 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:43.978000 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:43.910537 20726 rpcz_store.cc:276] 0709 12:22:41.381715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.754573 (+1372858us) service_pool.cc:224] Handling call
0709 12:22:42.779777 (+ 25204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:42.780064 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:42.808227 (+ 28163us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.862358 (+1054131us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.874259 (+ 11901us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:43.874337 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:43.874404 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.874506 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.874674 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":305147,"prepare.queue_time_us":31675,"prepare.run_cpu_time_us":607,"prepare.run_wall_time_us":1053,"raft.queue_time_us":369353,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":3176,"spinlock_wait_cycles":15236480,"thread_start_us":728249,"threads_started":2,"wal-append.queue_time_us":601048}
I20260709 12:22:44.046610 20882 rpcz_store.cc:276] 0709 12:22:41.673602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.178070 (+504468us) service_pool.cc:224] Handling call
0709 12:22:42.179166 (+  1096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:42.927768 (+748602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.975825 (+1048057us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:43.975987 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717574528}
I20260709 12:22:43.980614 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:43.981462 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.048200 20727 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
W20260709 12:22:44.050235 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72317.7us
I20260709 12:22:44.051450 21432 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:43.991269 20733 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:43.992363 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.052544 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:43.962102 21431 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:44.051080 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:44.007261 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.049336 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.054764 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.017127 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.055418 20727 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:44.055955 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.056176 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.057039 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.057811 20731 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:44.057946 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.058575 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.026250 21434 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:44.024820 20803 rpcz_store.cc:276] 0709 12:22:42.640439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.646878 (+  6439us) service_pool.cc:224] Handling call
0709 12:22:43.951592 (+1304714us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:43.951700 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725665536}
I20260709 12:22:44.054813 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:44.024597 20804 rpcz_store.cc:269] 0709 12:22:42.681145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.948675 (+1267530us) service_pool.cc:224] Handling call
0709 12:22:43.959091 (+ 10416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:44.059145 20731 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:44.015512 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.061590 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.062572 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.062870 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.064126 21427 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:44.014932 24225 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:44.066536 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.015949 23984 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:44.068444 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.061012 23772 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.015307 24296 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:44.024371 24053 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:44.065065 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 183143us
I20260709 12:22:44.039234 24066 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:44.092945 24066 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:44.095582 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 384188us
I20260709 12:22:44.064334 24101 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.072019 24197 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:44.029125 24289 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.076s
W20260709 12:22:44.097505 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.215909 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 147391us
W20260709 12:22:44.216226 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.217509 23772 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.218446 23772 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.104321 23383 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:44.224786 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:44.225301 23383 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:44.226086 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 511) took 1544 ms (client timeout 1633 ms). Trace:
I20260709 12:22:44.228338 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:44.227880 23383 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.229964 21428 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:44.229872 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1806) took 2369 ms (client timeout 1569 ms). Trace:
I20260709 12:22:44.230259 20878 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:44.230584 23383 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.232671 21431 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:44.234855 24066 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.098284 24101 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:22:44.174301 24299 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
I20260709 12:22:44.105000 24225 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.203284 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.067448 20734 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:44.206984 20727 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:44.064126 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.208142 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.249933 20734 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:44.209065 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1968) took 2168 ms. Trace:
I20260709 12:22:44.210004 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.251579 20730 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:44.252393 20735 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:44.253502 20730 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:22:44.090363 23984 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:44.213194 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 123689us
I20260709 12:22:44.003141 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:44.073899 24194 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.098088 24197 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:44.100903 24200 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.070751 24118 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:22:44.200500 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.088770 24296 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:44.169814 24300 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:22:44.173732 24293 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:44.273460 24293 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:44.166071 24292 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:22:44.224835 23819 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.275669 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:44.276409 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2122) took 2233 ms. Trace:
W20260709 12:22:44.226621 20801 rpcz_store.cc:269] 0709 12:22:42.681448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.223900 (+1542452us) service_pool.cc:224] Handling call
0709 12:22:44.225999 (+  2099us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:44.277556 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.230548 20876 rpcz_store.cc:269] 0709 12:22:41.859948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.915815 (+ 55867us) service_pool.cc:224] Handling call
0709 12:22:44.229213 (+2313398us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:44.229573 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239407872}
I20260709 12:22:44.251860 20875 rpcz_store.cc:276] 0709 12:22:42.040280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.238293 (+1198013us) service_pool.cc:224] Handling call
0709 12:22:43.268106 (+ 29813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:43.268556 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:43.978479 (+709923us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b18002ae0e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb25e5 00007f9e1ecb2568
0709 12:22:43.978735 (+   256us) log.cc:844] Serialized 14 byte log entry
0709 12:22:44.050665 (+ 71930us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:44.051741 (+  1076us) log.cc:844] Serialized 26 byte log entry
0709 12:22:44.051867 (+   126us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:44.052131 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 12:22:44.070335 (+ 18204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.070413 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:44.208392 (+137979us) raft_consensus.cc:1685] finished
0709 12:22:44.208439 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.208713 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7632,"prepare.queue_time_us":710176,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3206,"raft.queue_time_us":17905,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":2846,"spinlock_wait_cycles":5578368,"thread_start_us":71572,"threads_started":1,"wal-append.queue_time_us":125815}
I20260709 12:22:44.259307 20735 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:44.269637 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:44.255127 20730 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:44.272303 24191 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.289973 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1877) took 2912 ms. Trace:
I20260709 12:22:44.290505 24197 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.290623 20732 rpcz_store.cc:276] 0709 12:22:41.377110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.755804 (+1378694us) service_pool.cc:224] Handling call
0709 12:22:42.775349 (+ 19545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:43.690179 (+914830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:43.690206 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:43.690662 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:22:43.788238 (+ 97576us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:43.788296 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.788378 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:44.289356 (+500978us) raft_consensus.cc:1685] finished
0709 12:22:44.289411 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.289733 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":914352,"raft.queue_time_us":914965,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":97152,"threads_started":1,"wal-append.queue_time_us":324755}
I20260709 12:22:44.276996 20805 rpcz_store.cc:276] 0709 12:22:42.043107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.312023 (+268916us) service_pool.cc:224] Handling call
0709 12:22:42.434908 (+122885us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:42.759443 (+324535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.759483 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:43.907246 (+1147763us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:22:43.907847 (+   601us) log.cc:844] Serialized 52 byte log entry
0709 12:22:44.055298 (+147451us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:44.055348 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.055439 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:44.173913 (+118474us) spinlock_profiling.cc:243] Waited 895 us on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:44.275772 (+101859us) raft_consensus.cc:1685] finished
0709 12:22:44.275813 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.276135 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":199349,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":3720,"spinlock_wait_cycles":12507264,"thread_start_us":470985,"threads_started":2,"wal-append.queue_time_us":181672}
I20260709 12:22:44.241426 24053 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.306783 23987 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:44.311939 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 308702us
I20260709 12:22:44.311964 21431 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:44.312610 24246 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.313380 24246 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.314615 24182 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:44.314874 24109 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:44.315856 21433 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:22:44.315969 24109 consensus_queue.cc:1243] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:44.316792 23266 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:44.317273 23266 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:44.318856 23266 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.323057 24219 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:22:44.325027 23266 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.326000 24221 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:44.326635 24221 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:44.328202 24221 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.328573 24301 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:44.329576 24221 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.329739 24246 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.330520 24225 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.328369 24293 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.302264 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.344578 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:44.329989 24301 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:44.346273 24242 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:44.347402 24188 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:44.348101 24188 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:44.347311 24301 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:44.348392 24242 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:44.348955 24242 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:44.349566 24188 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.351125 21428 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:44.351480 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.354247 20731 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.350486 24301 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.354950 21428 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:44.358296 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88963.2us
I20260709 12:22:44.337276 23772 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.366178 20734 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:44.363551 24217 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:22:44.371179 23987 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.372177 24293 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.374152 23849 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:22:44.375439 24228 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:44.375556 23849 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:44.375948 24228 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.376214 23849 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:44.378350 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 99612.9us
I20260709 12:22:44.376420 24228 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.380628 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1953) took 2706 ms. Trace:
I20260709 12:22:44.381107 20878 rpcz_store.cc:276] 0709 12:22:41.674583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.336062 (+661479us) service_pool.cc:224] Handling call
0709 12:22:44.229570 (+1893508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:44.231094 (+  1524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.380348 (+149254us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54000a37d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:44.380537 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4345597312}
I20260709 12:22:44.382505 24066 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:22:44.388311 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.391633 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 775) took 1809 ms (client timeout 2039 ms). Trace:
I20260709 12:22:44.391862 24217 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:22:44.392154 21433 rpcz_store.cc:269] 0709 12:22:42.582152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.582292 (+   140us) service_pool.cc:224] Handling call
0709 12:22:44.391379 (+1809087us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54007201d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:44.391546 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3856244992}
I20260709 12:22:44.393175 24217 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:44.393644 24217 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:44.393888 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.363721 23984 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.407084 21432 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:44.378471 23849 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.406984 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:44.411281 24134 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.412421 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:44.412140 24134 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.410192 24051 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:44.350486 24242 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.417421 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:44.415411 24051 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:44.424834 23987 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.423516 24301 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.425328 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1874) took 3049 ms. Trace:
I20260709 12:22:44.426009 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.426728 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2121) took 2384 ms. Trace:
I20260709 12:22:44.426844 20801 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
W20260709 12:22:44.428553 24235 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.527s	user 0.000s	sys 0.004s
I20260709 12:22:44.427160 20807 rpcz_store.cc:276] 0709 12:22:42.042434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.042549 (+   115us) service_pool.cc:224] Handling call
0709 12:22:42.042844 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.482152 (+439308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.482195 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.482794 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.866292 (+383498us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:42.866355 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.418447 (+1552092us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400127354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:44.418659 (+   212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:44.426222 (+  7563us) raft_consensus.cc:1685] finished
0709 12:22:44.426237 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.426532 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160840,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":1009280896,"thread_start_us":383060,"threads_started":1,"wal-append.queue_time_us":573951,"wal-append.run_cpu_time_us":1603,"wal-append.run_wall_time_us":1001618}
I20260709 12:22:44.425833 20728 rpcz_store.cc:276] 0709 12:22:41.375595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.743557 (+367962us) service_pool.cc:224] Handling call
0709 12:22:41.743856 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.983556 (+239700us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:41.983603 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.989670 (+  6067us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.759873 (+770203us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:42.759921 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.759995 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:44.418211 (+1658216us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b2400185648. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf0fb0 00007f9e1e867488 00007f9e1e867be8 00007f9e1e83f398 00007f9e1e83a76a 00007f9e1e839bb6
0709 12:22:44.421576 (+  3365us) raft_consensus.cc:1685] finished
0709 12:22:44.421605 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.425149 (+  3544us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":12487936,"thread_start_us":1006271,"threads_started":2,"wal-append.queue_time_us":935644}
I20260709 12:22:44.436501 23987 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.437376 23987 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.438278 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.438700 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:44.438890 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43616us
I20260709 12:22:44.442627 24064 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:44.443918 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:44.442885 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.443150 24064 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.439056 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.446091 24064 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.446630 20882 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:44.396536 24217 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.447572 24296 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:44.451674 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.452451 24296 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.455950 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.457013 24191 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:44.457286 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.449179 23849 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.447829 23987 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.470427 20882 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:44.472326 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:44.466646 20700 connection.cc:256] RPC call timeout handler was delayed by -2.6738781290000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
I20260709 12:22:44.473623 24219 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:44.474058 24219 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.474433 24219 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.481055 23782 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:22:44.482440 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.482543 24154 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:44.482257 23782 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:44.456439 24242 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.483147 24154 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:44.488932 24154 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.490793 24154 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.491989 20807 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:44.425400 24051 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.472442 24134 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.495987 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.496750 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.497197 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.497637 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.497912 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.499222 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.499542 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.499733 20879 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
W20260709 12:22:44.500265 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.500558 20882 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:44.500281 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
W20260709 12:22:44.499733 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102331us
W20260709 12:22:44.504549 24307 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:22:44.511512 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.512655 20879 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:44.505205 24053 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:44.528244 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:44.529237 20809 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:44.529554 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.530824 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.531210 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:44.531553 20810 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:44.532655 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:44.532861 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.533545 24239 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.533967 24239 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:44.534354 24239 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.535470 23782 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:44.535732 24196 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.536173 23782 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:44.536454 24196 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.537335 23782 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.520328 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:44.541925 23876 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:22:44.542300 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76749.5us
I20260709 12:22:44.544040 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.544198 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:44.545389 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.556339 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.538957 24196 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.562271 20807 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:44.547214 24217 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:44.566676 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48998.2us
I20260709 12:22:44.571069 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1898) took 2104 ms. Trace:
W20260709 12:22:44.572556 20700 connection.cc:256] RPC call timeout handler was delayed by -2.2836603690002448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 12:22:44.568332 24134 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.573745 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.546411 24306 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:44.578138 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:44.579403 20700 connection.cc:256] RPC call timeout handler was delayed by -1.9362603690001379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
W20260709 12:22:44.607846 20700 connection.cc:256] RPC call timeout handler was delayed by -1.6903512700000647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 12:22:44.608201 20700 connection.cc:256] RPC call timeout handler was delayed by -1.5913512700003594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1761s for a response
W20260709 12:22:44.608505 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4592028040001424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 12:22:44.608769 20700 connection.cc:256] RPC call timeout handler was delayed by -1.3462132740000925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 12:22:44.609767 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:44.611455 24051 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.611846 24051 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:44.612215 24051 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.614207 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.616070 24293 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:44.581310 24308 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:22:44.589039 24228 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:44.590359 24173 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.004s
I20260709 12:22:44.618940 24173 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.621125 24173 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.621991 20729 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:22:44.592108 23984 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:44.623118 23984 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:44.623548 23984 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:44.624459 23984 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.625780 23984 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.592667 20807 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:44.568666 24053 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.595459 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.628943 20875 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:44.579833 20810 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:44.615082 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:44.630913 20879 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:44.633626 24228 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.636813 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:44.639430 20875 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.641705 24228 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.642495 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:44.647336 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 116994us
I20260709 12:22:44.571785 20802 rpcz_store.cc:276] 0709 12:22:42.466937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.285693 (+818756us) service_pool.cc:224] Handling call
0709 12:22:43.286066 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.286302 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:43.327927 (+ 41625us) log.cc:844] Serialized 14 byte log entry
0709 12:22:43.675060 (+347133us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b18003ba880. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:44.308190 (+633130us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b18003ba880. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9
0709 12:22:44.310446 (+  2256us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b18003ba880. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:22:44.463137 (+152691us) log.cc:844] Serialized 14 byte log entry
0709 12:22:44.463489 (+   352us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:44.463634 (+   145us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:44.502297 (+ 38663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.519576 (+ 17279us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.562571 (+ 42995us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b18003ba880. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:44.570875 (+  8304us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":64334,"prepare.run_cpu_time_us":176,"prepare.run_wall_time_us":14751,"raft.queue_time_us":331,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":190,"spinlock_wait_cycles":20417152,"thread_start_us":77960,"threads_started":1,"wal-append.queue_time_us":1131439}
I20260709 12:22:44.607445 24306 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:44.668511 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50844.4us
I20260709 12:22:44.664111 20803 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:44.673599 20879 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:44.650352 24228 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.561435 24239 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.662118 23383 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:44.670262 24306 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.703994 20733 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:44.703428 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.705519 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.686978 24228 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.687830 23383 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.712713 20876 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:44.713209 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.715541 20726 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:44.715994 20726 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:44.717006 20726 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.718034 20726 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.719168 23876 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:44.719475 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.719892 23876 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.722452 23876 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:44.725045 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.725932 23266 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.726639 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:44.727559 23266 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:44.727967 23266 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.728297 23266 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.728358 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.728797 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:44.729550 20726 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:22:44.729954 20726 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 6
I20260709 12:22:44.729843 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.730275 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:44.730770 24293 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.731510 24293 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:44.734387 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.740469 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.741336 20735 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.743310 24001 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:44.743883 24001 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:44.745164 24001 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.712448 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.746867 24001 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.766201 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.713253 23383 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.730322 20726 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:22:44.771941 20726 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:44.770911 23876 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.783659 20735 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:44.777161 20808 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:22:44.785068 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:44.761837 24242 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.788446 20803 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:44.787379 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:44.796068 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.797626 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:44.798039 23266 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:44.794150 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 250052us
I20260709 12:22:44.780562 23912 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.803010 20734 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:44.831756 23266 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.832974 24218 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.835414 20809 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.789228 24242 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:44.840418 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.842440 24219 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.844627 24219 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.847640 24161 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:44.848438 24161 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.837937 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:44.854122 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94385.9us
I20260709 12:22:44.856043 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:44.856206 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:44.857101 24111 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:44.857197 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62981.4us
I20260709 12:22:44.857555 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:44.857710 24111 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:44.858189 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.858693 24111 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:44.859251 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:44.860219 23984 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.775138 24064 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:44.861939 24197 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.004s
I20260709 12:22:44.862063 20810 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:44.861838 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.869486 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:44.870020 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:44.841113 24242 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:44.874895 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45985.5us
I20260709 12:22:44.851024 24161 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.888532 20877 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:44.892544 24246 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:44.893174 24246 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:44.894253 24246 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.896710 24064 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:44.898056 23782 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.871519 23266 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:44.899880 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 102367us
I20260709 12:22:44.900695 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:44.903151 23383 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.910463 20732 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:44.912366 23687 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:44.912900 21429 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:44.888437 24225 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.557s	user 0.006s	sys 0.009s
I20260709 12:22:44.919411 20875 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:44.925932 23383 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.871124 24197 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:44.930518 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68837.4us
W20260709 12:22:44.939751 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71516.3us
I20260709 12:22:44.924347 24219 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:44.875965 23912 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:44.941869 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 87794.6us
I20260709 12:22:44.945870 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:44.948764 21428 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:22:44.932317 23782 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.944736 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:44.954538 20805 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
W20260709 12:22:44.954833 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.934720 23266 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:44.955451 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.955840 24221 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.626s	user 0.007s	sys 0.007s
W20260709 12:22:44.956249 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.956328 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:44.957049 23912 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.957640 20802 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:44.958576 24231 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:44.959815 24231 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:44.959527 24192 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:44.939920 24218 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:44.938341 24239 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:44.965209 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:44.944386 24219 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:44.940881 24225 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:22:44.951114 23925 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.960286 24231 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:44.971415 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104494us
I20260709 12:22:44.967494 24219 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.987766 20726 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:44.988018 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:44.974995 24218 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:44.992195 24218 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:44.993093 24311 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.245s	user 0.000s	sys 0.004s
I20260709 12:22:44.988133 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:44.988159 20735 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:44.973052 23925 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:44.997756 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95631.7us
I20260709 12:22:44.955477 23266 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:44.982427 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.008277 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.019924 20875 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:22:45.004161 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:45.021381 20809 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:44.985450 24192 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.976758 24306 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.032902 23266 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.033561 21426 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:44.974555 23782 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:45.036872 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 77717.6us
I20260709 12:22:45.034925 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.016264 23925 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.984581 24196 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:22:44.997534 24239 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:45.074589 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:45.078846 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49806.7us
I20260709 12:22:45.041110 24306 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:45.098641 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54325.7us
I20260709 12:22:45.095307 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.099098 20875 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:45.093040 21429 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:45.094833 20807 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:45.095353 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50990us
I20260709 12:22:45.097832 23266 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:22:45.102792 24066 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:45.109308 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 110732us
I20260709 12:22:45.093945 23876 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:45.132388 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95728.1us
I20260709 12:22:45.134771 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:45.135501 21428 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:45.139421 24335 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:45.140141 24335 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.142603 23772 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.143612 24041 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:45.144089 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2176) took 1178 ms. Trace:
W20260709 12:22:45.144734 24051 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.003s
I20260709 12:22:45.144871 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:45.145505 21435 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:45.109908 24231 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.148116 24231 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.148720 24335 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.148113 21430 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:45.145171 24051 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.110075 24197 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.154348 20733 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:45.109606 24001 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:45.144483 20731 rpcz_store.cc:276] 0709 12:22:43.966118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.063818 (+ 97700us) service_pool.cc:224] Handling call
0709 12:22:45.143861 (+1080043us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54001d6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:45.144015 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242893056}
I20260709 12:22:45.160413 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:45.150946 24301 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.725s	user 0.007s	sys 0.005s
W20260709 12:22:45.168385 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58367.7us
I20260709 12:22:45.123389 24053 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:45.093520 24239 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.167429 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:45.122242 23687 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:45.172721 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 234524us
I20260709 12:22:45.171486 20804 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
W20260709 12:22:45.150076 23266 consensus_queue.cc:1175] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 7 status { last_received { term: 7 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 7 index: 7 } }
I20260709 12:22:45.124559 24196 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:22:45.175956 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.190929 24192 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:45.191625 24192 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:45.191691 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71493.2us
I20260709 12:22:45.192737 24192 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.193570 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:45.197690 24317 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:45.198199 24317 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:45.198632 24317 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.151594 24066 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 12:22:45.207690 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.208155 24066 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.166479 24001 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.208575 24066 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:45.209543 24066 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.210321 24001 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.210733 24066 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.212396 21430 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:45.218650 24219 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.219892 24051 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.222038 23925 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.233914 23925 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:45.235343 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58154.8us
I20260709 12:22:45.236364 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.240150 24219 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.242349 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.245167 24239 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.186339 23987 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:45.247895 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 923) took 1475 ms (client timeout 1650 ms). Trace:
W20260709 12:22:45.248386 21434 rpcz_store.cc:269] 0709 12:22:43.772281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.773309 (+  1028us) service_pool.cc:224] Handling call
0709 12:22:45.247508 (+1474199us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:45.247692 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254913920}
W20260709 12:22:45.250901 24315 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.694s	user 0.002s	sys 0.000s
I20260709 12:22:45.251875 24196 consensus_queue.cc:1243] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:45.176921 24161 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 12:22:45.258003 21428 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:45.258625 20728 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:45.261089 21435 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 12:22:45.269593 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 109382us
I20260709 12:22:45.279297 24318 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:45.279927 24318 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:45.280529 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44274.1us
I20260709 12:22:45.281011 24318 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.282557 24318 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.286902 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 472) took 1369 ms. Trace:
I20260709 12:22:45.287408 20729 rpcz_store.cc:276] 0709 12:22:43.918536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.203058 (+284522us) service_pool.cc:224] Handling call
0709 12:22:44.203859 (+   801us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:44.622394 (+418535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.286601 (+664207us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54000e74d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:45.286832 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":960764672}
I20260709 12:22:45.291121 24211 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.281495 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.291625 24211 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:45.296983 24246 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:45.297281 24211 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.298213 24246 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:45.298578 24246 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:45.298789 24211 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.301615 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.302752 23912 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:45.273600 24239 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:45.303707 23912 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:45.302906 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:45.304508 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.304782 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:45.303766 24219 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.305959 23925 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:45.306409 23925 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.307161 23925 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.307255 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.305389 21428 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:45.307596 24246 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.308986 24246 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.309926 23782 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:45.306339 23266 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:45.311501 23266 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.311904 23266 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:45.311926 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.316349 24197 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:45.316685 24231 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:45.292575 24296 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.001s
I20260709 12:22:45.324113 20807 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:45.322283 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:45.327521 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.268136 24228 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.321317 24219 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:45.277170 24242 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.321153 24197 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:22:45.352664 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:45.354179 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47155.7us
I20260709 12:22:45.344730 23782 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:45.347198 24306 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:45.326153 23772 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:22:45.363569 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.367133 20806 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:45.367333 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:45.368086 20803 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:45.355329 20729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:45.371531 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.371829 20729 raft_consensus.cc:606] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:45.375787 20801 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:45.376770 20804 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:45.377453 24289 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:22:45.372869 24225 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:45.378953 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:45.384469 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.384487 24053 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.385241 24053 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.385779 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.387722 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.389008 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:45.389196 20875 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:45.394781 24053 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.398314 24330 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.398904 24330 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.401160 24330 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.362555 24196 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:45.402283 24196 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.402658 24196 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:45.403630 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.405021 23687 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.405162 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:45.405804 20807 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:45.405546 23687 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.361346 24239 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:45.407802 23687 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.407737 24239 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.408207 20803 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:45.409865 24239 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:45.410051 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:45.410444 24306 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:45.410699 20803 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:45.411489 24306 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.351929 24296 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:22:45.423035 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:45.424175 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.426388 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:45.427214 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:45.433686 20810 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.382931 23925 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.415856 24289 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:45.454854 21432 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:45.459470 21428 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:45.437889 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.439610 24228 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.440826 20807 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:45.441421 23925 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.442157 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.442659 21431 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:45.444607 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:45.452654 24196 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.466845 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:45.463840 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 107949us
I20260709 12:22:45.462796 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.465397 21427 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.470111 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:45.469410 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:45.471304 23984 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.466229 24239 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:22:45.466379 23782 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:22:45.471933 23984 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.472415 20883 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:45.472617 24196 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.474037 23984 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.474824 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:45.476480 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:45.477470 24111 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:45.477933 24111 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:45.478763 24111 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.479243 24051 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:45.480043 24051 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.480583 24217 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:45.481068 24217 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:45.421357 23266 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.485759 21427 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:45.479223 23765 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:45.491086 20810 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:45.496601 20883 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:22:45.499346 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.499519 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.504024 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:45.506299 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.506920 20879 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.508749 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:45.510205 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.510766 21435 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:45.512598 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.514067 24242 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.514658 24228 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.515123 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.515230 24228 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:45.516580 24228 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.517958 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.518353 24228 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.518810 20880 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:45.519323 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 1916) took 1119 ms. Trace:
I20260709 12:22:45.519768 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.520069 23765 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:45.520347 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51027.7us
I20260709 12:22:45.520666 21430 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:45.521018 23765 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.521613 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:45.521864 21430 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:45.519797 23984 raft_consensus.cc:2764] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:22:45.524307 23383 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.524426 24289 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:45.524708 23383 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:45.524821 24289 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:45.525264 24289 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.522806 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.525488 23383 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.526629 24185 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:45.528630 23912 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.529201 23912 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.519723 20805 rpcz_store.cc:276] 0709 12:22:44.389384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.393593 (+  4209us) service_pool.cc:224] Handling call
0709 12:22:45.508408 (+1114815us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b5400126954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:45.508497 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287459072}
I20260709 12:22:45.532732 21432 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:22:45.532989 24340 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 12:22:45.533715 24313 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.785s	user 0.002s	sys 0.000s
I20260709 12:22:45.533872 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:45.534463 21434 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:45.535523 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:45.534971 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 1978) took 1031 ms. Trace:
I20260709 12:22:45.535941 24317 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.537690 20802 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:45.536615 20882 rpcz_store.cc:276] 0709 12:22:44.503531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.503627 (+    96us) service_pool.cc:224] Handling call
0709 12:22:44.504096 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:44.520850 (+ 16754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:44.520895 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:44.539946 (+ 19051us) log.cc:844] Serialized 26 byte log entry
0709 12:22:44.693602 (+153656us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:44.693656 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.693749 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:45.533042 (+839293us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:45.534680 (+  1638us) raft_consensus.cc:1685] finished
0709 12:22:45.534703 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:45.534913 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":916,"raft.queue_time_us":10769,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10570,"spinlock_wait_cycles":29317888,"thread_start_us":153196,"threads_started":1,"wal-append.queue_time_us":208547}
I20260709 12:22:45.482067 24217 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.502866 24296 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:45.538425 23687 consensus_queue.cc:1175] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 8 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:22:45.540357 24314 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.798s	user 0.005s	sys 0.000s
I20260709 12:22:45.538998 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:45.541775 20807 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:45.543203 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.543646 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:45.544680 20810 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:45.526960 23383 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.548545 21431 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:45.551155 21434 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:45.543226 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:45.554769 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.564389 21427 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:22:45.538437 24217 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.571501 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.573069 23912 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.573671 23772 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:45.565755 21428 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:45.567222 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.579317 24318 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:45.579721 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.580425 21431 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:45.549928 24196 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.577696 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.582712 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:45.592340 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 124360us
I20260709 12:22:45.502130 24231 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:22:45.507977 23782 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:22:45.558866 24001 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:45.550796 24242 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:45.602659 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:45.554608 24246 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:45.607044 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:45.611970 24231 consensus_queue.cc:1243] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:22:45.612553 24231 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:45.612874 24231 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:45.613683 24231 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.614626 23782 consensus_queue.cc:1243] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:22:45.616334 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.618089 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.618741 20875 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
W20260709 12:22:45.620112 23984 consensus_queue.cc:1175] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 8 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:45.622049 24318 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:45.622830 24053 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:45.627552 24246 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.629825 24246 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.631624 24051 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.632181 24196 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.634286 24196 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.647625 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.670699 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:45.648135 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.648255 20880 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:45.647403 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.671533 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:45.684235 20730 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:45.684767 20875 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:22:45.599627 24197 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:45.672276 20876 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:22:45.690587 20876 pending_rounds.cc:85] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:22:45.690920 20876 pending_rounds.cc:107] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:22:45.691250 20876 raft_consensus.cc:2889] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:45.589300 23772 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:22:45.599920 23912 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:45.684228 23984 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:22:45.615082 24231 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.680263 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:45.601172 24317 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.678747 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:45.698311 20735 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.679502 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:45.699338 20731 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:45.678203 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:45.686196 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:45.695492 23912 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.693909 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.701638 20882 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:45.707276 20734 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:45.733186 20728 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:45.678296 24242 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:45.677835 24041 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:45.678148 24001 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:45.689654 24192 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:45.749821 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 156424us
W20260709 12:22:45.750406 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101319us
I20260709 12:22:45.752955 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:45.754041 20731 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:45.751452 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 101249us
I20260709 12:22:45.707868 23912 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.759828 20726 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:22:45.749567 24310 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:22:45.765446 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.766058 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:45.737104 24185 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:45.773829 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:45.775792 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 154025us
I20260709 12:22:45.752182 24192 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:45.789546 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.788511 20735 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:45.791527 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.794704 20880 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:45.794791 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 206564us
I20260709 12:22:45.796865 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:45.798074 20726 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:45.796765 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.797977 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:45.803802 20809 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:45.804292 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:45.805227 20807 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:45.810017 20882 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:45.816642 20735 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:45.818380 20879 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:45.763912 23984 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:22:45.793218 24330 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:45.797403 24192 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.842423 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:45.778443 24317 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:22:45.843860 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.844427 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:22:45.846266 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49603.8us
I20260709 12:22:45.803539 24218 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:45.854596 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:45.815387 23266 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:45.860893 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.870934 20879 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:45.861546 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108761us
I20260709 12:22:45.866717 24231 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.861420 20880 raft_consensus.cc:2376] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9. Re-sending same reply.
W20260709 12:22:45.868456 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.814658 24239 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:22:45.850785 24330 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:45.877439 24330 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.878557 24330 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:45.880594 21399 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.537s (SENT)
I20260709 12:22:45.881093 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:45.881985 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 120625us
W20260709 12:22:45.883692 24345 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:22:45.883831 24192 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.884639 24351 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:45.885746 24346 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:22:45.886217 24192 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:45.886576 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.857869 24051 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:45.891011 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.852556 24242 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.822829 23782 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:22:45.809044 24185 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:22:45.857625 24246 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:45.897557 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.849591 24306 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:22:45.915827 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:45.906145 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:45.899412 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44907.2us
W20260709 12:22:45.918172 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.926627 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:45.927165 20735 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:45.936503 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:45.898139 24246 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:45.947588 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.859403 24001 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:45.953203 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70454.9us
I20260709 12:22:45.951016 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:45.954072 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:45.954653 20807 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:45.952828 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.921703 24355 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:45.932788 24218 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:45.958045 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60586.5us
I20260709 12:22:45.937676 23687 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:45.917548 24231 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:45.929128 24289 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:45.966620 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 90606us
I20260709 12:22:45.949393 24246 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.971114 20802 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:45.984783 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:45.984968 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.986925 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:45.991011 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:45.988967 24242 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.993360 24242 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:45.996194 20807 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:22:45.999424 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:45.999466 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:46.000146 21427 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:46.000434 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:46.001050 20880 raft_consensus.cc:2376] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8. Re-sending same reply.
I20260709 12:22:46.003413 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:46.004431 21426 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:46.009835 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:46.035413 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:45.991345 24362 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:46.005425 24041 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:46.042570 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:46.034188 24336 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.707s	user 0.003s	sys 0.001s
W20260709 12:22:46.044339 21399 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.602s (SENT)
W20260709 12:22:46.044658 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 230900us
I20260709 12:22:46.044823 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:45.966272 24218 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.050420 20735 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:46.033244 24217 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:46.051438 20735 raft_consensus.cc:606] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:46.037096 24219 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:46.052181 20735 raft_consensus.cc:614] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:46.057524 24289 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:46.058167 24348 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:22:46.059165 24218 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.061017 24231 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.012156 24351 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:22:46.041221 24296 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:46.033555 23687 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 12:22:46.071636 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61875us
W20260709 12:22:46.054340 24364 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:22:46.073105 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73713.6us
I20260709 12:22:46.049316 24355 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:46.044023 24041 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.074432 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:46.086899 20880 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:46.051286 24362 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:46.079965 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:46.123807 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:46.148149 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:46.154791 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.070314 24318 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.084157 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:46.175762 24242 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:46.191332 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98162us
W20260709 12:22:46.210418 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166107us
I20260709 12:22:46.215924 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:46.063401 24217 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:46.219769 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:46.078709 24306 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:46.225731 20880 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:46.209520 24318 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.233744 24192 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:22:46.234536 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 150473us
I20260709 12:22:46.180351 24041 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:46.064095 23984 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.155s
W20260709 12:22:46.239681 24352 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.248s	user 0.001s	sys 0.001s
I20260709 12:22:46.240629 20808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:22:46.241691 20807 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:22:46.241914 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 190290us
I20260709 12:22:46.244341 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:46.236423 24185 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:46.258668 20883 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:46.262356 20878 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:22:46.281543 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 201652us
I20260709 12:22:46.271344 20808 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:46.284405 20808 raft_consensus.cc:614] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:46.286514 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:46.286717 20801 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:46.287859 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.288017 24362 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.289283 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.290671 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:46.291429 20729 raft_consensus.cc:2376] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7. Re-sending same reply.
I20260709 12:22:46.291824 23772 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:46.292344 23772 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:46.253446 24296 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:46.250926 23925 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.294968 20875 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:46.239066 20727 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:46.319144 20735 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:22:46.353145 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56751.9us
I20260709 12:22:46.357888 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.359709 20728 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:22:46.365935 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83380.5us
I20260709 12:22:46.365981 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:46.369704 20728 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:46.369571 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.371405 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:46.373018 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:46.329401 24336 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:22:46.377493 20735 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 12:22:46.189779 24355 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.387815 24185 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:46.389485 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1896) took 1272 ms. Trace:
I20260709 12:22:46.389901 20734 rpcz_store.cc:276] 0709 12:22:45.116926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.117117 (+   191us) service_pool.cc:224] Handling call
0709 12:22:45.123476 (+  6359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:45.736899 (+613423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.389261 (+652362us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54003aded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:46.389426 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1321267456}
I20260709 12:22:46.355986 24351 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.393110 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:46.336452 24296 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:46.293859 24366 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:22:46.426796 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:46.370116 23925 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.427835 21427 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:46.429168 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:46.427395 23912 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.656s	user 0.015s	sys 0.002s
I20260709 12:22:46.412155 24296 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:46.412518 24368 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.412002 23687 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:46.308470 23772 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:46.464020 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 109550us
W20260709 12:22:46.467587 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:46.479692 20803 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.000s
I20260709 12:22:46.394243 24231 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:22:46.488607 20803 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:46.490012 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:46.504915 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:46.508191 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:46.513841 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 593) took 1622 ms. Trace:
I20260709 12:22:46.396790 24217 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:46.529603 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100546us
I20260709 12:22:46.514446 21433 rpcz_store.cc:276] 0709 12:22:44.884164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.892050 (+  7886us) service_pool.cc:224] Handling call
0709 12:22:44.931935 (+ 39885us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:44.973401 (+ 41466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:44.973441 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:44.996709 (+ 23268us) log.cc:844] Serialized 52 byte log entry
0709 12:22:45.131490 (+134781us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:45.150869 (+ 19379us) log.cc:844] Serialized 14 byte log entry
0709 12:22:45.178548 (+ 27679us) log.cc:844] Serialized 14 byte log entry
0709 12:22:45.178767 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.178845 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:45.871268 (+692423us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:46.506456 (+635188us) raft_consensus.cc:1685] finished
0709 12:22:46.506481 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.506658 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12211,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":17942,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":15394,"spinlock_wait_cycles":4911488,"thread_start_us":86415,"threads_started":1,"wal-append.queue_time_us":232248}
I20260709 12:22:46.526876 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:46.531319 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.532079 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:46.534578 20735 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:46.500697 24371 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:46.399403 24351 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:46.518966 24349 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 12:22:46.509493 24196 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:46.537179 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:46.461802 23912 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.549243 20730 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:22:46.550395 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.552109 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.567942 24296 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.570122 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.542255 24318 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.490662 24368 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:46.570611 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.571564 20807 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9.
I20260709 12:22:46.542773 24371 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:46.575695 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:46.576362 24355 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.576925 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.573259 24218 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:46.579985 23912 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.579701 24218 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.581550 24318 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.581931 24218 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.582062 24318 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.569278 24211 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:46.583663 24211 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.585055 24371 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:46.555763 24196 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:46.586544 24211 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.587340 24367 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.563606 24379 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.600718 20802 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:46.614496 20729 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:22:46.602905 20803 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:46.607865 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70792us
W20260709 12:22:46.643131 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.524294 24185 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:46.613543 24228 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.095s	user 0.010s	sys 0.015s
W20260709 12:22:46.621497 24374 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 12:22:46.619501 24356 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.461s	user 0.001s	sys 0.001s
W20260709 12:22:46.615803 24360 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.448s	user 0.001s	sys 0.000s
I20260709 12:22:46.668931 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1934) took 1135 ms. Trace:
I20260709 12:22:46.616027 24391 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.618305 24351 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.616544 24394 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:46.687764 20850 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:46.689175 24246 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.000s
I20260709 12:22:46.689510 24246 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.690294 21431 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:46.692018 20876 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:46.604139 24367 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.675493 20806 rpcz_store.cc:276] 0709 12:22:45.533020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.542375 (+  9355us) service_pool.cc:224] Handling call
0709 12:22:45.542687 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:45.593523 (+ 50836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:45.593571 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:45.615759 (+ 22188us) log.cc:844] Serialized 26 byte log entry
0709 12:22:45.771628 (+155869us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:45.772103 (+   475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.772226 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:46.668727 (+896501us) raft_consensus.cc:1685] finished
0709 12:22:46.668746 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.668868 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5779,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":155345,"threads_started":1,"wal-append.queue_time_us":244900}
I20260709 12:22:46.611784 24379 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:46.703994 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.610875 24041 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.708617 24041 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.683792 24365 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.710341 24365 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.714167 24365 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.716706 24372 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:46.716830 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:46.717799 24041 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.718290 24372 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:46.718811 24372 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:46.719704 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:46.720507 24211 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:46.720902 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.719844 24372 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.721012 24211 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:46.722527 24211 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.732054 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.734273 20701 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:46.738052 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.738847 20803 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
W20260709 12:22:46.740598 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101764us
I20260709 12:22:46.741511 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:46.691258 24246 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.745217 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:46.718648 24041 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.750516 24382 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:46.750945 24382 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.751160 24041 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.751333 24382 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:46.758373 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.746199 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:46.758905 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71220.8us
I20260709 12:22:46.771572 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.772518 20728 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:46.743319 23782 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:46.785899 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:46.790189 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:46.745461 23925 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.795522 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 793) took 1231 ms. Trace:
I20260709 12:22:46.767813 24375 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.763458 24365 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:46.691573 24391 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.798223 21434 rpcz_store.cc:276] 0709 12:22:45.564043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.591219 (+ 27176us) service_pool.cc:224] Handling call
0709 12:22:45.596901 (+  5682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:45.697047 (+100146us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:22:45.778981 (+ 81934us) log.cc:844] Serialized 14 byte log entry
0709 12:22:46.461169 (+682188us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b18003c9be0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:22:46.793301 (+332132us) log.cc:844] Serialized 14 byte log entry
0709 12:22:46.794117 (+   816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:46.794188 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:46.794241 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.794326 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.795443 (+  1117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":587,"prepare.queue_time_us":24042,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":9050,"raft.queue_time_us":12717,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":10787,"spinlock_wait_cycles":11253504,"thread_start_us":327427,"threads_started":2}
I20260709 12:22:46.832979 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:46.766863 24318 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:46.834668 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:46.837386 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:46.694720 24393 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 12:22:46.799345 24330 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.918s	user 0.009s	sys 0.004s
I20260709 12:22:46.735790 24380 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:22:46.846731 24399 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:46.768970 24351 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 12:22:46.847549 24356 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:22:46.853451 20810 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:22:46.857805 20810 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:22:46.852145 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:46.858302 20810 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:22:46.858604 20810 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:46.855270 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88576us
I20260709 12:22:46.846995 23782 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:46.845223 24365 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:46.859519 24330 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:46.849063 24399 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.877215 24399 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:46.878420 24371 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.832512 24392 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:22:46.850092 24394 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:46.872797 24391 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:46.864486 23782 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:46.845827 24228 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:22:46.873714 24379 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.865695 24330 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.874089 24390 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:46.876223 24368 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.849978 24375 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.873745 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.898998 24375 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.899624 24365 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.900054 24394 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.865451 24318 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:22:46.926394 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.888361 23782 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:46.894258 24373 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:22:46.889777 24330 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:46.932902 20803 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
W20260709 12:22:46.943375 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60650.6us
W20260709 12:22:46.944585 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 158807us
I20260709 12:22:46.947871 20802 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:46.949149 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:46.949874 20802 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:46.953323 20805 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:46.935268 24410 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.938750 24366 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.957623 20805 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:46.939363 24318 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:22:46.956286 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:46.960078 24400 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:22:46.960093 20802 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:46.965431 24382 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:46.965942 24296 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:46.970170 24416 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:46.970623 24416 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:46.970610 24318 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:46.972705 20806 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:22:46.971268 24318 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:46.988890 24417 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:46.956251 24367 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:46.972026 24410 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:46.996559 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 601) took 1099 ms. Trace:
I20260709 12:22:47.017858 20876 rpcz_store.cc:276] 0709 12:22:45.897072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.897223 (+   151us) service_pool.cc:224] Handling call
0709 12:22:46.996240 (+1099017us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54000a41d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:46.996461 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812005248}
I20260709 12:22:46.998826 24318 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.021903 24318 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:47.001294 24396 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:22:47.001454 20806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:22:47.024633 20806 raft_consensus.cc:606] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:47.024770 24358 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.986s	user 0.007s	sys 0.005s
I20260709 12:22:47.025060 20806 raft_consensus.cc:614] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:46.991492 24416 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.005452 24196 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.008373 24393 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.012925 24417 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:46.971359 24366 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.014151 24382 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.014735 24388 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:47.015110 24399 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.107688 24399 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.032985 24388 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:47.108929 24388 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.109567 24399 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:47.017195 24362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.006s	sys 0.005s
I20260709 12:22:47.024127 24410 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:47.113773 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.114060 20876 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:47.120398 20875 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:47.122975 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.110198 24388 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:47.129412 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.129428 20876 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 12:22:47.119105 24362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.006s	sys 0.005s
I20260709 12:22:47.130868 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.131719 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:47.132661 20882 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:47.121986 24196 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:47.134169 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 187942us
I20260709 12:22:47.135339 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.135761 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:47.137101 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:47.137533 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.138808 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.140856 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:47.141987 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:47.142763 20727 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:22:47.144866 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.143893 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.145622 20876 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:47.144618 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.133698 24367 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.147078 20875 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:47.030078 24393 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.134903 24410 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:47.148468 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.110231 24399 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.139272 24416 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.147497 24185 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.156303 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.157872 20876 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:47.160024 24185 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.161466 24393 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:47.162031 24393 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:47.162467 24185 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.162690 24393 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:47.155308 24422 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:47.153028 24367 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.160074 24362 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:47.164543 24366 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.164849 24393 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:47.138377 24417 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.168797 24417 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.173377 24422 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:47.174007 24422 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:47.175076 24422 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.175231 24367 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:47.183360 24423 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:22:47.201424 24239 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:47.184978 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.187985 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.196650 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:47.196862 20876 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:22:47.171882 24393 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.205623 24239 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.206001 24423 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:47.206728 24423 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:47.207931 24239 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.207643 24423 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.209090 24423 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:47.213963 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.214833 20876 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:47.222337 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.223306 20876 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:47.223165 20882 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:47.221414 24318 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.227581 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.228180 20876 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.229298 24366 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.229696 24399 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.229856 24366 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.232268 24366 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.249420 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.251361 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.251964 20878 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:47.254401 23876 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:47.254993 23876 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.255564 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.256361 20802 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:22:47.257077 23876 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:47.257700 20801 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:47.255569 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.258677 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.259900 24221 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:47.260540 24397 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 12:22:47.260390 24221 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.262862 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.268201 20878 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
W20260709 12:22:47.268563 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45034.3us
I20260709 12:22:47.269459 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:47.273311 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.271040 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:47.277421 20880 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:22:47.277595 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 135664us
I20260709 12:22:47.274034 21433 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:47.281071 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.283180 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:47.284258 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.285110 21427 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:47.286278 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:47.287179 21426 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 7: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:47.289276 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.289948 21434 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:47.290963 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.292140 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.292687 20880 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:47.249675 23782 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.294765 24318 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:47.297111 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44599.9us
I20260709 12:22:47.297613 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:47.299613 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.300856 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:47.297771 21399 leader_election.cc:336] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.712s (SENT)
I20260709 12:22:47.301908 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.303251 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.295037 23782 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.276136 23876 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.281956 24239 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.280844 24417 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.283288 24296 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.295091 23925 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:47.295315 24367 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:47.318468 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2004) took 1073 ms. Trace:
I20260709 12:22:47.277495 24317 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:22:47.310073 24219 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:47.324322 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.309660 24419 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:47.322429 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.328060 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.323861 20879 rpcz_store.cc:276] 0709 12:22:46.239046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.239408 (+   362us) service_pool.cc:224] Handling call
0709 12:22:46.239724 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.240069 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:46.486006 (+245937us) log.cc:844] Serialized 14 byte log entry
0709 12:22:47.312193 (+826187us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b180030a180. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:22:47.312294 (+   101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:47.312340 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:47.312384 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.312452 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.312608 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4851,"prepare.queue_time_us":318740,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2336,"raft.queue_time_us":72257,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":7462,"spinlock_wait_cycles":15193984,"thread_start_us":296586,"threads_started":2}
I20260709 12:22:47.330495 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.330138 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:47.334280 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:47.337374 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.313640 23876 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.323602 23925 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.343595 23876 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:47.343704 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:47.344302 21433 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:47.344205 23876 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:47.344585 20809 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:47.345221 23876 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:47.345876 23876 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:47.346190 23876 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:47.347095 24221 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:47.343464 23925 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.343240 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:47.350545 21434 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.317950 24239 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.353140 24239 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:47.337992 21426 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:47.353696 24330 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.319160 24394 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:47.323267 24367 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.306072 23687 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:22:47.339425 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.311700 24423 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:22:47.377228 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:47.360386 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:47.378052 20726 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:47.327149 24219 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:47.367739 20880 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:47.368916 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:47.384809 20732 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:47.378187 20808 raft_consensus.cc:2410] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:47.309165 24185 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:22:47.339092 21428 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:47.382807 21434 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:47.323246 24417 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.394413 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:47.396631 20731 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:47.323570 24296 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.374666 24367 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:47.400790 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.401441 21426 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:47.402441 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:47.334144 24393 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:22:47.400658 24380 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:47.406881 24296 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.409119 20808 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:47.434922 21434 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:47.411659 20801 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:47.416198 24330 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.416693 20875 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:22:47.417137 24423 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:47.418111 24427 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:47.388833 24219 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.421618 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:47.432644 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:47.433998 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:47.445598 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:47.445669 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:47.436035 21399 connection.cc:256] RPC call timeout handler was delayed by -1.1642334949997348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 12:22:47.446467 24317 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:47.435356 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.446848 24317 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.438737 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.447300 24317 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.447489 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.447690 21426 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:47.448158 20880 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
W20260709 12:22:47.441252 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 107045us
I20260709 12:22:47.447064 20808 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:47.450225 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.430960 24221 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.368497 23876 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.487331 23876 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.491216 24416 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:47.491809 24416 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.493465 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.494066 24416 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:47.481580 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.496811 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.442440 24427 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.503645 24427 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:47.504842 21434 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:47.506558 24416 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:47.512431 24417 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:47.439459 24296 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.513510 24296 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.494125 20807 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:47.483721 20808 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:47.520102 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.447347 24380 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:22:47.524828 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.526324 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:47.527262 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.447355 24246 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.485826 24242 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:22:47.524047 24409 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.014s
I20260709 12:22:47.563025 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.555064 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.548110 23876 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:47.573010 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.517287 24221 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:47.564131 24409 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.009s	sys 0.014s
I20260709 12:22:47.579471 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:47.555575 24380 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:47.545589 24330 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 12:22:47.584689 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.584372 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:47.556216 24246 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.569191 24318 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:47.576210 23876 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.602445 24318 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.604382 24219 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.605916 24246 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:47.579977 24242 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:47.607597 24242 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:47.608125 24242 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:47.609249 21428 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:22:47.609064 24242 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.612011 24318 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:47.618000 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.619201 21428 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:47.620780 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:47.621655 24416 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:47.622401 24416 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:22:47.622906 24416 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:47.623567 24416 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:47.624091 24416 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.577383 24434 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:47.577672 24435 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:47.633916 20876 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:47.589215 24433 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:47.643653 24380 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.644742 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:47.646943 24239 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:47.647384 23782 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:47.647832 23782 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.648207 23782 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:47.648461 24239 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:47.649832 24239 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:47.650211 24239 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:47.637899 24385 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:22:47.646746 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:47.651862 24306 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:47.653012 24246 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:47.586463 23687 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:22:47.596691 23876 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:47.646775 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:47.706616 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59926.4us
I20260709 12:22:47.656951 24427 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.657670 24219 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:47.661676 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:47.597878 24296 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:47.655721 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 794) took 1240 ms. Trace:
I20260709 12:22:47.667328 23925 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.673709 24317 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.675673 24221 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:47.677747 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.678716 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.680881 20806 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:47.661298 24376 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.737726 21429 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:47.693115 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.670506 23782 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.715893 20879 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:22:47.740918 20849 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9.
I20260709 12:22:47.741274 20849 leader_election.cc:403] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:47.677054 24053 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:47.741443 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.741986 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.729606 21427 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:47.726696 21431 rpcz_store.cc:276] 0709 12:22:46.415595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.421215 (+  5620us) service_pool.cc:224] Handling call
0709 12:22:46.421552 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.690634 (+269082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.655476 (+964842us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400750154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:47.655659 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617629056}
I20260709 12:22:47.730266 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.730513 20808 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:22:47.733400 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.744936 21430 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:22:47.735476 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.745287 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.745880 20880 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:47.676487 24367 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.746343 20877 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:47.745918 20884 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:22:47.745142 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 160847us
W20260709 12:22:47.747890 20632 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 196) took 31091 ms (client timeout 15000 ms). Trace:
I20260709 12:22:47.752367 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.752774 24185 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.753244 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:47.753532 24185 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.756018 20883 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:47.756330 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.725601 24318 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:47.759021 20802 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:47.762192 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.748195 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:47.748624 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.765137 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.748769 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.750820 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.766371 20879 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:47.746855 21430 pending_rounds.cc:85] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 7
I20260709 12:22:47.767382 21430 pending_rounds.cc:107] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:22:47.767729 21430 raft_consensus.cc:2889] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:47.747751 24053 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:47.785356 20881 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:22:47.785109 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.786754 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:47.786666 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.787870 20884 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:47.789283 20877 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:47.725370 24417 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:22:47.790220 21427 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:47.791757 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:47.792554 20880 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:47.772876 24318 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.775941 24296 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.775739 24388 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:47.776086 24434 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:47.751989 20632 rpcz_store.cc:269] 0709 12:22:16.656069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.656324 (+   255us) service_pool.cc:224] Handling call
0709 12:22:47.747664 (+31091340us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:36.763272 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:36.764001 (+   729us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:36.764056 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:22:36.764077 (+    21us) tablet.cc:662] Decoding operations
0709 12:22:36.765121 (+  1044us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:36.765213 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:36.765244 (+    31us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:22:36.766343 (+  1099us) tablet.cc:701] Row locks acquired
0709 12:22:36.766395 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:22:36.766549 (+   154us) write_op.cc:270] Start()
0709 12:22:36.766687 (+   138us) write_op.cc:276] Timestamp: P: 1783599756766482 usec, L: 0
0709 12:22:36.766707 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:22:36.833262 (+ 66555us) log.cc:844] Serialized 4363 byte log entry
0709 12:22:37.557689 (+724427us) op_driver.cc:464] REPLICATION: finished
0709 12:22:38.371962 (+814273us) write_op.cc:301] APPLY: starting
0709 12:22:38.429474 (+ 57512us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:38.754092 (+324618us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:38.787089 (+ 32997us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:39.173378 (+386289us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:39.257772 (+ 84394us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:22:39.262020 (+  4248us) write_op.cc:312] APPLY: finished
0709 12:22:39.287372 (+ 25352us) log.cc:844] Serialized 83 byte log entry
0709 12:22:39.470261 (+182889us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:39.488282 (+ 18021us) write_op.cc:454] Released schema lock
0709 12:22:39.495544 (+  7262us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:39.565818 (+ 70274us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:39.571959 (+  6141us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":165466,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":533997,"apply.run_cpu_time_us":53942,"apply.run_wall_time_us":1257593,"num_ops":11,"prepare.queue_time_us":103546,"prepare.run_cpu_time_us":19921,"prepare.run_wall_time_us":256120,"raft.queue_time_us":212292,"raft.run_cpu_time_us":14605,"raft.run_wall_time_us":548793,"replication_time_us":720854,"thread_start_us":910390,"threads_started":5,"wal-append.queue_time_us":400517,"wal-append.run_cpu_time_us":46245,"wal-append.run_wall_time_us":2471578}]]}
I20260709 12:22:47.799422 24410 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9.
I20260709 12:22:47.811518 24410 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:47.812209 24410 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.755743 24376 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:47.821938 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 124971us
I20260709 12:22:47.816054 20805 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:47.798630 24318 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:47.817262 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:47.838388 21427 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:47.808957 24392 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.007s	sys 0.015s
I20260709 12:22:47.801203 24388 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:47.926162 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.815156 24410 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:47.964659 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:47.965672 21428 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:47.894248 24392 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.007s	sys 0.016s
I20260709 12:22:47.787961 24239 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:47.937240 24388 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:47.804555 24296 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:48.008494 24446 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:22:48.009904 21427 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:48.010208 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:48.013239 20880 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:22:48.018836 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 311052us
I20260709 12:22:48.020447 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:47.809814 24434 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.030390 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:48.004092 24427 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.025580 24239 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.022666 24185 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.052947 21434 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:48.054961 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:48.039820 23925 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:48.067082 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.069285 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:48.024665 24415 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:22:48.070081 21434 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:48.034147 24380 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:22:48.059392 24317 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.088004 24317 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:48.088063 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:48.088609 24317 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:48.088690 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:48.089160 24317 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:48.092103 24444 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:22:48.097278 23772 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:48.100080 24289 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:48.100872 24289 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:48.105146 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.105978 24377 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.106518 24466 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.107419 24466 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.113268 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:48.114199 20729 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:48.118062 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:48.119935 24452 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:22:48.074491 24367 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:48.079661 23925 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:48.120656 20729 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:48.121152 23925 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.080344 24416 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.081871 24427 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:48.123277 23925 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.123771 24367 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.072641 24465 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:48.124217 24367 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:48.124634 24465 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:48.126562 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.127424 24382 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:48.127732 20878 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:48.127969 24382 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.129202 24416 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.130147 20878 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:48.130515 20878 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:48.131479 20878 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:48.132401 20878 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.133154 20733 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:48.136113 24422 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:48.136740 24422 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:48.137734 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2206) took 1794 ms. Trace:
I20260709 12:22:48.138123 20727 rpcz_store.cc:276] 0709 12:22:46.343695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.352234 (+  8539us) service_pool.cc:224] Handling call
0709 12:22:46.352654 (+   420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:47.143183 (+790529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.137484 (+994301us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54001d8b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:48.137670 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816435456}
I20260709 12:22:48.137907 24422 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.140107 20878 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:22:48.140465 20878 pending_rounds.cc:85] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:22:48.140657 20878 pending_rounds.cc:107] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:22:48.140856 20878 raft_consensus.cc:2889] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:48.083551 24053 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:48.119706 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:48.121845 24296 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:48.078248 24381 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.624s	user 0.006s	sys 0.000s
I20260709 12:22:48.146437 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2164) took 1197 ms. Trace:
I20260709 12:22:48.149362 20805 rpcz_store.cc:276] 0709 12:22:46.949153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.958631 (+  9478us) service_pool.cc:224] Handling call
0709 12:22:46.958900 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.837805 (+878905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.140204 (+302399us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54003ca354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:48.146362 (+  6158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970627840}
I20260709 12:22:48.148133 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:48.158473 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:48.163481 20733 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
W20260709 12:22:48.168835 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.170640 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:48.193475 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:48.164655 24439 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 12:22:48.106676 24377 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:48.194164 20734 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:48.144788 23876 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.164803 24435 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:48.213538 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 375862us
I20260709 12:22:48.622969 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:48.441042 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.405819 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.499763 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:48.311631 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:48.483224 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1967) took 1057 ms. Trace:
I20260709 12:22:48.291975 20735 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:48.274888 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:48.634920 20727 raft_consensus.cc:2368] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10.
W20260709 12:22:48.212675 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.675s (SENT) [suppressed 15 similar messages]
W20260709 12:22:48.644788 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.620733 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 407838us
I20260709 12:22:48.651762 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:48.656591 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:48.596095 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 493) took 1235 ms. Trace:
I20260709 12:22:48.658881 20728 rpcz_store.cc:276] 0709 12:22:47.361011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.361496 (+   485us) service_pool.cc:224] Handling call
0709 12:22:47.361819 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.383871 (+ 22052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.383910 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.384436 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:22:47.564743 (+180307us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:47.564781 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.564851 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:48.443904 (+879053us) raft_consensus.cc:1685] finished
0709 12:22:48.443934 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.595993 (+152059us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13135,"raft.queue_time_us":23546,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":6603,"thread_start_us":159297,"threads_started":1,"wal-append.queue_time_us":329239}
I20260709 12:22:48.630565 20809 rpcz_store.cc:276] 0709 12:22:47.425569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.089265 (+663696us) service_pool.cc:224] Handling call
0709 12:22:48.089567 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:48.089765 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:48.112181 (+ 22416us) log.cc:844] Serialized 14 byte log entry
0709 12:22:48.482709 (+370528us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:48.482791 (+    82us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:48.482855 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.482951 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.483126 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20470,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":809,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":47677,"thread_start_us":350906,"threads_started":1}
I20260709 12:22:48.680605 20734 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:48.192173 24289 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:48.192649 24453 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:48.691898 20809 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:22:48.694703 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:48.192584 24422 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:48.321156 24462 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:22:48.358961 24330 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.704193 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:48.250929 24376 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:48.320729 24221 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:48.712086 20801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.011s	sys 0.027s
I20260709 12:22:48.713455 20809 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:48.283048 23782 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:48.718078 20801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.011s	sys 0.027s
I20260709 12:22:48.721273 20731 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:22:48.732021 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.764766 20809 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:48.699894 24225 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.760097 24221 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:48.704150 23925 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:48.773010 23876 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.774847 20809 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:48.784139 20731 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:48.717412 24434 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.885535 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1917) took 2114 ms. Trace:
I20260709 12:22:48.885766 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 610) took 1205 ms. Trace:
I20260709 12:22:48.886934 21432 rpcz_store.cc:276] 0709 12:22:47.680071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.680282 (+   211us) service_pool.cc:224] Handling call
0709 12:22:47.685239 (+  4957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.793330 (+108091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.793376 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:47.793435 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:47.793490 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.885557 (+1092067us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54005f3a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:48.885593 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.885711 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247945088}
I20260709 12:22:48.888208 20730 rpcz_store.cc:276] 0709 12:22:46.771495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.773210 (+  1715us) service_pool.cc:224] Handling call
0709 12:22:46.793751 (+ 20541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.903690 (+109939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.927127 (+ 23437us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:46.927182 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:46.927251 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.885201 (+1957950us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54000c53d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:48.885265 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.885455 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252337024}
I20260709 12:22:48.716974 24330 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:48.892999 24462 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:22:48.686256 24435 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:48.879371 24469 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.899348 24371 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:48.893602 23876 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:48.893347 24376 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:48.896343 24362 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:48.870522 24225 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:48.919266 20804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.025s
W20260709 12:22:48.920058 20804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.013s	sys 0.025s
I20260709 12:22:48.918597 24377 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:48.872478 24365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.000s	sys 0.038s
W20260709 12:22:48.927901 24365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.000s	sys 0.039s
W20260709 12:22:48.918902 20878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.011s	sys 0.025s
W20260709 12:22:48.928871 20878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.011s	sys 0.025s
W20260709 12:22:48.918541 23984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.010s	sys 0.030s
W20260709 12:22:48.929848 23984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.010s	sys 0.030s
I20260709 12:22:48.878592 23925 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:48.939642 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.940809 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.018s	sys 0.026s
W20260709 12:22:48.941273 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.018s	sys 0.026s
W20260709 12:22:48.944978 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.949265 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:48.917802 23876 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:48.944576 24365 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:48.951074 23772 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.853s	user 0.010s	sys 0.004s
W20260709 12:22:48.958285 24419 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:22:48.919983 24362 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:48.916898 24376 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.985726 21435 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:22:48.986119 21435 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 7
I20260709 12:22:48.986419 21435 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:22:48.986696 21435 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:48.909688 24469 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:48.985405 24419 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.996155 24306 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.012130 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1966) took 1587 ms. Trace:
I20260709 12:22:49.012691 20807 rpcz_store.cc:276] 0709 12:22:47.425237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.807917 (+382680us) service_pool.cc:224] Handling call
0709 12:22:47.848188 (+ 40271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.896969 (+ 48781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.897015 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.953322 (+ 56307us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.596439 (+643117us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:48.596499 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.596596 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.011884 (+415288us) raft_consensus.cc:1685] finished
0709 12:22:49.011914 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.012062 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1318,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":26425,"thread_start_us":171336,"threads_started":1,"wal-append.queue_time_us":913682}
W20260709 12:22:49.014681 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.998457 24380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.004s	sys 0.052s
I20260709 12:22:49.001766 24306 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:49.014808 24468 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:22:49.020407 24477 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:22:49.012125 24463 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:48.985657 24365 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:49.029340 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 377676us
I20260709 12:22:49.023444 24221 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:49.022296 24380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.005s	sys 0.055s
I20260709 12:22:49.029009 24463 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:49.033463 24330 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.042165 20807 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:49.042737 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2014) took 1330 ms (client timeout 1503 ms). Trace:
W20260709 12:22:49.044672 20881 rpcz_store.cc:269] 0709 12:22:47.712512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.730901 (+ 18389us) service_pool.cc:224] Handling call
0709 12:22:49.042526 (+1311625us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b540005b254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.042670 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32831872}
W20260709 12:22:49.043385 24471 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:22:49.045974 24365 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.046708 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1871) took 1231 ms. Trace:
I20260709 12:22:49.001444 24419 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.014482 23876 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.045892 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:49.049566 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 836842us
I20260709 12:22:49.055004 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.056916 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1968) took 1631 ms. Trace:
I20260709 12:22:49.044371 24463 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:49.061101 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1868) took 1279 ms. Trace:
I20260709 12:22:49.047089 20884 rpcz_store.cc:276] 0709 12:22:47.815682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.815866 (+   184us) service_pool.cc:224] Handling call
0709 12:22:47.816256 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.837162 (+ 20906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.837199 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.895609 (+ 58410us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.162588 (+266979us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:48.162623 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.162701 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.046471 (+883770us) raft_consensus.cc:1685] finished
0709 12:22:49.046496 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.046649 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":207972,"spinlock_wait_cycles":12681088,"thread_start_us":136302,"threads_started":1,"wal-append.queue_time_us":898980}
I20260709 12:22:49.062702 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:49.063331 20803 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:22:49.063458 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:49.065009 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.066841 20884 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.065521 20881 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:49.073487 20807 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:22:49.073496 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.074736 20881 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.062251 20877 rpcz_store.cc:276] 0709 12:22:47.782121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.790070 (+  7949us) service_pool.cc:224] Handling call
0709 12:22:47.800021 (+  9951us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.804777 (+  4756us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:47.804810 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.837304 (+ 32494us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.283890 (+446586us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:48.283939 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.284016 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.060909 (+776893us) raft_consensus.cc:1685] finished
0709 12:22:49.060928 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.061048 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200019,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13308,"thread_start_us":320077,"threads_started":1,"wal-append.queue_time_us":821053}
W20260709 12:22:49.062880 20880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.013s	sys 0.016s
I20260709 12:22:49.077644 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2029) took 1044 ms. Trace:
I20260709 12:22:49.080709 20803 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:49.077996 20880 rpcz_store.cc:276] 0709 12:22:48.033804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.035010 (+  1206us) service_pool.cc:224] Handling call
0709 12:22:48.053295 (+ 18285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:48.054216 (+   921us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:48.442925 (+388709us) log.cc:844] Serialized 14 byte log entry
0709 12:22:49.077315 (+634390us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:49.077372 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:49.077407 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.077468 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.077571 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":841658,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":1001931,"threads_started":2}
I20260709 12:22:49.089893 24306 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.090703 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 555) took 1266 ms. Trace:
W20260709 12:22:49.095100 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134010us
W20260709 12:22:49.101991 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.105989 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.106710 20803 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:49.098367 20804 rpcz_store.cc:276] 0709 12:22:47.824239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.159539 (+335300us) service_pool.cc:224] Handling call
0709 12:22:48.173674 (+ 14135us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.174112 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.174152 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.195721 (+ 21569us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.920639 (+724918us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:48.920696 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.920790 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.090416 (+169626us) raft_consensus.cc:1685] finished
0709 12:22:49.090445 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.090624 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19712,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":741,"thread_start_us":609576,"threads_started":1,"wal-append.queue_time_us":771003}
W20260709 12:22:49.099680 24479 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:22:49.109566 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.109215 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.113895 20732 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.115442 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.117967 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:49.121719 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.102622 24369 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:49.107601 24376 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:49.113585 24239 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.058s	user 0.012s	sys 0.001s
I20260709 12:22:49.127084 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.128175 20730 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.129750 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.111274 24435 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.115672 24111 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.115585 24289 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.113238 23984 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:49.133038 20879 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:49.113090 24377 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:22:49.140304 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.140594 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.142186 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.144203 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.145807 20803 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:49.146790 20734 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:49.123742 24464 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.150529 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.149621 24377 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.155982 24419 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.125808 24188 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:22:49.157976 24377 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.158551 24377 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:49.159276 24365 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:22:49.160830 24478 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:22:49.161705 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.162329 20884 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:49.162443 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2032) took 1015 ms. Trace:
W20260709 12:22:49.163257 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.162998 20878 rpcz_store.cc:276] 0709 12:22:48.146626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.161869 (+ 15243us) service_pool.cc:224] Handling call
0709 12:22:48.162157 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.162368 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.162392 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.171752 (+  9360us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.929399 (+757647us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:48.929457 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.929553 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.162185 (+232632us) raft_consensus.cc:1685] finished
0709 12:22:49.162214 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.162372 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":658061,"threads_started":1,"wal-append.queue_time_us":802062}
I20260709 12:22:49.168944 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.140749 24362 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.180768 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.183386 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.145460 24239 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.184253 20803 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
W20260709 12:22:49.185886 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.186683 20803 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
W20260709 12:22:49.187362 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 63747.9us
I20260709 12:22:49.131815 24369 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.166392 24464 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.191720 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:49.205353 24393 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.205907 24393 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.206362 24393 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.206492 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.207818 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.208796 24419 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.209648 24419 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:49.212514 24410 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:49.217553 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.188397 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.218318 20807 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:49.191262 24306 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.219347 24306 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.219959 24306 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.220055 24410 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.220839 24410 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.189690 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.192901 24365 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:22:49.223517 24365 consensus_queue.cc:1243] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:49.191529 23772 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:49.223822 24410 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.224378 23772 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.197860 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2177) took 1136 ms. Trace:
W20260709 12:22:49.199069 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2018) took 1485 ms (client timeout 1855 ms). Trace:
W20260709 12:22:49.226670 20883 rpcz_store.cc:269] 0709 12:22:47.713734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.714195 (+   461us) service_pool.cc:224] Handling call
0709 12:22:49.198836 (+1484641us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b540019d7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.199000 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29919616}
I20260709 12:22:49.195442 24394 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.200107 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2215) took 1082 ms. Trace:
W20260709 12:22:49.201153 21399 connection.cc:256] RPC call timeout handler was delayed by -1.0667221529997732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 12:22:49.233013 21399 connection.cc:256] RPC call timeout handler was delayed by -1.0406202699996356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2542s for a response
I20260709 12:22:49.149108 23984 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.234879 23772 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.235504 23772 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.236048 23772 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.144932 24111 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.229485 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.230659 24464 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.260015 24111 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.236670 23984 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.260917 24480 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.261312 24376 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.261510 24480 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.261819 24376 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.219132 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:49.263569 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44537us
I20260709 12:22:49.264286 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.264587 24367 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.265147 24367 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:49.265585 24367 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:49.265630 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:49.238353 24388 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.213s	user 0.011s	sys 0.001s
W20260709 12:22:49.239280 24318 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.287s	user 0.014s	sys 0.000s
I20260709 12:22:49.146967 24289 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.193571 24239 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.225435 20805 rpcz_store.cc:276] 0709 12:22:48.061299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.161540 (+100241us) service_pool.cc:224] Handling call
0709 12:22:48.550141 (+388601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:49.028221 (+478080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:49.028254 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:49.061620 (+ 33366us) log.cc:844] Serialized 26 byte log entry
0709 12:22:49.143535 (+ 81915us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:49.143570 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.143650 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.197532 (+ 53882us) raft_consensus.cc:1685] finished
0709 12:22:49.197561 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.197765 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":75247,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":31711,"thread_start_us":538629,"threads_started":2,"wal-append.queue_time_us":122429}
I20260709 12:22:49.199962 24369 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.252595 24377 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:49.255810 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.282341 20730 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:49.282590 20883 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:49.232187 20735 rpcz_store.cc:276] 0709 12:22:48.117581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.117727 (+   146us) service_pool.cc:224] Handling call
0709 12:22:49.199913 (+1082186us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.200035 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397939456}
W20260709 12:22:49.240294 24317 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.151s	user 0.013s	sys 0.001s
W20260709 12:22:49.280732 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67680.7us
I20260709 12:22:49.283931 24492 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:49.288350 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.289955 24492 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.290357 24492 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.290505 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:49.271874 23984 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:49.300902 24499 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:49.307273 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.274850 24111 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:22:49.273861 24318 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.274159 24388 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.266496 24367 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.313352 24496 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:22:49.321193 24450 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.317s	user 0.007s	sys 0.016s
I20260709 12:22:49.323065 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.323949 20883 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:49.324330 24289 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.275025 24394 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.284428 24463 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.328761 24463 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.330499 24318 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.330294 24306 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.304945 24393 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.331317 24376 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.331907 24376 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.332253 24376 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.332580 24376 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.332923 24393 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.303390 24419 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:49.334542 24483 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:49.334227 24419 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.335060 24483 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.337167 24419 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.337368 24483 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.338475 20876 raft_consensus.cc:2368] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:49.338989 24239 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.340616 24306 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.340719 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:49.341924 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:49.342129 24422 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.636s	user 0.002s	sys 0.007s
I20260709 12:22:49.342551 20805 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.345173 24388 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.295943 24317 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.347625 20884 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:49.347776 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:49.348022 20884 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:49.348450 20803 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.348946 24317 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.348830 20884 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.349519 24496 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.349927 24496 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.350153 20884 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:49.351378 20882 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.347043 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:49.352015 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61612.6us
I20260709 12:22:49.352783 20878 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:22:49.358814 20732 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:49.369103 20803 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
W20260709 12:22:49.370628 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 104207us
I20260709 12:22:49.370437 20805 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:49.371363 24504 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:49.372278 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.374660 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.326984 24394 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.376362 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.376674 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.379130 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:49.379830 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.380458 20730 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:49.380939 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.376418 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.387373 20884 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:22:49.387950 20884 pending_rounds.cc:85] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 7
I20260709 12:22:49.388248 20884 pending_rounds.cc:107] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:22:49.388653 20884 raft_consensus.cc:2889] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:49.388633 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:49.388191 24503 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.389487 24464 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.389791 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.389837 24503 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.391829 24464 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.392021 24503 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.392736 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.393991 20731 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:49.388633 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:49.395177 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.395519 20803 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:49.317144 24480 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.395784 20735 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:49.413389 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.415128 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:49.417524 20884 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.419638 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.420444 20733 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.423601 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.423712 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.424301 20730 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:49.424454 20727 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.427604 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.410181 24376 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.428902 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.434216 20734 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:49.430073 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.393867 24463 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:49.432189 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.435056 20730 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
W20260709 12:22:49.436602 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 135631us
I20260709 12:22:49.437284 20729 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:49.438004 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.440297 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:49.445168 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.446002 20734 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:49.310781 24499 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.444453 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:49.449227 20727 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:49.461194 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:49.462602 20733 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:49.448511 24500 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:22:49.466413 24500 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:49.466751 24500 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:49.466987 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.467512 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.467746 20726 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.467595 24500 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.468739 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.468942 24500 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.469308 20730 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:49.476125 20726 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:49.475695 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.471025 24239 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.484189 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.492972 20727 raft_consensus.cc:2368] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:49.486248 24197 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:49.490216 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.486349 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.437263 24502 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:49.443022 24417 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:49.442739 24463 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:49.487231 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:49.497437 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 126093us
I20260709 12:22:49.487720 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.498310 21432 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 5.
I20260709 12:22:49.498461 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.489526 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.499306 21430 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:49.479547 24422 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:49.459982 24317 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:49.481509 24466 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.482497 24376 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.482676 24465 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.491672 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.502133 21432 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:22:49.502488 21434 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:49.492282 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.485394 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.495679 20878 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:49.496320 20729 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:49.496204 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.497592 20884 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:49.487274 24318 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:49.500590 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52375.3us
I20260709 12:22:49.495378 24197 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:49.505044 20730 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:22:49.504974 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.501147 24317 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.501780 24419 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:22:49.525913 24448 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.510s	user 0.013s	sys 0.009s
I20260709 12:22:49.497246 24239 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.527508 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 609) took 1864 ms. Trace:
I20260709 12:22:49.529278 24239 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:49.530632 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.531416 21429 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.507318 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.535029 21432 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:49.536018 24365 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.536526 24365 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.538672 21432 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:49.520109 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.540944 21431 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.503722 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.505518 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.542117 21430 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:49.508239 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.507711 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:49.543931 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.508152 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.509408 24318 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:49.508388 24422 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:49.519812 24496 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.506245 24410 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:22:49.506289 24393 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:22:49.510092 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.552464 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.527954 21435 rpcz_store.cc:276] 0709 12:22:47.663162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.663447 (+   285us) service_pool.cc:224] Handling call
0709 12:22:47.680783 (+ 17336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:48.995884 (+1315101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.527244 (+531360us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:49.527428 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3000118528}
I20260709 12:22:49.547056 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1947) took 3265 ms. Trace:
W20260709 12:22:49.556346 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 606) took 1896 ms (client timeout 2116 ms). Trace:
W20260709 12:22:49.556943 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46933.6us
I20260709 12:22:49.550802 21431 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:49.551944 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.557901 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:49.555991 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.558957 21432 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:49.546824 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.560946 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.551505 24221 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:49.563601 24221 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.563988 24221 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.564467 23925 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.565057 23925 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:49.564854 24492 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.565555 23925 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:49.566444 23925 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.565243 20775 leader_election.cc:400] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:22:49.572572 20775 leader_election.cc:403] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:49.556744 20810 rpcz_store.cc:276] 0709 12:22:46.281616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.282794 (+  1178us) service_pool.cc:224] Handling call
0709 12:22:46.322671 (+ 39877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.859083 (+536412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.546846 (+2687763us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:49.546999 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214364032}
I20260709 12:22:49.574327 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.570685 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.572183 21427 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:22:49.556811 21428 rpcz_store.cc:269] 0709 12:22:47.659780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.661309 (+  1529us) service_pool.cc:224] Handling call
0709 12:22:49.556128 (+1894819us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.556285 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549719424}
I20260709 12:22:49.575639 21434 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:49.579486 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.580034 20728 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.581182 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.581805 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.582499 20805 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:49.582609 20803 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:22:49.552006 24466 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:22:49.568414 24197 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.553366 24390 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.584086 24390 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.584448 24390 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.584367 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.585368 20804 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:49.587030 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.543764 24317 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.622262 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.592957 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.624176 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.624629 20805 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.625108 20810 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:49.623466 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.623235 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.554011 24296 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:49.621330 20733 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:49.626271 21433 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:49.621763 21429 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:49.592777 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:49.626330 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.631625 20803 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:49.624691 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.632504 21435 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.631589 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2030) took 1554 ms. Trace:
I20260709 12:22:49.634399 20882 rpcz_store.cc:276] 0709 12:22:48.076771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.088461 (+ 11690us) service_pool.cc:224] Handling call
0709 12:22:49.631356 (+1542895us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.631517 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2901911808}
I20260709 12:22:49.640390 21430 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
W20260709 12:22:49.640875 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.641081 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.567472 24419 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:49.502059 24502 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.572327 24464 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.567843 24239 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:49.643673 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51469.2us
I20260709 12:22:49.645349 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.645476 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:49.665163 20882 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.651961 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.652024 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.666394 20728 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:49.567144 24492 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.584088 24197 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.643272 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.672938 21433 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:49.593416 24458 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:49.587626 24435 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:22:49.570163 24499 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.593761 24416 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:49.592864 24388 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:49.624019 24111 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.688902 20734 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.625653 24493 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.592793 24377 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:49.629777 24296 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:49.658190 24289 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:49.651479 24419 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:49.657913 24365 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.702298 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.713275 24499 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.720676 21429 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:49.722134 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.725113 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:49.725461 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.725754 20809 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.726110 21435 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:49.726380 20805 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:49.727094 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.727730 21435 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:49.729898 21429 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:49.730446 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.732451 24502 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.733420 20809 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:49.733922 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.734670 20810 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:49.712961 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:49.737751 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.644116 24221 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:49.743525 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 150121us
I20260709 12:22:49.710415 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.701730 21433 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:49.744606 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:49.703456 24296 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:49.593899 24463 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.703986 20804 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:22:49.743391 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 150672us
I20260709 12:22:49.744750 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.679608 24496 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:49.710481 20882 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:49.751725 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:49.694340 24111 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:49.741858 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.765010 20805 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:49.680658 24464 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.707681 24419 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:49.704416 24289 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:49.766652 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:49.766718 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.705545 24416 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:49.766431 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.778527 21430 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:49.767280 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.779891 20882 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.782339 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.787631 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:49.788409 20809 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:49.743068 24376 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:49.793409 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55710.1us
I20260709 12:22:49.750272 24500 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:49.794809 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:49.796247 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.796860 20732 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:49.703773 24388 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.686877 24318 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.708235 24377 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.766124 24111 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.800647 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.703114 24367 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 12:22:49.802743 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50902.1us
I20260709 12:22:49.704707 24493 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.706302 24422 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.812217 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.814062 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.815259 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.815541 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.815941 20730 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:49.816618 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.816937 24388 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:49.817126 20730 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:49.818080 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:49.818974 20730 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:49.824298 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:49.825443 20730 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
W20260709 12:22:49.820014 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1851) took 2671 ms (client timeout 2135 ms). Trace:
I20260709 12:22:49.820458 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:49.821645 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1869) took 2039 ms (client timeout 1861 ms). Trace:
I20260709 12:22:49.823074 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 1965) took 2409 ms. Trace:
I20260709 12:22:49.804345 24197 raft_consensus.cc:2764] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:22:49.829463 20808 rpcz_store.cc:276] 0709 12:22:47.413895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.699864 (+285969us) service_pool.cc:224] Handling call
0709 12:22:47.729811 (+ 29947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.744198 (+ 14387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.822881 (+2078683us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b5400126950. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.823013 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64328448}
W20260709 12:22:49.830353 20876 rpcz_store.cc:269] 0709 12:22:47.782455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.786495 (+  4040us) service_pool.cc:224] Handling call
0709 12:22:49.821387 (+2034892us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540019bed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.821560 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3564946176}
W20260709 12:22:49.827872 20875 rpcz_store.cc:269] 0709 12:22:47.148117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.148263 (+   146us) service_pool.cc:224] Handling call
0709 12:22:49.819724 (+2671461us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:49.819923 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605293312}
I20260709 12:22:49.832501 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:49.794070 24502 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:22:49.793419 24433 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:22:49.839563 24388 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:49.808701 24514 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:49.845726 20726 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:49.847497 21432 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:49.860579 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.808194 24376 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:22:49.855594 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:49.883496 20875 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:49.890105 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89512.3us
I20260709 12:22:49.891736 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:49.893296 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.837265 24394 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:49.895819 24228 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:49.896438 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.884863 24501 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.901965 24228 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:49.840497 24513 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:22:49.909525 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:49.898133 24500 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:49.811640 24289 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:49.900605 24460 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.851552 24416 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:49.898064 24514 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:22:49.906876 24501 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:49.920996 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 124815us
I20260709 12:22:49.929883 20734 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:49.912799 24394 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:49.914412 24500 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:49.927590 24289 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:49.984958 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 96014.5us
W20260709 12:22:49.935563 24509 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:22:49.990924 20803 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:49.935281 24423 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.936520 24499 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:49.944172 24433 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.944690 24501 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:49.999603 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 167177us
I20260709 12:22:50.000486 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.992514 24500 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.016782 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:50.021296 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:50.021633 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:50.021536 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:50.024602 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:50.025445 20807 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:22:50.024607 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:50.026264 20806 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:50.026535 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:50.021986 20805 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:49.996966 24376 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:50.023777 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:49.983520 24513 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 12:22:50.034158 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48480us
I20260709 12:22:50.033339 20808 raft_consensus.cc:2368] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:22:50.035414 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:50.000360 24522 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:50.039011 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.043382 24514 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.043951 24514 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.044346 24514 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.928112 24460 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.054565 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:50.058283 24517 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:22:50.059772 24197 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.060009 24522 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:50.060562 24522 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:50.060670 24197 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.060992 24522 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.009703 24388 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:50.013273 24464 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.064503 20735 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5374e2302374917828294b0a4e3313a"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:50.013388 24221 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.002844 24410 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:22:50.070299 20735 raft_consensus.cc:606] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:50.038925 24499 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:50.063369 24526 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:22:50.095877 24525 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:50.097954 24525 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:50.098470 24525 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:50.013792 24433 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.099996 24388 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.039778 24393 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.000991 24228 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.040022 24377 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.008769 24423 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:50.108403 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52917.6us
W20260709 12:22:50.122895 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68422.9us
I20260709 12:22:50.040732 24367 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:22:50.098944 24525 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.080231 24464 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:50.134281 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:50.031838 24523 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:22:50.142854 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.143976 20729 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:50.144544 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.145299 21435 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:50.124006 24393 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.146337 24367 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:50.147269 24393 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:50.147472 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.147837 24530 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.151393 24528 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:50.152014 24528 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.153030 24423 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.154158 24528 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.154594 24536 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:50.155176 24536 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.140915 24493 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.156299 24483 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:50.156389 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.156988 24483 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:50.157332 20802 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 548) took 2750 ms (client timeout 3062 ms). Trace:
I20260709 12:22:50.157691 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:50.157793 20802 rpcz_store.cc:269] 0709 12:22:47.406478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.445443 (+ 38965us) service_pool.cc:224] Handling call
0709 12:22:50.157129 (+2711686us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b540006d1d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:50.157254 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713168640}
I20260709 12:22:50.158394 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:50.159770 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.157245 24536 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.110543 24433 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.161144 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:50.169238 20878 raft_consensus.cc:2376] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8. Re-sending same reply.
I20260709 12:22:50.157361 20875 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:22:50.166431 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.167080 20876 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:50.168320 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.111611 24499 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:50.171403 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:50.170876 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.172343 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.176621 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:50.181995 20884 raft_consensus.cc:2376] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9. Re-sending same reply.
W20260709 12:22:50.179692 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45561.6us
I20260709 12:22:50.180579 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.182801 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.183346 21430 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:50.183511 20700 leader_election.cc:400] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:22:50.183900 20700 leader_election.cc:403] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:50.180914 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:50.185688 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.147387 24367 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:50.179929 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.188635 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:50.190546 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:50.190764 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.187572 20881 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:50.141983 24464 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.154672 24528 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:50.192020 24196 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:50.206908 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.193434 24536 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.191726 24367 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:50.223819 21433 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:50.206913 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:50.228499 21429 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:50.230275 24534 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:50.232376 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:50.234495 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:50.236172 24382 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.216815 24289 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:22:50.216599 24526 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:50.168486 24514 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:50.244298 24317 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.602s	user 0.006s	sys 0.005s
I20260709 12:22:50.245162 24514 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.244890 24317 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:50.248651 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2033) took 2037 ms (client timeout 1934 ms). Trace:
I20260709 12:22:50.248296 24367 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:50.249078 20879 rpcz_store.cc:269] 0709 12:22:48.210866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.656294 (+445428us) service_pool.cc:224] Handling call
0709 12:22:50.248480 (+1592186us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b5400305f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:50.248585 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093996416}
I20260709 12:22:50.250195 24367 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:50.252703 24390 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.002s
I20260709 12:22:50.253162 24390 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.253832 24501 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.259473 20728 raft_consensus.cc:2368] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
W20260709 12:22:50.260186 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50523.6us
I20260709 12:22:50.260623 24188 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:50.261155 24188 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:50.264901 24111 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.267397 21426 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:50.269351 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:50.269452 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:50.270393 20807 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:50.270744 20805 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:50.271768 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:50.272846 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:50.273340 20805 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:50.276402 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69561.5us
I20260709 12:22:50.277447 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:50.279896 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.294979 21427 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:50.282188 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:50.262223 24188 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.199318 24464 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.291157 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:50.283452 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.270068 20808 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:50.304230 21429 raft_consensus.cc:2376] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6. Re-sending same reply.
I20260709 12:22:50.307819 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:50.209028 24537 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.313737 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.196717 24419 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.258701 24296 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.300840 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.315912 21427 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:50.217732 24196 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:50.282281 24524 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:50.318625 20699 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:22:50.274458 24522 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.302681 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.207839 24528 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.325485 20699 leader_election.cc:403] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:50.327122 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.327630 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.328322 21429 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:50.333318 20805 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
W20260709 12:22:50.334623 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.344216 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:50.347223 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.302225 24480 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:50.323695 24196 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.325403 24423 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:50.359025 24188 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:50.349256 24514 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:22:50.351868 24526 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.319983 24539 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.348076 24503 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.331239 24317 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:50.306222 24417 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:22:50.325026 24524 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:50.357689 24480 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.330274 24499 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.330150 24502 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:22:50.369500 24296 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.375408 24423 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.387482 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.376304 24501 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:50.371915 24522 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:50.381273 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:50.430148 20729 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:50.366876 24427 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:50.431118 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:50.431408 24427 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:50.432224 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.432343 24427 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.406324 24524 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:50.435035 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.435741 21434 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:50.436935 21434 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:50.438011 21434 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:50.412292 24525 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.440095 24423 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.423758 24532 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.441763 24525 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.368628 24528 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.396581 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:50.404770 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:50.445382 20728 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:50.405973 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:50.445732 24524 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.446523 20727 raft_consensus.cc:2368] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:50.374523 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.446709 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.447443 21426 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:50.377895 24514 consensus_queue.cc:1243] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:22:50.448473 24526 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:50.448813 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:50.449015 24526 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:50.364089 24523 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.450115 24526 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.450609 24524 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.367336 24483 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.451745 24435 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:50.451611 24483 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:50.369702 24390 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.385481 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.453786 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:50.454909 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69569.4us
I20260709 12:22:50.456068 21435 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:50.456367 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:50.415560 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 225482us
I20260709 12:22:50.389590 24539 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:50.366499 23772 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:50.399276 24392 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:22:50.390177 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.375286 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:49.057489 20801 rpcz_store.cc:276] 0709 12:22:47.425902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.090862 (+664960us) service_pool.cc:224] Handling call
0709 12:22:48.091273 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:48.091542 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:48.110998 (+ 19456us) log.cc:844] Serialized 14 byte log entry
0709 12:22:49.056170 (+945172us) log.cc:844] Serialized 14 byte log entry
0709 12:22:49.056453 (+   283us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:49.056519 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:49.056589 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.056679 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.056836 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19020,"prepare.queue_time_us":19829,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":160945,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":4186,"thread_start_us":944326,"threads_started":2,"wal-append.queue_time_us":810162}
I20260709 12:22:50.458220 24536 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:50.458356 24390 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:50.465111 24390 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:50.465574 24390 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:50.421995 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:50.457762 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.467240 20735 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:22:50.463632 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 67090.8us
I20260709 12:22:50.423704 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:50.426719 24480 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.401572 24197 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:50.370898 24419 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:50.367374 24111 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.372893 24469 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.363555 24538 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.380347 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:50.478488 21429 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:50.461294 21431 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.363171 24500 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.471283 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.473568 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.475014 24525 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:50.476939 24503 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:50.460011 24539 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.469677 21434 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:50.460343 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:50.518925 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46110.3us
I20260709 12:22:50.522899 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:50.532940 21431 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:50.551653 21427 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:50.554652 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.555361 20881 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:50.559437 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:50.560640 20876 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:50.561954 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:50.562355 21432 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:50.562736 20875 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.551072 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.567363 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:50.578559 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.010s	sys 0.015s
W20260709 12:22:50.579005 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.010s	sys 0.015s
W20260709 12:22:50.579560 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 540574us
I20260709 12:22:50.580399 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:50.580591 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 120332us
I20260709 12:22:50.584867 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.585736 20774 leader_election.cc:400] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:50.586097 20877 raft_consensus.cc:2410] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:50.586179 20774 leader_election.cc:403] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:22:50.588532 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:50.589066 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:50.549252 24466 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:50.592624 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.597306 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:50.597714 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:50.597995 20734 raft_consensus.cc:2376] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8. Re-sending same reply.
I20260709 12:22:50.598407 20730 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.599690 20735 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:50.601970 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.507283 24196 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.608264 20734 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:50.563848 24525 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:50.604828 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:50.555668 24510 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:50.610464 21400 leader_election.cc:400] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:50.610553 20875 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:50.612385 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:50.614464 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:50.611614 21400 leader_election.cc:403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:50.615768 20728 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:50.614917 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:50.591166 24480 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:50.594558 24536 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:50.619817 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:50.618032 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.620457 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.594236 24197 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:50.625200 20877 raft_consensus.cc:2376] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8. Re-sending same reply.
I20260709 12:22:50.617767 24528 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:50.622892 24296 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:22:50.629191 24528 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.629662 24528 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.611636 24293 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:50.630685 24293 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:50.611157 24525 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:50.631448 24525 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.631654 24293 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.632315 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:50.634212 24196 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:50.623170 24306 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:22:50.630038 24528 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.617785 24433 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:50.636605 24433 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:50.637538 24433 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.607767 24521 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.608040 24530 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.643215 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:50.646456 21431 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:22:50.611481 24557 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:50.650851 20805 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:50.661943 21429 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:50.653050 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:50.656119 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:50.656196 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:50.660362 20734 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:50.617626 24501 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:50.625187 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:50.669744 20875 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:50.675902 20875 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:50.685415 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66120.2us
I20260709 12:22:50.679472 20875 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.689790 24536 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.683029 20882 raft_consensus.cc:1217] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[8.6-12.8]   Dedup: 8.6->[10.7-12.8]
I20260709 12:22:50.690536 20882 pending_rounds.cc:85] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 6
I20260709 12:22:50.690779 20882 pending_rounds.cc:107] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:22:50.690361 24536 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.690999 20882 raft_consensus.cc:2889] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:50.692682 24536 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.692883 24296 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:50.693382 24296 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.695919 24296 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.689632 20875 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.621558 24480 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:50.641215 24523 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:50.556896 24390 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:50.721217 20730 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:50.721316 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:50.723560 20877 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:50.722011 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:50.722880 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:50.786749 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61209.3us
I20260709 12:22:50.787053 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:50.790635 20735 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:50.699590 24557 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:50.787081 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64883.5us
I20260709 12:22:50.617995 24510 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.811159 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.812855 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:50.805260 20875 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:22:50.723209 24480 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.818266 20875 pending_rounds.cc:85] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 8
I20260709 12:22:50.823338 20875 pending_rounds.cc:107] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:22:50.823688 20875 raft_consensus.cc:2889] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:50.808822 24367 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
W20260709 12:22:50.810321 24552 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:22:50.702694 24464 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:50.803612 24557 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:50.816929 24504 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.798929 24388 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:22:50.819361 24412 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:22:50.910169 24532 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:50.881348 24522 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.885984 24499 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:22:50.919550 24558 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:22:50.881711 24504 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:50.924294 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111556us
W20260709 12:22:50.907136 24549 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
W20260709 12:22:50.924063 24554 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:22:50.892190 24197 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:22:50.888052 24500 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:50.924206 24532 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:50.966056 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:50.949195 24564 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:22:50.946678 24412 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:50.986183 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:50.987771 20735 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:22:50.988251 20735 pending_rounds.cc:85] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 8
I20260709 12:22:50.988534 20735 pending_rounds.cc:107] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:22:50.988827 20735 raft_consensus.cc:2889] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:50.983573 24412 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:50.976840 24510 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:50.977890 24504 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:51.007222 24555 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
I20260709 12:22:51.017365 24464 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:51.018254 24228 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:51.005839 24567 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:22:50.971318 24579 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:22:51.026590 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2230) took 1915 ms. Trace:
I20260709 12:22:50.996376 24522 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:22:51.027079 20731 rpcz_store.cc:276] 0709 12:22:49.111261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.111418 (+   157us) service_pool.cc:224] Handling call
0709 12:22:51.026369 (+1914951us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54003b5454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:51.026522 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623721728}
I20260709 12:22:51.031612 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:51.032049 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.032364 21433 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:51.033375 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.035523 20879 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:51.036401 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.037161 20879 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:51.038316 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.039250 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.039904 20875 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.039574 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:51.038265 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.043893 20883 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:51.043679 23782 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:51.044732 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.043311 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:51.045295 20883 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:51.045617 21434 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:22:51.046175 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.047420 20877 raft_consensus.cc:2368] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:51.043363 24392 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:51.048234 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.048406 24392 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:51.049090 20877 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:51.049343 24392 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.050222 20877 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:22:51.050585 20877 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 8
I20260709 12:22:51.050802 20877 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 9.9
W20260709 12:22:51.054669 24423 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.000s
W20260709 12:22:51.057253 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.059439 24365 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.060031 24365 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.002840 24412 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.060576 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:51.068207 24225 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.068379 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:51.051033 20877 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:51.069162 24111 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:51.063692 24306 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:51.070394 24306 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:51.070792 24525 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.065238 24412 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:51.071336 24306 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.072674 24306 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.073164 24525 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:51.074470 24486 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.041910 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.075042 24486 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.076251 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.076812 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.047171 20883 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:51.062182 24365 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:51.076850 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:51.084991 24503 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:51.086264 23925 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:51.086393 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:51.087217 24503 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:51.086799 23925 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:51.088770 23925 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.089725 23925 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:51.091164 24503 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:51.092432 24514 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:51.092759 24514 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:51.093532 24514 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.094872 24514 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.095564 24427 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.097849 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.099177 24415 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:51.099370 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.099637 24415 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:51.100275 20883 raft_consensus.cc:2368] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:51.100612 24415 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.101122 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.101437 20880 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:51.101635 24415 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.102300 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.102850 20883 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.101817 24538 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:51.105000 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.106186 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:51.106410 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:51.108640 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:51.109071 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:51.110776 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.111893 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:51.106725 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:51.117899 20774 leader_election.cc:400] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:51.118264 20774 leader_election.cc:403] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:22:51.118925 20735 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:22:51.119115 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 77516.1us
I20260709 12:22:51.121790 21427 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:22:51.119871 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.125727 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.129931 20880 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:51.133996 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.135365 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.141839 24228 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:51.143836 24228 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:51.144338 24228 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.144366 24221 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:51.144907 24534 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:51.144929 24221 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:51.147437 24221 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:51.150161 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:51.151011 20882 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:51.151516 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.152540 20882 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:51.154331 20877 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:51.055177 24423 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.155296 24530 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.156070 20729 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:51.156498 24480 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.156978 24423 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:51.158659 24480 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.159344 20802 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:51.159951 24539 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.603s	user 0.005s	sys 0.006s
I20260709 12:22:51.160387 24539 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.161839 24480 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:51.162521 24539 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:51.162091 24221 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.164403 24221 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:51.164672 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.168936 20735 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:51.170096 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:51.171828 20731 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:51.173233 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:51.171731 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:51.174993 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:51.171123 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:51.184497 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.182576 20880 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:51.188449 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.198468 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:51.175840 20735 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:51.187584 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:51.204353 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.211453 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.213013 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:51.214876 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:51.215966 20733 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:22:51.217941 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106171us
I20260709 12:22:51.221089 20726 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:51.222594 20731 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:51.223559 20775 leader_election.cc:400] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
W20260709 12:22:51.219957 24563 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:22:51.225669 20875 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:51.039776 24474 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:51.218497 24423 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:51.129204 24528 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.164484 24532 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.288884 24532 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:51.289113 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.293361 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:51.301271 20875 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
W20260709 12:22:51.325099 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.242689 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:51.219985 24510 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:51.221843 23782 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:51.231019 24526 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:51.279286 24362 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:51.267048 24458 raft_consensus.cc:2764] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:22:51.269009 24423 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.273666 24474 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:51.282320 24576 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:22:51.285693 24453 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.311875 24539 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:51.159324 24557 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.375679 20875 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:51.377122 21434 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:22:51.370885 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 128288us
I20260709 12:22:51.286448 20883 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:51.387837 20882 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:51.392310 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.394212 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:51.396246 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.369220 24510 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.399730 24453 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.381374 24532 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:51.381136 24526 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:51.381798 23782 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:51.382874 24423 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.433374 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.440264 24435 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:51.455808 23782 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.403744 24221 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:22:51.460439 24221 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:51.460915 24221 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:51.461992 24221 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.446861 24486 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.477423 24460 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.478030 24221 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:51.478300 20729 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:51.480427 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2251) took 1098 ms. Trace:
I20260709 12:22:51.480832 20732 rpcz_store.cc:276] 0709 12:22:50.381898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.423363 (+ 41465us) service_pool.cc:224] Handling call
0709 12:22:51.480273 (+1056910us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54001d8654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:51.480371 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733860224}
I20260709 12:22:51.490172 24503 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:51.492660 24523 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:51.493196 24523 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:51.494344 24577 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:51.494809 24577 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:51.494220 24523 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.495960 24523 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:51.464542 24228 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.497654 24474 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:51.498353 24228 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.500072 20875 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:51.467867 24557 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.470281 20876 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:22:51.470480 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.508602 20802 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:51.471048 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.471837 20807 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:51.398972 24433 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:51.541464 21428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.012s	sys 0.032s
I20260709 12:22:51.495649 24577 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:51.401235 24582 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 12:22:51.463845 24587 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:22:51.503446 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70228.8us
I20260709 12:22:51.504719 21431 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:22:51.548388 21428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.012s	sys 0.033s
W20260709 12:22:51.552008 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.443001 24539 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:51.402622 24510 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.512633 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.509635 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:51.565639 20803 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:51.506191 24225 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.510303 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.443720 24526 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:51.499847 24576 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:22:51.532666 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:51.502450 24532 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:51.401607 24575 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.145s
I20260709 12:22:51.552968 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:51.557348 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.0883e+06us
I20260709 12:22:51.524499 24521 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:22:51.519984 24289 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:22:51.405160 24480 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:51.570541 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 355860us
I20260709 12:22:51.525094 24492 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:22:51.583534 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:51.584985 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.550041 24556 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:22:51.590217 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.611552 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.612290 20807 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.622090 20810 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:51.600571 23925 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.608846 24534 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:51.619896 24480 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:51.639262 24573 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:51.642192 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:22:51.635393 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45177.8us
I20260709 12:22:51.639444 24521 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.627019 24528 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.653731 20808 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:51.648628 20803 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:51.686825 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2062) took 1117 ms. Trace:
I20260709 12:22:51.608222 24433 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:22:51.612133 24557 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:22:51.656688 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:51.687290 20881 rpcz_store.cc:276] 0709 12:22:50.569305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.625886 (+ 56581us) service_pool.cc:224] Handling call
0709 12:22:50.628703 (+  2817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.628915 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:51.093333 (+464418us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b180041e120. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf
0709 12:22:51.093466 (+   133us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.686390 (+592924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:51.686461 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:51.686536 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.686608 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.686763 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157072,"prepare.queue_time_us":174009,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":4590,"spinlock_wait_cycles":28098304,"thread_start_us":612091,"threads_started":2,"wal-append.queue_time_us":539971}
I20260709 12:22:51.688859 20807 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:51.685354 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:51.687281 24573 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:51.723387 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 568) took 1075 ms. Trace:
I20260709 12:22:51.724093 20801 rpcz_store.cc:276] 0709 12:22:50.648201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.649560 (+  1359us) service_pool.cc:224] Handling call
0709 12:22:50.649930 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:50.663281 (+ 13351us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:50.724358 (+ 61077us) log.cc:844] Serialized 14 byte log entry
0709 12:22:50.912008 (+187650us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:50.912583 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 12:22:50.912747 (+   164us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:50.921380 (+  8633us) log.cc:844] Serialized 14 byte log entry
0709 12:22:50.921695 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.921796 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:51.044002 (+122206us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b18003fbc80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:51.723029 (+679027us) raft_consensus.cc:1685] finished
0709 12:22:51.723072 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.723305 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12781,"prepare.queue_time_us":71098,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":19741,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":20154,"spinlock_wait_cycles":15933056,"thread_start_us":89721,"threads_started":1,"wal-append.queue_time_us":303516}
W20260709 12:22:51.696197 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 112794us
I20260709 12:22:51.728343 20850 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:51.636091 24539 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.700793 24317 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:51.729055 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:51.690056 24289 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:51.749787 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:51.696695 24557 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:51.656246 24532 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:51.722226 20734 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.009s	sys 0.001s
I20260709 12:22:51.732023 24433 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:51.689687 24306 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:22:51.750546 24317 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:51.721633 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.782325 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:51.758683 20734 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.010s	sys 0.001s
I20260709 12:22:51.759047 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:51.786357 20729 raft_consensus.cc:2368] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:51.759582 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:51.787388 20727 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.788491 24453 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.760481 20731 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:22:51.690951 24521 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:51.780084 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51867.9us
I20260709 12:22:51.756150 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:51.795617 24577 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:51.785599 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.799044 20732 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:51.755028 24534 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.801766 20727 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:51.752980 24557 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:51.755275 24575 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:51.798019 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:51.767654 24433 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:51.816831 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 206823us
I20260709 12:22:51.800438 24317 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.804591 24417 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:51.804152 24435 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:51.756732 24528 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:22:51.758519 23925 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:22:51.813752 24537 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:51.775641 24480 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:51.818005 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:51.757573 24289 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:51.828570 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.840276 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:51.842998 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:51.815214 24390 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.006s	sys 0.014s
I20260709 12:22:51.806216 24225 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 12:22:51.845137 24390 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.006s	sys 0.015s
I20260709 12:22:51.780756 24503 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:22:51.846285 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.829854 24417 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.819427 24433 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:51.827137 24586 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:22:51.814566 24534 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:51.844697 20878 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:51.845533 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:51.812335 24575 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:51.857807 20882 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.856308 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:51.845445 24225 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:51.832163 24435 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:51.829777 24453 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
W20260709 12:22:51.864406 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46513.9us
I20260709 12:22:51.866310 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:22:51.824931 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:51.854063 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 212020us
I20260709 12:22:51.871385 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:51.869388 20875 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:51.875134 20875 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:51.852319 24417 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:51.854156 24419 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:22:51.857679 24585 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:22:51.871716 21432 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:22:51.886190 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.892343 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:51.893203 21432 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
W20260709 12:22:51.897513 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 373638us
I20260709 12:22:51.859623 24575 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:51.903157 20882 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:51.904714 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:51.905004 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:51.906872 20875 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.917009 20875 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.862325 24225 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:51.922276 24600 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:22:51.922447 24225 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.923925 24225 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.925570 24197 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:51.926000 24197 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:51.926843 24197 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.927927 24197 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:51.930016 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 173969us
I20260709 12:22:51.876585 23772 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:51.883700 24589 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 12:22:51.930866 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75173.4us
W20260709 12:22:51.931089 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64888.3us
I20260709 12:22:51.931871 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:51.873795 24496 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.869007 24435 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.898747 24419 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.867336 24464 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:22:51.873651 23925 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:22:51.940647 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:51.863978 24534 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:51.932202 24590 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:22:51.937194 24611 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.883869 24603 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:51.941722 24221 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:51.942062 24583 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:22:51.951296 24595 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:51.949647 24592 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:22:51.954276 23925 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:51.954669 24607 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:51.955674 24521 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:51.941926 23772 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.956228 24390 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:51.958782 24557 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.975620 24534 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:51.846653 24573 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:51.945551 24419 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.979288 24369 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:51.979488 21427 raft_consensus.cc:1217] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[6.7-10.9]   Dedup: 6.7->[8.8-10.9]
I20260709 12:22:52.038741 21427 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: [6.7-10.9]
I20260709 12:22:51.966866 24423 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.040320 20775 leader_election.cc:400] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:22:52.040707 20775 leader_election.cc:403] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:22:51.941745 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:51.934424 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:51.941394 24503 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:51.943857 24528 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:51.975008 24599 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:22:52.015729 24595 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.941107 24415 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 12:22:52.028937 24581 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:22:51.943681 24496 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.037710 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:52.074079 20875 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 12:22:52.042261 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 107902us
I20260709 12:22:52.050206 21435 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:52.050058 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.054458 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.027025 23925 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:52.062716 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 121098us
I20260709 12:22:52.087883 20875 pending_rounds.cc:85] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 8
I20260709 12:22:52.088204 20875 pending_rounds.cc:107] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:22:52.018117 24221 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.026444 24526 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:52.040755 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:52.093067 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53074.6us
I20260709 12:22:52.093101 21434 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:52.086217 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.059623 24228 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:52.088521 20875 raft_consensus.cc:2889] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:52.088994 20876 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:22:52.090147 20806 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:52.048247 20628 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:52.065333 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 647) took 1095 ms. Trace:
I20260709 12:22:52.050750 24528 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:52.102806 21428 rpcz_store.cc:276] 0709 12:22:50.970425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.970660 (+   235us) service_pool.cc:224] Handling call
0709 12:22:50.998396 (+ 27736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:50.998685 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.998704 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:51.015681 (+ 16977us) log.cc:844] Serialized 26 byte log entry
0709 12:22:51.552044 (+536363us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:51.552087 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.552158 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:52.065029 (+512871us) raft_consensus.cc:1685] finished
0709 12:22:52.065059 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.065254 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3372,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":94,"thread_start_us":514958,"threads_started":1,"wal-append.queue_time_us":612088}
I20260709 12:22:52.063401 24111 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:52.064517 24614 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:52.064335 24534 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
W20260709 12:22:52.118413 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63557.4us
W20260709 12:22:52.115000 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77362.7us
I20260709 12:22:52.106881 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.062011 24595 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:52.065475 24503 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:22:52.118178 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166793us
I20260709 12:22:52.128340 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:52.048353 24369 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.134486 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:52.136097 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 200332us
I20260709 12:22:52.113415 24557 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:52.138720 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.146175 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.114218 24528 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:52.151964 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:52.153908 20730 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:52.118829 24577 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:52.113970 23772 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:52.115931 24458 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:52.094650 24480 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:22:52.115576 24289 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:52.156251 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:52.145118 20628 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 2 to 5, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:52.121050 24365 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:52.246210 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 157921us
W20260709 12:22:52.140307 24601 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 12:22:52.177567 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:52.264653 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.183300 24617 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.094s	user 0.002s	sys 0.001s
I20260709 12:22:52.150236 24612 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:52.157344 24427 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:22:52.187965 24618 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:22:52.189646 24625 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:22:52.145134 24460 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:22:52.095849 24526 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.272228 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:52.272886 21427 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:22:52.157330 24602 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:22:52.148734 24486 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:52.275406 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 211544us
I20260709 12:22:52.262203 23772 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:22:52.281292 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.212564 24221 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:52.158035 24614 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:52.273205 24612 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:52.180200 24608 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 12:22:52.311643 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:52.178442 24532 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 12:22:52.275602 24577 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.152814 24228 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.146s
I20260709 12:22:52.273888 24365 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.203102 24435 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:52.320538 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 167569us
I20260709 12:22:52.305052 20628 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 8, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 8 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:52.304312 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 148121us
I20260709 12:22:52.222003 24575 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.349696 24575 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:52.328030 24526 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:52.328843 24577 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.350550 24597 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:52.329672 23925 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.332013 23772 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.352593 20730 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:22:52.352388 23772 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.353175 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:52.354468 23772 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:52.335448 24632 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:52.355084 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.339797 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1903) took 1176 ms. Trace:
I20260709 12:22:52.355772 20876 rpcz_store.cc:276] 0709 12:22:51.163761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.515582 (+351821us) service_pool.cc:224] Handling call
0709 12:22:51.515970 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:52.102032 (+586062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.339547 (+237515us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:52.339722 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317043584}
I20260709 12:22:52.198874 24317 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.356601 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.357498 23772 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.316684 24472 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:52.338469 24419 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.358634 20884 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:52.358723 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 214616us
W20260709 12:22:52.359552 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 213456us
I20260709 12:22:52.360030 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:52.367383 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:52.323575 24458 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:52.362540 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:52.363729 20727 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:52.364625 20809 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:22:52.364625 20731 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:52.367338 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:52.369609 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.323460 24612 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:52.369997 20733 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:52.377259 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:52.382579 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.382125 20628 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 4 to 5, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:52.317395 24225 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:22:52.391481 20728 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:52.393790 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.395674 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:52.323127 23782 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:22:52.306143 24503 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.360826 24221 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:52.400851 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 262095us
I20260709 12:22:52.359150 24614 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:52.379106 24603 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:52.407269 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:52.357976 23772 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.394392 24427 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:52.393126 24472 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:52.420146 24317 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:52.420333 24620 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:22:52.420948 24472 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:52.422791 24510 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.846s	user 0.008s	sys 0.003s
I20260709 12:22:52.414346 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1897) took 1322 ms. Trace:
I20260709 12:22:52.423211 24510 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.423552 20880 rpcz_store.cc:276] 0709 12:22:51.092004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.220164 (+128160us) service_pool.cc:224] Handling call
0709 12:22:51.220508 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:51.371449 (+150941us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:51.390092 (+ 18643us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.755568 (+365476us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:51.756034 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:22:51.756160 (+   126us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:51.756190 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.756253 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:52.414070 (+657817us) raft_consensus.cc:1685] finished
0709 12:22:52.414101 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.414269 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68297,"prepare.queue_time_us":16962,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":97645,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":21319,"thread_start_us":421634,"threads_started":2,"wal-append.queue_time_us":426795}
I20260709 12:22:52.392984 24423 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:52.394205 24594 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:52.405864 24556 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:52.409778 24225 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:52.407848 24577 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:52.403350 24629 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.427802 24629 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.430373 24633 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:52.430215 24614 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.392328 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:52.377216 24458 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:52.432303 24458 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.417079 24603 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:52.434254 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 521) took 1712 ms. Trace:
I20260709 12:22:52.434625 24603 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.427070 24225 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:52.436975 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 91888us
W20260709 12:22:52.442132 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46528us
I20260709 12:22:52.444145 24573 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:52.411167 24496 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:52.445544 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76422.6us
I20260709 12:22:52.394519 24597 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:22:52.444389 24629 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:52.434644 20734 rpcz_store.cc:276] 0709 12:22:50.721566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.791369 (+ 69803us) service_pool.cc:224] Handling call
0709 12:22:50.791766 (+   397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:50.802291 (+ 10525us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:51.171080 (+368789us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b18001e22c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:22:51.802572 (+631492us) log.cc:844] Serialized 14 byte log entry
0709 12:22:52.040584 (+238012us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:52.060072 (+ 19488us) log.cc:844] Serialized 52 byte log entry
0709 12:22:52.060216 (+   144us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:52.060479 (+   263us) log.cc:844] Serialized 14 byte log entry
0709 12:22:52.060751 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.064000 (+  3249us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:52.182772 (+118772us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:52.191670 (+  8898us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18001e22c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:52.335202 (+143532us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b18001e22c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:52.434016 (+ 98814us) raft_consensus.cc:1685] finished
0709 12:22:52.434043 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.434192 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135968,"prepare.queue_time_us":28407,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":7686,"raft.queue_time_us":21197,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":21484,"spinlock_wait_cycles":82926848,"thread_start_us":829130,"threads_started":2,"wal-append.queue_time_us":259253}
I20260709 12:22:52.462713 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.447712 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.449538 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.452302 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.463866 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:52.464531 20731 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:52.465396 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.467059 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:52.467816 21435 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:52.468529 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:52.469251 21431 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:52.465415 20877 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:52.397749 24486 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:22:52.439253 24556 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:52.459898 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.445029 24573 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:52.465518 20876 raft_consensus.cc:2368] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:52.503129 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:52.472656 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.472941 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.505885 20808 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:52.474874 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.475680 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.475977 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:52.476456 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59630.3us
I20260709 12:22:52.506983 20880 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:52.508064 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:52.458583 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66332.2us
W20260709 12:22:52.437507 24433 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.004s
I20260709 12:22:52.469906 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:52.483151 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:52.484567 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:52.457712 24594 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:52.494050 20879 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:52.445190 24510 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:52.486126 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.512197 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:52.510550 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:52.511620 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:52.539376 21429 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.502256 20628 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:52.416441 24221 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.460070 24577 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:52.545243 21435 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:22:52.477054 24369 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:52.548100 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48655us
I20260709 12:22:52.549196 21431 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:22:52.548776 20628 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:52.490851 24496 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:52.459570 24607 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.079s
W20260709 12:22:52.539912 24417 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.000s
I20260709 12:22:52.502167 24573 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:52.562594 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:52.568583 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:52.585896 21430 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:52.565194 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:52.570199 20805 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:52.572038 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.543407 24633 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:52.577283 20802 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:52.578339 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:52.581110 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.584944 20628 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 4 to 5, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 5 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:52.601260 21429 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:52.586737 21431 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.560051 24390 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:52.631254 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:52.506855 24611 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:22:52.619797 21431 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:52.550747 24539 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:52.560787 24525 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.662194 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:52.634420 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:52.676543 21430 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:52.561944 24577 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:52.678707 21431 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:22:52.567037 24607 consensus_queue.cc:1243] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:52.563959 24526 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.630400 24633 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:52.685047 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.536266 24433 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.689841 20628 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 4, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 4 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:52.589331 24538 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:52.693938 24615 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:22:52.694379 24640 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:52.540088 24225 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.705986 23772 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:52.691659 24631 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:22:52.781243 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 269109us
I20260709 12:22:52.754630 24633 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.783644 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.707374 24577 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.566700 24496 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.675669 24435 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:52.679301 24365 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:52.834268 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 251977us
I20260709 12:22:52.844501 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.846542 24660 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:52.847321 24660 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.847801 24660 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.849848 24660 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:52.850369 24660 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.579034 24417 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.849756 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1906) took 1007 ms. Trace:
I20260709 12:22:52.683974 24575 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:52.856276 20806 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:52.867354 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:52.584455 24595 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.853920 21435 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:52.854930 20884 rpcz_store.cc:276] 0709 12:22:51.842523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.842645 (+   122us) service_pool.cc:224] Handling call
0709 12:22:52.849483 (+1006838us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:52.849677 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183529344}
I20260709 12:22:52.691529 24419 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:52.867347 20733 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:22:52.784233 24537 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.945s	user 0.010s	sys 0.002s
I20260709 12:22:52.754208 24640 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:52.876866 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 5 to 6, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 6 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:52.894170 21435 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
W20260709 12:22:52.895852 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 314756us
I20260709 12:22:52.896852 21399 leader_election.cc:400] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9.
I20260709 12:22:52.898900 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:52.850826 24660 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.900210 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:52.901607 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:52.678063 24539 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.689405 24525 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:52.929667 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90108.1us
I20260709 12:22:52.932437 20628 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 3, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:52.755549 24526 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.899322 24435 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.893258 24537 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:52.708590 24573 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.892309 24640 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.780730 24538 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.936290 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:52.853628 24458 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.944829 20728 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:52.877308 24575 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.948872 20628 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 4 to 5, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:52.874199 24612 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:52.957913 20628 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 5, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 5 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:52.944816 24435 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:52.962023 20628 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:52.971982 21428 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:52.963084 24221 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:52.972483 23772 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:52.983071 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 116474us
I20260709 12:22:52.975049 20628 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 3 to 4, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193), VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194), VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195), VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 6 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:22:52.984937 24528 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.000s
I20260709 12:22:52.983767 23772 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:53.000635 24651 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:22:53.071273 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:53.000069 24643 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 12:22:53.004642 24433 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.003893 24496 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:52.996338 24435 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:52.989547 24669 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:53.088547 24538 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:52.997231 24539 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.035748 23772 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:53.086938 24197 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.159s	user 0.011s	sys 0.002s
W20260709 12:22:53.093750 24667 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:22:53.089848 24660 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:53.089171 24595 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:53.103499 24639 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:22:53.095348 24221 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.115546 24609 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:53.116091 24656 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:53.116926 24675 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:22:53.117080 24646 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:53.032500 24528 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.135833 24417 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:53.137589 24653 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:22:53.173221 21432 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:53.173255 20803 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
W20260709 12:22:53.141539 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.156629 24480 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:53.175431 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.175518 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.158012 24646 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.199476 24612 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.206751 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 580) took 1170 ms. Trace:
I20260709 12:22:53.222745 20807 rpcz_store.cc:276] 0709 12:22:52.036016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.036181 (+   165us) service_pool.cc:224] Handling call
0709 12:22:52.036536 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:52.268700 (+232164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:52.268747 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:52.269654 (+   907us) log.cc:844] Serialized 26 byte log entry
0709 12:22:52.446893 (+177239us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:52.446946 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:53.123172 (+676226us) spinlock_profiling.cc:243] Waited 77.8 ms on lock 0x7b5400126954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:53.123244 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:53.205566 (+ 82322us) raft_consensus.cc:1685] finished
0709 12:22:53.205598 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:53.206678 (+  1080us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79780,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":178963840,"thread_start_us":278715,"threads_started":2,"wal-append.queue_time_us":274079}
I20260709 12:22:53.134289 24458 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.131973 24435 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:53.209728 24646 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.210690 23925 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:53.225716 24675 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:22:53.236629 24525 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:53.238430 24648 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.392s	user 0.002s	sys 0.000s
I20260709 12:22:53.178128 24659 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.215337 24480 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.239540 24659 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.241353 24480 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.178586 20808 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:22:53.218075 24642 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.366s	user 0.001s	sys 0.001s
I20260709 12:22:53.206804 20805 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:53.243948 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 734683us
I20260709 12:22:53.247716 23782 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:53.250743 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:53.253566 24474 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.254161 24474 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.258409 24668 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:53.261032 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.262241 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.262528 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:53.263085 20879 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:53.264467 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:53.265002 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:53.266059 20880 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:53.267923 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:53.268976 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:53.229511 24557 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.275681 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.276029 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:53.269093 20805 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:53.271486 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:53.277216 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.277849 20808 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:53.263567 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:53.279170 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.273846 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:53.282229 20882 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:53.284447 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:53.131958 24526 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.276608 21435 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.263808 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:53.277359 20801 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.273504 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:53.287570 20884 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:53.272296 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:53.289324 21426 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:53.290731 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:53.294488 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:22:53.286108 20876 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:22:53.285773 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.297470 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:53.300932 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.301360 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.301644 20805 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:22:53.301975 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.302676 20808 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:53.242056 24659 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:53.303805 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.304607 20875 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:53.306012 21435 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:53.306104 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.306773 20808 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:53.258843 24668 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.134295 24656 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.278648 24597 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:53.286655 24458 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:53.335472 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:53.297426 24480 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:53.322217 20801 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:53.132453 24221 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.325752 24597 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.326521 24433 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:53.329239 24656 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:53.330760 24514 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.331228 24663 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:22:53.331677 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 347206us
I20260709 12:22:53.332948 24317 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:53.360052 20883 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:53.361112 24528 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.365475 24221 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.368669 24597 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.372704 24573 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:53.374503 24633 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.002s
I20260709 12:22:53.351748 24480 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:53.313218 24365 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:22:53.376235 24480 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.380972 24627 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:53.381101 23772 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.382447 21434 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:53.383731 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:53.384675 21430 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:53.383767 24663 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.313433 24423 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:22:53.389542 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:53.390175 24474 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:53.308324 24669 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:53.390980 24669 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.297747 24595 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:53.391394 24659 raft_consensus.cc:2764] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:22:53.391671 24595 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.353060 24514 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.394053 24595 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.394753 24514 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.388928 24663 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.349632 24458 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.398984 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:53.399959 20880 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:53.344700 24640 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.400801 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.228559 24435 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.307983 24609 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.419517 21433 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:22:53.419858 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.345404 20882 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:53.310217 24492 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.425331 24492 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.351176 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.313632 24668 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:53.323289 24607 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.354339 20875 raft_consensus.cc:1217] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[6.7-11.9]   Dedup: 6.7->[9.8-11.9]
I20260709 12:22:53.287297 24557 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.355300 24433 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:53.359475 24317 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:53.337041 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:53.390039 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:53.399080 24458 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.402029 20880 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:53.402822 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.403028 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:53.341185 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.339476 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.409937 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:53.410729 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:53.342372 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:53.583330 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:53.723592 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:53.656852 21426 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:53.428052 21428 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:53.429265 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:53.746141 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2270) took 1380 ms. Trace:
W20260709 12:22:53.738034 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 474531us
I20260709 12:22:53.426039 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:53.756541 21434 raft_consensus.cc:2368] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
W20260709 12:22:53.756459 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 173291us
I20260709 12:22:53.594893 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.757356 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:53.746238 20808 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:22:53.426826 20804 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:22:53.667546 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.442566 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.721650 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2219) took 1367 ms. Trace:
I20260709 12:22:53.443807 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.754393 20729 rpcz_store.cc:276] 0709 12:22:52.365999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.366113 (+   114us) service_pool.cc:224] Handling call
0709 12:22:52.366411 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:52.406167 (+ 39756us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:52.436503 (+ 30336us) log.cc:844] Serialized 14 byte log entry
0709 12:22:52.766520 (+330017us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:52.852325 (+ 85805us) log.cc:844] Serialized 52 byte log entry
0709 12:22:52.852487 (+   162us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:52.852714 (+   227us) log.cc:844] Serialized 14 byte log entry
0709 12:22:53.259174 (+406460us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b1800184fc0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:53.736673 (+477499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:53.736790 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:53.745912 (+  9122us) raft_consensus.cc:1685] finished
0709 12:22:53.745935 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:53.746090 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":570673,"prepare.queue_time_us":33161,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":92,"raft.queue_time_us":626170,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":30770,"spinlock_wait_cycles":4130176,"thread_start_us":766739,"threads_started":2,"wal-append.queue_time_us":435845}
I20260709 12:22:53.759364 20806 rpcz_store.cc:276] 0709 12:22:52.354729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.354859 (+   130us) service_pool.cc:224] Handling call
0709 12:22:53.721384 (+1366525us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54003ca854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:53.721569 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150523776}
I20260709 12:22:53.760274 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.760746 20728 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:22:53.749743 20875 pending_rounds.cc:85] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 7
W20260709 12:22:53.751529 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 445045us
I20260709 12:22:53.761591 20875 pending_rounds.cc:107] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:22:53.762053 20875 raft_consensus.cc:2889] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.739765 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:53.748327 20734 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.755535 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:53.594692 24221 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:53.765780 24221 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.767486 24557 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.768162 24221 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.769388 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.748101 20884 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:53.581151 24673 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.757771 20735 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:53.771651 24673 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.358843 24594 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:53.749878 21429 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:53.773223 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:53.773924 24673 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.774497 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.375025 24633 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.778488 20734 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:53.776167 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.780911 20735 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:22:53.761200 24607 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:53.785897 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 627) took 1335 ms (client timeout 1751 ms). Trace:
W20260709 12:22:53.786386 20883 rpcz_store.cc:269] 0709 12:22:52.450687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.450835 (+   148us) service_pool.cc:224] Handling call
0709 12:22:53.785678 (+1334843us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:53.785834 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1961496576}
I20260709 12:22:53.786725 20732 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:53.787751 24632 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:53.787418 24492 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.788841 20735 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:53.791949 24225 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.594398 24663 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.795843 24603 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:53.798369 24435 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:53.797750 24317 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.829137 24317 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.749238 24433 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:53.803280 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.593624 24661 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.391417 24669 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:53.808179 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68550.5us
W20260709 12:22:53.809165 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 379958us
W20260709 12:22:53.836699 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.817608 24658 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:53.836699 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.837894 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:53.809590 24474 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:53.809794 24594 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:53.838217 20735 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:22:53.427167 23772 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:53.832490 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75218.5us
I20260709 12:22:53.704892 24530 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:53.552636 24655 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.419907 24597 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:53.803898 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48487.1us
I20260709 12:22:53.428261 24514 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.839850 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:53.514650 24609 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.842717 24594 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.843987 24221 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:53.845026 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:53.847844 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:53.848518 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.850427 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:53.847543 24632 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:53.849751 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:53.850847 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:53.862524 24632 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.862972 24632 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:53.870720 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:53.856060 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:53.873696 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:53.785941 24607 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:53.594277 24646 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.277s
I20260709 12:22:53.849046 24530 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:53.853614 24474 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:53.858071 24656 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:53.888756 24640 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:53.901558 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.875535 24528 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:22:53.889539 23925 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.002s
I20260709 12:22:53.907792 24595 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:53.908892 20883 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:53.897240 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:53.916831 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:53.846414 24663 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:53.911535 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51013.7us
I20260709 12:22:53.918310 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:53.880501 24221 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:53.849416 24661 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.874258 24514 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.885619 23772 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:53.888497 24573 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:53.925118 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2089) took 1433 ms. Trace:
I20260709 12:22:53.880074 24629 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:22:53.858148 24597 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:53.940052 20882 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:22:53.856704 24655 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.919782 20880 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:53.880286 24532 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:22:53.922156 24630 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.923411 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.932008 20877 rpcz_store.cc:276] 0709 12:22:52.492080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.497083 (+  5003us) service_pool.cc:224] Handling call
0709 12:22:52.508745 (+ 11662us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:53.243949 (+735204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:53.243987 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:53.244529 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:22:53.441707 (+197178us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:53.441767 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:53.441865 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:53.843398 (+401533us) raft_consensus.cc:1685] finished
0709 12:22:53.843420 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:53.917096 (+ 73676us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b54001fe350. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:53.925059 (+  7963us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":734731,"raft.queue_time_us":738456,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":52,"spinlock_wait_cycles":26099968,"thread_start_us":196792,"threads_started":1,"wal-append.queue_time_us":597158}
I20260709 12:22:53.934749 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.938880 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:53.963559 20732 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.939312 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:53.964378 20730 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.940485 20882 pending_rounds.cc:85] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:22:53.940687 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.941992 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.965097 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:53.965966 20734 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:53.940187 24503 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:53.936652 24694 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:53.943924 20728 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:53.935392 24673 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.965132 20882 pending_rounds.cc:107] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 8.6
W20260709 12:22:53.957387 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 194598us
I20260709 12:22:53.968082 20735 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:22:53.949450 24228 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:54.001657 24228 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:54.002473 24663 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:54.002589 24228 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.931771 24634 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:22:53.960840 24530 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.005908 24609 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:53.971707 20877 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:53.903357 24464 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:22:53.948652 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:54.007411 24474 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.978412 24694 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.008966 24694 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:53.885696 24433 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.997525 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:53.997655 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:53.999517 24458 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:54.025321 20883 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:53.993813 20882 raft_consensus.cc:2889] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.970665 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:53.992347 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.032557 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:53.992658 21435 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:54.033147 20882 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.984448 20730 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:54.011969 20877 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:54.018028 20732 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:54.023358 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:54.023833 20884 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:22:54.039515 20884 pending_rounds.cc:85] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 3
I20260709 12:22:54.039906 20884 pending_rounds.cc:107] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:22:54.040223 20884 raft_consensus.cc:2889] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.924899 24595 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.043206 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:54.043671 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73105.6us
I20260709 12:22:54.044747 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:54.043992 20879 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:54.050561 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:54.052007 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103423us
I20260709 12:22:54.053684 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:53.978958 24659 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:54.011963 24557 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.064492 20882 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:54.068956 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:54.070647 20883 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:53.992509 24656 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:54.024796 24633 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.078033 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:54.083027 24686 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:22:54.059135 24640 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:54.061439 23772 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:54.042644 24609 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:54.088135 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 217511us
I20260709 12:22:54.067165 24503 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:54.010262 24673 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:53.975278 24668 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.090783 24693 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:54.099778 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:53.978444 24630 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.049099 24458 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.110226 20876 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:54.096505 24633 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:54.069226 24659 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.075286 24557 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.114464 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:54.135558 21432 raft_consensus.cc:2368] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12.
I20260709 12:22:54.112562 24221 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:54.091367 24607 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:22:54.112020 24289 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.111013 24464 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:54.111449 24693 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:54.084935 24656 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.102366 24609 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.124873 24317 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:54.139447 24472 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:54.153748 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:54.197122 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:54.198118 20732 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:54.152926 24575 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.156422 24693 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:54.155258 24221 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:54.195377 24225 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:54.155916 24289 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.156155 24464 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.198801 24628 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:54.208024 24604 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:54.209153 24514 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.210351 24472 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:54.213791 24597 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:54.231652 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 354933us
I20260709 12:22:54.213986 24655 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:54.194351 24317 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.218705 24225 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:54.247517 20774 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:54.247987 20774 leader_election.cc:403] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:22:54.249123 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:54.227264 24595 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:54.233110 24687 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:22:54.239823 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:54.226341 24604 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:54.255935 24656 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.258612 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2087) took 1767 ms. Trace:
I20260709 12:22:54.273753 20881 rpcz_store.cc:276] 0709 12:22:52.491394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.494502 (+  3108us) service_pool.cc:224] Handling call
0709 12:22:52.537826 (+ 43324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:53.249603 (+711777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:53.249632 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:53.250100 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:22:53.581480 (+331380us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:53.581519 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:53.581577 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:54.258333 (+676756us) raft_consensus.cc:1685] finished
0709 12:22:54.258360 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:54.258536 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":711349,"raft.queue_time_us":712465,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":302155,"threads_started":1,"wal-append.queue_time_us":686845}
I20260709 12:22:54.275372 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:54.259869 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:54.261051 24609 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.261390 24629 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:54.262250 24668 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:54.264242 24464 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.264601 24530 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:54.239743 24575 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.231272 24628 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.249693 24632 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.247604 24225 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:54.271793 24659 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.259676 24458 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:54.277443 20883 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 5.
I20260709 12:22:54.246857 24221 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:54.299149 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 414985us
I20260709 12:22:54.248989 24655 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:54.252283 24528 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:54.334674 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:54.376117 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:54.376850 20806 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:22:54.359282 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 110235us
I20260709 12:22:54.378252 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:54.379424 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.380125 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:54.380946 20810 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:54.383175 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:54.383962 21426 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 5.
I20260709 12:22:54.381129 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:54.385198 21435 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:54.243973 24472 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.273511 24656 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:22:54.419512 21435 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:54.422123 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:54.423030 20804 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:54.424084 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:54.431643 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 97108.3us
I20260709 12:22:54.358951 24629 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:54.412601 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:54.442476 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:54.412190 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:54.457451 24632 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:54.342592 24575 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.365564 24221 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.400712 24709 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:54.507582 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:54.414703 24609 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 12:22:54.379366 24655 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.519785 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 662) took 1138 ms. Trace:
I20260709 12:22:54.520479 21432 rpcz_store.cc:276] 0709 12:22:53.381149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.402696 (+ 21547us) service_pool.cc:224] Handling call
0709 12:22:54.519478 (+1116782us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54004d9c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:54.519671 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895843200}
W20260709 12:22:54.523432 24701 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:22:54.534293 24604 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.540459 24628 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:54.551456 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.554072 24575 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:54.489023 24661 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:54.556800 24668 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.558836 24629 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.569275 24655 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.572006 24221 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.510506 24709 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:54.574936 24669 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.002s
I20260709 12:22:54.557605 24668 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:54.586854 20847 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.026s	sys 0.038s
W20260709 12:22:55.189821 20847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.112s	user 0.029s	sys 0.039s
I20260709 12:22:54.434710 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:54.411233 20810 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
W20260709 12:22:54.802539 24738 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.052s	user 0.000s	sys 0.002s
W20260709 12:22:55.061316 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.002s	sys 0.010s
W20260709 12:22:55.083242 20728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.000s	sys 0.017s
W20260709 12:22:55.248657 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.003s	sys 0.011s
W20260709 12:22:55.248781 20728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.001s	sys 0.017s
W20260709 12:22:55.249315 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 806861us
W20260709 12:22:55.095052 20880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.006s	sys 0.002s
W20260709 12:22:55.250495 20806 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.873s	user 0.009s	sys 0.008s
W20260709 12:22:55.250777 20880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.007s	sys 0.002s
I20260709 12:22:55.251085 20806 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
W20260709 12:22:55.101104 21426 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.007s	sys 0.019s
W20260709 12:22:55.252071 21426 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.007s	sys 0.020s
I20260709 12:22:55.253096 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1977) took 1056 ms. Trace:
I20260709 12:22:55.103631 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 996) took 1703 ms. Trace:
W20260709 12:22:55.125727 20726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.002s	sys 0.014s
W20260709 12:22:55.147357 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.042s	user 0.000s	sys 0.019s
W20260709 12:22:55.255240 20726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.002s	sys 0.016s
W20260709 12:22:55.255895 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.150s	user 0.000s	sys 0.020s
I20260709 12:22:55.256464 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1976) took 1060 ms. Trace:
W20260709 12:22:55.256570 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.15943e+06us
I20260709 12:22:55.255049 21427 rpcz_store.cc:276] 0709 12:22:53.399768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.400070 (+   302us) service_pool.cc:224] Handling call
0709 12:22:53.400422 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:53.400815 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:53.401115 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 12:22:53.779011 (+377896us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b18002a89e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:53.799441 (+ 20430us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b18002a89e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:53.830654 (+ 31213us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b18002a89e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:22:54.583880 (+753226us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:54.644732 (+ 60852us) log.cc:844] Serialized 26 byte log entry
0709 12:22:54.644999 (+   267us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:54.645061 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:54.645176 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:55.102985 (+457809us) raft_consensus.cc:1685] finished
0709 12:22:55.103021 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:55.103360 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10625,"prepare.queue_time_us":18609,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":196334,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":8513,"spinlock_wait_cycles":603747328,"thread_start_us":752842,"threads_started":2,"wal-append.queue_time_us":987479}
I20260709 12:22:55.256946 20726 rpcz_store.cc:276] 0709 12:22:54.196088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.196228 (+   140us) service_pool.cc:224] Handling call
0709 12:22:54.196567 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:54.196907 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:54.196932 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:54.197382 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 12:22:54.269512 (+ 72130us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b180035c3e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:22:55.255914 (+986402us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:55.255967 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.256088 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:55.256126 (+    38us) raft_consensus.cc:1685] finished
0709 12:22:55.256136 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:55.256407 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":691,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":11678,"spinlock_wait_cycles":6819072,"thread_start_us":986100,"threads_started":1,"wal-append.queue_time_us":601430}
W20260709 12:22:55.155432 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.156427 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.102s	user 0.010s	sys 0.015s
W20260709 12:22:55.260538 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.206s	user 0.010s	sys 0.015s
W20260709 12:22:55.262177 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 1.20846e+06us
W20260709 12:22:55.160816 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.197312 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.633646 24633 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:55.273146 24633 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.691737 24646 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:54.639523 24655 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:55.428349 24655 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:55.429977 24655 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:54.727402 24668 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:54.599998 24669 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.441416 24669 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:54.596316 24690 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.028s	sys 0.036s
I20260709 12:22:54.611577 24317 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:54.712483 24472 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:55.113045 24604 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:54.952698 24624 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:55.520241 24624 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:55.521503 24624 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.522755 24624 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:54.724323 24640 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.003s
W20260709 12:22:54.871770 24688 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.830s	user 0.001s	sys 0.004s
I20260709 12:22:55.051033 24673 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:54.600685 23772 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:54.739426 24693 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.002s
I20260709 12:22:55.564855 24693 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.820214 24225 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.570355 24225 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:55.122184 24629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.001s	sys 0.010s
I20260709 12:22:54.646380 24306 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:55.574637 24693 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:54.643603 24423 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.020s	sys 0.046s
W20260709 12:22:55.577730 24423 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.490s	user 0.022s	sys 0.047s
I20260709 12:22:55.109634 24492 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:55.580196 24492 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:55.581296 24492 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:54.812103 24528 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:54.899103 24557 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:54.714951 24689 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.008s	sys 0.000s
W20260709 12:22:55.116879 24702 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.585s	user 0.000s	sys 0.003s
I20260709 12:22:54.594959 24709 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:54.618383 24711 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:55.596678 24711 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:55.597759 24711 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:55.108781 24704 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.614s	user 0.003s	sys 0.000s
I20260709 12:22:54.672530 24717 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.602911 24717 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.605408 24717 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.212597 24609 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:55.607897 24609 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.610316 24609 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:54.697566 24694 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.688s	user 0.016s	sys 0.000s
I20260709 12:22:55.612442 24694 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.613687 24709 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:54.983049 24707 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.404s	user 0.001s	sys 0.001s
I20260709 12:22:55.253541 20728 rpcz_store.cc:276] 0709 12:22:54.196466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.196584 (+   118us) service_pool.cc:224] Handling call
0709 12:22:54.197021 (+   437us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:54.197515 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:54.197561 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:54.204502 (+  6941us) log.cc:844] Serialized 52 byte log entry
0709 12:22:55.251784 (+1047282us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:55.252283 (+   499us) log.cc:844] Serialized 14 byte log entry
0709 12:22:55.252603 (+   320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.252745 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:55.252793 (+    48us) raft_consensus.cc:1685] finished
0709 12:22:55.252811 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:55.252964 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1913,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":4567,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":2620,"thread_start_us":1019107,"threads_started":1,"wal-append.queue_time_us":473101}
I20260709 12:22:55.286607 24630 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.308451 24743 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.308677 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2111) took 1352 ms. Trace:
W20260709 12:22:55.313504 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45543.7us
W20260709 12:22:55.325055 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65294.4us
I20260709 12:22:55.617354 24743 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.617223 20880 rpcz_store.cc:276] 0709 12:22:53.956623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.956784 (+   161us) service_pool.cc:224] Handling call
0709 12:22:53.962522 (+  5738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:53.962730 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:22:54.400756 (+438026us) log.cc:844] Serialized 14 byte log entry
0709 12:22:55.307948 (+907192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:55.308091 (+   143us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:55.308189 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.308334 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:55.308578 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":570633,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":105257,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":196,"thread_start_us":1313346,"threads_started":3,"wal-append.queue_time_us":344090}
I20260709 12:22:55.620391 24743 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:55.339843 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:55.622670 21427 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:55.624009 24306 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:55.624241 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.624738 24306 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:55.348264 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.003s	sys 0.006s
I20260709 12:22:55.350473 24746 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:55.627578 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.627709 24746 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.628216 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.628633 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.630182 24746 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.356432 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:55.631111 21426 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:55.357128 24752 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.632479 24752 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.633882 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.635053 20807 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:22:55.361671 24745 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.635418 24743 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:55.636341 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.363158 24634 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.637132 20801 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
W20260709 12:22:55.386183 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.362s	user 0.025s	sys 0.034s
I20260709 12:22:55.397938 24646 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:55.398983 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:55.417936 23925 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:55.432669 24754 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.447005 21429 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:22:55.477357 20875 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.636s	user 0.021s	sys 0.037s
I20260709 12:22:55.479990 24221 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:55.480716 24435 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:55.484634 24433 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:55.487160 24690 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.421s	user 0.032s	sys 0.037s
W20260709 12:22:55.495261 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.11655e+06us
I20260709 12:22:55.283440 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:55.640319 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:55.641116 20806 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:55.641321 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 378208us
I20260709 12:22:55.637470 24694 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.643970 24694 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:55.644680 24306 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.525966 24640 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:55.535789 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 631) took 2954 ms. Trace:
I20260709 12:22:55.646800 23925 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:55.647239 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.624s	user 0.025s	sys 0.038s
I20260709 12:22:55.646914 20878 rpcz_store.cc:276] 0709 12:22:52.581605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.581745 (+   140us) service_pool.cc:224] Handling call
0709 12:22:52.582061 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:53.254234 (+672173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:53.254274 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:53.254727 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:22:53.581992 (+327265us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:53.582047 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:53.582135 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:54.870606 (+1288471us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:55.535282 (+664676us) raft_consensus.cc:1685] finished
0709 12:22:55.535304 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:55.535604 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":671789,"raft.queue_time_us":673039,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":27824512,"thread_start_us":282527,"threads_started":1,"wal-append.queue_time_us":673064}
I20260709 12:22:55.648195 23925 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:55.648730 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 1.65102e+06us
I20260709 12:22:55.649844 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:55.550117 24317 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:55.651415 24423 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:55.651510 24317 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.653802 24317 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:55.557323 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.12296e+06us
I20260709 12:22:55.558303 24708 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.656497 24528 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:55.558370 24575 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:55.658268 24575 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.660602 24575 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.571558 24532 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.662158 24532 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.571626 24758 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.663558 24758 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.665959 24758 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:55.573345 24629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.003s	sys 0.010s
I20260709 12:22:55.562508 23772 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.579396 24225 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:55.669927 24225 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.672245 24225 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.582983 24492 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:55.590020 24689 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.682s	user 0.009s	sys 0.000s
I20260709 12:22:55.606361 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2272) took 2272 ms. Trace:
I20260709 12:22:54.692641 24723 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:55.675437 20731 rpcz_store.cc:276] 0709 12:22:53.333603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.333735 (+   132us) service_pool.cc:224] Handling call
0709 12:22:53.334126 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:53.748294 (+414168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:53.748325 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:53.763918 (+ 15593us) log.cc:844] Serialized 52 byte log entry
0709 12:22:54.140531 (+376613us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:54.151375 (+ 10844us) log.cc:844] Serialized 14 byte log entry
0709 12:22:54.153876 (+  2501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:54.153974 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:54.272235 (+118261us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b1800270a60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:22:55.107934 (+835699us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:22:55.605828 (+497894us) raft_consensus.cc:1685] finished
0709 12:22:55.605853 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:55.606142 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4607,"prepare.queue_time_us":10449,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":36443,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":11281,"spinlock_wait_cycles":77148800,"thread_start_us":704350,"threads_started":2,"wal-append.queue_time_us":558685}
I20260709 12:22:55.677109 24723 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.677838 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:55.675758 20806 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:55.679746 24723 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.615319 20809 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
I20260709 12:22:55.680665 24754 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.682302 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 603) took 1305 ms. Trace:
I20260709 12:22:55.618615 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:55.329185 24289 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.682725 20809 rpcz_store.cc:276] 0709 12:22:54.376909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.378966 (+  2057us) service_pool.cc:224] Handling call
0709 12:22:55.682028 (+1303062us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400127854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:55.682203 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687652224}
I20260709 12:22:55.637354 24634 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.636023 24745 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:55.218619 24706 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.591s	user 0.005s	sys 0.000s
W20260709 12:22:55.639545 20875 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.798s	user 0.022s	sys 0.038s
I20260709 12:22:55.547829 24472 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:55.627743 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.003s	sys 0.007s
W20260709 12:22:55.683506 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 284761us
W20260709 12:22:55.684062 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65590us
W20260709 12:22:55.694540 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66020.2us
I20260709 12:22:55.693307 24735 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:55.683126 24754 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.692862 24655 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.693266 24609 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:55.690274 20776 leader_election.cc:400] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:22:55.720367 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:22:55.717650 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 80284.7us
W20260709 12:22:55.686374 21429 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 671) took 1664 ms (client timeout 1753 ms). Trace:
W20260709 12:22:55.727417 21429 rpcz_store.cc:269] 0709 12:22:54.021972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.022973 (+  1001us) service_pool.cc:224] Handling call
0709 12:22:55.686079 (+1663106us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:55.686286 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3244517248}
I20260709 12:22:55.685220 24624 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.685034 24289 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:55.715884 24735 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.720084 24745 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.720531 24225 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:55.697779 24532 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.736142 20701 leader_election.cc:400] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12.
W20260709 12:22:55.736907 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 94428.3us
I20260709 12:22:55.740950 20701 leader_election.cc:403] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:55.743953 20809 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:55.728456 24609 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:55.694175 24472 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.696913 24433 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:22:55.773211 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.776553 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.777293 20883 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:55.777585 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.778076 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:55.778412 20876 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:55.778870 20881 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:55.779582 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.780117 24225 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:55.780279 20879 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:55.781519 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.782246 20884 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:55.777501 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.784005 20882 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
W20260709 12:22:55.784184 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.788776 24745 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:22:55.788964 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.789734 20882 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:55.752281 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.792111 24761 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:55.792786 24761 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:55.793658 24634 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.752457 24710 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:55.752521 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.794395 24710 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.794638 20880 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:22:55.794857 24710 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:55.795622 24423 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:55.752521 24659 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:55.736559 24735 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:55.797201 24659 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.798172 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.799131 20804 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:22:55.793902 24761 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.684887 24629 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:22:55.729326 24723 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:55.756474 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.745862 24752 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.807667 23925 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:55.809614 24754 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:55.809470 23925 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.720343 24655 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.814136 24723 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.816021 24655 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.816570 24723 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:55.758962 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68949.2us
I20260709 12:22:55.817507 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:55.818111 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 89407.9us
I20260709 12:22:55.818370 20776 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:22:55.818950 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:55.734508 24289 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:55.819849 20877 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:55.820050 24289 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:55.822469 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.684747 24646 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.823879 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.824318 24646 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:55.763016 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.825976 20809 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:22:55.763480 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.826427 24609 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.766285 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.755766 24709 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.804386 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:55.804549 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.804893 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.805137 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.804512 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:55.806456 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44696.3us
W20260709 12:22:55.804497 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50428.2us
I20260709 12:22:55.806109 24433 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:55.812080 24690 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:22:55.758015 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 291) took 31134 ms (client timeout 15000 ms). Trace:
I20260709 12:22:55.813563 24472 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.819355 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.826957 20808 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:55.764649 20810 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:55.828637 20806 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:55.829981 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.818128 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:55.697194 20630 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 6 to 11, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 11 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:55.837348 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.833498 20804 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:55.840319 20881 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:55.843596 24711 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:55.843971 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:55.844419 21400 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
I20260709 12:22:55.844638 23772 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:55.845184 23772 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:55.845436 24472 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.844079 24711 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:55.844776 21400 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:55.846552 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.808092 24532 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:55.857798 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.854678 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:55.860356 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:55.854359 24711 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:55.836172 24557 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.833213 24690 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:55.864042 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46022.7us
I20260709 12:22:55.861992 24532 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.868700 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:55.869249 24659 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.874686 20881 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:55.875993 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.873159 24754 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:55.851917 24709 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:55.842705 24433 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.839859 24492 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 12:22:55.890583 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61303.5us
I20260709 12:22:55.894157 20806 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: [8.5-9.6]
I20260709 12:22:55.887722 24634 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:55.900218 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:55.903728 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:55.874398 24575 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:22:55.904299 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.890734 24221 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:55.897382 20630 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 9, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 9 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:55.899899 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:55.907792 23925 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.908653 21429 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:55.854178 23772 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.898284 24655 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:55.898401 24624 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:55.887854 24557 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.911837 24221 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:55.906539 24634 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.936009 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:55.833439 20628 rpcz_store.cc:269] 0709 12:22:24.742205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.775508 (+ 33303us) service_pool.cc:224] Handling call
0709 12:22:55.757707 (+30982199us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:54.226089 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:54.453360 (+227271us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:54.453441 (+    81us) write_op.cc:435] Acquired schema lock
0709 12:22:54.453473 (+    32us) tablet.cc:662] Decoding operations
0709 12:22:54.503715 (+ 50242us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:54.503858 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:54.503915 (+    57us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:22:54.505674 (+  1759us) tablet.cc:701] Row locks acquired
0709 12:22:54.505734 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:22:54.505996 (+   262us) write_op.cc:270] Start()
0709 12:22:54.506207 (+   211us) write_op.cc:276] Timestamp: P: 1783599774505905 usec, L: 0
0709 12:22:54.506234 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:22:54.592599 (+ 86365us) log.cc:844] Serialized 5672 byte log entry
0709 12:22:55.300839 (+708240us) op_driver.cc:464] REPLICATION: finished
0709 12:22:55.333895 (+ 33056us) write_op.cc:301] APPLY: starting
0709 12:22:55.334055 (+   160us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:55.334510 (+   455us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:55.334662 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:55.354270 (+ 19608us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:55.379401 (+ 25131us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:22:55.379461 (+    60us) write_op.cc:312] APPLY: finished
0709 12:22:55.380031 (+   570us) log.cc:844] Serialized 101 byte log entry
0709 12:22:55.380288 (+   257us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:55.380846 (+   558us) write_op.cc:454] Released schema lock
0709 12:22:55.380924 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:55.381630 (+   706us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:55.381686 (+    56us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":324479,"reported_tablets":42,"spinlock_wait_cycles":28544,"child_traces":[["op",{"apply.queue_time_us":32651,"apply.run_cpu_time_us":6477,"apply.run_wall_time_us":47704,"num_ops":14,"prepare.queue_time_us":419442,"prepare.run_cpu_time_us":13963,"prepare.run_wall_time_us":592462,"raft.queue_time_us":52610,"raft.run_cpu_time_us":12606,"raft.run_wall_time_us":36001,"replication_time_us":794231,"thread_start_us":496222,"threads_started":4,"wal-append.queue_time_us":134322}]]}
I20260709 12:22:55.937618 20810 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:55.939100 24532 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.939575 24746 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:55.940511 24690 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:55.941962 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49020.1us
I20260709 12:22:55.955852 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:55.896376 24659 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.957988 23925 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:55.958376 23925 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:55.952958 20877 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:55.937546 20804 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:55.934629 24624 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.964955 21429 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:55.962703 23925 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.963711 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:55.912770 24709 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.906111 24754 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:55.951284 24221 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:55.954715 24746 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.970548 24690 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:55.970690 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:55.986294 20804 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:55.970999 24723 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:55.971728 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.968261 20630 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 4 to 7, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 7 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:55.973320 24575 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:55.989193 23925 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:55.975211 24668 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.995484 24747 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:55.976276 24492 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.952705 24634 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:55.964665 24710 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:56.003502 24532 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:56.004034 24532 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:56.004416 24532 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.004945 24710 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.984272 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:56.006455 20803 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:56.007385 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:55.949998 24630 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:56.008292 24630 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.008728 24630 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.008559 24492 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.009824 24289 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:55.976620 24557 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.988898 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.011750 20809 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:56.012812 24717 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:56.013589 24717 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:56.013926 24717 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:56.014324 24717 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.014374 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.015511 24306 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.015654 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.016043 24629 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:56.016423 24433 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.016141 24306 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.016739 24629 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:56.016932 24735 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.018038 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.018843 20730 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 6: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:55.990633 20630 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:56.021003 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:55.964490 24693 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:56.022153 24693 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.023151 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:56.013952 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.023866 20733 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.024418 24693 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:55.996172 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.025981 20810 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 6.
I20260709 12:22:56.026436 24754 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:55.990321 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:56.008316 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.028216 24709 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.028416 20803 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
W20260709 12:22:55.990767 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.989591 24575 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.964723 24624 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.000818 24752 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:56.017235 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:56.021740 20630 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 6 to 10, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 10 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:56.029888 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.030658 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:55.994915 24530 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.996173 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:56.010815 24492 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.016628 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.082196 20726 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:56.082619 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.048954 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.083611 20877 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:22:56.059733 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.990397 24746 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.030643 24306 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.088042 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.089243 20880 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:56.090454 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.092188 20729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:22:56.093048 20729 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:56.095424 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.097002 20810 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:56.097260 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.103354 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.104557 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:55.990034 24723 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.084622 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:56.064627 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 100997us
I20260709 12:22:55.997555 24659 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.069494 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:56.070314 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.072152 24289 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.072286 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.078537 20733 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:56.086637 24575 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.112730 20804 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:22:56.098978 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.083793 20731 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:56.120043 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.018952 24735 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:56.129720 20801 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:22:56.131855 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 133677us
I20260709 12:22:56.138409 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.018903 24629 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.133222 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:56.128665 21426 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:56.151482 20807 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:56.126569 20883 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:56.156471 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:56.165388 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:56.166173 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.167583 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:56.168380 20877 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:56.168910 21432 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.165902 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
W20260709 12:22:56.183979 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.177816 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2094) took 2849 ms. Trace:
I20260709 12:22:56.195464 24735 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:22:56.195878 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43650.9us
I20260709 12:22:56.196242 24735 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.196728 24735 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.197297 24745 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12.
I20260709 12:22:56.077785 24747 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:56.195485 24752 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:56.200320 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 98098.4us
I20260709 12:22:56.200876 24747 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.195613 20805 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:22:56.195711 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.119115 24723 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.203006 20805 pending_rounds.cc:85] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 8
I20260709 12:22:56.211659 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:56.211747 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:56.216375 20807 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 9
W20260709 12:22:56.216718 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:56.215528 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47242us
I20260709 12:22:56.217698 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.177687 24594 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:56.165553 24709 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:56.220039 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.221860 20733 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:22:56.221695 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:56.220732 20880 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:56.211701 20805 pending_rounds.cc:107] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:22:56.220466 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:56.226434 20805 raft_consensus.cc:2889] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:56.227147 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.228020 20806 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:22:56.169643 24640 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:56.231781 20731 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:56.237172 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.238054 20804 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:56.239528 21434 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:56.118528 24711 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.192859 20875 rpcz_store.cc:276] 0709 12:22:53.328239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.337744 (+  9505us) service_pool.cc:224] Handling call
0709 12:22:53.354111 (+ 16367us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:53.762967 (+408856us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:55.693235 (+1930268us) log.cc:844] Serialized 14 byte log entry
0709 12:22:55.739038 (+ 45803us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:55.824305 (+ 85267us) log.cc:844] Serialized 52 byte log entry
0709 12:22:55.827704 (+  3399us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:56.088603 (+260899us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b1800254020. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf
0709 12:22:56.088756 (+   153us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.122695 (+ 33939us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.177158 (+ 54463us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b1800254020. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:56.177378 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:56.177515 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:56.177561 (+    46us) raft_consensus.cc:1685] finished
0709 12:22:56.177579 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:56.177759 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72665,"prepare.queue_time_us":828424,"prepare.run_cpu_time_us":178,"prepare.run_wall_time_us":11210,"raft.queue_time_us":135515,"raft.run_cpu_time_us":454,"raft.run_wall_time_us":20409,"spinlock_wait_cycles":45663488,"thread_start_us":2134254,"threads_started":3,"wal-append.queue_time_us":117164}
I20260709 12:22:56.243615 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.244390 20880 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:22:56.168568 24530 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:56.245220 24777 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:22:56.245373 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.246222 20880 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:22:56.246567 24530 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.247298 20809 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:56.248482 20732 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:22:56.201153 24752 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.253147 24221 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:56.177539 24761 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:22:56.234520 24640 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:56.255425 24640 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:56.255931 24594 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.243749 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.258237 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:56.258713 20801 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:56.259332 24768 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:22:56.261377 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:56.262044 20875 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.247090 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.220160 24709 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.263178 20630 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:56.280586 20875 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:56.271311 20880 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:56.272961 20729 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:22:56.353171 20630 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:56.955279 20630 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 4 to 9, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 9 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:56.957032 20630 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 8, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 8 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:56.552443 24787 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.571336 20727 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
W20260709 12:22:56.574581 24783 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 12:22:56.958505 24787 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.955056 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.663988 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:56.667127 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.627362 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:57.021965 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 394657us
I20260709 12:22:57.022759 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:56.420539 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.037266 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.038331 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.038964 20804 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:56.562134 20808 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:22:57.041137 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.042155 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:56.335588 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.044602 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.045567 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.046461 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:56.521950 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.539307 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.058391 20806 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:22:56.314232 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.556151 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:56.340731 20876 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:56.677790 20879 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:22:57.066473 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 651) took 1025 ms. Trace:
I20260709 12:22:57.067066 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2130) took 1003 ms. Trace:
I20260709 12:22:57.067369 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:57.067489 20879 rpcz_store.cc:276] 0709 12:22:56.063741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.677162 (+613421us) service_pool.cc:224] Handling call
0709 12:22:56.677536 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:57.066813 (+389277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.067016 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:57.068543 24761 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:56.399886 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 188289us
I20260709 12:22:56.361582 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:56.340142 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.075623 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:57.076843 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:57.077888 20804 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:22:57.078274 20804 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 9
I20260709 12:22:56.566542 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:57.078025 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:56.310117 21432 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:57.080135 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.081148 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1002) took 1042 ms. Trace:
I20260709 12:22:57.081476 21432 rpcz_store.cc:276] 0709 12:22:56.039169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.039395 (+   226us) service_pool.cc:224] Handling call
0709 12:22:57.081011 (+1041616us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b54007b2ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:57.081099 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118725504}
I20260709 12:22:57.082579 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:56.312306 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:56.343739 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:56.339689 21456 debug-util.cc:398] Leaking SignalData structure 0x7b0800303660 after lost signal to thread 19606
I20260709 12:22:57.086745 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:56.277853 24221 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.618384 24433 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.093614 24433 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.155589 24423 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 12:22:56.356751 24532 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.101825 24532 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.611493 24557 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.103646 24557 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.105927 24557 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.360922 24575 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.108685 24575 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.267563 24630 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:56.351060 24624 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.111368 24630 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.111910 24624 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.112218 20802 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:56.266212 24640 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.119822 20808 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:57.120373 24640 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:56.578706 24646 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.000s
I20260709 12:22:56.359439 24655 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7.
I20260709 12:22:56.226569 24634 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.129891 20810 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:22:57.131178 24634 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.352926 24659 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.580744 24689 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:57.133697 24689 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:57.134083 24689 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:56.570850 24690 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:57.135752 20803 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:57.135906 24690 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.137123 21428 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:57.138047 24690 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:56.289412 24694 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:56.340380 24708 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:57.140364 24708 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.142457 24708 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.299953 24709 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.631237 24711 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:56.543769 24717 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.529s	user 0.003s	sys 0.009s
I20260709 12:22:57.147631 24717 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.623456 24723 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:56.564117 24735 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.151233 20806 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:57.151481 21430 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:57.151671 24735 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.347152 24746 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.153504 20806 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:22:56.280035 24749 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:56.359565 24758 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.159636 24758 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.160601 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:57.160161 24709 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.162482 24758 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:56.661814 24778 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.381s	user 0.003s	sys 0.001s
I20260709 12:22:56.806077 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.166307 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.166478 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:57.166981 20733 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:57.167784 24708 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:57.166461 24709 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:56.809110 20877 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:56.273260 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.171967 20726 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:57.173026 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.173310 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:57.173655 20726 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:57.174547 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:56.917896 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.014s	sys 0.010s
W20260709 12:22:57.175380 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.014s	sys 0.010s
I20260709 12:22:56.855866 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.176642 20730 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:57.177778 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.178618 24634 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.931929 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.179140 24634 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.179549 21434 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:57.181237 21434 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:57.181406 24634 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.182183 24492 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:57.182655 21434 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:56.934667 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.183640 20805 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:22:56.949270 24776 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.682s	user 0.003s	sys 0.003s
I20260709 12:22:56.623173 20735 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:22:57.184839 24634 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.185411 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.185315 24634 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.186513 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.963618 20630 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:57.187820 20630 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 7, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 7 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:57.189922 20630 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 8 to 9, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 9 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:22:56.966265 24787 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:56.966526 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 756573us
I20260709 12:22:57.192896 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:57.194646 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:56.968936 20731 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:57.195816 24634 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.195930 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.196457 24634 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:57.196839 24634 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.197162 24634 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:57.197234 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.198479 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:56.971351 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.200230 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.201609 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.202594 20729 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:56.973536 24629 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.203612 24629 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.206197 24629 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.982326 24594 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.999272 24317 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:56.989537 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.009163 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.207768 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.208441 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.208803 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.209038 20727 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:57.209352 20732 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:57.210459 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.009244 24669 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:57.211477 24669 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.213443 24669 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.025424 24802 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:57.213966 24802 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:57.214339 24802 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:57.216590 20727 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:57.217514 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.217351 24634 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.218633 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.027480 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:57.030177 24779 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.774s	user 0.005s	sys 0.000s
I20260709 12:22:57.031637 24803 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:57.220479 24803 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:57.220950 24803 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:57.221446 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:57.033495 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.223031 24624 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:57.223220 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:57.223548 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 190126us
I20260709 12:22:57.224241 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.224470 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:57.225081 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.225585 24809 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:57.226454 24809 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:57.065441 20884 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:57.226830 24809 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.227685 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.182699 24492 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.228840 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.229620 24492 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.065945 24472 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:57.069032 24761 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.229981 24809 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:57.230252 24472 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.232079 24761 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.066895 20876 rpcz_store.cc:276] 0709 12:22:56.041395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.330301 (+288906us) service_pool.cc:224] Handling call
0709 12:22:56.340351 (+ 10050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.066191 (+725840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.066416 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:57.232400 24472 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:57.074223 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:56.640268 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.233847 24575 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:57.234280 24575 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:57.234649 24575 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:57.234898 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.235540 21431 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:22:57.078554 20804 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:22:57.236370 20804 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:57.079517 20849 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:22:57.237228 24708 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:57.237525 20849 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:57.237802 24708 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:57.238140 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 158772us
I20260709 12:22:57.079442 21426 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:57.238716 24708 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.239346 24761 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:22:57.240087 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.082814 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:57.241092 24646 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:57.241524 24646 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:57.187070 20735 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:57.241894 24646 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.242479 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.083587 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.243681 20875 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:57.178995 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.244587 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.244851 24805 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:57.245381 24805 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:57.245981 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.246306 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.247104 20726 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:57.083985 21435 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:57.085186 20879 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:57.247961 24634 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.085132 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:57.087301 21456 debug-util.cc:398] Leaking SignalData structure 0x7b08003039a0 after lost signal to thread 20667
I20260709 12:22:57.090273 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.250746 21432 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:22:57.251260 21456 debug-util.cc:398] Leaking SignalData structure 0x7b0800303ec0 after lost signal to thread 20759
W20260709 12:22:57.251576 21456 debug-util.cc:398] Leaking SignalData structure 0x7b08001a1700 after lost signal to thread 20760
I20260709 12:22:57.252013 24749 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:57.251888 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:57.252605 21456 debug-util.cc:398] Leaking SignalData structure 0x7b080030f5c0 after lost signal to thread 20834
W20260709 12:22:57.252936 21456 debug-util.cc:398] Leaking SignalData structure 0x7b0800366800 after lost signal to thread 20835
I20260709 12:22:57.252748 24749 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:57.113077 20728 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:57.253623 24810 raft_consensus.cc:2764] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:22:57.254037 21456 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5800 after lost signal to thread 20908
W20260709 12:22:57.254334 21456 debug-util.cc:398] Leaking SignalData structure 0x7b0800250ae0 after lost signal to thread 20909
I20260709 12:22:57.254218 24749 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:57.255357 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 546) took 1243 ms. Trace:
W20260709 12:22:57.255607 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 172896us
I20260709 12:22:57.255766 20728 rpcz_store.cc:276] 0709 12:22:56.011792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.015346 (+  3554us) service_pool.cc:224] Handling call
0709 12:22:57.255140 (+1239794us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003b4f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:57.255275 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2521819392}
I20260709 12:22:57.246363 24805 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.257105 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.257670 20728 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:22:57.258525 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.114264 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2052) took 1142 ms. Trace:
I20260709 12:22:57.258977 24810 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:22:57.259541 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74016.5us
I20260709 12:22:57.259371 20802 rpcz_store.cc:276] 0709 12:22:55.971720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.986623 (+ 14903us) service_pool.cc:224] Handling call
0709 12:22:57.114065 (+1127442us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003ca354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:57.114203 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317550848}
I20260709 12:22:57.115000 20801 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:57.121610 20808 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:57.132447 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.138269 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 617) took 1098 ms. Trace:
I20260709 12:22:57.263614 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.142463 24433 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.143764 20734 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 10. Ops: []
I20260709 12:22:57.152611 21429 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:57.154232 24746 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:56.666626 24747 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.154978 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.155591 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.155480 24694 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:57.134997 24689 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.156368 24717 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.151021 24723 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.157541 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2129) took 1094 ms. Trace:
I20260709 12:22:57.158226 20806 pending_rounds.cc:85] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:22:57.348814 20806 pending_rounds.cc:107] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:22:57.349200 20806 raft_consensus.cc:2889] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:57.174607 24709 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:56.282464 20807 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:57.384003 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 620) took 1216 ms. Trace:
I20260709 12:22:57.384373 20807 rpcz_store.cc:276] 0709 12:22:56.167266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.175425 (+  8159us) service_pool.cc:224] Handling call
0709 12:22:57.383942 (+1208517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:57.048934 24752 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:57.233083 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.385560 24752 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:57.385962 24752 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:22:57.233371 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159206us
I20260709 12:22:57.386191 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.386873 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:57.387353 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:57.387797 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.388403 24749 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:57.388770 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:57.388922 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.389678 24717 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:57.390060 24717 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:57.390193 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.390429 24717 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.390793 20876 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.619282 23925 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.093091 20880 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:57.397924 24803 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.229991 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.265605 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.398360 23925 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.219897 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.266711 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.399535 20729 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:57.399832 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.400115 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.400336 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.400866 20729 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:22:57.401235 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.401834 20729 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:57.268208 21456 debug-util.cc:398] Leaking SignalData structure 0x7b080035f120 after lost signal to thread 21459
I20260709 12:22:57.272892 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:57.403898 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 131123us
I20260709 12:22:57.404634 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.405283 24689 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:57.405475 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.406075 24221 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:57.406737 24221 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:57.245204 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.407378 24433 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:57.243619 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.407644 20810 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:57.408913 20735 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:57.398351 24717 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.264380 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:22:57.411561 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 152812us
I20260709 12:22:57.412163 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:57.262872 20803 rpcz_store.cc:276] 0709 12:22:56.040229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.079879 (+ 39650us) service_pool.cc:224] Handling call
0709 12:22:56.080398 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.136155 (+1055757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.138020 (+  1865us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540027a954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:57.138187 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2426598784}
I20260709 12:22:57.412391 24575 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.265661 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.415227 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.199025 20731 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:22:57.407857 24433 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:57.416926 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:57.417375 24433 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.281752 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.418735 23925 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.418876 20808 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:22:57.277349 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:57.419426 23925 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:57.280509 20630 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 5, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 5 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.291716 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 852) took 1324 ms. Trace:
I20260709 12:22:57.419970 23925 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:57.420127 21429 rpcz_store.cc:276] 0709 12:22:55.967635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.967779 (+   144us) service_pool.cc:224] Handling call
0709 12:22:57.291529 (+1323750us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:57.291664 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2586600704}
I20260709 12:22:57.284188 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:57.292694 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.293605 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.297235 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:57.407598 20730 raft_consensus.cc:2368] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
W20260709 12:22:57.422519 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 125397us
I20260709 12:22:57.299042 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2281) took 1049 ms. Trace:
I20260709 12:22:57.302181 21427 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:22:57.303571 20877 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:57.425433 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1953) took 1066 ms. Trace:
I20260709 12:22:57.305600 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.426842 20732 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:22:57.305968 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:57.309108 20733 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:57.309986 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.309827 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.406855 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:57.429273 20802 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:57.278790 24557 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:57.434978 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 150906us
I20260709 12:22:57.435806 20733 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:57.436267 20729 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:22:57.436640 20729 pending_rounds.cc:85] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 5
I20260709 12:22:57.436867 20729 pending_rounds.cc:107] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:22:57.437120 20729 raft_consensus.cc:2889] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:57.436949 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:57.322880 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.438674 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:57.323315 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:57.330930 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.440007 20883 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:22:57.287439 24690 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:22:57.291011 24472 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:22:57.299329 24761 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:22:57.278445 24809 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.324842 24746 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:57.338873 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:22:57.352286 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 157708us
I20260709 12:22:57.347779 20882 rpcz_store.cc:276] 0709 12:22:56.063387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.653154 (+589767us) service_pool.cc:224] Handling call
0709 12:22:56.653608 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.144236 (+490628us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:57.144566 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 12:22:57.156742 (+ 12176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:57.156789 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:57.156829 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.157343 (+   514us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:57.157384 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.157489 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":168,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":45,"raft.queue_time_us":1209,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":1127762688,"thread_start_us":11814,"threads_started":1,"wal-append.queue_time_us":15939}
I20260709 12:22:57.343348 24492 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:57.355129 24802 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.355520 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.357863 24624 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:57.351172 24709 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:57.503428 21433 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:57.439481 24575 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.348302 24747 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.367234 24646 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.375535 24758 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.375897 24630 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.378233 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.386874 24752 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:57.248062 21435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 675) took 1193 ms (client timeout 1567 ms). Trace:
I20260709 12:22:57.382237 24814 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:22:57.572647 21435 rpcz_store.cc:269] 0709 12:22:56.054576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.134021 (+ 79445us) service_pool.cc:224] Handling call
0709 12:22:57.248005 (+1113984us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:57.580462 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 141852us
I20260709 12:22:57.393769 24813 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.398478 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1952) took 1040 ms. Trace:
I20260709 12:22:57.585384 20880 rpcz_store.cc:276] 0709 12:22:56.358418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.361230 (+  2812us) service_pool.cc:224] Handling call
0709 12:22:57.398415 (+1037185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:57.399005 20884 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:57.142875 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:57.586601 21428 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:57.407119 24221 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.423444 20734 rpcz_store.cc:276] 0709 12:22:56.249890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.251035 (+  1145us) service_pool.cc:224] Handling call
0709 12:22:56.267722 (+ 16687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.279508 (+1011786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.298728 (+ 19220us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54001d6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:57.298954 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014312192}
I20260709 12:22:57.425872 20877 rpcz_store.cc:276] 0709 12:22:56.358778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.808697 (+449919us) service_pool.cc:224] Handling call
0709 12:22:56.808997 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.425081 (+616084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.425366 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:57.438154 20630 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 1 to 7, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 7 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.398101 20876 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:57.438154 21434 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:57.602540 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.449257 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1006) took 1281 ms. Trace:
I20260709 12:22:57.449965 24820 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:57.435917 24629 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:57.577708 24814 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:57.454141 20733 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:57.454736 24803 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.583739 24758 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.660369 24747 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.604677 24423 raft_consensus.cc:2764] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:22:57.455626 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.438047 24799 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:57.456197 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:57.664017 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 207863us
I20260709 12:22:57.664884 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:57.462456 24749 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:57.485924 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.666790 24749 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:57.667431 24749 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:57.468670 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56578.1us
I20260709 12:22:57.668540 24749 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.669183 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.429613 24723 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:57.358546 24810 raft_consensus.cc:993] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
W20260709 12:22:57.556679 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52763.2us
I20260709 12:22:57.561959 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1010) took 1283 ms. Trace:
I20260709 12:22:57.603067 24492 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:22:57.465387 24796 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:57.444710 24557 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.570772 20726 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:57.468626 24746 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:57.572865 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.661175 24758 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.605964 21427 rpcz_store.cc:276] 0709 12:22:56.168299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.176184 (+  7885us) service_pool.cc:224] Handling call
0709 12:22:57.292068 (+1115884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:57.430625 (+138557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.449038 (+ 18413us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400453150. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:57.449178 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482024192}
I20260709 12:22:57.499675 24669 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:22:57.672053 24717 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:57.646677 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48528.6us
I20260709 12:22:57.671533 24723 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:57.710421 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:57.649075 20630 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 4 to 6, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 6 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.648027 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.453557 23925 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.605296 24221 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:57.638691 24743 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.438239 24816 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:57.661677 20881 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:57.640424 24708 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:57.563642 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 140318us
I20260709 12:22:57.638396 24575 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:22:57.605283 24813 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.658083 24752 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:57.654320 24629 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.585059 24630 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.690183 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:57.557840 24709 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:57.503072 24787 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
W20260709 12:22:57.708001 20756 debug-util.cc:398] Leaking SignalData structure 0x7b0800366420 after lost signal to thread 19603
W20260709 12:22:57.762936 20756 debug-util.cc:398] Leaking SignalData structure 0x7b080010e3c0 after lost signal to thread 19606
W20260709 12:22:57.764297 20756 debug-util.cc:398] Leaking SignalData structure 0x7b08000b70e0 after lost signal to thread 20667
W20260709 12:22:57.766091 20756 debug-util.cc:398] Leaking SignalData structure 0x7b080030b760 after lost signal to thread 20759
I20260709 12:22:57.671629 24659 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:57.687357 24803 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:57.688141 24747 raft_consensus.cc:1081] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
W20260709 12:22:57.719547 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.776305 20727 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:57.729305 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:57.730029 20804 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:22:57.675222 21430 rpcz_store.cc:276] 0709 12:22:56.278341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.161636 (+883295us) service_pool.cc:224] Handling call
0709 12:22:57.161909 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.162133 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:57.336201 (+174068us) log.cc:844] Serialized 14 byte log entry
0709 12:22:57.561594 (+225393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:57.561638 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:57.561685 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.561748 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.561886 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6717,"prepare.queue_time_us":142423,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":26624,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":12552,"thread_start_us":367216,"threads_started":2,"wal-append.queue_time_us":204851}
I20260709 12:22:57.686944 24805 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:22:57.750406 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:57.728077 24723 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:57.768231 20756 debug-util.cc:398] Leaking SignalData structure 0x7b080038c780 after lost signal to thread 20760
I20260709 12:22:57.730500 24624 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:22:57.784425 20756 debug-util.cc:398] Leaking SignalData structure 0x7b08002371a0 after lost signal to thread 20834
W20260709 12:22:57.784893 20756 debug-util.cc:398] Leaking SignalData structure 0x7b08004091a0 after lost signal to thread 20835
I20260709 12:22:57.760650 24629 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:57.788261 20756 debug-util.cc:398] Leaking SignalData structure 0x7b0800363100 after lost signal to thread 20908
I20260709 12:22:57.789309 24646 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:57.790596 24630 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.790796 24646 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:57.791440 24646 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:57.792403 24646 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.793678 20884 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:57.797547 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:57.798372 20756 debug-util.cc:398] Leaking SignalData structure 0x7b080031a820 after lost signal to thread 20909
I20260709 12:22:57.767865 24810 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:57.776032 24803 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:57.774498 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84385.1us
W20260709 12:22:57.784587 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 110854us
I20260709 12:22:57.780391 20883 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
W20260709 12:22:57.799821 20756 debug-util.cc:398] Leaking SignalData structure 0x7b0800193dc0 after lost signal to thread 21459
W20260709 12:22:57.800913 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 353083us
I20260709 12:22:57.728120 20630 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 5 to 7, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 7 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:22:57.813391 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 406561us
I20260709 12:22:57.826097 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:22:57.812592 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62307.9us
I20260709 12:22:57.827857 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:57.808375 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 160477us
W20260709 12:22:57.817989 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 153200us
I20260709 12:22:57.811981 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:57.745637 24743 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.858058 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:22:57.859711 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47826.3us
I20260709 12:22:57.762529 24820 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:22:57.860500 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:57.862859 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.787559 24809 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:57.865128 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:57.867827 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.869449 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:57.870335 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:57.872295 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:57.873389 20810 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:57.880487 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.881248 20879 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:57.745891 24814 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:57.875123 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:57.882668 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:57.891544 20806 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:57.882838 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70943.9us
I20260709 12:22:57.886631 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 1993) took 1084 ms. Trace:
I20260709 12:22:57.824679 24803 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:57.894402 20727 rpcz_store.cc:276] 0709 12:22:56.802203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.402042 (+599839us) service_pool.cc:224] Handling call
0709 12:22:57.402287 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.776722 (+374435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.886366 (+109644us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54000c44d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:57.886548 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850007424}
I20260709 12:22:57.900992 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:57.903347 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:57.904587 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:57.810266 24827 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:57.908407 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:57.915460 23925 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:57.922004 23925 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.864359 24758 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:57.922541 23925 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:57.925421 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:57.921605 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:57.929287 20884 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:22:57.782178 24746 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.824939 24810 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.882423 24659 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:22:57.945101 24669 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:22:57.946174 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77007.9us
I20260709 12:22:57.988708 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:22:57.949376 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67959.6us
I20260709 12:22:57.943380 24708 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:57.959544 20806 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:22:57.940788 24805 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:22:57.957412 20731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.007s	sys 0.003s
I20260709 12:22:57.971237 20630 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:22:57.993040 20731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.008s	sys 0.004s
I20260709 12:22:57.993300 20630 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 8, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 8 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.995162 24708 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:57.995621 20630 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 2 to 7, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 7 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.997478 20630 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.999761 20630 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.994210 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.001593 24630 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:58.014333 20727 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:58.015074 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:58.021248 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:58.024840 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:58.026223 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:57.991096 24838 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:58.028189 20734 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:58.012180 20733 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:58.001485 24805 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:58.029927 20732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.005s	sys 0.011s
I20260709 12:22:58.008409 24557 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.032517 21430 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:58.055780 20630 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 5 to 9, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 9 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:22:58.028132 20729 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:58.005226 24669 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:22:58.031562 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 163135us
I20260709 12:22:58.061455 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.062992 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:22:58.031417 24630 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.010957 24831 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:22:58.053272 20732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.005s	sys 0.011s
I20260709 12:22:58.063180 20630 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 4, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 4 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:58.011162 24834 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:58.065685 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2298) took 1099 ms. Trace:
I20260709 12:22:58.035604 24532 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:22:58.025601 24802 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:58.033748 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45112.7us
W20260709 12:22:58.061071 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.069200 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.091504 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:58.091753 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
W20260709 12:22:58.093502 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51927.4us
I20260709 12:22:58.081413 23925 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.070313 20732 rpcz_store.cc:276] 0709 12:22:56.966750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.427660 (+460910us) service_pool.cc:224] Handling call
0709 12:22:57.463276 (+ 35616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.463520 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:57.463806 (+   286us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.065395 (+601589us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.065444 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:58.065484 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.065543 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.065636 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14420,"prepare.queue_time_us":4862,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"thread_start_us":601248,"threads_started":1}
I20260709 12:22:58.094993 24796 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:58.099108 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.058028 24805 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:58.095780 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.095721 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:58.088759 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:58.083684 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 175332us
W20260709 12:22:58.153127 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.154371 20732 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:58.068441 24838 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.157539 21427 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:22:58.071300 24630 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:58.162184 20735 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.007s	sys 0.007s
W20260709 12:22:58.165074 20735 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.007s
I20260709 12:22:58.157297 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:58.166517 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:58.096767 24813 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.095386 24690 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:58.061286 24809 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 12:22:58.055998 24849 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.129s
I20260709 12:22:58.180889 23925 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.180841 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:58.179076 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.190127 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:58.190569 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:22:58.191533 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:58.192824 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:58.193444 20726 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 12:22:58.193935 20726 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:22:58.194162 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:58.177593 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:58.170181 24689 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.764s	user 0.011s	sys 0.010s
I20260709 12:22:58.159840 24833 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:58.199477 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:58.165145 24839 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:58.201809 20732 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:22:58.187765 24823 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.387s	user 0.001s	sys 0.000s
I20260709 12:22:58.171633 24835 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:58.208798 20732 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:22:58.206427 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 125151us
W20260709 12:22:58.373942 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 216713us
I20260709 12:22:58.217581 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:58.200137 24752 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:58.205572 24802 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:58.209120 24839 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.168962 24838 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.203464 24833 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:58.212733 24810 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:58.398245 24629 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.001s
I20260709 12:22:58.399067 24864 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.399269 24624 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:58.399653 24864 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.401572 24624 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:58.297312 20879 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:22:58.402060 24624 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:58.185274 24848 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 12:22:58.310613 24855 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:58.403331 24833 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:58.403839 24855 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:58.404820 24855 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.391898 24694 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:58.304968 24840 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:22:58.204108 24831 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 12:22:58.409550 24655 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.410187 24831 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:58.410146 24655 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.410701 24831 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:58.411639 24831 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.412942 24831 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:58.413694 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.206326 24669 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.421386 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 664) took 1157 ms. Trace:
I20260709 12:22:58.421881 20878 rpcz_store.cc:276] 0709 12:22:57.264274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.574522 (+310248us) service_pool.cc:224] Handling call
0709 12:22:57.646308 (+ 71786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:57.812827 (+166519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:57.812872 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:57.879886 (+ 67014us) log.cc:844] Serialized 26 byte log entry
0709 12:22:58.084467 (+204581us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:58.084527 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.084623 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.421178 (+336555us) raft_consensus.cc:1685] finished
0709 12:22:58.421198 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.421326 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7561,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":286239,"threads_started":2,"wal-append.queue_time_us":494743}
I20260709 12:22:58.423246 24749 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:58.423682 24749 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:58.424527 24749 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.427784 24851 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.428535 24851 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.395023 24839 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:58.320658 24842 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.433126 24842 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.330801 24743 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.347409 21430 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:22:58.436334 20881 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:58.436939 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:22:58.437562 24836 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
I20260709 12:22:58.439371 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2306) took 1147 ms. Trace:
I20260709 12:22:58.439749 20728 rpcz_store.cc:276] 0709 12:22:57.291620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.561065 (+269445us) service_pool.cc:224] Handling call
0709 12:22:57.578658 (+ 17593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:57.579146 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:57.579207 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:57.606404 (+ 27197us) log.cc:844] Serialized 26 byte log entry
0709 12:22:57.753979 (+147575us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:57.788460 (+ 34481us) log.cc:844] Serialized 14 byte log entry
0709 12:22:57.808775 (+ 20315us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18002c6140. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:22:57.959403 (+150628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.963165 (+  3762us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.439064 (+475899us) raft_consensus.cc:1685] finished
0709 12:22:58.439158 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.439314 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26908,"prepare.queue_time_us":33810,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":97550,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":25155,"spinlock_wait_cycles":23051520,"thread_start_us":226820,"threads_started":2,"wal-append.queue_time_us":285353}
I20260709 12:22:58.441826 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.442390 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:58.443822 24850 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:58.444466 24850 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:58.444167 24863 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:58.445336 24850 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:58.320493 24843 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
W20260709 12:22:58.343871 24852 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:22:58.342106 24709 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.000s
I20260709 12:22:58.197751 24827 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:58.177078 24834 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:58.472239 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:58.378368 24802 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:58.377738 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:58.388073 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2294) took 1422 ms. Trace:
W20260709 12:22:58.348610 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141971us
I20260709 12:22:58.392661 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2297) took 1426 ms. Trace:
I20260709 12:22:58.396921 24532 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:58.400031 24746 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:58.421375 24838 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.481894 20735 rpcz_store.cc:276] 0709 12:22:56.965566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.409644 (+444078us) service_pool.cc:224] Handling call
0709 12:22:57.409967 (+   323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:57.410355 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:57.410380 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:57.410969 (+   589us) log.cc:844] Serialized 52 byte log entry
0709 12:22:57.561703 (+150734us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:58.165564 (+603861us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.387491 (+221927us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.387761 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.387847 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.387887 (+    40us) raft_consensus.cc:1685] finished
0709 12:22:58.387898 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.388014 (+   116us) inbound_call.cc:177] Queueing success response
0709 12:22:58.436463 (+ 48449us) spinlock_profiling.cc:243] Waited 7.12 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
Metrics: {"mutex_wait_us":22572,"prepare.queue_time_us":80139,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":81,"raft.queue_time_us":131874,"raft.run_cpu_time_us":2676,"raft.run_wall_time_us":145779,"spinlock_wait_cycles":17491584,"thread_start_us":924205,"threads_started":3,"wal-append.queue_time_us":273071}
I20260709 12:22:58.469332 24575 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.452634 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:22:58.375644 24221 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.627s	user 0.007s	sys 0.003s
W20260709 12:22:58.341646 24837 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:22:58.471367 20732 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:22:58.375149 24723 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.564s	user 0.005s	sys 0.005s
I20260709 12:22:58.471302 20883 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:22:58.371981 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 174264us
I20260709 12:22:58.545517 20849 leader_election.cc:400] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:22:58.545895 20849 leader_election.cc:403] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:58.472987 20875 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:22:58.398721 24629 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.480563 20881 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 12:22:58.484359 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 106728us
I20260709 12:22:58.483665 20731 rpcz_store.cc:276] 0709 12:22:56.966461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.417509 (+451048us) service_pool.cc:224] Handling call
0709 12:22:57.417755 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.417964 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:57.418241 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.392325 (+974084us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.392374 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:58.392423 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.392489 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.392606 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6222,"prepare.queue_time_us":152,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":279734,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":13207,"spinlock_wait_cycles":1792,"thread_start_us":973545,"threads_started":2,"wal-append.queue_time_us":761293}
I20260709 12:22:58.489037 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.493717 21427 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:58.402076 24864 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.563140 20884 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:22:58.494859 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2006) took 1331 ms. Trace:
I20260709 12:22:58.564417 20726 rpcz_store.cc:276] 0709 12:22:57.163601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.247632 (+ 84031us) service_pool.cc:224] Handling call
0709 12:22:58.494614 (+1246982us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001d8654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:58.494768 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347200384}
I20260709 12:22:58.565408 20878 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:58.492461 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:58.447975 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:58.493520 24844 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.297s	user 0.000s	sys 0.021s
I20260709 12:22:58.431103 24851 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.468732 24799 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:58.511241 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.334388 20877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.002s	sys 0.005s
W20260709 12:22:58.573647 20877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.002s	sys 0.006s
I20260709 12:22:58.574421 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1961) took 1230 ms. Trace:
I20260709 12:22:58.427047 24833 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.465893 24827 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:22:58.522770 24803 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.004s
I20260709 12:22:58.469157 24717 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:22:58.472239 24709 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.568173 20728 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:58.571079 20726 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:58.577055 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.577374 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:58.579455 20883 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:58.632176 20735 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:58.632278 24492 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:58.632846 24492 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.633219 24492 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:58.633879 20883 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:22:58.580054 24629 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.634644 24867 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:22:58.540395 24221 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.636722 24842 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.639643 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:58.640272 21430 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:58.581501 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89157us
I20260709 12:22:58.488005 24805 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.583513 20731 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:58.590654 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:58.646349 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57154.8us
I20260709 12:22:58.647038 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:58.500358 24532 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.572463 24743 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.602722 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.570225 24752 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:22:58.597860 24813 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:58.638814 24221 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.644119 21435 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:58.537523 24575 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:58.721352 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59244us
I20260709 12:22:58.647181 20801 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:58.648272 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.458194 24835 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.580611 24846 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:58.562558 24723 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.487380 24834 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.592836 24827 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:58.719935 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 142655us
I20260709 12:22:58.768676 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:58.770334 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:58.729229 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.729755 23925 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:58.772321 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:58.774108 23925 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:58.581004 24799 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:58.767153 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.592998 24851 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 12:22:58.720278 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:58.600445 24803 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:58.771257 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.520s (SENT) [suppressed 23 similar messages]
I20260709 12:22:58.788195 21426 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:58.789915 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67299.3us
I20260709 12:22:58.715240 24629 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:58.719669 24849 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:58.715704 24813 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.582576 24802 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:58.771912 24630 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.000s
I20260709 12:22:58.657310 24867 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
W20260709 12:22:58.808810 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 160598us
W20260709 12:22:58.771409 24865 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.419s	user 0.001s	sys 0.000s
I20260709 12:22:58.756251 20878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54476
I20260709 12:22:58.821470 20878 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:58.772459 24855 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:58.714865 24743 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.817860 21433 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:22:58.756028 24829 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:58.836191 21427 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:58.756572 24841 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:22:58.818877 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.819687 21435 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:58.816833 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:22:58.821940 20878 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:58.729075 24557 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:22:58.809580 24813 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:58.802389 24629 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.843977 21431 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:58.808043 24816 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:58.847844 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.848479 21435 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:22:58.849637 21430 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:58.818513 24717 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:58.808617 24646 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:22:58.798103 24861 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:58.856572 24717 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.859334 24861 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:58.859660 21430 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:58.859833 24861 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:58.788141 24221 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:22:58.845631 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.813118 24877 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:58.818912 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 165965us
I20260709 12:22:58.842087 21433 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:58.823313 24864 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:22:58.846846 24629 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:58.866019 24877 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.868254 24877 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.869611 24575 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:58.871909 24875 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:22:58.851594 24810 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.875599 24882 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:58.876046 24803 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:22:58.876880 24871 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:22:58.877118 24834 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.878239 24532 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.879060 24799 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:58.879572 20727 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:22:58.880014 20727 pending_rounds.cc:85] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 5
I20260709 12:22:58.879591 24835 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.876080 24723 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.882532 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.884218 24874 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:58.886161 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.887142 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.888291 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.850623 24816 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:58.874940 24624 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.892467 24816 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:58.864290 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.820544 24630 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.863072 21430 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:22:58.891978 24433 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:58.903332 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.880293 20727 pending_rounds.cc:107] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:22:58.904012 21433 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
W20260709 12:22:58.902899 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57385.5us
I20260709 12:22:58.904330 20727 raft_consensus.cc:2889] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:58.894752 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:22:58.905032 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.906054 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:58.906642 24624 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.908046 20880 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:22:58.907688 24492 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.908610 24557 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:58.908921 24814 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:58.903116 24433 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:22:58.909965 24492 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.910207 24433 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.911049 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:58.911290 24433 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:58.912186 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:58.912451 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:58.912767 20880 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:22:58.913633 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:58.914134 20880 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:58.916678 24630 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:58.917644 24709 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:58.926884 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:22:58.928411 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.928265 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:58.931967 24629 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.933478 20880 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:22:58.941875 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 665) took 1603 ms. Trace:
I20260709 12:22:58.950419 20882 rpcz_store.cc:276] 0709 12:22:57.338140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.584986 (+246846us) service_pool.cc:224] Handling call
0709 12:22:57.641412 (+ 56426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:57.778943 (+137531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:57.778986 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:57.800522 (+ 21536us) log.cc:844] Serialized 26 byte log entry
0709 12:22:58.163808 (+363286us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:58.163860 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.163924 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.941662 (+777738us) raft_consensus.cc:1685] finished
0709 12:22:58.941681 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.941813 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19046,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":500115,"threads_started":2,"wal-append.queue_time_us":531712}
I20260709 12:22:58.950412 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:58.945825 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:58.947826 20806 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:58.957667 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:58.919335 24717 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:58.959836 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:58.960486 20880 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:58.892999 24810 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:58.965659 20807 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:22:58.896895 24723 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:22:58.894495 24659 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:22:58.944175 24532 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:58.974088 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:58.975514 24884 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:58.976895 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:58.978019 20805 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:58.978477 24857 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.628s	user 0.000s	sys 0.005s
I20260709 12:22:58.978617 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:58.981069 24646 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:58.981195 24709 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:58.956319 24841 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.966941 24834 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:22:58.983711 24878 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:22:58.987686 20880 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:58.946378 24492 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:22:58.958950 24839 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.003s
I20260709 12:22:58.990849 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.028891 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:22:59.028937 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:58.997512 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:58.951310 24805 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.030375 20808 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:58.991698 20878 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:58.981833 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:58.999150 20802 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:59.001348 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:58.982517 24815 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:58.999116 24799 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:59.001271 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:59.054615 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:58.984639 24717 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:22:59.034232 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 145995us
I20260709 12:22:59.033811 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:58.999315 24646 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.001111 24827 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.043310 24881 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.031401 24492 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.031423 24813 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.032749 20728 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:22:59.042929 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.049077 20808 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:59.052999 20730 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:59.078465 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.062274 24879 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.062641 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:59.070920 20726 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:59.084182 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:59.073017 20878 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.079113 20735 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:59.086398 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.087693 20726 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:59.089051 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:59.089321 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 318083us
I20260709 12:22:59.074774 24646 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:59.023278 24841 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.096436 20876 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:59.093492 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:59.100760 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:59.098688 20807 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:59.032087 24839 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.103778 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.108048 20726 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.090274 24867 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.111507 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.088812 24835 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:59.114265 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.077790 24659 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:22:59.100963 24796 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:59.072559 24799 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.119585 20804 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:22:59.066737 24815 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.124724 20876 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:22:59.094238 24423 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:22:59.126946 20733 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:59.105307 24689 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:59.094815 24831 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:22:59.095461 24758 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:22:59.082628 24890 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:22:59.110997 24557 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:59.169747 24557 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.159576 20726 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:59.171465 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.171895 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.172717 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.173023 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.173611 20726 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:22:59.173951 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:59.174471 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:22:59.180047 24887 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:22:59.118564 24809 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.125265 24796 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:59.175101 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:22:59.189584 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.131983 24689 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:59.185865 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:22:59.081254 24881 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.123268 24835 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.221092 24796 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:59.252594 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:22:59.224690 24689 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:59.253552 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.254863 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.256459 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.254573 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 565) took 1259 ms. Trace:
I20260709 12:22:59.292797 20734 rpcz_store.cc:276] 0709 12:22:57.994933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.195697 (+200764us) service_pool.cc:224] Handling call
0709 12:22:58.209762 (+ 14065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.791911 (+582149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:58.791958 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.792016 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:58.792069 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.254266 (+462197us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54003ad9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:59.254332 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.254494 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1338102656}
I20260709 12:22:59.246379 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:22:59.228188 24885 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:22:59.298213 20726 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:59.322101 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63259.6us
I20260709 12:22:59.322793 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.322746 23925 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:59.228660 24752 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:59.236196 24694 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:59.208726 24758 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:22:59.243757 24829 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.364243 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.255457 24872 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:59.344966 24634 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.233572 24809 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.371841 20734 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:59.356323 24694 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:22:59.345238 24805 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:59.345685 24855 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.381309 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:59.345602 24752 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:59.362874 24717 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.392795 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.393669 20732 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:59.400540 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.401470 20734 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.380270 24694 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.383586 24831 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.358080 24864 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:22:59.410670 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.389195 24752 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:59.371462 24846 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:22:59.383365 24805 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:59.410526 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 309837us
W20260709 12:22:59.419028 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 161532us
W20260709 12:22:59.408566 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 306567us
I20260709 12:22:59.390954 24827 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:59.397612 24813 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.438779 23925 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:59.439594 23925 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.429124 20726 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:59.430867 20622 maintenance_manager.cc:419] P 9f77a9352772433bb5809bf72fd4f0ee: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033392
I20260709 12:22:59.455963 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:59.456887 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.465585 20728 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:59.465952 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.470494 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.425344 24805 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:59.458330 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:58.574702 20877 rpcz_store.cc:276] 0709 12:22:57.344510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.598227 (+253717us) service_pool.cc:224] Handling call
0709 12:22:57.648853 (+ 50626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.649067 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:22:57.748585 (+ 99518us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.574100 (+825515us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.574151 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:58.574218 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.574277 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.574375 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12549,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":104,"thread_start_us":922969,"threads_started":2,"wal-append.queue_time_us":668534}
W20260709 12:22:59.478317 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54268.8us
I20260709 12:22:59.435335 24839 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:59.397140 24717 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.492519 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:59.464159 24861 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.604s	user 0.003s	sys 0.009s
I20260709 12:22:59.496670 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.416958 24831 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.460083 24834 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:59.466287 24827 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:59.466445 24841 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:59.515303 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 92399.2us
I20260709 12:22:59.460073 24723 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:22:59.486872 24813 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:59.496398 24659 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:22:59.533447 24433 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.622s	user 0.006s	sys 0.005s
I20260709 12:22:59.506191 24890 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:59.509666 24848 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:22:59.512831 24827 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:59.513291 24799 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:22:59.534217 24889 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
I20260709 12:22:59.515503 20808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "986267de598f431fb25840d9d741436d"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:22:59.585907 20808 raft_consensus.cc:606] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:59.586277 20808 raft_consensus.cc:614] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:59.543810 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:22:59.544572 20735 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:22:59.543808 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:22:59.534488 24894 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 12:22:59.550966 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80576.5us
I20260709 12:22:59.506527 24861 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.510236 24902 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:22:59.534628 24834 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.566946 24831 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:59.573494 20802 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:59.574245 24805 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.578151 20806 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:59.582015 24907 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:59.582273 24905 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:59.582034 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84332.4us
I20260709 12:22:59.600243 20806 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:59.543817 24815 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:59.588416 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86213us
I20260709 12:22:59.601326 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.588753 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.554466 24835 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.494774 24850 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:59.602958 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2265) took 2315 ms (client timeout 2811 ms). Trace:
I20260709 12:22:59.606864 24853 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:22:59.607353 24895 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:22:59.612718 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:22:59.625989 24851 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:59.626561 24851 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:22:59.613947 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70225.8us
I20260709 12:22:59.628614 24851 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:59.616518 20806 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:59.592442 24659 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.619948 20802 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:22:59.630911 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:22:59.617714 20801 rpcz_store.cc:269] 0709 12:22:57.281146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.292361 (+ 11215us) service_pool.cc:224] Handling call
0709 12:22:59.596608 (+2304247us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003a1454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:59.596733 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2804091264}
I20260709 12:22:59.631593 21430 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:59.611939 24847 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.651413 24902 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:59.619134 24809 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:59.655443 24861 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:59.611867 24758 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:22:59.620262 24890 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:59.634438 24913 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:59.681562 24913 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:22:59.682088 24913 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:59.688686 24646 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.583s	user 0.005s	sys 0.005s
I20260709 12:22:59.689330 24646 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.690923 20805 raft_consensus.cc:2368] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:22:59.654177 20735 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:59.664381 21430 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:59.664834 24689 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:59.674746 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46716.8us
I20260709 12:22:59.633144 24557 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.633339 24877 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.635807 24723 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:59.701822 24723 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:59.634770 24831 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:59.658963 20729 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:59.594204 24848 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.704668 20734 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:59.694972 20732 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:22:59.707576 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.709064 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.709617 24492 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:59.700469 24689 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.709678 24833 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:59.705286 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.709632 20729 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:22:59.716214 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:22:59.716732 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:59.717005 24848 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.711745 24758 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:22:59.715483 24905 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:22:59.718366 20877 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:59.717267 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:22:59.720185 20875 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:59.721482 20732 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:59.717587 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:59.722484 20732 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:22:59.723743 20729 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:59.721591 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.725242 20735 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:22:59.671496 24799 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:22:59.730258 20735 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:59.731925 20735 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:59.704281 24831 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:59.679728 24690 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:22:59.727902 24758 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:59.707612 24911 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:59.691111 24913 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.724406 20732 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.745949 20732 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:59.746626 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1971) took 1564 ms (client timeout 1519 ms). Trace:
W20260709 12:22:59.747105 20876 rpcz_store.cc:269] 0709 12:22:58.181711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.181878 (+   167us) service_pool.cc:224] Handling call
0709 12:22:59.746378 (+1564500us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b54001ff254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:59.746513 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061364992}
I20260709 12:22:59.754365 20732 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:22:59.755105 20803 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:22:59.755499 20804 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:22:59.741124 24689 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:22:59.758363 24689 raft_consensus.cc:2764] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:22:59.758468 24796 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.756829 24655 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:22:59.759595 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.760666 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.760934 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:22:59.761444 20803 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:22:59.761201 24827 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.761982 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1025) took 1304 ms. Trace:
I20260709 12:22:59.762740 20803 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:59.762847 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:59.763602 20804 raft_consensus.cc:2368] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:22:59.762336 21428 rpcz_store.cc:276] 0709 12:22:58.457788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.471223 (+ 13435us) service_pool.cc:224] Handling call
0709 12:22:58.471591 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:58.471948 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:59.422526 (+950578us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b180041f620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:22:59.463669 (+ 41143us) log.cc:844] Serialized 14 byte log entry
0709 12:22:59.624549 (+160880us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:59.625268 (+   719us) log.cc:844] Serialized 26 byte log entry
0709 12:22:59.625527 (+   259us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:59.625585 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.625696 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:59.761709 (+136013us) raft_consensus.cc:1685] finished
0709 12:22:59.761811 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.761931 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23573,"prepare.queue_time_us":484769,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":192,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":10750,"spinlock_wait_cycles":12353920,"thread_start_us":79251,"threads_started":2,"wal-append.queue_time_us":124038}
W20260709 12:22:59.764809 24915 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:22:59.764866 20801 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:22:59.715427 24694 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:22:59.768392 24694 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:22:59.769179 24802 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:59.774623 24816 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.852s	user 0.007s	sys 0.001s
I20260709 12:22:59.775745 20881 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:59.775745 24834 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.781437 24890 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:59.782677 24890 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:59.742328 24911 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:22:59.783025 24890 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:59.783736 24831 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.783984 24911 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.784789 24874 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:59.785624 24874 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:59.762267 24827 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:59.785983 24874 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:22:59.787227 24874 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.784440 24890 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.790210 24890 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:59.702788 24723 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:22:59.791558 20733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2015) took 1591 ms (client timeout 1519 ms). Trace:
W20260709 12:22:59.791936 20733 rpcz_store.cc:269] 0709 12:22:58.200189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.216317 (+ 16128us) service_pool.cc:224] Handling call
0709 12:22:59.791363 (+1575046us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b54002986d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:59.791493 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1728885120}
I20260709 12:22:59.785043 24831 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.796486 20875 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:22:59.800386 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:59.800738 21430 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:22:59.802840 21430 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
W20260709 12:22:59.803088 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.804570 20733 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:22:59.806015 20735 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:22:59.806761 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.807756 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:59.808784 21430 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:22:59.796718 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:59.811524 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:22:59.813743 20729 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:22:59.818222 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 101565us
I20260709 12:22:59.807073 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.819075 20735 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:22:59.819063 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:22:59.821864 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:22:59.797618 24874 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:59.823276 24832 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:59.823764 24832 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:22:59.824158 24832 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:59.828419 20728 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 12:22:59.830319 20730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 569) took 1834 ms (client timeout 2011 ms). Trace:
I20260709 12:22:59.831678 20726 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:22:59.785259 24911 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:59.830744 20730 rpcz_store.cc:269] 0709 12:22:57.995913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.062029 (+ 66116us) service_pool.cc:224] Handling call
0709 12:22:59.830128 (+1768099us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400226fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:22:59.830250 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2029921024}
I20260709 12:22:59.837293 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:22:59.820005 24867 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.000s
I20260709 12:22:59.841009 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:22:59.830154 24492 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:59.844309 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2156) took 1184 ms. Trace:
I20260709 12:22:59.844679 20882 rpcz_store.cc:276] 0709 12:22:58.659774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.011654 (+351880us) service_pool.cc:224] Handling call
0709 12:22:59.036019 (+ 24365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.162855 (+126836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:59.162890 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:59.162964 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:59.163006 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.844112 (+681106us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54000965d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:22:59.844155 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.844255 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291069312}
W20260709 12:22:59.846212 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.827955 24655 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:22:59.848302 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:59.851507 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.832512 24905 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:59.832645 24805 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.886512 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:59.889539 24802 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.887605 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:59.775133 24816 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.848529 24867 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:59.905905 24796 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:22:59.885903 24813 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:59.910369 24816 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:22:59.893059 24803 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:59.911306 20733 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:59.840757 24809 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 12:22:59.896342 24900 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.434s	user 0.004s	sys 0.000s
I20260709 12:22:59.915320 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:22:59.890096 24834 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.918902 20878 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:59.894915 24802 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:22:59.922220 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:22:59.884377 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 164893us
W20260709 12:22:59.937175 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 91306.8us
I20260709 12:22:59.937125 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.938005 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.922374 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:59.940220 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:22:59.940382 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:22:59.946139 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:59.948381 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:59.948450 20879 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:22:59.954617 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:59.920971 24796 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:00.012349 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66328us
I20260709 12:22:59.936704 24805 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:00.010458 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51763.4us
I20260709 12:23:00.010730 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.011520 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:22:59.985030 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.018817 20877 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:00.019167 20880 raft_consensus.cc:2410] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:59.957829 24659 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.919620 24803 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:22:59.953884 24752 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 12:23:00.022086 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:00.009871 24900 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:22:59.939582 24813 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.952840 24850 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.957696 24723 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
W20260709 12:23:00.022260 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84331.8us
I20260709 12:23:00.024614 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:00.013780 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:22:59.985193 24557 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:00.029800 24867 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:00.015743 24796 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.920715 24834 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.015795 24864 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:59.955410 24853 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 12:23:00.122596 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:22:59.956189 24863 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:00.054551 20877 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:59.936028 24816 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:00.028159 24690 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:00.009248 24847 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:23:00.025490 24861 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:23:00.046644 24839 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:23:00.042204 24752 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.045956 24803 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.048763 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:00.105279 24860 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:00.087397 20849 leader_election.cc:400] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:23:00.206103 20849 leader_election.cc:403] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:00.056396 24913 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:00.206658 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.071810 24557 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:00.208945 24557 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:00.209661 24925 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:23:00.112394 24689 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:00.211472 24689 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:00.211865 24924 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.159s	user 0.002s	sys 0.000s
W20260709 12:23:00.061839 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 228191us
W20260709 12:23:00.158483 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.057276 24930 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:00.110447 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:00.114907 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 576) took 1010 ms. Trace:
I20260709 12:23:00.254123 20726 rpcz_store.cc:276] 0709 12:22:59.104929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.445202 (+340273us) service_pool.cc:224] Handling call
0709 12:22:59.831370 (+386168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.832038 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.114664 (+282626us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400226fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:00.114822 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887130880}
I20260709 12:23:00.120661 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2016) took 1907 ms. Trace:
W20260709 12:23:00.072175 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58488us
I20260709 12:23:00.255652 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:00.085692 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1990) took 1119 ms. Trace:
I20260709 12:23:00.106488 24918 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:00.256446 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:00.255501 20727 rpcz_store.cc:276] 0709 12:22:58.212966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.414701 (+201735us) service_pool.cc:224] Handling call
0709 12:22:58.414985 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.904889 (+489904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.120349 (+1215460us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b5400298bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:00.120584 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067584896}
I20260709 12:23:00.259734 24918 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:00.260334 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:00.260689 24918 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.261385 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:00.262413 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:00.263441 24860 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:00.263661 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:00.263937 24860 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:00.264931 24860 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.206177 24863 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:00.268226 24863 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.268798 24433 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:00.269641 24433 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:00.270093 24433 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:00.187104 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:00.273824 20729 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:00.120133 24787 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.040122 24659 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.114295 20880 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:23:00.063772 24796 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:00.218600 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.064294 24905 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:00.116694 24802 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.173242 24655 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:00.285923 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65860.9us
W20260709 12:23:00.259048 24904 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.659s	user 0.000s	sys 0.003s
W20260709 12:23:00.248806 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 162452us
I20260709 12:23:00.078728 24867 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.256695 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:00.256973 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:00.110702 24846 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 12:23:00.279347 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:00.280989 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.256875 20884 rpcz_store.cc:276] 0709 12:22:58.966631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.054806 (+ 88175us) service_pool.cc:224] Handling call
0709 12:22:59.062703 (+  7897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:59.063207 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:59.107382 (+ 44175us) log.cc:844] Serialized 14 byte log entry
0709 12:22:59.356199 (+248817us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:59.372254 (+ 16055us) log.cc:844] Serialized 26 byte log entry
0709 12:22:59.372399 (+   145us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:59.372433 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.372499 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:59.581738 (+209239us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b18001b20e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e22b7358f 00007f9e1bbb2571 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd
0709 12:22:59.838391 (+256653us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:00.085499 (+247108us) raft_consensus.cc:1685] finished
0709 12:23:00.085518 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:00.085631 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9032,"prepare.queue_time_us":34580,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":104291,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":15817,"spinlock_wait_cycles":21945856,"thread_start_us":245755,"threads_started":2,"wal-append.queue_time_us":269935}
W20260709 12:23:00.282850 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 172505us
I20260709 12:23:00.222358 24928 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.285485 20732 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:23:00.249107 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:00.321327 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:00.330454 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.333187 21435 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.333693 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:00.321930 21433 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:00.321282 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58070.4us
I20260709 12:23:00.207036 24816 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.220305 24690 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.094254 24929 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.186s
W20260709 12:23:00.340713 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.257293 24926 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:23:00.281661 24492 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:00.345592 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:00.343045 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 298515us
I20260709 12:23:00.270960 24433 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.354511 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:00.361227 20807 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:00.359920 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:00.321537 24802 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:00.343976 24890 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.005s
I20260709 12:23:00.380107 20727 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:00.363070 21431 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:00.334158 24853 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:00.377333 24867 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.335047 24920 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.292973 24861 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:23:00.382182 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:00.400797 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106226us
I20260709 12:23:00.280700 24787 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.335436 24913 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:00.395076 21434 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:23:00.409180 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2309) took 1444 ms. Trace:
W20260709 12:23:00.397625 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106907us
I20260709 12:23:00.382331 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:00.422163 20732 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:00.335304 24903 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:23:00.360898 24492 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:00.418398 20731 rpcz_store.cc:276] 0709 12:22:58.965133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.965704 (+   571us) service_pool.cc:224] Handling call
0709 12:22:58.966022 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:59.003084 (+ 37062us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:59.003118 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:59.003635 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:22:59.412846 (+409211us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:59.412888 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.108239 (+695351us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b54001d8154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:00.108290 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:00.214411 (+106121us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:00.408969 (+194558us) raft_consensus.cc:1685] finished
0709 12:23:00.408990 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:00.409121 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":272,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":18190,"spinlock_wait_cycles":111220480,"thread_start_us":340246,"threads_started":1,"wal-append.queue_time_us":521616}
I20260709 12:23:00.428392 21431 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:23:00.342979 24936 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 12:23:00.433794 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:00.441926 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:00.452147 24833 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:00.453153 20807 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:00.401196 24853 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:00.457672 21429 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:00.400868 24861 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:00.407902 24905 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.347265 24928 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.410799 24867 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:23:00.472786 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 112910us
I20260709 12:23:00.452706 24833 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:00.467705 24831 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.004s
I20260709 12:23:00.476017 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:00.404763 24920 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.431860 24913 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.462556 24890 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:23:00.475867 24841 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:00.519598 24874 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.001s
I20260709 12:23:00.473300 24802 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:00.485494 24833 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:00.484318 24834 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.519903 24947 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:00.433081 24655 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:00.531565 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129329us
I20260709 12:23:00.540129 20883 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:00.517767 24956 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:00.550896 24949 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:23:00.551448 24834 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:00.551981 24920 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:00.517204 24752 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:00.620963 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:00.628232 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:00.630028 20810 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:00.533718 24947 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:00.483815 24492 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.576012 24659 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:00.624745 24874 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.541887 24940 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:00.647854 24816 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.656852 24816 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.657476 24816 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:00.658618 24805 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.000s
I20260709 12:23:00.659086 24805 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:00.659947 24805 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.658912 24796 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.661573 24743 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:00.649219 24947 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.662398 24947 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:00.665716 20881 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:00.646342 24913 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.635675 24939 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:00.647254 24935 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:00.678004 21435 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:00.529659 24853 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.630561 24956 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.520728 24861 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.672430 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:00.648900 24492 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.677191 20729 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
W20260709 12:23:00.649600 24923 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
W20260709 12:23:00.691282 24949 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:23:00.655409 24933 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:00.632550 24936 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 12:23:00.707247 24954 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:23:00.707336 24861 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:00.707648 24956 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.709527 24903 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:00.711385 24787 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:00.692051 24935 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.729996 24935 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:00.713282 24816 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.713681 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.732183 20880 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:00.732793 24646 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:00.735387 24948 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:23:00.714810 24689 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.717262 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.718357 24939 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:00.719319 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.720705 21428 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:00.723958 21433 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:00.695096 24796 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:00.724363 20729 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:23:00.674880 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 312516us
I20260709 12:23:00.650141 24874 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.723255 24743 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:00.739605 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.744695 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.752628 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:00.753683 24913 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:00.763350 24752 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:00.764667 24752 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:23:00.765730 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 267481us
I20260709 12:23:00.745880 20880 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:00.746747 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:00.750998 20879 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:00.768785 20883 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:00.650429 24940 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.785203 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.786517 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2060) took 1797 ms. Trace:
I20260709 12:23:00.787787 20809 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:00.779042 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:00.779508 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:00.781111 20883 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:00.781002 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:00.782215 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:00.784524 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.784721 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.786966 20805 rpcz_store.cc:276] 0709 12:22:58.989218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.000973 (+ 11755us) service_pool.cc:224] Handling call
0709 12:23:00.786312 (+1785339us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540027bd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:00.786456 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461982592}
I20260709 12:23:00.727131 24903 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:00.811895 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:00.696439 24802 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.742049 24956 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:00.813750 20879 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:00.794940 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:00.833303 21428 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:23:00.814870 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:00.833907 20732 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:00.826088 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:00.834712 20808 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:00.827630 20700 leader_election.cc:400] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:23:00.835397 20700 leader_election.cc:403] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:23:00.827593 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:00.830081 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.832064 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:00.838315 20807 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:00.833104 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:00.816188 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:00.840173 20726 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:00.839602 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:00.844802 20731 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:00.789284 24933 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:00.796608 24954 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:23:00.794777 24962 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.855738 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.812574 24920 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:00.869743 20731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:00.870347 20731 raft_consensus.cc:606] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:00.869118 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:00.870618 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:00.835161 24863 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:00.869755 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2282) took 1115 ms. Trace:
I20260709 12:23:00.871376 21428 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:00.871991 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:00.881815 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 110037us
I20260709 12:23:00.870774 20731 raft_consensus.cc:614] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:00.850358 24873 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:23:00.885648 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.794629 24743 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.785353 24930 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:23:00.876550 20806 rpcz_store.cc:276] 0709 12:22:59.754631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.757332 (+  2701us) service_pool.cc:224] Handling call
0709 12:22:59.757674 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:59.758039 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:59.758069 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:59.758664 (+   595us) log.cc:844] Serialized 52 byte log entry
0709 12:22:59.988655 (+229991us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:59.988712 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.988805 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:00.869523 (+880718us) raft_consensus.cc:1685] finished
0709 12:23:00.869546 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:00.869699 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1314,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":123,"thread_start_us":229534,"threads_started":1,"wal-append.queue_time_us":425026}
I20260709 12:23:00.904904 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:00.843467 24841 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:00.916326 20801 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:00.916036 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.920854 20877 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:00.869974 24920 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:00.876312 24796 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:00.872834 24848 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:00.927232 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:00.974668 20806 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:00.952123 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:00.933076 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.858507 24816 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:23:00.960374 24974 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.962245 20726 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
W20260709 12:23:00.964151 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71440.3us
I20260709 12:23:00.858713 24835 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:23:00.970268 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:00.948374 24827 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.162s	user 0.014s	sys 0.000s
I20260709 12:23:00.973836 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:00.974747 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2035) took 1169 ms. Trace:
I20260709 12:23:00.959484 24433 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.911919 24629 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:00.923178 24935 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:00.869316 24809 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:23:00.866583 24956 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.851689 24928 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:00.978420 24832 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.154s	user 0.009s	sys 0.007s
I20260709 12:23:00.986501 20884 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:00.994259 20879 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:23:00.993973 24841 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:00.997035 24743 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:00.997186 24979 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.974771 24971 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:00.870813 24933 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.032249 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:00.986673 20735 rpcz_store.cc:276] 0709 12:22:59.805612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.833951 (+ 28339us) service_pool.cc:224] Handling call
0709 12:22:59.834367 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:59.842966 (+  8599us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:00.156161 (+313195us) log.cc:844] Serialized 14 byte log entry
0709 12:23:00.353662 (+197501us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:00.379853 (+ 26191us) log.cc:844] Serialized 26 byte log entry
0709 12:23:00.380050 (+   197us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:00.380101 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.380199 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:00.974526 (+594327us) raft_consensus.cc:1685] finished
0709 12:23:00.974546 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:00.974683 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12699,"prepare.queue_time_us":449587,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":11991,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":35061,"spinlock_wait_cycles":44909440,"thread_start_us":401309,"threads_started":2,"wal-append.queue_time_us":301189}
I20260709 12:23:01.033914 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:00.874294 24962 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:01.003121 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:01.047955 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.983559 24796 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:01.051410 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:00.986459 24972 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:01.009413 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:00.993081 24831 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:00.974310 24873 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:23:00.982900 24848 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:00.912175 24903 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.001190 24982 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:01.006245 24935 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.025719 24913 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:01.043440 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:00.997776 24433 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.050371 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:01.057752 24831 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.052950 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:01.053958 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44640.7us
I20260709 12:23:01.016371 24690 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:01.061308 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.061679 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:01.058969 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:01.115671 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:01.115912 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57050.5us
I20260709 12:23:01.045984 24874 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.026700 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.073412 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:00.984190 24974 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.025385 24872 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:01.060644 24629 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:01.065310 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:01.133188 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89987.9us
W20260709 12:23:01.136169 20734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2317) took 1486 ms (client timeout 1947 ms). Trace:
I20260709 12:23:01.074661 24935 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.140403 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:01.020358 24832 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.194675 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.198436 20806 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:01.102059 24903 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:01.167021 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:01.132728 24905 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:01.154429 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.212052 20807 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:01.200213 24758 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:23:01.227226 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.137748 24872 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:01.195674 20734 rpcz_store.cc:269] 0709 12:22:59.649625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.707308 (+ 57683us) service_pool.cc:224] Handling call
0709 12:23:01.026504 (+1319196us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:23:01.135986 (+109482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8454016}
I20260709 12:23:01.220541 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.141902 24629 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.193845 24873 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:01.046983 24971 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:01.371318 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.232589 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:01.111815 24940 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.121171 24831 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.248689 21428 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:01.237169 21431 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:01.357285 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:01.358076 20850 leader_election.cc:336] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.520s (SENT)
I20260709 12:23:01.031198 24979 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.226789 24758 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:01.373087 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:01.074399 24982 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.380704 20807 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:01.391386 24978 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.404s	user 0.001s	sys 0.001s
I20260709 12:23:01.395736 24940 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:01.404963 24940 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.405494 24940 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:01.364212 24629 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:01.419853 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.371619 24827 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.428562 20807 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:01.219161 24905 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:01.430835 24975 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.457s	user 0.000s	sys 0.010s
W20260709 12:23:01.557152 24998 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
W20260709 12:23:01.557886 24985 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 12:23:01.558254 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.558944 20731 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
W20260709 12:23:01.559763 24958 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 12:23:01.560129 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:01.449803 24981 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 12:23:01.562141 24805 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.902s	user 0.012s	sys 0.001s
I20260709 12:23:01.562674 24805 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:01.566633 24861 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.840s	user 0.008s	sys 0.000s
W20260709 12:23:01.456223 20808 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.606s	user 0.007s	sys 0.006s
W20260709 12:23:01.467350 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 309511us
W20260709 12:23:01.475970 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 356742us
I20260709 12:23:01.465543 24799 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.366605 24796 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.507270 24743 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.476289 24758 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.356609 24872 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.455299 24827 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.476097 24982 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:01.531438 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.537909 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.525740 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.482486 24956 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.544719 24968 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:01.527041 24984 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.456s	user 0.001s	sys 0.001s
I20260709 12:23:01.474910 24749 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:01.448928 21428 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:01.603552 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.604158 21428 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 7: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:01.552835 24905 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:01.554607 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:01.554955 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.610911 20726 raft_consensus.cc:606] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:01.611508 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56650.7us
I20260709 12:23:01.612217 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.556542 24995 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.567898 24832 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:01.569073 24974 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.609392 24433 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.454553 20803 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:01.623687 24971 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:01.583741 24928 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:23:01.624750 20803 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:01.625955 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 1994) took 1678 ms. Trace:
I20260709 12:23:01.626518 20875 rpcz_store.cc:276] 0709 12:22:59.947094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.947224 (+   130us) service_pool.cc:224] Handling call
0709 12:23:00.718920 (+771696us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5800003698. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:23:00.719135 (+   215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:00.719554 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:00.719589 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:00.720101 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:23:00.919042 (+198941us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:00.919081 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.919145 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:01.625527 (+706382us) raft_consensus.cc:1685] finished
0709 12:23:01.625559 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.625871 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2923,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":4092,"spinlock_wait_cycles":39500032,"thread_start_us":148866,"threads_started":1,"wal-append.queue_time_us":265982}
I20260709 12:23:01.630043 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.575378 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:01.632310 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.634012 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.635962 24994 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.637739 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:01.582161 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2036) took 1776 ms. Trace:
I20260709 12:23:01.636482 24994 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.543058 24962 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.591795 20735 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:23:01.592118 20734 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:01.526769 24940 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.551265 24848 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:01.585932 24833 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.033s	user 0.007s	sys 0.006s
I20260709 12:23:01.605288 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:01.590358 24743 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.608453 24872 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:01.608707 24968 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.588519 24799 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.586632 24967 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.609642 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:01.611548 20726 raft_consensus.cc:614] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:01.629099 24979 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.621219 24935 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.537350 24933 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:01.637938 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62666.4us
I20260709 12:23:01.691639 24935 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.637948 24873 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:01.582698 21432 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.139s	sys 0.228s
I20260709 12:23:01.640970 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:01.698256 21432 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.139s	sys 0.228s
I20260709 12:23:01.640139 24433 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:01.590858 24629 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:01.638305 24951 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.000s	sys 0.008s
I20260709 12:23:01.640995 20728 rpcz_store.cc:276] 0709 12:22:59.805848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.835676 (+ 29828us) service_pool.cc:224] Handling call
0709 12:22:59.844179 (+  8503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:59.844551 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:00.123073 (+278522us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18002877a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:23:00.344300 (+221227us) log.cc:844] Serialized 14 byte log entry
0709 12:23:00.767487 (+423187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:00.778810 (+ 11323us) log.cc:844] Serialized 26 byte log entry
0709 12:23:00.779347 (+   537us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:00.779384 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.779448 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:01.551761 (+772313us) raft_consensus.cc:1685] finished
0709 12:23:01.581893 (+ 30132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.582079 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69082,"prepare.queue_time_us":536324,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":2627,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":19328,"spinlock_wait_cycles":3548928,"thread_start_us":563380,"threads_started":2,"wal-append.queue_time_us":466046}
I20260709 12:23:01.641674 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:01.649721 21428 raft_consensus.cc:2376] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10. Re-sending same reply.
I20260709 12:23:01.604890 24758 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:01.614167 25007 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:01.637574 24861 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:23:01.661221 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67695us
I20260709 12:23:01.714983 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.627911 24995 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.691699 24799 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.582120 20808 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:01.637218 24850 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:01.695525 23925 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.641230 24835 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:01.645833 24962 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.705684 24967 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.713948 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.734190 21432 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:01.716018 21428 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:01.735081 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.694507 24914 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:01.727466 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:01.658841 24833 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:01.723276 24951 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.002s	sys 0.011s
W20260709 12:23:01.724159 24492 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.023s	user 0.006s	sys 0.011s
I20260709 12:23:01.721882 20807 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:01.713209 24982 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:23:01.730192 24861 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:01.734644 24947 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.072s	user 0.009s	sys 0.009s
W20260709 12:23:01.741910 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 100371us
I20260709 12:23:01.712951 24974 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:23:01.742278 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:01.733599 24835 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:01.762184 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52051.4us
I20260709 12:23:01.735370 24850 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:01.745100 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.745463 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.770579 20805 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:01.774335 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.770433 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:01.775372 21428 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:01.744508 24971 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:01.779939 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:01.763682 24973 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.388s	user 0.000s	sys 0.002s
I20260709 12:23:01.730509 24962 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.781944 21432 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:01.788153 24995 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.789541 24629 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.791433 21428 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:01.744278 24831 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:01.793013 24972 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.722s	user 0.006s	sys 0.005s
I20260709 12:23:01.793505 24972 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:01.794111 24831 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:01.798074 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:01.762050 24743 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.771263 24861 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:01.803645 24861 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:01.773890 24835 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.804209 25010 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:01.805920 24835 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.773797 24492 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:01.806525 24492 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.806916 24492 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:01.821137 20735 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 12:23:01.780431 24990 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:23:01.781617 24433 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:23:01.780210 24947 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:01.836376 24947 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:01.836658 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.808051 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.836410 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.838682 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.808686 24717 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.809095 24833 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.842114 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.842860 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:01.843302 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.844163 20729 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:23:01.837531 24947 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.845197 20729 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:23:01.846752 20729 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:23:01.847314 24743 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.847805 24935 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.847968 24743 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.816977 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.777866 24968 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:01.820621 20803 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:01.805826 25010 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:01.821653 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:01.745886 24860 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:01.835371 24997 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:23:01.831100 24995 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:01.839775 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.815009 24816 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:01.815907 24940 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.894901 20735 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:23:01.895896 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.885766 24850 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:01.886085 24933 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:01.888408 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:01.889122 24743 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.898720 24928 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:01.846531 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.899931 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:01.831957 24982 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:01.830181 24971 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:01.746071 24914 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:01.906157 24743 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:01.906873 24994 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:01.906773 24743 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.909061 24743 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.902071 24816 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:01.912220 24816 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.913565 24816 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.894500 24995 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.915699 24995 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:01.880430 25010 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:01.919960 24971 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.920149 25010 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.921584 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2290) took 1563 ms. Trace:
W20260709 12:23:01.893616 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72085.4us
I20260709 12:23:01.897616 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:01.848377 24935 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.923122 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.844172 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:01.903052 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:01.924109 20728 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:01.925328 20727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2042) took 1980 ms (client timeout 2067 ms). Trace:
W20260709 12:23:01.925751 20727 rpcz_store.cc:269] 0709 12:22:59.945334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.260068 (+314734us) service_pool.cc:224] Handling call
0709 12:23:01.925117 (+1665049us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54003ad4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:01.925257 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273920768}
I20260709 12:23:01.922242 20801 rpcz_store.cc:276] 0709 12:23:00.358815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.359023 (+   208us) service_pool.cc:224] Handling call
0709 12:23:00.359398 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.923250 (+563852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.921235 (+997985us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b540019afd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:01.921499 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1231539968}
I20260709 12:23:01.846482 24492 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:01.928073 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:01.921439 25010 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.929270 20801 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:01.897302 24968 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:01.934919 24492 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:01.901384 24832 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:23:01.935807 20731 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:01.936120 24832 consensus_queue.cc:1243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:01.931676 20801 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:01.937989 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:01.939975 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:01.941080 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:01.905911 24982 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:01.941243 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 146071us
I20260709 12:23:01.941665 21428 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:01.950388 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:01.942342 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:01.953857 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56402.3us
I20260709 12:23:01.897766 24860 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:01.957891 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:01.958663 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:01.952692 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:01.943099 24758 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:01.979943 20803 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:01.948859 24867 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:01.951267 24994 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:01.982002 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.949589 24832 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.959985 24743 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:01.988517 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:01.939148 24968 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:23:01.992447 21429 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:01.993621 24898 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.993654 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 877) took 1276 ms. Trace:
I20260709 12:23:01.994333 21429 rpcz_store.cc:276] 0709 12:23:00.717581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.719032 (+  1451us) service_pool.cc:224] Handling call
0709 12:23:01.993484 (+1274452us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:01.993597 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2871553920}
I20260709 12:23:01.996234 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:01.952016 24982 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:01.961030 24972 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.988945 24743 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.001174 20729 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:01.984067 24831 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:01.988339 24832 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.019120 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:02.023293 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.016569 21428 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:01.987075 24994 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:01.992457 24749 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:02.028542 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:02.032788 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 633) took 2943 ms. Trace:
I20260709 12:23:02.033268 20809 rpcz_store.cc:276] 0709 12:22:59.089846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.528808 (+438962us) service_pool.cc:224] Handling call
0709 12:23:00.787581 (+1258773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.788255 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.032532 (+1244277us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540027bd50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:02.032689 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2894018816}
I20260709 12:23:02.026489 20729 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:02.038143 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.042377 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:02.038967 20729 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:02.045486 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:02.044674 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.049017 20729 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:01.949126 25014 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:23:02.050671 20801 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:02.056315 21428 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:02.057081 21428 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:02.022219 24743 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:02.025548 24903 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.815s	user 0.008s	sys 0.004s
I20260709 12:23:02.021996 24831 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.063606 25016 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:01.994372 24898 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.036518 24994 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:02.035851 24749 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.035669 24867 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:02.052774 24872 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:02.073063 21428 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.085086 21428 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:02.070499 24831 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:02.074239 24933 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:02.013679 24850 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.070578 24962 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.079865 25016 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.023203 24972 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.051954 24861 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.085333 24935 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.092398 24832 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.079619 24749 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:02.094849 24920 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:02.098925 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 1995) took 2151 ms (client timeout 1864 ms). Trace:
I20260709 12:23:02.099205 21428 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:23:02.106155 24746 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:02.108887 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2087) took 1067 ms. Trace:
I20260709 12:23:02.089318 24933 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.079842 24867 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:02.115571 21428 pending_rounds.cc:85] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 9
I20260709 12:23:02.134428 21428 pending_rounds.cc:107] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:23:02.134722 21428 raft_consensus.cc:2889] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:02.087587 24872 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:02.129668 20807 rpcz_store.cc:276] 0709 12:23:01.041777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.552540 (+510763us) service_pool.cc:224] Handling call
0709 12:23:01.721624 (+169084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:01.745568 (+ 23944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.108594 (+363026us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54003b6d50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:02.108793 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305228544}
W20260709 12:23:02.109967 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.138513 24933 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:02.114750 20881 rpcz_store.cc:269] 0709 12:22:59.947637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.947751 (+   114us) service_pool.cc:224] Handling call
0709 12:23:02.098667 (+2150916us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:02.098821 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484259072}
I20260709 12:23:02.106215 24922 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:02.080976 24905 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.110644 25016 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:02.133303 20726 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:02.144060 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.154768 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:02.155750 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.169238 20881 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 5.
I20260709 12:23:02.165493 24689 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:02.169164 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:02.129570 24746 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:02.141637 24867 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:02.204751 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.205821 20881 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:02.166375 24809 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:23:02.213747 24934 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:02.143496 24922 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:02.229254 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.184288 24890 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:02.182363 24920 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:02.231269 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:02.226114 24934 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:02.239970 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:02.243742 25014 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.253582 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.254144 20727 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:02.235641 24907 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:02.237797 24866 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.240192 24932 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:02.240831 24809 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:02.242517 24890 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:02.266665 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:02.272868 20875 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:23:02.255605 24902 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:02.259004 24907 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:02.289021 25018 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.289656 25018 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.289186 24846 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.293012 20731 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:23:02.242063 24746 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.273370 24758 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.295403 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.269048 25014 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.296495 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2000) took 1413 ms. Trace:
I20260709 12:23:02.291806 24846 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.298996 20879 rpcz_store.cc:276] 0709 12:23:00.883313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.885258 (+  1945us) service_pool.cc:224] Handling call
0709 12:23:02.293363 (+1408105us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54001fe854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:02.296390 (+  3027us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234815232}
I20260709 12:23:02.244614 24872 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.297683 24905 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:02.294560 25020 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:23:02.280839 24689 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:23:02.286428 24971 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:02.287446 24934 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.309671 24995 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.285441 24881 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.267795 24809 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.304702 24746 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:02.302713 24902 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:02.315706 24995 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.337603 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2001) took 1453 ms. Trace:
I20260709 12:23:02.316754 25014 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.317950 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.308921 24827 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.313370 25016 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.265014 24866 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.304940 24861 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.311363 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:02.333083 24986 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.385s	user 0.004s	sys 0.000s
I20260709 12:23:02.333729 24994 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.321097 24890 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.337296 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1036) took 2065 ms. Trace:
I20260709 12:23:02.316005 24932 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:02.312252 24934 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:02.345633 20879 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 4.
I20260709 12:23:02.305119 24758 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.354070 21426 rpcz_store.cc:276] 0709 12:23:00.271489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.272141 (+   652us) service_pool.cc:224] Handling call
0709 12:23:00.272473 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:02.100811 (+1828338us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b18003cb7a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1ec97bfb 00007f9e1eca264b
0709 12:23:02.100956 (+   145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:02.100986 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:02.101525 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 12:23:02.246827 (+145302us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:02.246882 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.246982 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:02.326689 (+ 79707us) raft_consensus.cc:1685] finished
0709 12:23:02.326713 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:02.337195 (+ 10482us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31691,"raft.queue_time_us":122027,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":6364,"spinlock_wait_cycles":33922944,"thread_start_us":246859,"threads_started":2,"wal-append.queue_time_us":170424}
I20260709 12:23:02.344798 20883 rpcz_store.cc:276] 0709 12:23:00.883738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.886282 (+  2544us) service_pool.cc:224] Handling call
0709 12:23:00.886654 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.948972 (+ 62318us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:01.049915 (+100943us) log.cc:844] Serialized 14 byte log entry
0709 12:23:01.072148 (+ 22233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:01.072207 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:01.072259 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.335840 (+1263581us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b1800415180. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:23:02.337324 (+  1484us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:02.337383 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:02.337537 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":643,"prepare.queue_time_us":65503,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1045,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":28083,"spinlock_wait_cycles":145845888,"thread_start_us":89466,"threads_started":2,"wal-append.queue_time_us":80707}
I20260709 12:23:02.367156 20883 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:23:02.367316 20879 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:02.350626 25016 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.280902 24922 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.368245 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2008) took 1360 ms. Trace:
I20260709 12:23:02.368247 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2180) took 1338 ms. Trace:
I20260709 12:23:02.372232 24846 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.375620 20883 rpcz_store.cc:276] 0709 12:23:01.029572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.366383 (+1336811us) service_pool.cc:224] Handling call
0709 12:23:02.366835 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:02.367878 (+  1043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.368170 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:02.386924 24926 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:02.388875 24972 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.389070 24926 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:02.364248 24994 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.347023 24902 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.365806 25025 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:02.373538 20879 rpcz_store.cc:276] 0709 12:23:01.008229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.359580 (+1351351us) service_pool.cc:224] Handling call
0709 12:23:02.360056 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.367890 (+  7834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.368170 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:02.388530 24867 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.387247 24809 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.363953 24932 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.405858 24932 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:02.410825 24867 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.402181 20879 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:02.358125 24907 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.383076 24933 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.417291 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:02.418346 24933 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:02.424799 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.426023 21426 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:02.393275 24922 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:02.433681 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.434139 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:02.434721 21426 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:02.436836 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.434473 24907 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:02.411908 24926 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.440303 24809 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.449605 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1040) took 1708 ms. Trace:
I20260709 12:23:02.459334 21431 rpcz_store.cc:276] 0709 12:23:00.741448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.742206 (+   758us) service_pool.cc:224] Handling call
0709 12:23:00.742548 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:01.389965 (+647417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.449310 (+1059345us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b5400729a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:02.449521 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1115730304}
I20260709 12:23:02.450717 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2178) took 1628 ms. Trace:
I20260709 12:23:02.461459 20882 rpcz_store.cc:276] 0709 12:23:00.822601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.822895 (+   294us) service_pool.cc:224] Handling call
0709 12:23:00.854275 (+ 31380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:01.400630 (+546355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:01.400678 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:01.400737 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:01.400782 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.450462 (+1049680us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54000a32d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:02.450518 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:02.450655 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255801216}
W20260709 12:23:02.477661 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:02.460908 24557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.176s	user 0.050s	sys 0.062s
W20260709 12:23:02.489413 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:02.490161 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1035) took 2251 ms (client timeout 2900 ms). Trace:
W20260709 12:23:02.487044 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 220452us
I20260709 12:23:02.412220 25025 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:02.490556 21434 rpcz_store.cc:269] 0709 12:23:00.222059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.222560 (+   501us) service_pool.cc:224] Handling call
0709 12:23:02.470879 (+2248319us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54004d9c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:02.473748 (+  2869us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294151424}
I20260709 12:23:02.424423 24749 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.486456 24926 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:02.495718 24557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.215s	user 0.051s	sys 0.062s
I20260709 12:23:02.496222 24867 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:02.468801 24809 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:02.534298 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:02.561887 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.532352 24832 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.559528 24850 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:02.585207 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 871) took 2146 ms. Trace:
I20260709 12:23:02.586237 21430 rpcz_store.cc:276] 0709 12:23:00.435076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.435252 (+   176us) service_pool.cc:224] Handling call
0709 12:23:00.435711 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:00.700993 (+265282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:00.701024 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:00.705816 (+  4792us) log.cc:844] Serialized 52 byte log entry
0709 12:23:00.923822 (+218006us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:00.923896 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.518653 (+1594757us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b540013d154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:02.518720 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:02.560582 (+ 41862us) raft_consensus.cc:1685] finished
0709 12:23:02.560611 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:02.580877 (+ 20266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552,"raft.queue_time_us":11451,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":253,"spinlock_wait_cycles":607730048,"thread_start_us":172028,"threads_started":1,"wal-append.queue_time_us":302862,"wal-append.run_cpu_time_us":2889,"wal-append.run_wall_time_us":1493418}
W20260709 12:23:02.579592 24982 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.002s
I20260709 12:23:02.559696 24867 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.568989 24932 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.579110 25026 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:02.620060 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.618413 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:23:02.598251 24743 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.009s
I20260709 12:23:02.569218 24831 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.623513 20881 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
W20260709 12:23:02.614404 24816 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.000s
W20260709 12:23:02.628248 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.628870 24974 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:02.630573 24994 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:02.631763 24749 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.632330 20883 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:02.633893 24831 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:02.635090 24960 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:23:02.582213 24832 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.613921 24867 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:02.638154 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:02.590044 24850 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:02.640089 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.640394 21430 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:02.641945 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.642050 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.643745 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.599993 24982 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:02.649390 24839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.363s	user 0.010s	sys 0.003s
W20260709 12:23:02.649930 24839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.367s	user 0.011s	sys 0.003s
I20260709 12:23:02.635637 24838 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:02.657660 24838 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.658129 24838 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:02.631181 24809 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.614197 24932 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.662627 24932 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.664909 24832 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.666363 25025 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.668397 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.669134 20877 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:02.646270 24867 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.653863 20879 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:02.654621 21434 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 9. Ops: []
I20260709 12:23:02.673784 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.675781 21433 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:23:02.676679 24983 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.157s	user 0.002s	sys 0.000s
I20260709 12:23:02.678025 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 584) took 2872 ms. Trace:
I20260709 12:23:02.678745 24860 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.678467 20733 rpcz_store.cc:276] 0709 12:22:59.805380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.815555 (+ 10175us) service_pool.cc:224] Handling call
0709 12:22:59.815903 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.912957 (+ 97054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.677786 (+2764829us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b540028d054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:02.677964 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218815744}
I20260709 12:23:02.680634 24832 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:02.678032 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:02.623018 25026 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.626674 24743 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.686808 24994 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.691282 20801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:52492
I20260709 12:23:02.692354 20801 raft_consensus.cc:606] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:02.692739 20801 raft_consensus.cc:614] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:02.689743 24557 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.693925 24557 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.696556 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.697223 20877 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:02.687392 24970 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:02.701131 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.701920 21434 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:02.684980 24842 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:02.689168 25043 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.117s	user 0.001s	sys 0.007s
I20260709 12:23:02.701622 24970 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.707641 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.709015 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:02.683941 24867 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.710846 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:02.717937 21433 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:02.718448 21428 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:23:02.718840 21428 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 10
I20260709 12:23:02.711866 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:02.714133 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:02.719615 21435 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:02.719062 21428 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:23:02.715561 24842 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.698649 24977 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.720357 21428 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:02.722658 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.724694 24863 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:02.725394 24850 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.697885 24994 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.685837 25025 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:23:02.745646 24994 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:02.747097 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73489us
I20260709 12:23:02.731551 24863 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:02.752238 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2160) took 3704 ms. Trace:
I20260709 12:23:02.752633 20878 rpcz_store.cc:276] 0709 12:22:59.048186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.123744 (+ 75558us) service_pool.cc:224] Handling call
0709 12:22:59.124086 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.919328 (+795242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.752011 (+2832683us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:02.752179 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1827348736}
I20260709 12:23:02.753985 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.754554 20878 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:02.759635 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2289) took 2402 ms. Trace:
I20260709 12:23:02.760089 20810 rpcz_store.cc:276] 0709 12:23:00.358103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.363444 (+  5341us) service_pool.cc:224] Handling call
0709 12:23:00.373705 (+ 10261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.630712 (+257007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.759411 (+2128699us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540006d454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:02.759576 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579913088}
I20260709 12:23:02.761232 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:02.761420 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:02.762169 20807 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:02.762428 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:02.762943 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:02.763527 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:02.765166 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:02.751104 21433 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:02.769683 20728 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
W20260709 12:23:02.798220 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.780208 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:02.770208 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.778264 21435 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:02.794463 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.818737 24810 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.780639 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:02.767299 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:02.821028 20878 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:02.821231 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:02.815210 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.821868 20728 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:02.826993 24832 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:02.827514 24832 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:02.829097 24832 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.839761 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:02.845249 25051 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:23:02.845745 24898 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.785310 24815 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:02.847110 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.848280 24815 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:02.849421 24815 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.847472 20878 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:02.751076 24867 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:02.733899 24977 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:02.901962 20728 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:02.944679 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:02.945114 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.945390 20728 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
W20260709 12:23:02.945778 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.915132 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:02.953653 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.954465 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.955367 20727 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:02.956622 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:02.958096 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:02.877054 25026 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.959034 20878 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:02.959617 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:02.961746 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:02.963394 21435 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:23:02.915614 24758 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.914959 24815 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:02.957607 24890 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:02.957449 24877 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:02.965555 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.956410 24967 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:02.957804 24922 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:02.975955 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2012) took 1392 ms (client timeout 1710 ms). Trace:
W20260709 12:23:02.968771 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 118856us
I20260709 12:23:02.966354 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.977139 20728 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:02.978034 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.971817 25026 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.851753 24982 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:02.976588 20875 rpcz_store.cc:269] 0709 12:23:01.583677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.640688 (+ 57011us) service_pool.cc:224] Handling call
0709 12:23:02.975713 (+1335025us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:02.975873 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320938752}
I20260709 12:23:02.978996 20728 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:02.985457 20728 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:02.850987 24842 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.965318 24919 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:02.971302 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:02.966315 24929 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:02.987844 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.991595 24758 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:02.995021 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:02.995935 21435 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:02.996786 20878 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:02.997928 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:02.918334 24743 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:02.925489 24970 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:02.981040 24813 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:02.966169 24831 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:23:03.004102 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:03.012013 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2090) took 1969 ms (client timeout 1835 ms). Trace:
W20260709 12:23:03.012652 20806 rpcz_store.cc:269] 0709 12:23:01.042599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.050003 (+  7404us) service_pool.cc:224] Handling call
0709 12:23:03.011809 (+1961806us) spinlock_profiling.cc:243] Waited 85.4 ms on lock 0x7b5400127d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:03.011932 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196326144}
W20260709 12:23:03.014508 20805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2296) took 2088 ms (client timeout 1812 ms). Trace:
W20260709 12:23:03.015127 20805 rpcz_store.cc:269] 0709 12:23:00.925912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.926041 (+   129us) service_pool.cc:224] Handling call
0709 12:23:03.014229 (+2088188us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54003c6c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:03.014412 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813016448}
I20260709 12:23:02.975430 24890 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.976298 24877 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.982259 24922 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.022015 24863 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.993671 24758 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:02.983701 25026 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.016086 24813 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:03.012076 24977 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.031982 24877 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.033367 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.024426 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:03.038336 20875 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:23:03.040735 24758 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:03.043036 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:03.043679 24813 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.044996 24813 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:02.998304 24929 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.032891 24918 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:03.038493 25049 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 12:23:03.066818 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 872) took 2631 ms. Trace:
I20260709 12:23:03.067258 21427 rpcz_store.cc:276] 0709 12:23:00.435518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.435629 (+   111us) service_pool.cc:224] Handling call
0709 12:23:00.458609 (+ 22980us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:00.529118 (+ 70509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:00.529169 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:00.552636 (+ 23467us) log.cc:844] Serialized 26 byte log entry
0709 12:23:00.885527 (+332891us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:00.885590 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.066465 (+2180875us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:03.066518 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:03.066558 (+    40us) raft_consensus.cc:1685] finished
0709 12:23:03.066568 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.066763 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10177,"raft.queue_time_us":9498,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4457,"spinlock_wait_cycles":98593152,"thread_start_us":332432,"threads_started":1,"wal-append.queue_time_us":475867,"wal-append.run_cpu_time_us":15763,"wal-append.run_wall_time_us":1806721}
I20260709 12:23:03.069886 24863 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:03.071576 25026 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:03.073280 24922 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:03.073977 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.076056 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.076910 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:03.078253 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.063752 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.064786 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:03.034108 24838 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.082069 20809 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 5. Current term is 7. Ops: []
I20260709 12:23:03.082365 25000 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:03.082908 25000 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:03.083827 25000 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.084336 24838 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.057379 20875 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:03.080626 21435 raft_consensus.cc:2376] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12. Re-sending same reply.
I20260709 12:23:03.086594 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.087267 20875 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:03.088117 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:03.088447 24929 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:03.089200 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.089849 20875 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:03.090643 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.091377 20875 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:03.092083 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.093055 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.093642 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:03.095165 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.154182 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:03.142824 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:03.136063 20727 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:03.144471 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:03.177846 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:03.190435 21434 raft_consensus.cc:2376] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12. Re-sending same reply.
I20260709 12:23:03.191995 20878 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.097981 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.191020 21427 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.193091 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:03.111541 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.154350 24922 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:03.176880 24950 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 2.660s	user 0.000s	sys 0.011s
I20260709 12:23:03.195685 21434 raft_consensus.cc:2368] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11.
W20260709 12:23:03.193768 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95874.8us
I20260709 12:23:03.080717 24834 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:23:03.198238 20776 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.543s (SENT)
I20260709 12:23:03.200496 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:03.199102 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:03.082136 24890 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.207578 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.094317 24863 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:03.208928 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.195956 24838 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:03.216020 24842 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:03.216570 24842 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:03.217022 24842 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:03.222594 24926 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.001s
I20260709 12:23:03.223711 20875 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:23:03.215452 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:03.226126 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.228837 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:03.230469 20878 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:03.231549 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.200240 24834 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:03.232504 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:03.214428 24963 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:03.236021 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2039) took 3291 ms. Trace:
I20260709 12:23:03.197400 24922 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.231524 24838 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:03.236454 20730 rpcz_store.cc:276] 0709 12:22:59.944333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.955077 (+ 10744us) service_pool.cc:224] Handling call
0709 12:23:00.015653 (+ 60576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:00.032628 (+ 16975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:00.032679 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:00.048860 (+ 16181us) log.cc:844] Serialized 26 byte log entry
0709 12:23:00.391097 (+342237us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:00.391147 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.391241 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:03.176230 (+2784989us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:03.233886 (+ 57656us) raft_consensus.cc:1685] finished
0709 12:23:03.233912 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.235941 (+  2029us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6485,"raft.queue_time_us":19576,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5246,"spinlock_wait_cycles":28690176,"thread_start_us":317846,"threads_started":1,"wal-append.queue_time_us":386711}
I20260709 12:23:03.238950 25021 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:03.225278 24982 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:03.241044 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:03.225673 25000 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:03.246104 21435 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:03.246799 20726 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:03.235610 24834 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.248431 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:03.234975 25018 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:03.249703 21435 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:03.236392 24977 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:03.236235 24963 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.242416 25021 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:03.246066 25000 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:03.255339 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.259776 21435 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:03.259804 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:03.261039 20878 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:23:03.254098 24963 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:03.254596 24977 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:03.255911 25000 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:03.269210 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.274108 21427 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:23:03.281549 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.267671 24989 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:03.282807 24989 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:03.282984 20876 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:03.283562 24655 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:03.284039 24655 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:03.283685 24989 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:03.284245 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2009) took 2168 ms (client timeout 2160 ms). Trace:
I20260709 12:23:03.285020 24968 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.285948 24922 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:03.284742 20876 rpcz_store.cc:269] 0709 12:23:01.115603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.609363 (+493760us) service_pool.cc:224] Handling call
0709 12:23:03.284058 (+1674695us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540005ad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:03.284173 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3664669952}
W20260709 12:23:03.286949 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2013) took 1699 ms (client timeout 1921 ms). Trace:
I20260709 12:23:03.284919 24655 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.287556 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:03.287360 20881 rpcz_store.cc:269] 0709 12:23:01.587307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.218061 (+630754us) service_pool.cc:224] Handling call
0709 12:23:03.286795 (+1068734us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54003f6e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:03.286895 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894427904}
I20260709 12:23:03.287828 25021 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.289048 24977 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.285607 24968 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.300086 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.298825 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:03.300940 20881 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:03.297466 20876 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:23:03.296833 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:03.302446 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.303375 20876 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:23:03.304781 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.285045 24989 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.308053 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:03.306057 24803 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:03.312383 20878 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:23:03.315980 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.317245 24995 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:03.318819 24952 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:03.319425 24952 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:03.320384 24952 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.326025 24803 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:03.326612 24803 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:03.262038 24838 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.330703 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:03.340008 20876 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:03.305600 24863 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:03.306583 25021 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:03.346119 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:03.347841 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.349080 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:03.350359 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.312526 24866 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:03.356781 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.356779 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.358049 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2004) took 2473 ms. Trace:
I20260709 12:23:03.361930 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.362577 20878 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:23:03.363786 20879 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:03.364346 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2007) took 2356 ms. Trace:
I20260709 12:23:03.365845 20879 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:03.364760 20884 rpcz_store.cc:276] 0709 12:23:01.007921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.030097 (+ 22176us) service_pool.cc:224] Handling call
0709 12:23:01.065407 (+ 35310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:01.065650 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:03.328233 (+2262583us) log.cc:844] Serialized 14 byte log entry
0709 12:23:03.363986 (+ 35753us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:03.364036 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:03.364077 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.364154 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.364286 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2262263,"prepare.queue_time_us":2262452,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":88,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":35090,"threads_started":1,"wal-append.queue_time_us":35402}
W20260709 12:23:03.366292 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50530.1us
I20260709 12:23:03.306030 24970 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:23:03.340734 25002 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:03.344079 24903 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.354158 24655 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.372311 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:03.370927 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.373209 20881 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:03.357959 24907 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:03.375223 24903 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.377477 24903 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:03.366662 25056 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:23:03.382688 20731 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:23:03.383641 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.358428 20880 rpcz_store.cc:276] 0709 12:23:00.884772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.887086 (+  2314us) service_pool.cc:224] Handling call
0709 12:23:00.887436 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.887690 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:00.917804 (+ 30114us) log.cc:844] Serialized 14 byte log entry
0709 12:23:03.324169 (+2406365us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18002f8720. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:23:03.324397 (+   228us) log.cc:844] Serialized 14 byte log entry
0709 12:23:03.357682 (+ 33285us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:03.357736 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:03.357792 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.357864 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.357988 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":901,"prepare.queue_time_us":2342145,"prepare.run_cpu_time_us":2473,"prepare.run_wall_time_us":33809,"raft.queue_time_us":140,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6027,"spinlock_wait_cycles":3001088,"thread_start_us":130536,"threads_started":2,"wal-append.queue_time_us":632930,"wal-append.run_cpu_time_us":2538,"wal-append.run_wall_time_us":1269940}
W20260709 12:23:03.386193 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.365573 24867 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:03.374727 24907 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.391780 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:03.397632 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.398557 20881 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 13 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:23:03.405256 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:03.406476 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2010) took 1825 ms. Trace:
I20260709 12:23:03.406611 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:03.407008 20883 rpcz_store.cc:276] 0709 12:23:01.581563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.379951 (+798388us) service_pool.cc:224] Handling call
0709 12:23:02.380267 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.632786 (+252519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.406203 (+773417us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:03.406411 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579191168}
I20260709 12:23:03.412132 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:03.413450 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:03.414410 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:03.418923 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.425352 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.426426 20801 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:03.428171 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.429870 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:03.430675 20805 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:03.367787 24842 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.440577 20884 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:03.440593 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:03.450961 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 136155us
I20260709 12:23:03.455350 20806 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:03.455497 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.409228 24863 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:03.453240 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.459558 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:03.458789 20733 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:03.439469 24723 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:03.416596 24717 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:03.440325 24903 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:03.471689 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.474860 24833 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:03.475294 24951 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:03.476347 25058 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:03.476935 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.432291 24841 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.492327 24841 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.493515 20880 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:03.478693 20808 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:03.481894 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:03.482916 21433 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:23:03.496378 21433 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 7
I20260709 12:23:03.496670 21433 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:23:03.496919 21433 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:03.500886 24803 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.485497 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:03.507454 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.478035 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2058) took 1521 ms. Trace:
I20260709 12:23:03.510207 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.494174 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.441479 24890 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:23:03.485411 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:03.505934 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:03.506937 20880 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:03.507411 20878 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:23:03.514362 20878 pending_rounds.cc:85] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 9
I20260709 12:23:03.514649 20878 pending_rounds.cc:107] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:23:03.514909 20878 raft_consensus.cc:2889] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:03.511368 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:03.516443 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:03.512048 20804 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:23:03.413165 24867 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.510159 20729 rpcz_store.cc:276] 0709 12:23:01.956381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.063316 (+106935us) service_pool.cc:224] Handling call
0709 12:23:02.068276 (+  4960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:02.068990 (+   714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:02.069037 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:02.080466 (+ 11429us) log.cc:844] Serialized 26 byte log entry
0709 12:23:03.071072 (+990606us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b18003e5ea0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:03.164405 (+ 93333us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:03.164466 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.164567 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:03.458311 (+293744us) raft_consensus.cc:1685] finished
0709 12:23:03.458328 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.477976 (+ 19648us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":286,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4020,"spinlock_wait_cycles":3382144,"thread_start_us":93062,"threads_started":1,"wal-append.queue_time_us":1133055}
I20260709 12:23:03.518064 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:03.508792 20810 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:23:03.518949 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.520586 20880 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:03.520926 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.517913 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:03.525192 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:03.490293 24922 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:03.530681 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.532688 24963 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.533530 24803 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.534247 20809 raft_consensus.cc:2368] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:23:03.533761 24834 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.534812 24963 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.536568 20803 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:03.538122 24717 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:03.538792 24717 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.539299 24717 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.536670 24834 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:03.543264 24723 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:03.544965 24842 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.547798 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.550817 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.552369 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:03.553180 20807 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:03.555653 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.557336 20804 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:03.557209 20810 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:03.561476 20810 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:03.568881 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 107472us
I20260709 12:23:03.529996 24898 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.575579 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.559142 24841 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:03.585343 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.581300 20776 leader_election.cc:400] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11.
I20260709 12:23:03.586767 20776 leader_election.cc:403] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:03.588440 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:03.589200 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.594733 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.597265 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:03.601763 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.559670 24922 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.604892 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:03.607095 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.613194 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.613879 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:03.623250 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.625226 24557 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:03.632992 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:03.598179 24952 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:03.604226 24841 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:03.642258 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.659499 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.662781 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:03.663964 20807 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:23:03.654965 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:03.650302 25065 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:03.676806 20810 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:03.682520 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2332) took 1428 ms. Trace:
I20260709 12:23:03.682744 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2062) took 1678 ms. Trace:
I20260709 12:23:03.658988 24918 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:03.683465 20734 rpcz_store.cc:276] 0709 12:23:02.004043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.145314 (+141271us) service_pool.cc:224] Handling call
0709 12:23:02.180872 (+ 35558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.181091 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:02.569663 (+388572us) log.cc:844] Serialized 14 byte log entry
0709 12:23:03.316636 (+746973us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b18004127e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:03.682240 (+365604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:03.682320 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:03.682409 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.682503 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.682668 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1112240,"prepare.queue_time_us":320447,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2083,"spinlock_wait_cycles":39177472,"thread_start_us":359463,"threads_started":1}
I20260709 12:23:03.683142 20735 rpcz_store.cc:276] 0709 12:23:02.253833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.253973 (+   140us) service_pool.cc:224] Handling call
0709 12:23:02.254326 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.254578 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:02.358377 (+103799us) log.cc:844] Serialized 14 byte log entry
0709 12:23:02.566959 (+208582us) log.cc:844] Serialized 14 byte log entry
0709 12:23:03.681825 (+1114866us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18003fe500. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:23:03.682033 (+   208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:03.682105 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:03.682171 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.682278 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.682441 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1091181,"prepare.queue_time_us":447874,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":2361,"raft.run_cpu_time_us":583,"raft.run_wall_time_us":10873,"spinlock_wait_cycles":23120256,"thread_start_us":301985,"threads_started":2,"wal-append.queue_time_us":274056,"wal-append.run_cpu_time_us":1985,"wal-append.run_wall_time_us":1003678}
I20260709 12:23:03.659287 24831 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:23:03.676569 24926 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:03.705237 20801 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
W20260709 12:23:03.708200 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.708511 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:03.709373 21428 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:03.711053 20729 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:23:03.712563 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:03.713094 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.713299 20729 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:03.713611 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:03.714349 20735 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:03.714537 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:03.715363 20730 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:03.689430 24918 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.717940 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:03.723142 21432 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:03.679992 24911 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:23:03.729425 20728 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:03.703348 24829 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:23:03.712136 24694 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.660950 24914 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.688732 24929 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:23:03.726984 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:03.740607 20732 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:03.703758 25065 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:03.731829 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:03.725658 20730 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:03.744549 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:03.743826 20728 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:03.718075 24816 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.727047 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:03.722350 25002 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:03.724465 24918 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.750432 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:03.740453 25062 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:23:03.751760 20734 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:03.752295 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:03.753229 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:03.760275 20728 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:03.755486 24831 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:03.756934 20732 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:03.749109 20735 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:03.761452 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:03.751614 24914 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.764645 20735 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:03.765515 20732 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:23:03.772929 20735 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:03.739687 25000 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.740700 24694 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:03.780931 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.763777 24831 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:03.751662 24717 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.787587 20806 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:03.788372 24832 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:03.788875 24694 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:03.789480 20734 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 6.
I20260709 12:23:03.789796 24831 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:03.791358 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.795456 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:03.803594 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:03.805023 25000 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.756536 24816 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.769853 24926 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:03.815157 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:03.815389 24926 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:03.816206 24816 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.816383 24929 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:03.809785 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.813273 20732 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.817481 21427 raft_consensus.cc:2368] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
W20260709 12:23:03.800858 25067 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:23:03.815213 25071 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:23:03.806809 24968 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:03.818414 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.819096 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:03.820878 24717 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:03.828269 24929 raft_consensus.cc:2764] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:23:03.818075 24911 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:03.830564 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:03.813750 25065 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.838007 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:23:03.847715 20732 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:23:03.841279 24690 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:03.863080 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.837834 25059 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:23:03.926265 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.933979 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:03.939359 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:03.952731 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.952998 20734 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:03.939417 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:03.955102 20732 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:03.980680 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:03.841193 24952 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11.
I20260709 12:23:03.969686 24967 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:03.939253 24717 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:03.970726 24690 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:03.971082 25059 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:03.980680 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:03.840943 24968 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.141s
W20260709 12:23:03.987679 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 157154us
I20260709 12:23:04.007568 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.009069 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:04.009266 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:04.009912 20876 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:04.010309 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:03.996282 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.997433 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2335) took 1229 ms. Trace:
I20260709 12:23:04.012400 20726 rpcz_store.cc:276] 0709 12:23:02.768114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.768253 (+   139us) service_pool.cc:224] Handling call
0709 12:23:02.768503 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:03.247278 (+478775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.993816 (+746538us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54003b4a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:03.997330 (+  3514us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099559808}
I20260709 12:23:03.992841 24914 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:04.013758 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:04.014357 24989 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
I20260709 12:23:04.015097 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:04.016934 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.017815 24960 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:03.991212 24717 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.019932 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.023432 20884 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:23:04.030740 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:04.019176 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:03.993389 24929 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.032667 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.025183 20877 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:04.041401 20876 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:04.044545 24890 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:23:04.044874 25078 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.045414 25078 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.047525 24723 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.048316 25078 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.049301 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2334) took 1282 ms. Trace:
I20260709 12:23:04.049817 20733 rpcz_store.cc:276] 0709 12:23:02.767215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.767338 (+   123us) service_pool.cc:224] Handling call
0709 12:23:03.458546 (+691208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:03.460493 (+  1947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.049083 (+588590us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54003b4050. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:04.049225 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588857600}
I20260709 12:23:03.994015 24918 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.034564 24960 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.013816 24914 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:04.064215 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:04.064980 20879 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:04.043704 24926 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.035265 24911 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:03.991501 24967 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:04.057971 25066 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.494s	user 0.007s	sys 0.010s
I20260709 12:23:04.042881 25075 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:04.062754 20875 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:04.054899 25078 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.077575 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:04.071625 20804 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:23:04.078444 25076 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:23:04.077028 25075 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.081648 25075 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.059013 25059 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.078397 25078 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:04.088269 25059 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:04.090346 25078 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:04.074111 24655 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.691s	user 0.013s	sys 0.000s
I20260709 12:23:04.052138 24929 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.093693 24717 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:04.094022 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.094429 24929 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.095510 24914 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:04.109257 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 400814us
I20260709 12:23:04.116147 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2029) took 1145 ms. Trace:
I20260709 12:23:04.116614 20883 rpcz_store.cc:276] 0709 12:23:02.970626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.408013 (+437387us) service_pool.cc:224] Handling call
0709 12:23:03.408366 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:03.415321 (+  6955us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:03.415355 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:03.415911 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:23:03.489372 (+ 73461us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:03.489443 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.489546 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:04.057316 (+567770us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:04.115842 (+ 58526us) raft_consensus.cc:1685] finished
0709 12:23:04.115869 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:04.116080 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6610,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":26448,"spinlock_wait_cycles":11288320,"thread_start_us":5829,"threads_started":1,"wal-append.queue_time_us":64848}
I20260709 12:23:04.122046 25000 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:04.123402 25065 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:04.124114 24963 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:04.124317 25065 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:04.124874 24963 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:04.125283 24433 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:23:04.124239 25072 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.367s	user 0.007s	sys 0.007s
I20260709 12:23:04.127468 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:04.125662 24433 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:04.129431 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.127496 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47112.6us
I20260709 12:23:04.132551 20880 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:23:04.133113 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.134569 21432 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:23:04.134940 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:04.135864 21427 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:23:04.122674 25000 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:04.149164 20776 leader_election.cc:400] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:23:04.149623 20776 leader_election.cc:403] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:23:04.136341 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.137470 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:04.150422 20801 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.133563 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:04.151657 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:04.150415 20805 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:04.138029 20849 leader_election.cc:400] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:23:04.155658 20849 leader_election.cc:403] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:04.160568 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:04.163043 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 130439us
I20260709 12:23:04.163172 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.161944 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:04.164248 20805 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:04.164016 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.168143 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:04.171814 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:04.173594 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.176419 25083 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:04.180342 24968 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:04.180572 25083 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:04.180792 24968 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.181052 25083 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:04.181188 24968 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:04.178298 24960 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:04.183218 24960 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:04.076083 24926 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.157022 25078 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:04.158560 24929 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:04.187858 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:04.167677 24723 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:23:04.188495 24708 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:04.201927 20880 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:04.189754 24690 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.190003 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:04.177014 24898 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.004s
I20260709 12:23:04.162523 24911 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
W20260709 12:23:04.192006 20731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2346) took 1361 ms (client timeout 1652 ms). Trace:
I20260709 12:23:04.189568 24918 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.201113 20879 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:04.202188 20883 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:04.210462 24841 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:04.195784 24929 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.218210 24841 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:04.218552 24929 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:04.218688 24841 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.178750 25084 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:04.213387 24898 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.210314 24863 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:04.214948 20731 rpcz_store.cc:269] 0709 12:23:02.830604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.846837 (+ 16233us) service_pool.cc:224] Handling call
0709 12:23:04.191838 (+1345001us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b5400298bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:04.191954 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1227906432}
I20260709 12:23:04.224466 24977 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:04.234277 20801 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
W20260709 12:23:04.247756 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74238.3us
I20260709 12:23:04.250727 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:04.251111 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:04.251489 21433 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:04.252012 21435 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
W20260709 12:23:04.252789 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73292.1us
I20260709 12:23:04.254021 24995 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:04.256429 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2345) took 1433 ms. Trace:
I20260709 12:23:04.260358 20727 rpcz_store.cc:276] 0709 12:23:02.823165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.971022 (+147857us) service_pool.cc:224] Handling call
0709 12:23:04.256275 (+1285253us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54000e6fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:04.256374 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341942656}
I20260709 12:23:04.257207 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:04.261540 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:04.222311 25084 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.257947 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:04.267412 21427 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:04.257597 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:04.204268 25078 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:04.252195 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.298684 25078 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:04.300475 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:04.301239 20884 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:04.303205 25015 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:04.300535 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.67962e+06us
I20260709 12:23:04.302824 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:04.268896 21428 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:04.256294 24898 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:04.275492 24907 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:04.265532 24911 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:04.276865 24929 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:04.328197 24926 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.328750 24929 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:04.238150 24977 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:04.332266 24977 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.333801 24977 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:04.216636 24433 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.336159 24433 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:04.336241 24967 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:04.337401 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:04.280403 21431 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:04.281458 25084 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:04.237998 24991 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8.
I20260709 12:23:04.282833 25000 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:04.284240 25088 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:23:04.287647 21434 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:04.289014 25075 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.291425 25083 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.293262 24922 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.294345 21429 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:04.313053 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:04.313791 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:04.323138 20883 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:23:04.320520 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:04.362296 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.317055 24898 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:04.342862 24810 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:04.321072 24911 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:04.366973 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.369388 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55665us
I20260709 12:23:04.382014 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.303752 25015 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:04.378216 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 217722us
W20260709 12:23:04.378602 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58204us
I20260709 12:23:04.341006 24863 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:04.341136 24492 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.404003 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:04.406164 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.405187 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.408206 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:04.409037 21427 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:04.410102 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:04.411054 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:04.470888 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.471479 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.472088 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.475323 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:04.476184 20806 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:04.477552 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.478763 20805 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:23:04.383042 25000 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:04.346292 25014 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:23:04.473428 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:04.508607 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:04.512164 24995 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:04.383800 24911 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:04.384824 25091 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:04.402594 20875 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:04.413755 24967 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:04.505344 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:04.360643 24805 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:23:04.514458 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.525606 20806 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:04.480238 24690 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:04.532557 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.532673 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59844.7us
I20260709 12:23:04.500871 24723 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:04.534104 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 553464us
I20260709 12:23:04.384174 24810 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.402529 24841 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.551689 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:04.552141 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 247414us
I20260709 12:23:04.553011 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.516098 24914 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:04.470023 24492 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.564126 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:04.519804 24974 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:04.536043 25000 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:04.533416 24863 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:04.527787 24995 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:04.568934 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.546990 25011 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.571290 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.573472 20728 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:04.574447 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.543951 24690 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.545188 24989 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:04.534970 25091 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:04.582422 20734 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:04.582839 20727 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:04.593202 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.762656 20732 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
W20260709 12:23:04.762677 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 225291us
I20260709 12:23:04.569872 20726 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:23:04.341527 24557 consensus_queue.cc:1175] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 13 status { last_received { term: 9 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.10-12.10]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:23:04.624972 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.625763 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.767653 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:04.561939 24929 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.561625 24872 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.636865 24989 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:04.775382 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 187287us
W20260709 12:23:04.666283 25087 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:23:04.746059 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 203033us
W20260709 12:23:04.747710 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 183704us
I20260709 12:23:04.778666 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:04.635088 25011 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.549342 24723 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.566778 24708 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:23:04.623979 24746 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:04.755812 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.755591 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.806653 20728 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:04.557315 25089 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:04.535424 24898 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.766285 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:04.746663 24839 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.780088 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:04.633126 24690 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:04.745658 24835 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.782145 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:04.826030 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 43982.7us
W20260709 12:23:04.790097 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.769519 24873 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:04.756932 24815 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:04.754981 24655 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:04.810940 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.814260 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.811645 24872 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:04.782382 24989 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:04.824357 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 272736us
W20260709 12:23:04.824645 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44680.3us
I20260709 12:23:04.888123 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:04.889046 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:04.889858 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:04.826821 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:04.828928 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.816560 25015 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.806106 25048 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:04.852190 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:04.880532 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.811865 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.781747 25058 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:23:04.781570 24982 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:23:04.814177 24926 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:04.895936 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:04.829936 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63732.5us
I20260709 12:23:04.849824 24947 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:04.966938 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61735.1us
I20260709 12:23:04.826481 25083 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:04.778311 24863 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:04.992682 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.539s (SENT) [suppressed 15 similar messages]
I20260709 12:23:04.831139 24816 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:04.832860 24839 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:04.835112 25014 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:04.884713 24898 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:04.994515 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.814594 24746 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:04.990742 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.821770 25075 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:04.991636 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:05.000998 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 655) took 1418 ms (client timeout 1806 ms). Trace:
W20260709 12:23:04.907081 25093 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.495s	user 0.014s	sys 0.012s
I20260709 12:23:04.830502 24873 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.006987 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.168383 20805 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:23:05.101570 25065 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:05.021677 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 125831us
W20260709 12:23:04.995430 25097 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.450s	user 0.000s	sys 0.001s
I20260709 12:23:04.913285 24853 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:05.171381 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:04.913687 24914 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:04.852658 24835 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.108557 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:05.186643 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78158.5us
I20260709 12:23:05.109566 20808 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:04.895867 24929 raft_consensus.cc:2764] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:23:04.987700 25048 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:05.108651 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:04.992892 20734 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:05.130748 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.200958 20733 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:23:05.000492 24918 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.020326 25035 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:05.106750 25086 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.540s	user 0.000s	sys 0.001s
I20260709 12:23:05.121037 20876 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Committing config change with OpId 7.8: config changed from index 3 to 8, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:04.900460 24815 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:04.892474 24655 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.009393 24810 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.154809 20810 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:23:04.866456 24995 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.131935 25109 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:05.157447 20804 rpcz_store.cc:269] 0709 12:23:03.583072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.586511 (+  3439us) service_pool.cc:224] Handling call
0709 12:23:05.000799 (+1414288us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54002a99d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:05.000931 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873551232}
I20260709 12:23:05.101042 24841 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.101833 24976 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:05.017343 24947 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:05.220444 25102 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:23:05.015664 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:05.107564 24974 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:05.248823 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83037.2us
I20260709 12:23:05.105078 25015 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:05.183913 25104 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 12:23:05.183787 24867 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.184725 25075 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.171969 24853 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.185634 24914 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.129508 24816 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:04.995508 25000 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.134656 25014 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.270515 25014 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:05.160652 25095 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.531s	user 0.000s	sys 0.002s
I20260709 12:23:05.272331 24816 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:05.273703 24989 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.154704 24492 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.173754 24873 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:05.266422 24914 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.275717 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2056) took 1075 ms. Trace:
I20260709 12:23:05.276157 20880 rpcz_store.cc:276] 0709 12:23:04.200802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.204883 (+  4081us) service_pool.cc:224] Handling call
0709 12:23:04.205311 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.205721 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.205754 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.206384 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.402415 (+196031us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:04.402461 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.402533 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.275493 (+872960us) raft_consensus.cc:1685] finished
0709 12:23:05.275513 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.275659 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2494,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":195582,"threads_started":1,"wal-append.queue_time_us":334326}
W20260709 12:23:05.213564 24977 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.005s
W20260709 12:23:05.228608 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 235969us
W20260709 12:23:05.280120 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.206858 25089 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:05.281258 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:05.282020 24717 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.213691 25035 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:05.255882 25099 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 12:23:05.283100 24717 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.283504 24974 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.248771 24976 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.285897 24976 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.011512 25091 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.283541 25035 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:05.192920 25065 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.284281 24873 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.288034 24989 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.281407 25083 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.281276 25089 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.239408 25109 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.290228 25083 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.291676 24839 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.292253 25109 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.290616 25089 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.293027 25083 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.293507 24655 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.238258 24947 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:05.296379 25059 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.208s	user 0.013s	sys 0.000s
I20260709 12:23:05.297611 24746 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.289405 25065 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.298216 24873 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.290361 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2059) took 1074 ms. Trace:
I20260709 12:23:05.249032 25011 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.266136 25075 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:05.300485 25100 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:23:05.300321 25089 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.265259 25048 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.216117 24918 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.302462 25075 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.250692 24723 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.304883 24918 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.299945 20879 rpcz_store.cc:276] 0709 12:23:04.215547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.245879 (+ 30332us) service_pool.cc:224] Handling call
0709 12:23:04.265481 (+ 19602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.316595 (+ 51114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.316643 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.317283 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.666523 (+349240us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:04.666565 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.666637 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.290120 (+623483us) raft_consensus.cc:1685] finished
0709 12:23:05.290140 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.290296 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14571,"raft.queue_time_us":52217,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":19439,"thread_start_us":301603,"threads_started":1,"wal-append.queue_time_us":437189}
I20260709 12:23:05.266815 24853 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:05.265799 24968 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.084s	user 0.012s	sys 0.000s
I20260709 12:23:05.286998 24717 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.302532 25089 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.295022 24815 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.302701 25048 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:05.321012 25084 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.963s	user 0.008s	sys 0.004s
I20260709 12:23:05.329126 24968 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.342123 25084 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.355593 25084 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.359934 25035 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.170861 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:05.431886 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 260879us
W20260709 12:23:05.438254 20630 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 300) took 23169 ms (client timeout 15000 ms). Trace:
W20260709 12:23:05.200376 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 373626us
I20260709 12:23:05.496281 25035 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.172235 20726 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:05.498726 24746 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:05.500957 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:05.501986 20726 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:05.502360 24746 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.360008 25014 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.506714 25014 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.343861 24690 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.509979 20735 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:23:05.510277 24690 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.169463 20728 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:05.441002 20731 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:23:05.217738 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:05.231777 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 103765us
I20260709 12:23:05.512521 20733 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:05.512316 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.514294 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.225423 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 664) took 1106 ms. Trace:
I20260709 12:23:05.449993 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2331) took 1673 ms. Trace:
I20260709 12:23:05.453126 20802 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:05.205655 20803 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:05.457922 20776 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.067s (SENT)
I20260709 12:23:05.458010 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2069) took 1739 ms. Trace:
W20260709 12:23:05.472378 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:05.261281 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73985.9us
I20260709 12:23:05.477044 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:05.491212 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:05.557967 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1071) took 2354 ms (client timeout 2067 ms). Trace:
W20260709 12:23:05.557981 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 81353us
I20260709 12:23:05.558537 20879 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:05.562948 25115 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:05.563548 25115 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:05.563997 25115 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:05.521427 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.522491 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.566848 20804 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
W20260709 12:23:05.572350 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 114735us
I20260709 12:23:05.548097 20729 rpcz_store.cc:276] 0709 12:23:03.717994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.718200 (+   206us) service_pool.cc:224] Handling call
0709 12:23:04.192328 (+474128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.534734 (+342406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.534785 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.535337 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.765887 (+230550us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:04.913216 (+147329us) log.cc:844] Serialized 14 byte log entry
0709 12:23:05.003916 (+ 90700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:05.004033 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.281400 (+277367us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b18002b8a60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:05.442456 (+161056us) raft_consensus.cc:1685] finished
0709 12:23:05.442656 (+   200us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.457534 (+ 14878us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b5400298bd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:05.457752 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":617508,"prepare.queue_time_us":466561,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":437056,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":99306,"spinlock_wait_cycles":1099338752,"thread_start_us":302466,"threads_started":2,"wal-append.queue_time_us":549168}
W20260709 12:23:05.558650 21434 rpcz_store.cc:269] 0709 12:23:03.203314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.203510 (+   196us) service_pool.cc:224] Handling call
0709 12:23:05.557324 (+2353814us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:05.557609 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432660224}
I20260709 12:23:05.515870 20807 rpcz_store.cc:276] 0709 12:23:03.776091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.776235 (+   144us) service_pool.cc:224] Handling call
0709 12:23:03.801401 (+ 25166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.120434 (+319033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.120463 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.149249 (+ 28786us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.518070 (+368821us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:04.518113 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.518185 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.449287 (+931102us) raft_consensus.cc:1685] finished
0709 12:23:05.449327 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.449720 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":318564,"raft.queue_time_us":318726,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":331474,"threads_started":1,"wal-append.queue_time_us":720876}
I20260709 12:23:05.525017 24914 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.532905 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 896) took 1279 ms. Trace:
I20260709 12:23:05.533712 24873 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.587090 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:05.536722 20884 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 11. Ops: []
I20260709 12:23:05.584321 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.554090 24815 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.503876 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.558506 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:05.600524 24815 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:05.601568 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43513.1us
I20260709 12:23:05.441249 20730 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 12:23:05.603042 25081 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.461s	user 0.006s	sys 0.004s
I20260709 12:23:05.603296 25108 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:05.603925 25108 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.604514 25108 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:05.606021 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2357) took 1031 ms. Trace:
I20260709 12:23:05.520365 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2030) took 2545 ms. Trace:
I20260709 12:23:05.539845 24492 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:05.594271 24873 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.591607 21435 rpcz_store.cc:276] 0709 12:23:04.253776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.253915 (+   139us) service_pool.cc:224] Handling call
0709 12:23:04.254280 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.254605 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.254630 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.255106 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.506830 (+251724us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:04.506907 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.507019 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.530228 (+1023209us) raft_consensus.cc:1685] finished
0709 12:23:05.530276 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.532594 (+  2318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66902,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":18242,"thread_start_us":251188,"threads_started":1,"wal-append.queue_time_us":1079859}
I20260709 12:23:05.597165 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:05.597445 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:05.580756 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:05.599598 25063 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.964s	user 0.012s	sys 0.009s
I20260709 12:23:05.582649 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.606545 20730 rpcz_store.cc:276] 0709 12:23:04.574074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.574199 (+   125us) service_pool.cc:224] Handling call
0709 12:23:05.605637 (+1031438us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:05.605893 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574666624}
I20260709 12:23:05.583344 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:05.577741 24968 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.583485 24835 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.515481 20809 rpcz_store.cc:276] 0709 12:23:04.118799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.128176 (+  9377us) service_pool.cc:224] Handling call
0709 12:23:04.143652 (+ 15476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.178627 (+ 34975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.178659 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.183694 (+  5035us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.264791 (+ 81097us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:04.264831 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.265685 (+   854us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.225193 (+959508us) raft_consensus.cc:1685] finished
0709 12:23:05.225216 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.225355 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29195,"raft.queue_time_us":40145,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"thread_start_us":80722,"threads_started":1,"wal-append.queue_time_us":159310}
I20260709 12:23:05.625960 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:05.575126 24694 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:23:05.652509 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:05.650645 20807 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.638516 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:05.620746 24746 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:05.661896 24433 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.325s	user 0.006s	sys 0.008s
I20260709 12:23:05.658756 20884 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:05.664070 20733 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:05.608217 20881 rpcz_store.cc:276] 0709 12:23:02.975016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.456032 (+481016us) service_pool.cc:224] Handling call
0709 12:23:03.468780 (+ 12748us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:03.469193 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:03.469239 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:03.469821 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.380423 (+910602us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b1800270820. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:04.755626 (+375203us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:04.755669 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.755742 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.519602 (+763860us) raft_consensus.cc:1685] finished
0709 12:23:05.519668 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.520098 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7822,"raft.queue_time_us":136,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8063,"spinlock_wait_cycles":16778496,"thread_start_us":357120,"threads_started":1,"wal-append.queue_time_us":1863525}
I20260709 12:23:05.656190 21426 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:05.668560 24746 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.668696 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:05.481026 20630 rpcz_store.cc:269] 0709 12:22:42.268919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.546913 (+277994us) service_pool.cc:224] Handling call
0709 12:23:05.437899 (+22890986us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:58.918718 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:59.119362 (+200644us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:59.127141 (+  7779us) write_op.cc:435] Acquired schema lock
0709 12:22:59.131139 (+  3998us) tablet.cc:662] Decoding operations
0709 12:22:59.338058 (+206919us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:59.368513 (+ 30455us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:59.371354 (+  2841us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:22:59.701925 (+330571us) tablet.cc:701] Row locks acquired
0709 12:22:59.701980 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:22:59.702150 (+   170us) write_op.cc:270] Start()
0709 12:22:59.702308 (+   158us) write_op.cc:276] Timestamp: P: 1783599779702085 usec, L: 0
0709 12:22:59.702333 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:22:59.777197 (+ 74864us) log.cc:844] Serialized 9082 byte log entry
0709 12:23:00.475526 (+698329us) op_driver.cc:464] REPLICATION: finished
0709 12:23:00.880844 (+405318us) write_op.cc:301] APPLY: starting
0709 12:23:00.890297 (+  9453us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:01.573511 (+683214us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:01.573650 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:01.799618 (+225968us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:01.800455 (+   837us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=46
0709 12:23:01.800526 (+    71us) write_op.cc:312] APPLY: finished
0709 12:23:01.801129 (+   603us) log.cc:844] Serialized 158 byte log entry
0709 12:23:01.905217 (+104088us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:01.936704 (+ 31487us) write_op.cc:454] Released schema lock
0709 12:23:01.952927 (+ 16223us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:01.995345 (+ 42418us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:02.003481 (+  8136us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1434446,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":369264,"apply.run_cpu_time_us":24090,"apply.run_wall_time_us":1114797,"num_ops":23,"prepare.queue_time_us":276227,"prepare.run_cpu_time_us":46943,"prepare.run_wall_time_us":1145741,"raft.queue_time_us":215760,"raft.run_cpu_time_us":3815,"raft.run_wall_time_us":333723,"replication_time_us":760552,"thread_start_us":831517,"threads_started":5,"wal-append.queue_time_us":351350,"wal-append.run_cpu_time_us":10908,"wal-append.run_wall_time_us":2352946}]]}
I20260709 12:23:05.669785 20805 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:23:05.570788 24646 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Committing config change with OpId 7.8: config changed from index 3 to 8, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:05.675432 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:05.671175 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.671864 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.681684 20732 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:05.675877 25059 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.682703 20884 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:05.683329 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.683028 25059 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:05.683707 25126 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 12:23:05.684211 24898 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.685225 24898 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.685885 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:05.675742 25014 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:05.686848 25014 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.686993 20876 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:05.672224 24839 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.687386 24746 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.687826 20734 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:05.688318 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.688072 24746 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.671818 24815 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.690158 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:05.690897 21432 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:05.691772 20733 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:05.692981 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.694332 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:05.687909 24839 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.685883 25084 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.695765 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.688638 20804 raft_consensus.cc:2376] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11. Re-sending same reply.
I20260709 12:23:05.696214 25084 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.690681 24746 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.681550 21434 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:05.699440 20805 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:23:05.700551 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.700791 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.703738 24951 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.704547 24951 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.705583 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.706207 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:05.703830 24898 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.707176 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:05.704946 20726 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:23:05.710213 24911 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.169s	user 0.014s	sys 0.004s
I20260709 12:23:05.710650 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:05.711910 24968 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:05.712423 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.712661 24968 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.713822 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.715981 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.717051 20730 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:05.718143 24968 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.718242 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.701572 25014 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:05.719808 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.720860 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2181) took 3077 ms. Trace:
I20260709 12:23:05.721213 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.722754 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:05.723500 24853 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.726999 20726 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:23:05.726284 20730 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:05.733052 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:05.735921 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.739646 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:05.721374 20882 rpcz_store.cc:276] 0709 12:23:02.642959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.643085 (+   126us) service_pool.cc:224] Handling call
0709 12:23:02.645390 (+  2305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:02.650493 (+  5103us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:03.352969 (+702476us) log.cc:844] Serialized 14 byte log entry
0709 12:23:03.524571 (+171602us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:03.534985 (+ 10414us) log.cc:844] Serialized 26 byte log entry
0709 12:23:03.535150 (+   165us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:03.535222 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.535287 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.598348 (+2063061us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:05.720331 (+121983us) raft_consensus.cc:1685] finished
0709 12:23:05.720465 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.720654 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":716047,"prepare.queue_time_us":702048,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":44552,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":33489,"spinlock_wait_cycles":4054784,"thread_start_us":105267,"threads_started":1,"wal-append.queue_time_us":218400}
I20260709 12:23:05.743491 20727 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:05.743630 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:05.745915 24907 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:05.746325 24907 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.746711 24907 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:05.749231 24926 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:05.749797 24926 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:05.750370 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:05.755776 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2334) took 1600 ms. Trace:
I20260709 12:23:05.758508 20734 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:05.740357 20732 raft_consensus.cc:2376] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11. Re-sending same reply.
I20260709 12:23:05.765237 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.731916 24835 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.710747 24911 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.753113 24853 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:05.781790 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118660us
I20260709 12:23:05.757465 24951 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.756268 20801 rpcz_store.cc:276] 0709 12:23:04.131265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.254610 (+123345us) service_pool.cc:224] Handling call
0709 12:23:04.254916 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.258108 (+  3192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.258139 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.258591 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:23:05.229824 (+971233us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b1800459700. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:05.489304 (+259480us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:05.489374 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:05.489552 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.697189 (+207637us) raft_consensus.cc:1685] finished
0709 12:23:05.697211 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.732057 (+ 34846us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2763,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":5151,"spinlock_wait_cycles":7423360,"thread_start_us":259035,"threads_started":1,"wal-append.queue_time_us":1230519}
I20260709 12:23:05.777977 21434 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:05.791885 25108 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.795289 24947 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.800274 24947 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.800052 25115 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.796564 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:05.773504 24914 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.797431 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:05.802243 24976 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:05.777702 24694 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.843879 21432 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:05.802364 25115 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:05.795163 24816 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.004s
I20260709 12:23:05.828088 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:05.848572 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.814203 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:05.812901 24947 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.788141 24898 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:05.875178 20801 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:23:05.853652 25108 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:05.802856 24557 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:23:05.885607 25084 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.886559 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:05.880309 24816 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.883108 24911 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:05.892526 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 78436.2us
I20260709 12:23:05.867564 24976 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:05.882570 24694 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.897996 20877 raft_consensus.cc:2368] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:23:05.897887 25089 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:05.898399 24890 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.903077 20735 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:05.852545 24835 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:05.917022 24835 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.892047 24951 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:05.918331 24694 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:05.918195 24951 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:05.908421 25096 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.403s	user 0.004s	sys 0.000s
I20260709 12:23:05.911681 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.911512 24890 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.897615 24976 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:05.925684 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:05.930222 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:05.930982 21435 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:05.931540 24890 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:05.901733 25108 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:05.893631 24947 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:05.949592 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:05.955111 21435 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:05.961010 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:05.942706 24853 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:05.926296 24690 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:05.922094 24898 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.004838 21434 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:06.010066 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47329.3us
I20260709 12:23:05.963446 24890 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.958647 25108 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:05.967751 25098 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
W20260709 12:23:06.015146 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 225562us
I20260709 12:23:06.017776 24951 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:06.023653 20808 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:06.008852 24853 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.027225 20849 leader_election.cc:400] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:23:06.045372 20849 leader_election.cc:403] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:06.028417 24898 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:06.029109 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2064) took 1014 ms. Trace:
I20260709 12:23:06.030417 24952 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:23:06.030937 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 665) took 1911 ms. Trace:
I20260709 12:23:06.031412 24890 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:06.050328 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 85744.2us
I20260709 12:23:06.032187 24557 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.031926 25108 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.036919 20801 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:23:06.039633 24831 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.041477 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.043409 20807 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:06.052095 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.050434 24911 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:23:06.056674 21426 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:06.058676 24557 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.059873 20803 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:06.061484 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2362) took 1077 ms. Trace:
I20260709 12:23:06.061685 24557 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.061829 20728 rpcz_store.cc:276] 0709 12:23:04.983612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.991291 (+  7679us) service_pool.cc:224] Handling call
0709 12:23:06.061155 (+1069864us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b5400249fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:06.061309 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27651840}
I20260709 12:23:06.014055 25089 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:06.047835 20802 rpcz_store.cc:276] 0709 12:23:04.129994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.132280 (+  2286us) service_pool.cc:224] Handling call
0709 12:23:04.143175 (+ 10895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.516174 (+1372999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.030644 (+514470us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540006d1d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:06.030878 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3009384704}
I20260709 12:23:05.960076 24907 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.045929 20877 rpcz_store.cc:276] 0709 12:23:05.014998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.015331 (+   333us) service_pool.cc:224] Handling call
0709 12:23:06.028848 (+1013517us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b540005ad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:06.029033 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320412928}
I20260709 12:23:06.050415 24890 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:06.069047 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.048210 24853 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:06.081833 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.066812 24816 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.024547 24690 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.094983 24952 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:06.121026 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.097684 24951 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:06.099570 24991 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:06.100889 24835 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:06.112056 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.113082 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.069442 25089 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:06.114383 21427 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:06.114703 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:06.115496 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.115928 24947 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.116230 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2350) took 1548 ms. Trace:
I20260709 12:23:06.116458 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.113131 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.117947 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:06.120231 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.123473 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.148222 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.148510 20882 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:06.143534 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.130882 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:06.150691 21429 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:06.151207 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:06.151602 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:06.152148 21435 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.069331 25117 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:06.153126 25117 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:06.153626 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.154172 25117 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.154949 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:06.155139 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.155601 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.156255 24492 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.156342 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.156733 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:06.157428 20879 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.130280 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.085614 24557 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:23:06.125901 21434 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:06.132586 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.157938 21431 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
W20260709 12:23:06.173892 25134 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 12:23:06.148948 20731 rpcz_store.cc:276] 0709 12:23:04.568132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.568317 (+   185us) service_pool.cc:224] Handling call
0709 12:23:04.568791 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:05.511438 (+942647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.115897 (+604459us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54001d7754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:06.116158 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003023744}
I20260709 12:23:06.176476 24839 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:06.177152 24839 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.180508 24951 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:06.177958 24839 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.185114 24839 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:06.186146 20731 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:06.187991 24690 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.188066 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:06.190956 25089 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.191675 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:06.191902 21426 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:23:06.192850 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.192986 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:06.194351 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:06.194774 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.195393 24492 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:06.195500 20735 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:06.196501 25115 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:06.196713 24746 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:06.197206 24746 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:06.197598 24746 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.198172 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:06.190246 24829 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.190474 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.200035 25059 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.200148 24829 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.200908 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.201525 20735 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:06.130066 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:06.202443 24829 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:06.160949 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:06.197710 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.161355 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.161886 24914 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:06.160835 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83659.2us
I20260709 12:23:06.164518 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:06.205935 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:06.207466 25084 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.207621 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.208586 20731 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:06.209854 20728 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:06.209650 25084 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:06.210248 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.147553 24873 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.212129 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.212694 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.213066 20733 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:06.203102 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.213994 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.214768 20730 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:06.215781 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.167068 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.216964 20727 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.168401 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.219633 24926 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.165470 21433 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:23:06.216420 20726 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:06.222129 20876 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:06.222503 24926 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.223225 24926 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.214172 24914 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:06.223986 24914 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:06.223762 24926 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.224968 24914 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.226137 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:06.228220 20876 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:06.230290 20876 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:06.231410 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.232245 20728 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:06.233347 20730 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:06.233824 20802 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:06.236660 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.236838 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.235368 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.235679 20809 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:06.237493 20804 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:06.237709 20728 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:06.231441 24746 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.238785 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.239223 20879 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:06.239675 24746 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.239986 20882 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:06.205591 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:06.158552 25138 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.110s	user 0.000s	sys 0.003s
I20260709 12:23:06.242575 20876 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:06.240664 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.078946 25014 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.243840 20774 leader_election.cc:400] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:23:06.244632 20729 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:06.244830 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.245697 20728 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:06.248824 25014 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.152215 21429 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.110165 20631 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 10 to 11, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 11 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.251624 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.252490 20729 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:06.240563 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.254424 20878 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:23:06.242615 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47477.5us
I20260709 12:23:06.256453 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.240314 20731 raft_consensus.cc:2376] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7. Re-sending same reply.
I20260709 12:23:06.241227 25117 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:06.255831 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:06.257302 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.258725 20878 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:06.203114 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:06.240769 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.244871 20774 leader_election.cc:403] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:23:06.264422 20729 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:06.267470 20727 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:06.267827 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.268950 20728 raft_consensus.cc:2376] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11. Re-sending same reply.
I20260709 12:23:06.270635 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.271003 20876 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:06.271304 20729 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:06.275167 20879 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:06.252827 20631 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 5, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 5 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:06.279253 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.282076 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.287557 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.243834 24907 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:06.300483 21435 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:23:06.301185 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.305388 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.309418 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.310367 20805 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:06.307936 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.311142 21426 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:23:06.310547 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.320984 21433 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:23:06.322274 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.281034 24839 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:06.370194 20727 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.306783 24890 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.391814 20803 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:06.318688 24557 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.319394 24926 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:06.408484 24433 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:06.410295 24646 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.735s	user 0.009s	sys 0.008s
I20260709 12:23:06.411325 20729 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:06.749713 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.422708 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:06.750958 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.718086 21429 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:06.752485 21433 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:06.752869 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.752012 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.568532 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.755877 20735 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:06.756256 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.756520 25084 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.757176 20884 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:06.757251 25084 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.759624 24898 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:06.696789 24976 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.737s	user 0.007s	sys 0.002s
I20260709 12:23:06.760574 24898 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.760888 24976 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.762256 20810 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:06.764307 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2119) took 1059 ms. Trace:
W20260709 12:23:06.749293 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 493541us
I20260709 12:23:06.748821 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:06.494628 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95403.9us
W20260709 12:23:06.773123 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 451432us
I20260709 12:23:06.470968 20726 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:06.752059 20731 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:06.754419 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.775316 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.775964 20805 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:06.544761 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.392812 24911 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:23:06.318954 20631 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:06.533916 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.423424 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:06.748924 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.783977 20728 raft_consensus.cc:2376] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11. Re-sending same reply.
W20260709 12:23:06.759778 20849 connection.cc:256] RPC call timeout handler was delayed by -1.6990458229997785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:23:06.785279 20849 connection.cc:256] RPC call timeout handler was delayed by -1.539180277000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
I20260709 12:23:06.764683 20810 rpcz_store.cc:276] 0709 12:23:05.704487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.775163 (+ 70676us) service_pool.cc:224] Handling call
0709 12:23:05.811235 (+ 36072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.762659 (+951424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.764018 (+  1359us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:06.764218 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2186376064}
W20260709 12:23:06.787812 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 365212us
I20260709 12:23:06.792799 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.793720 21426 raft_consensus.cc:2368] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:23:06.794528 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.794950 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:06.802419 20849 connection.cc:256] RPC call timeout handler was delayed by -1.2071366039999702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 12:23:06.795214 20728 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:06.804029 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.494820 24926 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.822907 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:06.823320 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.825167 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.781391 25014 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:06.824388 20881 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.007s	sys 0.020s
W20260709 12:23:06.836642 20881 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.008s	sys 0.020s
I20260709 12:23:06.781180 24898 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:06.843236 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:06.803920 25108 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:06.804297 24976 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:06.849916 20727 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:06.857864 20802 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:06.422457 24433 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.471555 24890 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:06.803325 20849 connection.cc:256] RPC call timeout handler was delayed by -1.0062955369999145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 12:23:06.870939 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.860884 20631 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 6, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 6 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:06.872987 21433 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:06.871174 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:06.863556 24831 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.789s	user 0.013s	sys 0.000s
W20260709 12:23:06.884052 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48511.5us
I20260709 12:23:06.683933 21431 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Committing config change with OpId 7.8: config changed from index 3 to 8, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:06.860509 25089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.027s	sys 0.016s
I20260709 12:23:06.891755 20728 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:23:06.860839 24694 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.036s	sys 0.077s
I20260709 12:23:06.836072 24926 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.877295 24839 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.902460 20728 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:23:06.893699 25089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.028s	sys 0.016s
W20260709 12:23:06.898175 24694 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.037s	sys 0.077s
I20260709 12:23:06.909045 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.859025 25014 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.906108 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.897171 24898 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.929241 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:06.929350 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:06.930224 20877 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.931684 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:06.898993 25084 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:06.907230 20631 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 7, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 7 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:06.933940 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:06.939939 24963 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:23:06.941092 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:06.941212 24963 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:06.941864 24963 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:06.948344 20728 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:06.946486 20733 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:06.946363 20735 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:06.956048 20883 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:23:06.961687 21428 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:06.964408 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:06.903515 24976 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:23:06.927599 24898 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:06.957767 24853 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.726s	user 0.004s	sys 0.008s
I20260709 12:23:06.971068 20728 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:06.975420 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66990.3us
I20260709 12:23:06.980861 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.976573 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.979218 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:06.986256 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.987164 20808 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.983821 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.963903 24907 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.966322 24914 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:06.988945 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:06.928361 24839 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:06.935294 25059 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:23:06.991545 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:06.975415 24898 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.969807 24951 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:06.993288 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.993991 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:07.000267 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:06.994640 21434 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:06.995963 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:06.986081 20631 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 7, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 7 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:07.007588 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 98985.9us
I20260709 12:23:07.006531 20728 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:06.946817 24963 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:07.048516 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:07.006299 24805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.025s	sys 0.021s
I20260709 12:23:07.050460 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:07.050985 20806 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.050529 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:06.993078 25075 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.010s	sys 0.022s
W20260709 12:23:06.999799 24929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.010s	sys 0.044s
W20260709 12:23:07.055214 24805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.026s	sys 0.021s
I20260709 12:23:06.996033 24907 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.984411 24853 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:06.984584 24991 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 12:23:07.100152 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44495us
I20260709 12:23:07.101442 24433 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:07.071563 25075 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.011s	sys 0.022s
I20260709 12:23:07.073243 24963 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:07.103899 24926 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:07.107563 21434 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:07.008021 25084 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:07.108497 20733 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:07.119480 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.121631 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:07.122511 25089 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:07.123222 25089 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:07.123435 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:07.124670 25089 raft_consensus.cc:2764] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:23:07.125489 24853 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:07.112720 20735 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:07.075578 21428 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:23:07.055343 24929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.011s	sys 0.044s
I20260709 12:23:07.073558 25002 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:07.129752 20726 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:07.129832 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:07.137055 20805 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:07.139263 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:07.006775 24951 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.169605 24977 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.252s
W20260709 12:23:07.216193 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 160842us
W20260709 12:23:07.235958 24557 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.766s	user 0.019s	sys 0.001s
I20260709 12:23:07.236649 24557 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:23:07.238891 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:07.240196 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:07.240960 21434 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:07.215235 20631 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 8 to 9, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 9 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:07.130652 25002 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:07.245313 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 438725us
W20260709 12:23:07.253469 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 112938us
W20260709 12:23:07.250371 20732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.008s	sys 0.007s
W20260709 12:23:07.258337 20732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.008s	sys 0.007s
I20260709 12:23:07.209156 25089 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:07.251960 25151 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 12:23:07.268595 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2070) took 1735 ms. Trace:
I20260709 12:23:07.273141 20883 rpcz_store.cc:276] 0709 12:23:05.533734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.540938 (+  7204us) service_pool.cc:224] Handling call
0709 12:23:05.541531 (+   593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.961403 (+1419872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.268231 (+306828us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:07.268500 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3116107136}
I20260709 12:23:07.251395 25002 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:07.270920 20631 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:07.276823 21435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.005s	sys 0.012s
W20260709 12:23:07.298265 21435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.006s	sys 0.012s
I20260709 12:23:07.320364 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.260375 24911 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 12:23:07.336114 25159 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:23:07.336947 25152 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:23:07.277961 25014 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:07.318249 25137 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:23:07.341044 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77762.2us
I20260709 12:23:07.330091 20631 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 5 to 8, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 8 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:07.356787 25147 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
W20260709 12:23:07.358203 25150 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
I20260709 12:23:07.364244 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:07.365128 24898 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.366012 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.368983 20877 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:07.380062 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:07.295471 25089 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:07.296435 20727 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Committing config change with OpId 7.8: config changed from index 3 to 8, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:07.384990 20631 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:07.401294 21434 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:07.402683 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2115) took 1017 ms. Trace:
W20260709 12:23:07.391481 20879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.006s	sys 0.002s
W20260709 12:23:07.403427 20879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.006s	sys 0.002s
W20260709 12:23:07.403967 20734 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.003s	sys 0.007s
W20260709 12:23:07.408927 20734 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.003s	sys 0.007s
I20260709 12:23:07.412011 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2092) took 1245 ms. Trace:
I20260709 12:23:07.380525 24929 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:07.407100 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2091) took 1241 ms. Trace:
W20260709 12:23:07.410732 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 409856us
W20260709 12:23:07.391402 25108 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.006s	sys 0.002s
I20260709 12:23:07.419389 20734 rpcz_store.cc:276] 0709 12:23:06.166380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.183327 (+ 16947us) service_pool.cc:224] Handling call
0709 12:23:06.183657 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.183909 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:06.566421 (+382512us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.411484 (+845063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:07.411545 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:07.411601 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.411697 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.411949 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":781075,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"thread_start_us":1175331,"threads_started":3,"wal-append.queue_time_us":391827}
I20260709 12:23:07.403215 20730 rpcz_store.cc:276] 0709 12:23:06.385737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.386069 (+   332us) service_pool.cc:224] Handling call
0709 12:23:06.407601 (+ 21532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:06.848525 (+440924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:06.848602 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:06.877698 (+ 29096us) log.cc:844] Serialized 26 byte log entry
0709 12:23:07.295314 (+417616us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:07.295363 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.295451 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:07.402421 (+106970us) raft_consensus.cc:1685] finished
0709 12:23:07.402441 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.402621 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":389939,"raft.queue_time_us":441177,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":988,"thread_start_us":370219,"threads_started":1,"wal-append.queue_time_us":430069}
W20260709 12:23:07.405224 24646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.003s	sys 0.016s
I20260709 12:23:07.406585 24746 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:07.432560 20631 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 4 to 7, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 7 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:07.441170 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:07.411809 24492 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.447359 24839 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:07.449146 25089 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:07.434625 20732 rpcz_store.cc:276] 0709 12:23:06.165958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.169017 (+  3059us) service_pool.cc:224] Handling call
0709 12:23:06.169327 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.169611 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:06.634877 (+465266us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.406240 (+771363us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:07.406324 (+    84us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:07.406391 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.406831 (+   440us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.407035 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1364,"prepare.queue_time_us":781863,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":64,"thread_start_us":1150357,"threads_started":3,"wal-append.queue_time_us":650683}
I20260709 12:23:07.367415 24831 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:23:07.465252 20808 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:23:07.465493 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 465270us
W20260709 12:23:07.437449 25158 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
W20260709 12:23:07.482203 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 161881us
I20260709 12:23:07.406754 24898 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:07.474025 20775 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.021s	sys 0.039s
W20260709 12:23:07.494769 20775 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.021s	sys 0.041s
W20260709 12:23:07.457274 24646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.003s	sys 0.017s
W20260709 12:23:07.495091 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 667638us
I20260709 12:23:07.500362 25065 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.500944 25065 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.505113 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1091) took 1197 ms. Trace:
I20260709 12:23:07.504444 25065 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:07.495066 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.510219 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:07.512926 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:07.513566 20806 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:23:07.514149 24951 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:07.518234 25117 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.243s	user 0.003s	sys 0.007s
I20260709 12:23:07.519636 24890 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:07.520586 20631 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 5 to 7, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 7 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:07.521952 25117 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:07.438138 25108 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:07.524010 25108 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.525288 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:07.526579 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:07.505767 21435 rpcz_store.cc:276] 0709 12:23:06.307298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.307499 (+   201us) service_pool.cc:224] Handling call
0709 12:23:06.340970 (+ 33471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:06.401397 (+ 60427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:06.401444 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:06.569217 (+167773us) log.cc:844] Serialized 26 byte log entry
0709 12:23:07.298879 (+729662us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:07.298937 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.299031 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:07.504812 (+205781us) raft_consensus.cc:1685] finished
0709 12:23:07.504844 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.505031 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"thread_start_us":598323,"threads_started":1,"wal-append.queue_time_us":785937}
I20260709 12:23:07.522663 20631 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 6, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 6 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:07.439492 24929 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:07.440112 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 222962us
I20260709 12:23:07.536625 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2095) took 1259 ms. Trace:
I20260709 12:23:07.545708 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:07.546526 20805 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:07.434916 25075 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:07.526923 24963 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:07.572461 20803 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:07.550809 24898 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:07.572081 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45626us
I20260709 12:23:07.543932 20879 rpcz_store.cc:276] 0709 12:23:06.277065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.277282 (+   217us) service_pool.cc:224] Handling call
0709 12:23:06.748869 (+471587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.749133 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:06.804747 (+ 55614us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.535489 (+730742us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:07.536179 (+   690us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:07.536251 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.536356 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.536540 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47428,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":7435,"raft.queue_time_us":86647,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":25065,"spinlock_wait_cycles":49536,"thread_start_us":690045,"threads_started":2,"wal-append.queue_time_us":714100}
I20260709 12:23:07.582813 20810 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
W20260709 12:23:07.584574 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 91240.7us
W20260709 12:23:07.588502 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:07.545416 24929 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:07.631448 25157 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.512s	user 0.006s	sys 0.002s
I20260709 12:23:07.600064 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:07.593519 20631 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:23:07.634572 21431 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.000s
I20260709 12:23:07.605803 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:07.615816 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:07.636439 20877 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:07.638527 20880 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:07.640316 20882 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:07.637492 25186 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:07.634218 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:07.571549 25089 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:07.583999 24898 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.583469 24963 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:07.606335 25117 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:07.651825 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:07.648311 25173 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.002s	sys 0.002s
W20260709 12:23:07.696890 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 275881us
I20260709 12:23:07.697981 20849 leader_election.cc:400] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 6. Current term is 7.
I20260709 12:23:07.677838 20877 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:07.656711 20631 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 5 to 7, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 7 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:07.659830 20883 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:07.653831 25089 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.648872 24646 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.658457 24898 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:07.606556 25000 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 12:23:07.703004 20631 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 7 to 8, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 8 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:07.702850 25117 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:07.715286 25157 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:23:07.699107 24963 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:07.700909 25108 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:07.729184 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 123452us
I20260709 12:23:07.738035 20877 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:07.719251 25089 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.744057 20631 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 6 to 9, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 9 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:07.727341 25117 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:07.748883 20631 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 10, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 10 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:07.729123 25190 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.761252 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:07.762331 25174 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.753645 25190 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.770345 25190 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:07.775815 25176 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:07.775954 20631 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 3 to 5, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 5 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:07.777992 24907 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:07.783470 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 219799us
W20260709 12:23:07.784206 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 136063us
I20260709 12:23:07.784605 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:07.784967 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:07.785989 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.799059 24863 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:07.799541 24863 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:07.800446 24863 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.801927 25182 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:07.805051 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:07.813130 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.816491 25191 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:07.817129 25191 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:07.817529 25191 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:07.819433 24890 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:07.822073 24890 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:07.823565 25175 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.824304 25175 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.828670 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1095) took 1258 ms. Trace:
I20260709 12:23:07.829258 21431 rpcz_store.cc:276] 0709 12:23:06.569899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.570122 (+   223us) service_pool.cc:224] Handling call
0709 12:23:06.659457 (+ 89335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.683730 (+ 24273us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:07.635646 (+951916us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.827676 (+192030us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.827985 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.828095 (+   110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:07.828133 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:07.828177 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.828331 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.828585 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80159,"thread_start_us":47241,"threads_started":1}
I20260709 12:23:07.823009 24890 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:23:07.835073 25175 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:07.843582 25175 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:07.844261 25175 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:07.845754 25175 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.801460 24863 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.747978 25108 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:07.858088 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45044us
I20260709 12:23:07.748592 25002 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.813254 25176 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:07.814604 25190 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:07.868340 25156 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.660s	user 0.000s	sys 0.002s
W20260709 12:23:07.869076 20733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2372) took 1373 ms (client timeout 1557 ms). Trace:
I20260709 12:23:07.870255 20729 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:07.857669 25000 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:07.875094 20733 rpcz_store.cc:269] 0709 12:23:06.496049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.496301 (+   252us) service_pool.cc:224] Handling call
0709 12:23:07.868857 (+1372556us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002986d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:07.869013 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327885568}
W20260709 12:23:07.872717 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67734.4us
I20260709 12:23:07.880105 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:07.883667 24929 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:07.886327 24929 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:07.860857 25182 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:07.856714 25117 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:07.897454 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 110304us
I20260709 12:23:07.898213 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.892339 25000 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:07.911228 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:07.912011 25145 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.943s	user 0.000s	sys 0.004s
I20260709 12:23:07.909336 25200 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:07.915383 25089 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.923599 25174 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:07.925846 25174 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:07.923949 25002 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:07.933830 25174 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:07.924858 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:07.924609 25191 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:07.938948 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:07.953801 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62135.5us
I20260709 12:23:07.927444 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:07.907877 25000 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:07.991345 25002 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:07.993397 25191 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:07.994849 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:07.995097 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:08.013345 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 125560us
I20260709 12:23:07.995461 24963 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:07.957295 25182 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:07.994894 20803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2350) took 1606 ms (client timeout 1996 ms). Trace:
W20260709 12:23:08.019446 20803 rpcz_store.cc:269] 0709 12:23:06.388907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.966985 (+578078us) service_pool.cc:224] Handling call
0709 12:23:07.994729 (+1027744us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:07.994837 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251884288}
I20260709 12:23:07.997164 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:07.998085 25108 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:07.999488 25190 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:23:07.996485 24914 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:23:08.022866 24914 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:08.023329 24914 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:08.024421 24914 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:07.996371 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:08.006299 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67601.4us
I20260709 12:23:08.027030 25200 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.027977 25200 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.026515 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:07.997944 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:08.017654 20810 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:23:08.007153 25000 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.007512 25002 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:08.037848 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 128464us
I20260709 12:23:08.045334 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.039008 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:08.049013 21400 connection.cc:256] RPC call timeout handler was delayed by -1.0875690960001521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 12:23:08.016891 25197 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:08.038194 25084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.003s	sys 0.033s
I20260709 12:23:07.995674 24898 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.058634 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:08.064757 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:08.056950 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.062050 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2117) took 1264 ms. Trace:
I20260709 12:23:08.035393 25089 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:08.068148 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 573129us
I20260709 12:23:08.069089 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.070575 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.056197 25190 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:08.073391 24907 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:08.074146 24907 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:08.075002 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:08.075659 25190 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:08.076683 25190 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.044173 24947 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:08.079942 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2118) took 1240 ms. Trace:
I20260709 12:23:08.047825 25002 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.083850 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:08.084357 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:08.102037 20884 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:08.048890 25167 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.106443 25167 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.110870 24963 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.112772 25200 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.115486 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:08.115826 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:08.116622 25197 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:08.086933 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60544.3us
I20260709 12:23:08.117147 25197 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.117660 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:08.119350 25182 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.119791 25174 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:08.119899 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:08.060309 25117 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.120280 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:08.120615 25174 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:08.121665 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:08.121699 25174 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:08.122256 25174 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:08.122394 24898 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:08.122639 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:08.123337 21435 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:08.123631 20878 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:08.124366 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:08.123142 25174 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.124818 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:08.119922 25182 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.103869 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:08.104025 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:08.105463 20808 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:08.105684 25199 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:23:08.106716 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:08.161055 20732 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:23:08.066249 25084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.003s	sys 0.033s
I20260709 12:23:08.166107 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:08.127502 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:08.168527 20884 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:08.128727 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:08.169425 20733 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:08.062194 24890 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:23:08.137310 25197 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.139703 25174 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:08.140372 25179 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:08.140443 24746 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:08.060916 25191 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:08.125751 25117 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:08.125237 21427 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:08.086454 20729 rpcz_store.cc:276] 0709 12:23:06.839649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.863783 (+ 24134us) service_pool.cc:224] Handling call
0709 12:23:07.870012 (+1006229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:07.878953 (+  8941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:08.079722 (+200769us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54002986d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:08.079873 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270520064}
I20260709 12:23:08.063854 25108 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:08.197012 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:08.100582 24839 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:08.100144 25176 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:08.142350 24831 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:08.142446 25171 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:08.143420 25180 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:08.143893 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2355) took 1169 ms. Trace:
I20260709 12:23:08.144388 21433 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:08.145459 24863 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:08.147487 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2130) took 1860 ms. Trace:
I20260709 12:23:08.147981 25014 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:08.149096 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:08.149977 25002 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.150380 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.179250 25188 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:23:08.151690 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:08.152689 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 82210.4us
I20260709 12:23:08.116223 21434 raft_consensus.cc:2368] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:23:08.167915 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:08.168563 20726 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:08.086326 24947 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:08.065479 20731 rpcz_store.cc:276] 0709 12:23:06.797690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.844392 (+ 46702us) service_pool.cc:224] Handling call
0709 12:23:06.844884 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:06.895306 (+ 50422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:06.895362 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:06.906816 (+ 11454us) log.cc:844] Serialized 26 byte log entry
0709 12:23:07.277357 (+370541us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:07.333214 (+ 55857us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.945220 (+612006us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b1800487720. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:23:08.061460 (+116240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:08.061576 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:08.061643 (+    67us) raft_consensus.cc:1685] finished
0709 12:23:08.061773 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:08.061968 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":669823,"prepare.queue_time_us":57675,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":5373,"raft.queue_time_us":465772,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9656,"spinlock_wait_cycles":3598208,"thread_start_us":379937,"threads_started":2,"wal-append.queue_time_us":472427}
I20260709 12:23:08.180341 24646 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:23:08.180677 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.184671 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:08.186749 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:08.190099 20735 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:08.190359 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:08.190820 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:08.191145 21435 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:08.191777 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:08.193312 20878 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:23:08.191932 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:08.193900 21400 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.515s (SENT)
I20260709 12:23:08.197677 21431 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:08.167285 25084 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:08.208841 21426 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:08.213176 25171 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:08.245229 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.247094 21430 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:08.267170 25183 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.276180 25183 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.278337 25002 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:08.267880 24947 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:08.282363 21430 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:08.225199 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:08.151113 25167 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.183838 25210 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:08.268558 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88028us
I20260709 12:23:08.268867 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:08.269105 20879 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:08.285089 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.285841 25000 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.287503 25089 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:08.287992 25000 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.288666 24907 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:08.289281 24907 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:08.289532 25089 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:08.290571 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.291427 20806 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:23:08.269524 24646 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:23:08.269552 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.293689 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:08.292269 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.275332 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:08.258508 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:08.257268 25162 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.263952 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:08.266012 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.199513 24898 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:08.192493 24951 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:23:08.294413 25210 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:08.271811 21429 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:23:08.272387 20880 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:08.272858 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 81011.6us
I20260709 12:23:08.286695 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.180063 20875 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:08.192272 25065 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 12:23:08.273454 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 79585.4us
I20260709 12:23:08.315554 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:08.316355 20808 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
W20260709 12:23:08.316280 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 770970us
I20260709 12:23:08.237478 20801 rpcz_store.cc:276] 0709 12:23:06.287603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.304912 (+ 17309us) service_pool.cc:224] Handling call
0709 12:23:06.775383 (+470471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:06.802173 (+ 26790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:06.802218 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:06.857944 (+ 55726us) log.cc:844] Serialized 26 byte log entry
0709 12:23:07.250117 (+392173us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:07.250159 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.250242 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:08.146572 (+896330us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b24001686c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf0fb0 00007f9e1e867488 00007f9e1e867be8 00007f9e1e83f398 00007f9e1e83a76a 00007f9e1e839bb6
0709 12:23:08.147149 (+   577us) raft_consensus.cc:1685] finished
0709 12:23:08.147170 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:08.147424 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9434,"raft.queue_time_us":17092,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":12049,"spinlock_wait_cycles":26854528,"thread_start_us":346433,"threads_started":1,"wal-append.queue_time_us":456017}
I20260709 12:23:08.196508 25200 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 12:23:08.201017 25108 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:08.305156 20802 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
W20260709 12:23:08.305757 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 110770us
I20260709 12:23:08.308902 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.258708 20732 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:08.313350 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:08.314028 21427 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:08.296839 20879 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:08.325549 20735 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:08.216006 20804 rpcz_store.cc:276] 0709 12:23:06.972312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.979300 (+  6988us) service_pool.cc:224] Handling call
0709 12:23:08.126636 (+1147336us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540027b354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:08.141670 (+ 15034us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536157952}
I20260709 12:23:08.274464 25171 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.308732 24898 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:08.330870 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.334908 20804 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:23:08.279045 25183 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:08.328078 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:08.314415 25167 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:08.252414 25014 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.367746 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.337450 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:08.337970 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72248.8us
I20260709 12:23:08.338196 21400 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
I20260709 12:23:08.428112 20808 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:08.372576 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.428094 21400 leader_election.cc:403] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:23:08.390203 20806 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:23:08.394177 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107570us
I20260709 12:23:08.431154 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.433472 20801 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:08.430337 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 93849us
I20260709 12:23:08.433984 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:08.316615 25176 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:08.323449 25180 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:08.330286 25179 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.335079 25188 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:08.329633 25108 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.398425 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.444639 20803 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:08.274542 25084 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.316448 25162 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.408097 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:08.447764 20807 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.399845 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:08.448866 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49155.5us
W20260709 12:23:08.410977 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 220949us
I20260709 12:23:08.411396 25200 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:08.310760 24805 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:23:08.395944 24890 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:08.401847 25210 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:08.421631 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:08.433084 25175 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.427776 20802 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:08.429970 25183 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:08.466574 25162 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:08.426523 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 98593.7us
I20260709 12:23:08.491176 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:08.481585 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:08.454958 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:08.443835 24907 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:08.438166 25167 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:08.447052 25089 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:08.501525 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:08.505793 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:08.456465 25182 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.489866 25183 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.533035 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.570779 20735 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:23:08.588945 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 81806us
I20260709 12:23:08.467142 25200 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.605778 20807 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:23:08.632929 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 137192us
I20260709 12:23:08.687749 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.687976 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.688545 20802 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:08.688774 20807 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:08.487430 24890 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.642469 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:08.643208 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 161687us
I20260709 12:23:08.467952 25179 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.684367 20621 maintenance_manager.cc:643] P 9f77a9352772433bb5809bf72fd4f0ee: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.156s	user 0.225s	sys 0.052s Metrics: {"bytes_written":30565,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":226285,"dirs.run_cpu_time_us":575,"dirs.run_wall_time_us":12174,"drs_written":1,"lbm_read_time_us":59592,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":1,"lbm_write_time_us":87717,"lbm_writes_10-100_ms":3,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":187218,"threads_started":1,"update_count":23}
I20260709 12:23:08.427572 24898 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.507314 25162 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:08.639173 24831 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:08.591161 25183 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.508378 25213 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:08.688710 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:08.640304 25089 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:08.508299 25163 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:08.508109 25216 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.264s
I20260709 12:23:08.520851 24839 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:08.508280 25223 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:08.732971 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44322.2us
I20260709 12:23:08.745091 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:08.590299 25190 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.131s
W20260709 12:23:08.737977 20775 leader_election.cc:336] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 2.442s (SENT)
I20260709 12:23:08.508843 25214 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.224s
I20260709 12:23:08.643718 25225 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
I20260709 12:23:08.644200 24951 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:23:08.753234 20776 leader_election.cc:400] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9.
I20260709 12:23:08.737267 25002 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:08.749547 25163 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.755263 25002 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:08.753667 25216 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:08.755873 25002 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:08.757445 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:08.746726 24839 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.746595 25223 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.776691 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:08.777640 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:08.782099 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:08.779223 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:08.772344 25089 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.740283 25167 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:08.788813 20622 maintenance_manager.cc:419] P 9f77a9352772433bb5809bf72fd4f0ee: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8847 bytes on disk
I20260709 12:23:08.761487 24831 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.763034 24839 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:08.796610 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:08.775599 25239 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.757301 25232 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:08.824291 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:08.823657 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141320us
W20260709 12:23:08.795591 25174 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.607s	user 0.000s	sys 0.010s
I20260709 12:23:08.826467 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2205) took 2686 ms. Trace:
I20260709 12:23:08.763255 25223 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:08.789937 25230 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:08.821485 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44986.5us
I20260709 12:23:08.793817 24963 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:08.830524 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:08.790140 24929 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:08.833250 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:08.832805 20881 rpcz_store.cc:276] 0709 12:23:06.139661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.139883 (+   222us) service_pool.cc:224] Handling call
0709 12:23:06.140376 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:06.140959 (+   583us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:06.141299 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 12:23:06.837295 (+695996us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:06.858736 (+ 21441us) log.cc:844] Serialized 26 byte log entry
0709 12:23:06.858928 (+   192us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:06.859244 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:23:06.859569 (+   325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.859655 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:08.727832 (+1868177us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b2400162cc8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf0fb0 00007f9e1e867488 00007f9e1e867be8 00007f9e1e83f398 00007f9e1e83a76a 00007f9e1e839bb6
0709 12:23:08.826233 (+ 98401us) raft_consensus.cc:1685] finished
0709 12:23:08.826265 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:08.826405 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":395,"prepare.queue_time_us":288,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":10228,"raft.queue_time_us":297,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":17482,"spinlock_wait_cycles":182123392,"thread_start_us":695453,"threads_started":1,"wal-append.queue_time_us":858958}
I20260709 12:23:08.808880 25213 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:08.782837 24492 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 12:23:08.843425 25222 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 12:23:08.840456 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:08.847750 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:08.849004 25224 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 12:23:08.859215 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:08.865948 20802 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:08.867684 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:08.839303 25232 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:08.844643 25014 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:08.885757 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.859570 24963 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:08.869326 25174 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:08.891628 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51238.2us
I20260709 12:23:08.846693 24929 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.836474 25239 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:08.877588 24492 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:08.877345 25216 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:08.966984 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70709.8us
I20260709 12:23:08.967891 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.968053 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2121) took 2071 ms. Trace:
I20260709 12:23:08.849238 25243 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:23:08.968541 20727 rpcz_store.cc:276] 0709 12:23:06.896646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.901189 (+  4543us) service_pool.cc:224] Handling call
0709 12:23:06.911859 (+ 10670us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:07.273917 (+362058us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:07.561655 (+287738us) log.cc:844] Serialized 14 byte log entry
0709 12:23:08.571484 (+1009829us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:08.590294 (+ 18810us) log.cc:844] Serialized 52 byte log entry
0709 12:23:08.590481 (+   187us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:08.760565 (+170084us) log.cc:844] Serialized 14 byte log entry
0709 12:23:08.778366 (+ 17801us) log.cc:844] Serialized 14 byte log entry
0709 12:23:08.966934 (+188568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:08.967026 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:08.967828 (+   802us) raft_consensus.cc:1685] finished
0709 12:23:08.967844 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:08.967993 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":519848,"prepare.queue_time_us":157045,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":2228,"raft.queue_time_us":1552949,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":4970,"thread_start_us":1112186,"threads_started":5,"wal-append.queue_time_us":119938}
I20260709 12:23:08.977249 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:08.978188 25244 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:08.895390 25174 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:08.978904 25229 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:23:08.979478 25228 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:23:08.979499 25238 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:23:08.980113 25218 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 12:23:08.980594 25246 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:08.894202 25227 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:08.992995 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68471.7us
I20260709 12:23:09.006821 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:09.008366 25227 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.859746 25230 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.001042 20621 maintenance_manager.cc:643] P 9f77a9352772433bb5809bf72fd4f0ee: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.168s	user 0.012s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":427,"lbm_reads_lt_1ms":4}
I20260709 12:23:08.965471 24492 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:08.997100 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111446us
I20260709 12:23:09.008900 25227 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:08.999667 25174 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:09.021154 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:08.988473 25216 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:09.003669 25190 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:09.005716 25014 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:09.007081 25246 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.009560 25248 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:09.043771 25216 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:09.055748 25175 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
I20260709 12:23:09.043776 24963 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.029084 25190 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.025933 25238 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:09.030025 25233 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:09.069916 20875 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:09.072997 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51990.5us
W20260709 12:23:09.064072 25171 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.720s	user 0.001s	sys 0.012s
I20260709 12:23:09.077242 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.078138 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.040978 25237 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:23:09.076056 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60411.6us
I20260709 12:23:09.082980 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:09.081313 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:09.084465 20879 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:09.034560 25250 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:09.092288 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.094345 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.095427 20731 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:09.096841 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.097766 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:09.098656 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.004179 25244 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.039355 25246 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.102675 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.104261 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.104492 20733 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:09.105015 20726 raft_consensus.cc:2376] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9. Re-sending same reply.
I20260709 12:23:09.071859 25190 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:09.103667 20731 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:09.102052 20735 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:09.112877 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.081094 25243 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:09.003232 25213 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.082787 25239 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.039710 25251 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:23:09.125820 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.126528 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:09.072482 25233 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:09.132936 24839 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:09.133885 24947 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.854s	user 0.010s	sys 0.003s
I20260709 12:23:09.134292 24947 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.134065 25233 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.135550 25233 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:09.136008 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:09.136270 20809 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:09.078540 25248 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:09.050618 25232 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.083007 25171 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.137775 25248 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.126037 25237 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.139237 25237 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.120474 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.141453 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:09.142163 21427 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:09.155203 20804 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:09.126819 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.107601 25238 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:09.129695 25002 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:09.131275 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.116175 25259 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:23:09.074100 25254 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 12:23:09.136940 25108 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.001s
I20260709 12:23:09.106029 24492 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.116693 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.158324 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:09.159153 20731 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:09.160041 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.171059 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.163872 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:09.176633 21430 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:09.170082 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.177908 20735 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 6.
I20260709 12:23:09.173269 21428 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:23:09.199702 25176 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.200397 25176 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:09.210453 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.217658 25117 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:09.218309 25117 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:09.221467 25179 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.229892 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:09.230618 20810 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
W20260709 12:23:09.232693 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 902001us
I20260709 12:23:09.233472 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:09.234293 20808 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:09.235371 25214 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:09.235957 25214 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:09.236845 25214 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:09.239156 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.239522 25182 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.240044 25182 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.241881 25199 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:09.242185 25182 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.242489 25199 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.153126 25250 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.246901 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:09.232782 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:09.247840 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.242709 25182 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:09.248529 25117 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.249426 25244 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.253762 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.171540 25251 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.255590 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.267390 25251 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.267670 25255 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.268570 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.268483 25255 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.268934 25217 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:23:09.269431 20726 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:23:09.269594 25251 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.270437 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.270541 20726 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:09.270609 25255 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.231025 24947 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.267746 25230 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.267673 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.277801 21435 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:09.278730 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.247393 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2383) took 1065 ms. Trace:
W20260709 12:23:09.286098 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53401.3us
I20260709 12:23:09.285990 20730 rpcz_store.cc:276] 0709 12:23:08.181654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:08.181982 (+   328us) service_pool.cc:224] Handling call
0709 12:23:08.194630 (+ 12648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:08.293986 (+ 99356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:08.294059 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:08.294535 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:23:08.486208 (+191673us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:08.506504 (+ 20296us) log.cc:844] Serialized 14 byte log entry
0709 12:23:08.506794 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.246988 (+740194us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54001d8b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:09.247037 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:09.247086 (+    49us) raft_consensus.cc:1685] finished
0709 12:23:09.247177 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.247331 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20070,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":5490,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":59676,"spinlock_wait_cycles":144718720,"thread_start_us":191306,"threads_started":1,"wal-append.queue_time_us":390136}
I20260709 12:23:09.288975 20733 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:09.290683 20728 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:09.161749 25239 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:23:09.293865 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.300153 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:09.172925 25108 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.303702 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.312335 25261 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:09.316841 25261 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.317498 25261 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:09.311497 25251 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:09.297032 25264 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.172753 25238 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.320554 25264 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.321552 25246 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.322075 25238 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.322883 25264 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.302660 25167 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 12:23:09.324029 25183 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.586s	user 0.005s	sys 0.005s
I20260709 12:23:09.324123 25002 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.289331 24839 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.324836 25167 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.324594 25183 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.325322 25237 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.326161 25199 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.326730 25183 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.329632 25167 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.288403 24907 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.331518 24907 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.333680 24907 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.334972 25167 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.335573 25264 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.336323 25264 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.338877 25264 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.289374 24694 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.339794 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.322803 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.344883 20726 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:09.346241 25188 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.864s	user 0.006s	sys 0.007s
I20260709 12:23:09.346797 25188 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:09.346361 25261 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.351151 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:09.356434 20731 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:09.295789 24746 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:09.363384 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66981.9us
I20260709 12:23:09.304389 24492 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:09.372709 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:09.360771 25180 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.881s	user 0.012s	sys 0.000s
I20260709 12:23:09.397433 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.378062 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.358598 25117 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:09.340698 25246 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.408059 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:09.362179 25243 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.372682 25216 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.424214 25216 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.426434 25216 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.360400 24694 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:09.431716 25223 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.592s	user 0.005s	sys 0.006s
I20260709 12:23:09.432185 25223 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.432898 25108 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.433104 24694 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.410187 25180 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:09.397778 24746 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.433970 24746 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:09.434296 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.434506 25223 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.435525 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.436123 20730 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:09.436271 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:09.436884 21431 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:09.436976 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.438149 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.396919 25014 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:09.289177 25250 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.441053 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2381) took 1277 ms. Trace:
I20260709 12:23:09.441147 25250 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.341599 25176 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.443327 25176 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.445217 21429 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13.
I20260709 12:23:09.398291 25244 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.447621 25244 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.441519 20734 rpcz_store.cc:276] 0709 12:23:08.163859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:08.164197 (+   338us) service_pool.cc:224] Handling call
0709 12:23:08.187507 (+ 23310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:08.317625 (+130118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:08.317661 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:08.329463 (+ 11802us) log.cc:844] Serialized 26 byte log entry
0709 12:23:08.505539 (+176076us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:08.753577 (+248038us) log.cc:844] Serialized 14 byte log entry
0709 12:23:08.770811 (+ 17234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.440704 (+669893us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540028cb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:09.440752 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:09.440795 (+    43us) raft_consensus.cc:1685] finished
0709 12:23:09.440870 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.441000 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50744,"prepare.queue_time_us":214993,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":48256,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":11690,"spinlock_wait_cycles":247154048,"thread_start_us":256484,"threads_started":3,"wal-append.queue_time_us":270058}
I20260709 12:23:09.431641 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.449306 20734 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:09.450431 25243 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.451033 25162 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:09.451516 25162 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:09.452354 25162 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.452859 25243 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.453755 25162 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:09.449899 24890 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.450026 25244 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.459074 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:09.459401 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:09.460067 20801 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:09.460963 25171 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.462304 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2359) took 1358 ms. Trace:
I20260709 12:23:09.339144 25174 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.462683 20809 rpcz_store.cc:276] 0709 12:23:08.104173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:08.105963 (+  1790us) service_pool.cc:224] Handling call
0709 12:23:08.106272 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.136634 (+1030362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.462088 (+325454us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002b6bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:09.462246 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2368302976}
I20260709 12:23:09.464203 20726 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:09.464900 25174 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.466109 24746 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.466866 24914 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:09.484917 25227 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.486243 20731 raft_consensus.cc:2368] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
I20260709 12:23:09.467275 20735 raft_consensus.cc:2368] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
I20260709 12:23:09.467706 25185 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:09.468310 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:09.469022 24929 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.469038 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:09.487716 25185 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:09.488183 20809 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:09.488471 25261 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.488689 25185 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.489848 24863 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:09.469817 25200 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.470916 24898 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:09.477069 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:09.478163 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.491228 20730 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.491488 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:09.491537 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:09.493502 24929 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:09.493904 24929 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.481253 25174 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:23:09.494390 24929 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.495249 25204 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:09.495661 25204 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:09.496598 25204 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.497666 25204 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:09.501575 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.501832 24929 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.502748 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:09.505244 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:09.480970 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.407538 25167 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:09.508129 20802 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.000s	sys 0.018s
W20260709 12:23:09.509171 20802 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.000s	sys 0.019s
I20260709 12:23:09.512686 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:09.482810 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:09.482566 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:09.479977 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:09.514367 20804 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:09.503226 25176 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.503485 25216 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:09.518280 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 446646us
I20260709 12:23:09.512781 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:09.519122 20803 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:09.517288 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:09.482523 20726 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:09.507030 20732 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:09.478744 21431 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:09.536775 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:09.478936 24746 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.543133 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:09.543454 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:09.551952 20803 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:09.553166 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:09.511871 25200 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:09.559420 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:09.513803 24890 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:09.557668 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.019s	sys 0.031s
I20260709 12:23:09.510219 25108 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:09.579722 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.580344 20731 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.583305 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:09.562934 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:09.514230 25227 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:09.567514 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.019s	sys 0.031s
W20260709 12:23:09.586030 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 700705us
I20260709 12:23:09.570503 20801 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:09.540959 25199 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.605401 20730 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:09.543076 25182 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:09.505942 25190 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.557438 25217 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:09.616122 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:09.560701 25248 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:09.511443 25167 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.637156 24746 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:09.637465 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:09.580878 25014 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:09.576097 25108 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.559887 25200 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.581961 24890 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.540784 25176 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.542796 25243 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:23:09.577580 25216 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:09.657653 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:09.649592 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 136952us
I20260709 12:23:09.724673 20730 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:09.724834 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:09.664567 25200 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:09.677227 24890 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:09.585958 25238 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:23:09.652423 25248 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.694344 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:09.582151 25264 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 12:23:09.714329 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76954us
I20260709 12:23:09.588586 25213 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:23:09.654528 25182 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:09.663255 25233 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.003s
I20260709 12:23:09.662569 25108 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:09.637941 25162 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.582180 25180 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:09.662905 25217 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:09.638361 25199 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.665220 24746 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:09.666476 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 103637us
I20260709 12:23:09.771373 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:09.678828 24831 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:09.740895 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:09.714840 25259 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.723938 25174 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:09.777851 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:09.739965 25243 consensus_queue.cc:1243] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:09.756515 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.793797 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:09.757182 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.745381 25175 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:09.746567 25248 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.748945 24907 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.799007 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:09.799634 20877 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:09.745355 24929 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.779028 25002 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:23:09.780500 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:09.780651 25197 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:09.780925 25247 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.201s
I20260709 12:23:09.781564 25242 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:09.784085 25270 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:23:09.755852 25180 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.790496 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:09.794011 20730 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:09.794925 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:09.795502 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:09.858028 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:23:09.797410 20726 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:23:09.801759 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:09.801833 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:09.860939 21433 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:09.814174 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:09.787384 25233 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.817050 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:09.865175 21428 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:09.813058 24907 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.793264 25259 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:09.841121 25277 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:23:09.842864 25266 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:23:09.800092 25216 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.823675 24746 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.796051 25232 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:09.851738 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2144) took 1053 ms. Trace:
I20260709 12:23:09.796411 25000 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:09.797135 25175 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.764052 25162 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.886560 20802 rpcz_store.cc:276] 0709 12:23:08.797954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:08.882940 (+ 84986us) service_pool.cc:224] Handling call
0709 12:23:08.883266 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:08.883480 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:09.510544 (+627064us) log.cc:844] Serialized 14 byte log entry
0709 12:23:09.851252 (+340708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:09.851327 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:09.851398 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.851493 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.851664 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120889,"prepare.queue_time_us":410389,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":104481,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":3911,"spinlock_wait_cycles":100035328,"thread_start_us":863624,"threads_started":2,"wal-append.queue_time_us":286752}
I20260709 12:23:09.831627 25217 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.864648 20730 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:09.868256 25247 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:09.869028 25271 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:23:09.869148 25199 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.869496 25176 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:09.962071 25285 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 12:23:09.870106 25281 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:23:09.870663 25182 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.965348 25288 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.965961 25288 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.968278 25288 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:09.969558 24492 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.001s
I20260709 12:23:09.971045 20734 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:09.975052 20729 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:09.983529 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1107) took 1869 ms. Trace:
I20260709 12:23:09.983801 24805 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.983943 21432 rpcz_store.cc:276] 0709 12:23:08.114504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:08.121399 (+  6895us) service_pool.cc:224] Handling call
0709 12:23:09.983293 (+1861894us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:09.983437 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967442176}
I20260709 12:23:09.984397 24805 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.986652 24805 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.986779 25254 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:09.987438 25254 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.987859 25002 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.988525 25002 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.873042 20875 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:09.997361 25254 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.874977 25233 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.001881 25233 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:09.876667 24898 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.005447 25275 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:10.005931 25275 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:10.008432 25282 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:23:10.008718 25014 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:10.009058 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:10.010396 25254 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:10.010886 25254 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:09.890913 25274 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 12:23:09.891471 25276 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:23:09.895447 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:09.871887 25227 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:09.865114 25180 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:10.022289 25254 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.023032 20880 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:10.023031 25227 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:09.991060 25002 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.855489 25242 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:09.900172 25247 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.895500 25175 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:09.894187 25000 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:09.905445 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.060359 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:09.873088 25190 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:09.953020 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:10.089640 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 136704us
I20260709 12:23:10.090536 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:10.091276 24898 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:10.091357 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:10.091722 24898 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:10.092142 24898 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:09.905404 25197 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.093003 25197 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:09.954427 25259 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.095611 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.096262 20875 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:09.884527 25167 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:09.814960 25174 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.012161 25108 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:10.012778 25283 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
I20260709 12:23:10.117725 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.118990 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.006965 25275 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.122946 20801 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:23:10.033595 20732 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.002s
I20260709 12:23:10.032274 20877 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:10.064122 25233 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:10.066146 20880 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:10.069380 25261 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:10.070194 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.077365 20878 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:10.077510 20884 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:10.080120 20882 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:10.137108 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:10.081946 25204 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.000s
I20260709 12:23:10.081996 25162 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.087931 21428 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:10.095131 25197 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.096908 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 625) took 1020 ms. Trace:
I20260709 12:23:10.098503 20730 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:10.098862 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.097434 25174 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.159562 25174 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.160068 25174 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.160506 25174 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.160843 24646 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.161285 25248 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.161547 24646 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.113921 25288 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:10.163389 25288 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:10.163792 25288 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:10.120100 25084 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:10.114025 25000 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.115764 25247 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.117614 25108 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:10.109536 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:10.122224 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.124310 20732 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:10.132545 20729 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:10.058465 25242 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:10.127381 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:10.138769 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.139904 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.213860 20726 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:10.214002 20802 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:10.096364 25200 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.086879 25180 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.151667 21433 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:10.152837 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.221549 20881 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13.
I20260709 12:23:10.152930 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.153496 20727 rpcz_store.cc:276] 0709 12:23:09.076781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.076960 (+   179us) service_pool.cc:224] Handling call
0709 12:23:10.096658 (+1019698us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54000c44d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:10.096853 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006955904}
I20260709 12:23:10.123814 25272 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.155841 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.153823 21426 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
W20260709 12:23:10.176389 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:10.228608 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 119161us
I20260709 12:23:10.229696 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.176545 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.209050 20879 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:10.215047 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87755.9us
I20260709 12:23:10.232296 20776 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13.
I20260709 12:23:10.219560 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.222613 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:10.232638 20776 leader_election.cc:403] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:23:10.233189 20732 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:10.233527 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.234941 20732 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:23:10.212543 25300 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:23:10.174470 24907 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.174748 25084 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.244617 20733 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:10.245221 21429 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:10.142220 25233 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.224460 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:10.240506 20809 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:10.242995 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.243503 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:10.244073 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:10.244563 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.238582 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.302711 20802 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:10.342790 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.343780 20884 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:10.307785 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:10.235329 20775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.567s (SENT) [suppressed 30 similar messages]
W20260709 12:23:10.344559 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 564081us
I20260709 12:23:10.308139 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.345409 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.306957 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.347811 20810 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:10.231679 25226 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:10.150597 25261 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.349990 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.354800 20808 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:10.306795 25182 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:10.361047 25176 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:10.364962 20806 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:23:10.366539 25200 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:10.244163 25108 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:10.233392 24898 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.246025 25248 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.344751 20882 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:23:10.236738 25247 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:10.403254 25247 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.404161 25233 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:10.405637 25247 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:10.392771 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:10.393325 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 158802us
I20260709 12:23:10.237533 25272 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.246663 25084 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:10.408711 20809 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:10.302819 25204 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:10.391467 25226 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.358623 25175 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.413681 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.303648 25227 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.422405 20884 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
W20260709 12:23:10.427021 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 188507us
I20260709 12:23:10.392402 25199 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:10.430572 20809 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:10.503916 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.501180 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2149) took 1026 ms. Trace:
I20260709 12:23:10.504894 20733 rpcz_store.cc:276] 0709 12:23:09.474733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.639043 (+164310us) service_pool.cc:224] Handling call
0709 12:23:10.500899 (+861856us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400298bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:10.501099 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890627456}
I20260709 12:23:10.503960 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.501818 20735 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:10.439801 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:10.507474 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 99405.8us
I20260709 12:23:10.508186 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:10.506074 20810 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:10.509109 20806 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:10.461915 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:10.510871 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 275550us
I20260709 12:23:10.513437 20882 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:10.467761 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.516860 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.429620 25176 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:10.425086 25188 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:10.523929 20849 leader_election.cc:400] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
I20260709 12:23:10.524394 20849 leader_election.cc:403] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:10.425844 25226 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:10.518647 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 125948us
I20260709 12:23:10.526057 25246 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:10.526701 25299 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.527061 25234 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:10.527474 25260 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:23:10.426772 25204 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.529727 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2398) took 1187 ms. Trace:
I20260709 12:23:10.492748 25174 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.512367 24863 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:23:10.514674 25233 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:10.627209 20881 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:23:10.630717 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.598590 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.630772 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.531909 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:10.633143 20881 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:10.411559 25182 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.549810 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.551983 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.638401 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.522086 20884 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:10.558800 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:10.618631 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.636000 20878 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:10.571353 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.636688 20731 rpcz_store.cc:276] 0709 12:23:09.342739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.968505 (+625766us) service_pool.cc:224] Handling call
0709 12:23:09.968823 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:09.969192 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.969235 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:09.969733 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:23:10.239152 (+269419us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:10.239212 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.239287 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:10.529500 (+290213us) raft_consensus.cc:1685] finished
0709 12:23:10.529525 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.529666 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1031,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":269049,"threads_started":1,"wal-append.queue_time_us":541717}
W20260709 12:23:10.639710 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81015.4us
I20260709 12:23:10.539662 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:10.641589 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109717us
I20260709 12:23:10.642460 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.642580 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:10.548092 25247 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:23:10.644124 21426 raft_consensus.cc:2368] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
I20260709 12:23:10.648727 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.594542 25190 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.644304 20881 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:10.577781 25237 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:10.627656 25180 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.594535 25162 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:10.576100 25261 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.524032 24646 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.664408 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.659009 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.579048 25175 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:10.672016 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.577860 24947 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.664575 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.542093 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.667838 21429 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:23:10.578161 25199 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:23:10.670061 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.701028 20882 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
W20260709 12:23:10.653070 24890 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.000s
I20260709 12:23:10.709322 25163 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.709041 24890 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.695708 20876 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:10.672921 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.574156 25312 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:23:10.532434 25304 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:10.655308 25264 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:10.709175 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.653738 24907 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:10.712009 20879 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:10.668790 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.653312 25185 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:10.700788 25162 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.725910 25162 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.726106 25199 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:10.643963 25311 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:10.727880 20879 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:10.663686 25299 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:10.697633 25308 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.121s	user 0.001s	sys 0.001s
I20260709 12:23:10.666857 25237 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:10.721580 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.547727 25188 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.731328 20881 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:10.699028 25248 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.729362 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.698787 24898 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.671305 25246 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:23:10.759476 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.759819 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.720724 25304 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.743518 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.718192 24646 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.746209 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.763324 20882 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:23:10.741896 25311 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:10.730621 25264 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:10.736337 24907 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:10.788671 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.747630 25237 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.757323 25188 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.766906 20884 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:10.771449 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.711529 24947 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.727923 25199 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:10.665989 25190 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:10.798930 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 130183us
I20260709 12:23:10.799041 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.799634 20884 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:10.800657 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:10.800997 21431 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:10.801604 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:10.767611 25304 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:10.809644 20808 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:10.794445 25272 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.821069 20879 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:10.768168 25248 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.772645 24646 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.802680 25237 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.827272 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.766219 25299 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.828629 21431 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:10.829756 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.806399 25188 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:10.817821 25246 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:10.840715 25311 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.846928 25108 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:10.827265 25297 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:23:10.842039 25182 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.841904 24907 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:10.845600 25242 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.000s
I20260709 12:23:10.847855 24646 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.848341 25204 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.848534 25248 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.851497 25199 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.890139 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2147) took 1416 ms. Trace:
I20260709 12:23:10.891011 20810 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:10.853955 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.853968 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.854637 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.855443 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:10.864109 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:10.822150 24890 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.870689 20850 leader_election.cc:400] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
I20260709 12:23:10.874027 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:10.875020 21400 connection.cc:256] RPC call timeout handler was delayed by -1.1635464910000337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
I20260709 12:23:10.875818 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:10.880863 21433 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:23:10.881347 24947 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:10.882382 25313 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.906810 21429 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:10.871860 25242 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.888810 20805 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:10.892023 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.858378 25246 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:10.897374 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:10.902100 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48205.5us
I20260709 12:23:10.917114 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.920161 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.890537 20734 rpcz_store.cc:276] 0709 12:23:09.474214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.971839 (+497625us) service_pool.cc:224] Handling call
0709 12:23:09.972126 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:09.972519 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.972549 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:09.972983 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:23:10.177174 (+204191us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:10.177246 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.177336 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:10.889898 (+712562us) raft_consensus.cc:1685] finished
0709 12:23:10.889918 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.890067 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13333,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":203777,"threads_started":1,"wal-append.queue_time_us":680413}
I20260709 12:23:10.907271 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:10.907934 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:10.909338 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.922385 21429 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:10.832715 25264 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.913667 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.927747 20884 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
W20260709 12:23:10.914319 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96744.3us
I20260709 12:23:10.851550 25315 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:23:10.924592 21429 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:10.929009 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.938598 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:10.939280 21432 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:10.939405 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:10.895008 24646 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:10.936779 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:10.943950 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:10.944617 20729 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:10.945147 25322 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:23:10.946035 25217 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:10.946487 25217 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:10.912297 25248 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.949350 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:10.951840 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.954633 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.934276 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:10.941565 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:10.925631 24947 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:10.922909 25199 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:10.959084 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:10.969822 20731 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:10.961319 20807 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:23:10.940372 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:10.926674 25014 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.923720 25246 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:10.938542 25175 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:10.968111 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:10.937312 25321 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:10.938134 25210 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:10.962296 24890 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:10.980818 25324 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:10.981060 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73216.4us
I20260709 12:23:10.947472 25217 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:10.984022 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 43724.6us
I20260709 12:23:10.975333 25188 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:10.986018 21432 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:10.984112 25331 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:11.012259 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 112994us
I20260709 12:23:10.948998 25162 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:10.992650 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:10.964709 25220 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:11.039734 25220 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:11.040871 25220 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.048082 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2142) took 1775 ms. Trace:
I20260709 12:23:11.048640 20726 rpcz_store.cc:276] 0709 12:23:09.273128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.883622 (+610494us) service_pool.cc:224] Handling call
0709 12:23:10.213634 (+330012us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:10.214253 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.047821 (+833568us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54002e4fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.048018 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757886336}
I20260709 12:23:11.052405 25325 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.053165 25325 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:10.996122 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:11.055651 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 59591.9us
I20260709 12:23:11.055810 25188 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.056437 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:10.923429 25313 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.981088 25327 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:11.058706 25313 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.001421 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:11.059799 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58485.1us
I20260709 12:23:11.030256 25226 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:10.979113 25197 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.006361 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.013957 25190 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:11.015058 25316 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:23:10.968976 25180 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:23:11.016455 25242 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.020012 25289 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:23:10.956604 24646 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.031813 24890 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:10.988266 24898 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:23:11.000648 25321 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.036298 25299 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:10.992149 25304 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.037580 24947 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:10.994553 24746 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.990952 25175 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:11.005959 25288 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.842s	user 0.009s	sys 0.004s
I20260709 12:23:11.014010 25210 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:11.039155 25162 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.052166 25117 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:11.054862 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2397) took 1712 ms. Trace:
I20260709 12:23:10.948527 25250 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.060461 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:10.923532 25182 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:23:11.061410 25163 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.116629 25162 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.060947 25331 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.063795 25014 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:11.065662 25315 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:11.066208 25254 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.067801 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.069984 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:11.125443 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 176162us
I20260709 12:23:11.110728 24898 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:11.059103 25327 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:11.105603 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.077733 25313 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:23:11.116943 20735 rpcz_store.cc:276] 0709 12:23:09.342480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.959084 (+616604us) service_pool.cc:224] Handling call
0709 12:23:10.501474 (+542390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:10.506304 (+  4830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.054594 (+548290us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b5400298bd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.054781 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246367488}
W20260709 12:23:11.120738 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60330.6us
I20260709 12:23:11.117766 25210 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:11.110762 25084 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
I20260709 12:23:11.124548 20802 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:11.126277 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56480.2us
I20260709 12:23:11.154302 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:11.155529 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:11.071259 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:11.156942 20726 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 7.
I20260709 12:23:11.115216 25175 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:11.111325 24947 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.133613 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.121079 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.159317 20810 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:11.149971 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.153652 21432 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:11.160066 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:11.163570 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:11.124806 24746 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:11.171852 21431 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:11.130313 25315 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.127151 25299 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:11.178376 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:11.179073 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.173671 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:11.177881 21435 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:23:11.181525 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:11.130602 24898 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:11.131995 25327 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:11.117579 25288 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.154292 25210 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.124668 25242 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:23:11.147025 25325 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:11.167555 24947 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:11.125180 25261 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:23:11.157946 25204 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:11.218047 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132016us
I20260709 12:23:11.219991 20877 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:11.225643 20882 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:11.230679 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.180295 24746 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:11.186992 25315 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:11.154381 25185 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:23:11.212641 25327 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:11.242102 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.171227 25014 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:23:11.145824 25254 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.169152 25084 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:11.245532 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 91331.9us
I20260709 12:23:11.246644 20882 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:11.248858 20810 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:11.158973 25326 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:23:11.216321 25332 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:11.250706 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.297046 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.470175 20810 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.237107 24746 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.237004 25264 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:11.315477 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 160023us
I20260709 12:23:11.477329 20774 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13.
I20260709 12:23:11.225435 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:11.478441 20875 raft_consensus.cc:606] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:11.243880 25249 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.240s
W20260709 12:23:11.319928 25290 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.786s	user 0.003s	sys 0.000s
I20260709 12:23:11.320269 25331 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.322208 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.244988 25299 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:11.324688 24963 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:11.466223 25327 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.327033 25326 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.327942 25321 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.487509 25264 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.525743 25338 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.526216 25264 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:11.526373 25338 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.528833 25338 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.329562 25227 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:11.334075 25237 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.003s
I20260709 12:23:11.333971 24863 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:11.531037 24863 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:11.337911 25275 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:11.341946 25175 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.537333 25175 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.342943 25217 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:11.355562 25162 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.544734 25162 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.357167 25250 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:11.217322 25329 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 12:23:11.546515 25329 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:11.547030 25329 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:11.299322 25330 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:23:11.549058 25330 consensus_queue.cc:1243] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:11.234292 25204 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.296310 25325 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:23:11.243463 25223 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:11.551779 25223 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:11.552390 25223 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:11.373119 25247 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.727s	user 0.005s	sys 0.005s
I20260709 12:23:11.555614 25247 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
I20260709 12:23:11.374372 24646 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.383999 25239 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:11.559435 25239 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:11.560739 25239 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.385257 25288 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.392355 24898 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.395892 21433 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:23:11.401583 25230 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:11.565356 25230 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:11.566603 25230 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.568182 25230 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:11.402431 25065 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.572038 25065 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.407738 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2143) took 2134 ms. Trace:
I20260709 12:23:11.575404 20730 rpcz_store.cc:276] 0709 12:23:09.273401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.895305 (+621904us) service_pool.cc:224] Handling call
0709 12:23:10.098332 (+203027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:10.153665 (+ 55333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.407490 (+1253825us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54000c44d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.407662 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463646464}
I20260709 12:23:11.416683 25246 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.418057 25333 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:23:11.418284 25182 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:11.577044 25182 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:11.419790 25174 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:23:11.422489 25304 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.433271 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:11.435221 21432 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:23:11.436138 25220 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.578083 25182 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.580119 25261 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:11.580456 25182 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:11.580751 25261 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.440243 25000 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.444785 20633 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 7, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 7 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:11.459287 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.548224 25329 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.587988 25329 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:11.460191 25226 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.468391 25163 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:11.227259 25180 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:11.479847 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 229217us
I20260709 12:23:11.484898 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:11.589946 20881 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:11.590075 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.589764 25180 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.591094 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:11.592202 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:11.489014 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:11.489975 21428 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:11.489477 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 431855us
I20260709 12:23:11.490999 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:11.487625 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.000s	sys 0.010s
I20260709 12:23:11.593529 20879 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:11.594514 25259 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:11.594712 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:11.325547 25315 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.578508 25246 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:11.593792 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.000s	sys 0.010s
W20260709 12:23:11.599320 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 626213us
I20260709 12:23:11.600020 25315 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.517241 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:11.601631 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:11.602404 20883 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:11.593549 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:11.603839 25326 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.605732 20882 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:11.607613 25226 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:11.608326 25226 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:11.608619 20882 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:11.592237 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.609679 25226 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.612474 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.613723 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.615641 25247 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:11.615748 25246 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:11.616578 25247 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:11.618356 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:11.619822 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.625818 25261 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:11.626242 20803 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:11.592617 25180 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.626895 20882 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:11.627202 25065 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.628048 25247 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:11.631570 25246 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:11.632586 25246 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:11.633330 25325 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:11.633841 25325 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:11.606848 25326 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.562741 25239 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:11.634920 25325 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.640688 25247 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:23:11.644712 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.645943 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:11.646833 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:11.649163 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.649740 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.650360 20807 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:11.652235 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.657994 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:11.658922 20733 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:11.661702 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:11.662535 20735 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
W20260709 12:23:11.662746 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.664682 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:11.665513 20734 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:23:11.675115 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48424.4us
W20260709 12:23:11.678438 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.679819 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:11.681704 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:11.683837 25290 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 12:23:11.685140 25234 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.042s	user 0.013s	sys 0.004s
I20260709 12:23:11.685549 25234 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13.
I20260709 12:23:11.687613 25288 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.688153 25288 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.689003 24947 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.690332 25288 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.691105 21426 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:11.691931 25247 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:11.679515 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:11.692373 25247 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:11.692663 21433 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:11.692839 25247 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.694252 25204 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.691129 24947 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.696404 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2160) took 1573 ms. Trace:
I20260709 12:23:11.696794 20803 rpcz_store.cc:276] 0709 12:23:10.122997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.123125 (+   128us) service_pool.cc:224] Handling call
0709 12:23:10.123402 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:11.626687 (+1503285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.696080 (+ 69393us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540025d5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:11.696339 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3455369472}
I20260709 12:23:11.698015 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.699381 25315 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:11.700991 25084 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.701833 25249 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:11.702229 25249 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:11.702572 20804 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:11.700007 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.703018 25249 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.704233 25014 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.704339 25249 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.704378 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.704802 25014 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.707077 25014 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.708714 25246 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.709442 25246 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:11.711556 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 697) took 1469 ms (client timeout 1733 ms). Trace:
I20260709 12:23:11.712291 20880 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:11.712749 25325 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:11.712944 24898 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.713196 25327 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.713409 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2229) took 1188 ms. Trace:
I20260709 12:23:11.713754 20880 rpcz_store.cc:276] 0709 12:23:10.525179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.525504 (+   325us) service_pool.cc:224] Handling call
0709 12:23:11.713189 (+1187685us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.713326 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2463416192}
I20260709 12:23:11.714242 25117 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:11.714634 25117 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:11.513098 25197 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:11.446914 25210 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.717516 25210 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.719501 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:11.719220 20633 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 9, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 9 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:11.722153 20633 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:11.484066 25233 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:11.516803 25343 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.233s
I20260709 12:23:11.700997 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 925) took 1055 ms. Trace:
I20260709 12:23:11.743399 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:11.724917 25246 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:11.701241 21431 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:11.727629 25313 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:11.747712 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:11.730525 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 297354us
I20260709 12:23:11.731076 25299 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:11.482874 25332 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.732529 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:11.751142 21435 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:23:11.732645 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.752424 20805 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:11.711989 20804 rpcz_store.cc:269] 0709 12:23:10.242370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.242757 (+   387us) service_pool.cc:224] Handling call
0709 12:23:11.711382 (+1468625us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540027b854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.711497 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2971508608}
I20260709 12:23:11.734163 20877 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:11.715096 20876 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:23:11.753881 20876 pending_rounds.cc:85] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:23:11.682442 21427 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:11.754282 20807 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:23:11.737973 20810 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:11.742126 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
W20260709 12:23:11.742908 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60763.3us
I20260709 12:23:11.745040 21428 rpcz_store.cc:276] 0709 12:23:10.645465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.645622 (+   157us) service_pool.cc:224] Handling call
0709 12:23:11.700813 (+1055191us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b5400088c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.700942 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815591808}
W20260709 12:23:11.728886 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.754197 20876 pending_rounds.cc:107] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:23:11.760524 20876 raft_consensus.cc:2889] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:11.761333 20801 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:11.759660 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.745122 25174 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:11.757565 25234 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:11.765731 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:11.768491 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:11.769531 21428 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:11.774420 25339 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:11.760651 25117 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.774883 25339 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.775298 25339 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:11.777670 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:11.778795 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:11.779767 21428 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:11.757784 25014 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.764611 25242 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:11.785215 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.802703 20735 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:11.747121 25223 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.769155 25233 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:11.768805 25254 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:11.769389 25174 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:11.813326 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:11.795008 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:11.807821 25065 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 12:23:11.821082 20631 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 292) took 24404 ms (client timeout 15000 ms). Trace:
I20260709 12:23:11.824137 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:11.818724 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:11.826220 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.029s	sys 0.038s
I20260709 12:23:11.776584 25217 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:11.770848 25234 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:11.826103 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:11.757684 25313 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.830812 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:11.829130 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.029s	sys 0.039s
I20260709 12:23:11.814203 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.825115 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:11.831419 21435 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:11.832612 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:11.834020 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 699574us
I20260709 12:23:11.839308 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:11.839589 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:11.804096 25197 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:11.850937 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:11.845836 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.812003 25275 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:11.848023 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:11.825416 25311 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.023s	sys 0.044s
I20260709 12:23:11.828120 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:11.786060 25326 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:23:11.785944 25246 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 12:23:11.917857 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68119.6us
I20260709 12:23:11.830865 25217 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:11.917189 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 89168.7us
I20260709 12:23:11.809844 25242 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:11.826683 25327 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:11.803830 25014 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.933614 21427 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:23:11.811947 25171 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 12:23:11.934576 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95372.5us
I20260709 12:23:11.976817 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:11.909336 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.024s	sys 0.046s
W20260709 12:23:11.977924 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.025s	sys 0.047s
W20260709 12:23:11.978884 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:11.876420 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.935510 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:11.936599 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.035s	sys 0.094s
W20260709 12:23:11.980230 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.035s	sys 0.094s
I20260709 12:23:11.922349 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2235) took 1264 ms. Trace:
W20260709 12:23:11.864012 21399 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.028s	sys 0.046s
I20260709 12:23:11.846614 25335 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:11.917349 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59990.5us
W20260709 12:23:11.821580 20631 rpcz_store.cc:269] 0709 12:22:47.416234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.416476 (+   242us) service_pool.cc:224] Handling call
0709 12:23:11.820840 (+24404364us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:07.917910 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:07.918707 (+   797us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:07.918785 (+    78us) write_op.cc:435] Acquired schema lock
0709 12:23:07.918815 (+    30us) tablet.cc:662] Decoding operations
0709 12:23:07.929836 (+ 11021us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:07.929971 (+   135us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:07.930006 (+    35us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:23:07.949362 (+ 19356us) tablet.cc:701] Row locks acquired
0709 12:23:07.949427 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:23:07.998354 (+ 48927us) write_op.cc:270] Start()
0709 12:23:07.998500 (+   146us) write_op.cc:276] Timestamp: P: 1783599787998238 usec, L: 0
0709 12:23:07.998531 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:23:07.999518 (+   987us) log.cc:844] Serialized 7539 byte log entry
0709 12:23:08.284483 (+284965us) op_driver.cc:464] REPLICATION: finished
0709 12:23:08.414337 (+129854us) write_op.cc:301] APPLY: starting
0709 12:23:08.414438 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:08.464634 (+ 50196us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:08.464794 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:09.487837 (+1023043us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:09.488216 (+   379us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=19,key_file_lookups=19,delta_file_lookups=19,mrs_lookups=38
0709 12:23:09.488264 (+    48us) write_op.cc:312] APPLY: finished
0709 12:23:09.488893 (+   629us) log.cc:844] Serialized 248 byte log entry
0709 12:23:09.752178 (+263285us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:09.772806 (+ 20628us) write_op.cc:454] Released schema lock
0709 12:23:09.781043 (+  8237us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:09.781534 (+   491us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:09.781575 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30465,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":129570,"apply.run_cpu_time_us":64769,"apply.run_wall_time_us":1367129,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37620,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":2,"lbm_read_time_us":169956,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":4,"lbm_reads_lt_1ms":7,"num_ops":19,"prepare.queue_time_us":53275,"prepare.run_cpu_time_us":11744,"prepare.run_wall_time_us":107909,"raft.queue_time_us":174413,"raft.run_cpu_time_us":17009,"raft.run_wall_time_us":188117,"replication_time_us":254304,"thread_start_us":409845,"threads_started":5,"wal-append.queue_time_us":312395,"wal-append.run_cpu_time_us":33047,"wal-append.run_wall_time_us":2121475}]]}
I20260709 12:23:11.942919 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.990671 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:11.991312 20810 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:11.915899 25315 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:11.934285 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:11.935889 20633 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 9, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 9 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:11.809587 25324 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.004s	sys 0.039s
I20260709 12:23:11.935053 25330 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:11.939796 25338 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:11.940243 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:11.817339 25342 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:23:11.916770 24646 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.132s
I20260709 12:23:11.849813 25343 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:11.919234 25311 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.034s	user 0.023s	sys 0.044s
I20260709 12:23:11.822583 25210 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:23:11.981611 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:12.027793 20807 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:11.856132 25216 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.145s
I20260709 12:23:11.919427 25275 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.983531 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.040167 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.002234 25331 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:12.021587 25343 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.928604 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2244) took 1102 ms. Trace:
I20260709 12:23:11.981379 20878 rpcz_store.cc:276] 0709 12:23:10.658581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.658725 (+   144us) service_pool.cc:224] Handling call
0709 12:23:11.922154 (+1263429us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.922281 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978218368}
I20260709 12:23:11.994735 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:11.957367 21435 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:12.000128 25338 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:12.001669 20735 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:11.847288 25233 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 12:23:11.962787 25254 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:23:12.002097 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66673.5us
W20260709 12:23:12.002703 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62570.7us
I20260709 12:23:11.835477 25307 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:12.003230 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:12.036320 25259 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:12.002838 20633 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:11.975303 25314 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.996784 25171 consensus_queue.cc:1243] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:12.013070 25014 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.013588 25339 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.851483 24890 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.149s
I20260709 12:23:12.016243 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:11.916477 25197 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.093550 25339 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.094070 25197 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.073668 25307 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.094846 25307 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:12.094225 25339 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:23:12.086109 25259 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:12.083146 25233 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:23:12.061743 25345 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:12.079416 25254 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:12.084745 25314 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:11.919907 24963 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:23:11.972723 25327 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.019660 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:12.025081 20810 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
W20260709 12:23:11.982069 21399 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.028s	sys 0.055s
I20260709 12:23:11.994853 25335 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.984679 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:11.987156 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:12.110814 20633 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:12.049829 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.040856 20734 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:23:12.000504 20879 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:23:12.054752 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60067.7us
I20260709 12:23:11.999527 25330 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:12.023010 25239 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:12.024319 25324 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.004s	sys 0.040s
I20260709 12:23:12.144348 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:12.057909 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:12.059587 20876 rpcz_store.cc:276] 0709 12:23:10.825943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.826090 (+   147us) service_pool.cc:224] Handling call
0709 12:23:11.714760 (+888670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:11.761210 (+ 46450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.916641 (+155431us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b540026ded0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:11.928518 (+ 11877us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2042999040}
I20260709 12:23:12.072065 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:12.052490 25275 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:12.057865 25342 consensus_queue.cc:1243] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:12.081741 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.056830 25343 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:12.063055 25338 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:11.935784 25188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.021s	sys 0.063s
I20260709 12:23:12.089927 20731 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:12.087530 25347 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:12.092181 25329 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
W20260709 12:23:12.095168 24746 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.001s
I20260709 12:23:12.162532 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:12.168339 20807 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 6.
I20260709 12:23:12.113431 25313 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.099972 25259 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.103704 25345 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:12.105088 25254 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.105970 25247 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:12.116282 25220 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 12:23:12.080296 25315 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:12.035835 25333 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:23:12.037038 25217 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.119583 25332 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.082152 25065 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:12.120076 25299 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:12.121948 25242 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:12.017966 20727 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:12.133536 20728 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:12.136072 20633 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 6 to 8, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 8 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:12.134814 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:12.148464 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 76554.2us
I20260709 12:23:12.137742 20729 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:12.191831 20807 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:23:12.192498 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:12.193982 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:12.149281 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 944269us
I20260709 12:23:12.195749 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:12.196553 21435 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:12.133339 25327 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:12.150484 20732 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:12.151726 20735 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:12.199103 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:12.199946 20735 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:23:12.134961 25190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.013s	sys 0.013s
I20260709 12:23:12.155269 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:12.204046 20810 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 12:23:12.157649 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.152371 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 182818us
W20260709 12:23:12.162532 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.204888 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.095542 25223 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.138584 25330 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:12.145555 25216 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.147783 25335 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:12.163439 25188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.104s	user 0.021s	sys 0.063s
I20260709 12:23:12.152204 25233 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:12.182082 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 756) took 1026 ms. Trace:
I20260709 12:23:12.245235 20808 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:12.245285 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.246133 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:12.182034 20803 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:12.182564 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.248351 20731 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:12.201329 20734 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:12.202841 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.249231 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:12.207005 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.250355 20881 raft_consensus.cc:2368] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12.
I20260709 12:23:12.250671 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:12.212670 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.212507 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.251410 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.252223 20881 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:12.177754 25315 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:12.253902 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.255470 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:12.256217 20727 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:12.241264 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:12.245253 21429 rpcz_store.cc:276] 0709 12:23:11.155529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.156240 (+   711us) service_pool.cc:224] Handling call
0709 12:23:11.156597 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:11.157026 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:11.157073 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:11.178671 (+ 21598us) log.cc:844] Serialized 26 byte log entry
0709 12:23:11.360696 (+182025us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b1800372c40. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:11.831842 (+471146us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:12.163950 (+332108us) log.cc:844] Serialized 14 byte log entry
0709 12:23:12.168237 (+  4287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:12.168321 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:12.168360 (+    39us) raft_consensus.cc:1685] finished
0709 12:23:12.178504 (+ 10144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:12.182021 (+  3517us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1821,"raft.queue_time_us":71365,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":13038,"spinlock_wait_cycles":5034880,"thread_start_us":716307,"threads_started":2,"wal-append.queue_time_us":799261}
I20260709 12:23:12.242434 20633 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 5 to 7, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 7 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:12.237920 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.260555 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.208503 20880 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:23:12.261314 20729 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:12.261438 20884 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:12.252600 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:12.262408 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.253579 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:12.251755 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.263509 20875 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:12.263973 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.208187 25190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.014s	sys 0.013s
W20260709 12:23:12.271333 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.239145 25353 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:12.276393 20633 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 1 to 4, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 4 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:12.295962 25350 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:23:12.297338 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:12.256770 25325 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 12:23:12.298595 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:12.299180 20882 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:23:12.299780 20882 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 10
I20260709 12:23:12.300078 20882 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:23:12.300417 20882 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:12.302582 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.304804 25348 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:12.305338 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:12.305864 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.306533 20883 raft_consensus.cc:2376] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13. Re-sending same reply.
I20260709 12:23:12.307156 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.308408 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.309520 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.310142 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.310298 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:12.283700 25233 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:12.311471 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.312168 25233 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.312662 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.312772 25234 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.177487 25347 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.314585 25234 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:12.315831 25347 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.275244 25331 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:23:12.317862 25331 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:12.318347 25331 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:12.320640 25322 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:12.321072 25322 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:12.321938 25322 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.311017 20803 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:12.265903 20732 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:12.282701 20729 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:12.174489 24746 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.303560 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:12.304517 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.188412 25065 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:12.374456 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.342869 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:12.363955 20775 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.567s (SENT)
I20260709 12:23:12.380827 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:12.173766 25345 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:12.381623 20884 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:12.244830 25332 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.383533 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:12.394685 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.395314 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:12.385058 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.331600 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.326635 20803 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:23:12.396030 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70cdcdfe95804895979397c927fae46f"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:12.396575 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:12.399487 20879 raft_consensus.cc:606] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:12.399879 20879 raft_consensus.cc:614] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:12.353466 20734 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:12.237293 25242 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:12.397622 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.397136 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.401872 20883 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 7: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:12.405809 20881 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:12.368979 20727 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:12.408527 20727 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:12.401954 20732 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:12.419577 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.420900 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:12.323140 25322 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:12.423383 25343 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.424566 20732 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:12.274996 25210 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:12.333180 25358 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:12.281457 25226 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:12.427165 20728 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:12.456671 20735 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:12.428915 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:12.458081 20734 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:23:12.435153 20801 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:12.443841 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 79943us
I20260709 12:23:12.449501 21435 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:12.331256 25188 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:12.326216 25014 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:12.460964 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.456746 20726 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:12.462241 25249 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:12.461069 25014 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.464550 25014 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:12.332468 25247 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:12.474455 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:12.478466 20810 raft_consensus.cc:2376] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13. Re-sending same reply.
I20260709 12:23:12.426738 25239 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:12.476104 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.253513 25216 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.479884 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:12.496287 20810 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:12.520318 20803 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 12:23:12.520819 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.521737 20883 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:12.442268 24890 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:12.473703 25220 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:12.540047 20878 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:23:12.542935 20808 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:23:12.426581 25332 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.481501 25188 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:12.481244 24898 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:12.365061 25331 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.535329 25084 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:12.553866 20803 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:12.436817 25242 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:12.568602 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.543406 25354 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:12.542537 25065 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.485471 24863 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.542300 24890 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.543551 25314 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.534883 25190 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:23:12.549393 25182 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:12.621026 25261 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.563345 24898 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.626835 25065 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.630581 24863 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:12.538722 25315 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:12.652966 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.012s	sys 0.016s
W20260709 12:23:12.665853 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.012s	sys 0.016s
I20260709 12:23:12.673698 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:12.674430 20619 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.045s
I20260709 12:23:12.687587 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
W20260709 12:23:12.682391 20619 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.015s	sys 0.045s
I20260709 12:23:12.692167 25014 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.693077 25330 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.694617 25014 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.695413 25014 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.696238 25014 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.698798 25014 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.676261 25190 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:12.662317 25261 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.692389 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:12.659714 25188 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:12.703617 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.704377 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:12.804111 21433 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:12.768405 20882 raft_consensus.cc:1217] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Deduplicated request from leader. Original: 9.7->[11.8-11.8]   Dedup: 11.8->[]
W20260709 12:23:12.792930 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.796830 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.808950 20882 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:12.707974 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:12.814407 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 106503us
W20260709 12:23:12.784011 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.773887 20881 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:23:12.816277 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.795320 20810 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:12.817045 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.810011 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:12.817992 20882 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
W20260709 12:23:12.819571 20806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.010s	sys 0.028s
W20260709 12:23:12.820088 20806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.011s	sys 0.028s
I20260709 12:23:12.820407 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:12.817905 20881 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:12.799468 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 456667us
W20260709 12:23:12.731305 25359 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.336s	user 0.001s	sys 0.001s
I20260709 12:23:12.820204 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:12.806587 25348 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:12.828526 20807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.012s	sys 0.015s
I20260709 12:23:12.830673 25330 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:12.767891 25307 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.000s
W20260709 12:23:12.831285 20807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.012s	sys 0.015s
I20260709 12:23:12.831380 25307 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.833074 20804 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:12.835206 25307 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.776005 25117 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:12.788975 25313 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:23:12.844238 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.706058 25242 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.763406 25315 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:12.838012 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:12.742689 25275 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.001s
I20260709 12:23:12.842566 20881 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:12.828313 25254 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.013s	sys 0.013s
I20260709 12:23:12.757651 25332 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.704394 24890 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.703259 25261 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:12.681597 25247 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:23:12.671460 25324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.002s	sys 0.019s
I20260709 12:23:12.710551 25239 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.144s
I20260709 12:23:12.699128 25335 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.700129 25190 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.790681 25343 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:12.844218 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.752789 24898 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.735380 24746 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.700987 25216 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:12.797122 25217 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:12.699631 25014 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:12.960098 24898 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.960763 24898 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:12.851672 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.964340 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.965507 24898 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:12.966508 24898 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.967149 24898 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:12.969978 24898 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:12.975580 24898 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:12.858434 25275 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:12.980253 25275 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:12.824220 25325 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:12.776706 25318 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:23:13.010736 25318 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:13.011601 25014 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:13.012266 20882 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:12.828915 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.019s
W20260709 12:23:13.013381 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.007s	sys 0.019s
I20260709 12:23:13.013468 25014 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:13.013906 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 708594us
W20260709 12:23:13.018363 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.019860 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:12.862246 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:13.021160 20881 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:13.022291 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:12.855472 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:13.023445 20881 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:13.023717 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 168314us
I20260709 12:23:13.024839 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2422) took 1367 ms. Trace:
I20260709 12:23:13.024704 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:12.903375 21398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.013s	sys 0.016s
W20260709 12:23:13.025767 21398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.016s	sys 0.016s
W20260709 12:23:13.026522 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.027137 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.836166 25220 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.8, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:23:12.863087 20701 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:12.980960 24890 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:12.839280 25210 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:12.846761 25246 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.015s	sys 0.015s
W20260709 12:23:12.951179 25324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.009s	sys 0.019s
W20260709 12:23:12.849152 25259 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.012s	sys 0.015s
I20260709 12:23:12.851096 25326 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:12.848853 25313 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:12.897787 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.026163 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:13.055117 20881 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:12.856163 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.852905 25254 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.013s	sys 0.013s
W20260709 12:23:13.031242 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 176796us
I20260709 12:23:13.025342 20730 rpcz_store.cc:276] 0709 12:23:11.657146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.667753 (+ 10607us) service_pool.cc:224] Handling call
0709 12:23:11.749626 (+ 81873us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:11.758587 (+  8961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:11.758640 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:11.759373 (+   733us) log.cc:844] Serialized 52 byte log entry
0709 12:23:12.034611 (+275238us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:12.034655 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:12.034730 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.024618 (+989888us) raft_consensus.cc:1685] finished
0709 12:23:13.024643 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.024780 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":9746,"thread_start_us":248741,"threads_started":1,"wal-append.queue_time_us":431711}
W20260709 12:23:13.045132 25226 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.004s
I20260709 12:23:13.031801 25318 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:13.039253 25233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.007s	sys 0.040s
I20260709 12:23:13.057271 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:13.042085 24890 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:12.889806 25330 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.156s
W20260709 12:23:13.039422 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.013s	sys 0.042s
W20260709 12:23:13.070291 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.013s	sys 0.042s
I20260709 12:23:13.045281 25299 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:12.855600 25242 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:13.045570 25345 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
I20260709 12:23:13.069298 20731 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
W20260709 12:23:13.070591 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 799359us
I20260709 12:23:13.079903 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:13.074162 25233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.007s	sys 0.040s
I20260709 12:23:13.081910 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:13.082149 20877 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:23:13.084026 24746 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:23:13.041404 25246 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.015s	sys 0.015s
W20260709 12:23:13.052985 25259 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.014s	sys 0.016s
I20260709 12:23:13.048031 24963 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.071412 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:13.072082 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.022s	sys 0.037s
W20260709 12:23:13.094890 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.022s	sys 0.038s
I20260709 12:23:13.069694 25226 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:13.049856 25216 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.101048 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2425) took 1440 ms. Trace:
I20260709 12:23:13.054409 25326 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.082935 21399 leader_election.cc:400] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12.
I20260709 12:23:13.104177 21399 leader_election.cc:403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 12:23:13.068293 25338 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.898s	user 0.009s	sys 0.003s
I20260709 12:23:13.093089 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 708) took 1478 ms. Trace:
I20260709 12:23:13.066792 25220 consensus_queue.cc:1243] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:13.069878 25348 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:23:13.095615 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 797017us
I20260709 12:23:13.114389 20806 rpcz_store.cc:276] 0709 12:23:11.614398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.614676 (+   278us) service_pool.cc:224] Handling call
0709 12:23:11.615128 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:11.733193 (+118065us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:12.162863 (+429670us) log.cc:844] Serialized 14 byte log entry
0709 12:23:12.341607 (+178744us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:23:12.836171 (+494564us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:12.836737 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 12:23:12.836875 (+   138us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:12.836919 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:12.836994 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.085626 (+248632us) raft_consensus.cc:1685] finished
0709 12:23:13.092853 (+  7227us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.093013 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121977,"prepare.queue_time_us":547832,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":150015,"raft.run_cpu_time_us":4622,"raft.run_wall_time_us":605509,"spinlock_wait_cycles":6743296,"thread_start_us":1043209,"threads_started":2,"wal-append.queue_time_us":887836}
I20260709 12:23:13.119400 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:13.121448 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:13.049068 25332 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:13.099628 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2172) took 1295 ms (client timeout 1506 ms). Trace:
W20260709 12:23:13.122341 20804 rpcz_store.cc:269] 0709 12:23:11.804604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.845533 (+ 40929us) service_pool.cc:224] Handling call
0709 12:23:13.099407 (+1253874us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54003d68d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:13.099561 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070013184}
I20260709 12:23:13.100653 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2171) took 1306 ms. Trace:
I20260709 12:23:13.123697 20801 rpcz_store.cc:276] 0709 12:23:11.794063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.002942 (+208879us) service_pool.cc:224] Handling call
0709 12:23:13.100462 (+1097520us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54003c7154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:13.100596 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826466688}
I20260709 12:23:13.102619 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2412) took 1033 ms. Trace:
I20260709 12:23:13.124811 20810 rpcz_store.cc:276] 0709 12:23:12.068828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.534367 (+465539us) service_pool.cc:224] Handling call
0709 12:23:12.535605 (+  1238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:12.817285 (+281680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.102425 (+285140us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002b57d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:13.102564 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596084096}
I20260709 12:23:13.106642 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:13.108290 25384 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:13.102823 25370 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 12:23:13.129676 25373 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:13.074909 25318 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.128849 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:13.131850 25318 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.038738 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:13.111945 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:13.134094 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.134944 21427 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:13.090101 25299 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:13.136448 25364 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.491s	user 0.003s	sys 0.001s
I20260709 12:23:13.137143 25299 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.138334 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2401) took 1483 ms. Trace:
I20260709 12:23:13.096151 25239 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:13.139374 25239 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:13.140036 20879 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:13.138762 20802 rpcz_store.cc:276] 0709 12:23:11.654577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.654967 (+   390us) service_pool.cc:224] Handling call
0709 12:23:11.655401 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:11.812195 (+156794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:11.812232 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:11.844891 (+ 32659us) log.cc:844] Serialized 26 byte log entry
0709 12:23:12.264413 (+419522us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:12.433354 (+168941us) log.cc:844] Serialized 14 byte log entry
0709 12:23:12.433627 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:12.433732 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.093522 (+659790us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:13.138020 (+ 44498us) raft_consensus.cc:1685] finished
0709 12:23:13.138124 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.138270 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":263762,"prepare.queue_time_us":207368,"raft.queue_time_us":153228,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":30751,"spinlock_wait_cycles":16072960,"thread_start_us":295786,"threads_started":1,"wal-append.queue_time_us":634512}
I20260709 12:23:13.076977 25254 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:13.135953 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.167269 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.168145 20735 raft_consensus.cc:2376] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11. Re-sending same reply.
I20260709 12:23:13.106688 25326 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:13.172286 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.141227 20879 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:13.144438 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.101507 20733 rpcz_store.cc:276] 0709 12:23:11.660459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.664709 (+  4250us) service_pool.cc:224] Handling call
0709 12:23:11.665101 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:11.665357 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:12.163295 (+497938us) log.cc:844] Serialized 14 byte log entry
0709 12:23:13.100599 (+937304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:13.100672 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:13.100796 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.100874 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.100992 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":624389,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":352325,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5757,"thread_start_us":1434301,"threads_started":3,"wal-append.queue_time_us":815475}
W20260709 12:23:13.169983 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.112875 25264 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:13.163787 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:13.145498 25084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.031s	sys 0.036s
I20260709 12:23:13.112560 24746 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:13.175732 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.181962 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.090185 25345 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.176734 20732 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:23:13.177177 20729 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:13.203753 25380 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:23:13.204759 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.205469 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.204725 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.206318 25345 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:13.207172 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.208058 20732 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:13.174226 21429 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:13.174722 25220 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:13.187779 24746 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.209440 24746 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:13.209741 25326 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.207095 25345 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:13.212028 25345 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:13.212388 25345 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:13.212739 25182 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:13.213369 25182 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:13.188187 20728 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:13.121279 25338 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.174966 25332 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:13.215363 25332 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.215770 25332 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:13.217801 24746 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.180004 24890 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.200702 21428 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:13.201126 25084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.031s	sys 0.036s
I20260709 12:23:13.202327 25197 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:13.202291 25381 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:13.116070 25210 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.252097 21435 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:13.207047 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.178104 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.119552 25313 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.221402 25338 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.196378 25190 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.221994 25174 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.201948 25335 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:13.205914 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.225150 20876 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:13.098629 24963 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.170694 20734 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:13.229884 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.245937 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.266892 21428 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:13.246701 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:13.106313 25216 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.253042 20735 raft_consensus.cc:2376] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11. Re-sending same reply.
I20260709 12:23:13.127264 25226 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:23:13.260120 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.174685 25317 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:23:13.242027 25254 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:13.236896 25385 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:13.264014 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:13.266283 25327 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.018s	user 0.011s	sys 0.000s
I20260709 12:23:13.268456 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:13.281243 21433 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:13.261399 25197 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12.
I20260709 12:23:13.244225 25318 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:23:13.242728 25084 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:13.172628 25384 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:23:13.272444 25381 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:13.312412 20733 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:23:13.249068 25188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.010s	sys 0.021s
I20260709 12:23:13.274132 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.321919 20730 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:13.140650 25239 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.225678 25387 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 12:23:13.247963 25065 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.000s
I20260709 12:23:13.290119 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2150) took 1267 ms. Trace:
I20260709 12:23:13.338793 20731 rpcz_store.cc:276] 0709 12:23:12.023069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.313542 (+290473us) service_pool.cc:224] Handling call
0709 12:23:12.313834 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:13.079201 (+765367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.289797 (+210596us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54000e79d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:13.290039 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736923520}
I20260709 12:23:13.339932 21431 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:13.291170 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.268738 25313 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:13.309159 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.275408 25254 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:13.271896 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 473189us
I20260709 12:23:13.282096 25174 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:13.270025 21435 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:23:13.322465 25188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.010s	sys 0.021s
I20260709 12:23:13.292336 25335 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:13.352407 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.292351 25385 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:13.377676 21401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.027s
I20260709 12:23:13.307767 25338 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.308422 25376 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:13.283031 25327 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.341959 25326 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:13.385229 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.385293 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:13.383255 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.023s	sys 0.032s
W20260709 12:23:13.405421 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.024s	sys 0.032s
I20260709 12:23:13.405572 20731 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:23:13.405694 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 582747us
I20260709 12:23:13.406003 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.406641 21435 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:13.407774 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:13.385185 21401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.002s	sys 0.027s
I20260709 12:23:13.408581 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:13.408819 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.410424 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.383558 25384 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.395521 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:13.390157 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.396315 20878 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:13.398793 21426 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:13.413324 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:13.426975 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.420795 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.466969 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.467164 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.467736 20733 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:13.467931 21435 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:13.467917 20734 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:13.426788 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:13.396248 25331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.021s	sys 0.033s
I20260709 12:23:13.474097 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.474817 21434 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:13.474890 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:13.386006 24863 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.722s	user 0.006s	sys 0.006s
I20260709 12:23:13.470911 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.475400 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.471431 20880 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:13.471700 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.476761 20810 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:23:13.469623 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 348213us
I20260709 12:23:13.477711 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:13.422590 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:13.480536 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.481168 20728 raft_consensus.cc:2376] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15. Re-sending same reply.
I20260709 12:23:13.482133 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.399569 25239 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:13.483453 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.475683 20731 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:13.484635 21435 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:13.420396 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.486210 21434 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 12:23:13.488576 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.491587 20730 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:13.492818 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 324627us
I20260709 12:23:13.467877 25396 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:13.482738 25331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.021s	sys 0.034s
I20260709 12:23:13.417866 25117 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:23:13.398662 25197 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 12:23:13.380620 25254 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.419642 25317 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:23:13.493185 25324 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.504971 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.558307 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:13.558612 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.559217 20804 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:13.508693 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.513787 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:13.528930 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:13.503638 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 464916us
I20260709 12:23:13.504758 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:13.515286 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.398144 25376 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:13.487274 25217 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.397485 25335 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.568364 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:13.569023 20806 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:13.552400 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:13.560974 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47347.2us
I20260709 12:23:13.404958 25338 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.463361 25384 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.471357 25332 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.566253 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:13.580469 20730 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:13.584808 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.585729 20731 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:13.586284 20734 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:13.570648 20733 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
W20260709 12:23:13.531838 25377 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:23:13.572321 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:13.560976 20732 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:13.589088 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.576225 20808 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:13.427827 25383 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 12:23:13.511358 25403 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:13.533290 25396 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.531515 24746 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.567641 25376 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:13.595207 20810 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:23:13.597183 20804 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:13.587479 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:13.575953 25217 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.599978 25339 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.622812 20728 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:13.622649 25339 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.623742 20732 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:13.575013 25324 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.624220 20732 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:13.624550 25230 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:13.624847 25339 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:13.624980 25230 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:13.472390 25084 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.602540 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:13.607353 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.555586 25330 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:13.619014 20808 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:13.619843 25406 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:13.620163 25323 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:13.620878 25386 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:23:13.620985 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.623631 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.635154 20806 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:13.645340 20730 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:13.634300 21434 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:23:13.625094 20732 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.603149 21433 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:13.630291 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:13.640527 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.631372 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.643258 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:13.634024 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.591918 25234 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:13.650449 21426 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [8.8-11.9]
I20260709 12:23:13.548578 25389 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:23:13.636447 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.607384 25335 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.613237 24963 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:13.637351 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.651839 21435 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:13.626368 25216 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:13.630946 25402 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.618307 25254 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:13.633257 25394 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:23:13.657373 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:13.633356 25393 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 12:23:13.677624 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:13.608424 25065 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:23:13.664043 20810 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:13.685961 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.686874 20734 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:13.573328 25385 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.633538 25326 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:13.670921 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2409) took 1988 ms. Trace:
I20260709 12:23:13.652489 25339 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:23:13.662778 25399 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:13.710318 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:13.711570 20879 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:13.639460 25323 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:13.652335 25188 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:13.645489 25407 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:13.649686 25322 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:13.648161 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.675477 25398 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:23:13.634068 25382 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:13.644619 25324 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:13.682917 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 256245us
W20260709 12:23:13.715341 25400 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:23:13.693008 25396 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:13.652004 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.690814 25331 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:13.618515 25403 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.756428 25418 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:13.655932 24746 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:23:13.757289 25418 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.758005 24746 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:13.644629 25330 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.707144 20807 rpcz_store.cc:276] 0709 12:23:11.682075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.243869 (+561794us) service_pool.cc:224] Handling call
0709 12:23:12.278912 (+ 35043us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:12.279362 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:12.279406 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:12.280376 (+   970us) log.cc:844] Serialized 26 byte log entry
0709 12:23:12.831963 (+551587us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:12.834704 (+  2741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:12.834806 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.670615 (+835809us) raft_consensus.cc:1685] finished
0709 12:23:13.670649 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.670836 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27822,"raft.queue_time_us":813,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":98513,"thread_start_us":551198,"threads_started":1,"wal-append.queue_time_us":837553}
I20260709 12:23:13.644315 25259 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:13.723556 20730 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:13.759496 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2181) took 1267 ms. Trace:
I20260709 12:23:13.724969 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:13.760519 25330 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.761016 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.761207 25417 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:13.761698 20734 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:13.762054 25417 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:13.762440 25417 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:13.762650 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.671260 25401 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:23:13.678596 25216 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:13.649890 25406 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.666263 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.672047 25335 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:23:13.746245 20733 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:13.706641 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.630900 25230 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.746639 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:13.710126 25399 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.746232 25389 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.735340 25407 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:13.742278 25394 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:23:13.759754 25397 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:23:13.759950 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:13.759955 20881 rpcz_store.cc:276] 0709 12:23:12.492550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.115178 (+622628us) service_pool.cc:224] Handling call
0709 12:23:13.183629 (+ 68451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:13.241929 (+ 58300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:13.241958 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:13.242514 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:23:13.535552 (+293038us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:13.535603 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.535679 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.759263 (+223584us) raft_consensus.cc:1685] finished
0709 12:23:13.759292 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.759442 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60524,"raft.queue_time_us":61306,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3211,"thread_start_us":266203,"threads_started":1,"wal-append.queue_time_us":396833}
I20260709 12:23:13.691581 25234 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:13.771735 21435 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:13.804620 20730 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:13.789767 20879 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:13.676671 25402 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.792984 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:13.806833 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:13.807360 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:13.807976 20807 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:23:13.801654 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:13.808254 25402 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:13.808683 20804 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:13.793246 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:13.811066 20801 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:23:13.793856 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.812459 25299 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:13.794600 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:13.737289 25327 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:13.726945 25190 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:13.814519 25190 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.817921 25190 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.820426 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 714) took 1388 ms. Trace:
W20260709 12:23:13.761232 25409 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 12:23:13.798755 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.790910 25392 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
W20260709 12:23:13.798491 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51970.8us
I20260709 12:23:13.798460 25322 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.795799 25399 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:13.819903 25230 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:13.822751 20806 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:23:13.844218 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 279229us
I20260709 12:23:13.829718 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:13.812875 25299 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:13.846084 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 197065us
I20260709 12:23:13.820852 20805 rpcz_store.cc:276] 0709 12:23:12.432415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.574028 (+141613us) service_pool.cc:224] Handling call
0709 12:23:12.625774 (+ 51746us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:12.650551 (+ 24777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:12.650597 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:12.651151 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:23:13.103872 (+452721us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:13.103913 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.103973 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.820178 (+716205us) raft_consensus.cc:1685] finished
0709 12:23:13.820199 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.820366 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":442717,"threads_started":1,"wal-append.queue_time_us":647807}
I20260709 12:23:13.830047 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:13.848875 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:13.826130 25414 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.889532 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:13.802500 25407 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:13.803691 25234 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:13.770390 25348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.020s	sys 0.005s
I20260709 12:23:13.846340 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:13.802028 20733 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:13.832934 20807 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:13.822104 25415 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:13.888584 25359 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:23:13.770650 25331 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.921576 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2176) took 1558 ms. Trace:
I20260709 12:23:13.893946 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:13.804167 25389 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:13.899024 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50231us
I20260709 12:23:13.844732 25332 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:13.895530 25313 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:13.894457 25190 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.963958 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2172) took 1512 ms. Trace:
I20260709 12:23:13.918123 25402 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:13.947067 25382 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:13.889340 25338 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:13.891788 25396 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:13.821467 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:13.841449 25322 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:13.955945 25311 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:13.819243 25323 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:13.918190 25408 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:13.900311 25384 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:13.948314 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54497.6us
I20260709 12:23:14.071506 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.071653 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.072425 20807 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:13.964425 20875 rpcz_store.cc:276] 0709 12:23:12.451722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.539171 (+ 87449us) service_pool.cc:224] Handling call
0709 12:23:12.551596 (+ 12425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:12.571652 (+ 20056us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:12.624356 (+ 52704us) log.cc:844] Serialized 14 byte log entry
0709 12:23:13.106783 (+482427us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:13.107553 (+   770us) log.cc:844] Serialized 52 byte log entry
0709 12:23:13.107722 (+   169us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:13.107769 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.107859 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.735898 (+628039us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b180043b520. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:13.961754 (+225856us) raft_consensus.cc:1685] finished
0709 12:23:13.963707 (+  1953us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.963888 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19626,"prepare.queue_time_us":49483,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":14367,"raft.queue_time_us":27172,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":52631,"spinlock_wait_cycles":4225408,"thread_start_us":435995,"threads_started":1,"wal-append.queue_time_us":662088}
I20260709 12:23:13.936486 20803 rpcz_store.cc:276] 0709 12:23:12.363601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.568003 (+204402us) service_pool.cc:224] Handling call
0709 12:23:12.568409 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:12.573754 (+  5345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:12.573811 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:12.617820 (+ 44009us) log.cc:844] Serialized 26 byte log entry
0709 12:23:13.105933 (+488113us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:13.105995 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.106103 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.921321 (+815218us) raft_consensus.cc:1685] finished
0709 12:23:13.921357 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.921514 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4920,"raft.queue_time_us":31541,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3518,"thread_start_us":456011,"threads_started":1,"wal-append.queue_time_us":750596}
I20260709 12:23:13.970495 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:14.075131 20881 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:13.918462 25348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.021s	sys 0.005s
I20260709 12:23:13.851372 25299 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:13.917977 25327 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.898053 25234 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:13.960192 25313 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:14.008055 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 115116us
I20260709 12:23:14.074406 21435 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 10: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:13.894464 25414 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.104467 25322 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.104703 25414 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.069720 25216 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:14.105798 25182 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.892s	user 0.008s	sys 0.004s
I20260709 12:23:14.106240 25323 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.105964 25216 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.107851 20881 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 9
I20260709 12:23:14.108206 20881 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 11.10
W20260709 12:23:13.818923 25247 consensus_queue.cc:1175] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 10 status { last_received { term: 7 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: [9.10-9.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:13.982072 25239 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 12:23:14.106292 25182 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.110119 21430 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:13.972146 25423 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:14.112036 25420 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:23:14.112226 25423 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.114897 25423 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:14.124076 25422 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:23:14.124645 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:14.125926 25410 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:14.127647 20728 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:14.130151 25182 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.131717 25410 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:14.099499 25327 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.101464 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.133641 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.079658 25396 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.075670 25339 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:23:13.982235 25382 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.108462 20881 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:14.108599 25406 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.093514 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:14.156641 20875 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:14.117560 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2188) took 1057 ms. Trace:
I20260709 12:23:14.157857 20882 rpcz_store.cc:276] 0709 12:23:13.060321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.116725 (+1056404us) service_pool.cc:224] Handling call
0709 12:23:14.117037 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.117218 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:14.117246 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:14.117295 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:14.117335 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.117402 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:14.117504 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:14.119398 21432 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:14.119928 25313 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.077467 25348 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:13.981864 24963 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.172787 20875 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:23:14.122885 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.122807 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.174218 20735 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:14.174288 20733 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:14.135138 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:14.088018 25359 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:23:14.142305 25401 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:14.149196 25421 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:23:14.151223 25389 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.152292 25331 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:13.969460 25190 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.130625 25259 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:14.142594 25311 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:23:14.142658 25246 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:14.107093 25419 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:23:14.135253 25384 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:14.108600 25216 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.189988 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:14.192157 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111487us
I20260709 12:23:14.206112 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:14.193080 25401 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:14.206178 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.182305 25339 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:14.209240 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.195699 25259 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:14.221786 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 121509us
I20260709 12:23:14.226301 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.208863 25246 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:14.227239 20733 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:14.227118 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:14.210639 25389 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:14.234453 20735 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.228068 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.201237 25404 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.234630 25246 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.235210 25404 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.237239 25404 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:14.179114 24963 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:14.220983 25174 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.840s	user 0.013s	sys 0.000s
I20260709 12:23:14.215873 25307 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:14.211796 25326 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:14.243053 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.243443 20735 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:14.243908 20809 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:23:14.240168 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.219743 25327 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.218640 25339 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.258339 24890 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.247794 25326 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.259425 25326 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.261108 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2187) took 1233 ms. Trace:
I20260709 12:23:14.261499 20881 rpcz_store.cc:276] 0709 12:23:13.027903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.095820 (+1067917us) service_pool.cc:224] Handling call
0709 12:23:14.107617 (+ 11797us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:14.154090 (+ 46473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:14.154126 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:14.154720 (+   594us) log.cc:844] Serialized 52 byte log entry
0709 12:23:14.228050 (+ 73330us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:14.228093 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.228162 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:14.260880 (+ 32718us) raft_consensus.cc:1685] finished
0709 12:23:14.260902 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:14.261049 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9167,"raft.queue_time_us":9296,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":72956,"threads_started":1,"wal-append.queue_time_us":104487}
W20260709 12:23:14.266681 25315 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.413s	user 0.025s	sys 0.021s
I20260709 12:23:14.268548 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:14.269771 25324 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:14.269886 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.270207 25324 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:14.270619 25324 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:14.272354 25325 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:14.274494 25345 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.062s	user 0.012s	sys 0.000s
I20260709 12:23:14.279042 25403 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.279613 25345 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.281196 25417 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.281217 20727 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 7.
I20260709 12:23:14.284623 25337 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:14.285063 25337 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:14.245515 25430 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.285912 25337 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.286808 25430 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.287552 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.288218 25337 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.246213 25307 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.290130 25307 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:14.227399 25182 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:14.296496 25382 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.298923 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:14.299505 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.299688 20807 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.299899 24863 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:14.300132 20735 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:14.300369 24863 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:14.301046 25314 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:14.301447 25314 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:14.301295 24863 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.302320 24863 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.302256 25314 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.304067 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.304450 24963 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.304981 24963 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:14.305356 24963 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.305686 24963 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.305946 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.306587 20735 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:14.307554 25401 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.312448 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.282934 25417 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.252169 25343 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.316016 25299 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.316622 25343 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.317111 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.318217 20805 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:14.318518 25299 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.319728 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41760 (request call id 2) took 1674 ms. Trace:
I20260709 12:23:14.320251 20878 rpcz_store.cc:276] 0709 12:23:12.645549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.687424 (+ 41875us) service_pool.cc:224] Handling call
0709 12:23:14.319484 (+1632060us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec99853 00007f9e23c6760a 00007f9e1d1ed7d4 00007f9e1d1ed762 00007f9e1d1ed691 00007f9e1d1ed60c 00007f9e1d1ec8c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:14.319596 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628033792}
I20260709 12:23:14.321477 20883 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:14.323019 25431 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:23:14.325328 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.326067 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2402) took 2669 ms. Trace:
I20260709 12:23:14.327484 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:14.327630 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.328161 20803 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:14.323051 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.330976 25311 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.331223 25014 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:23:14.332691 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.333397 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.321398 20876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2247) took 2119 ms (client timeout 1834 ms). Trace:
W20260709 12:23:14.335161 20876 rpcz_store.cc:269] 0709 12:23:12.202372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.202585 (+   213us) service_pool.cc:224] Handling call
0709 12:23:14.321133 (+2118548us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b540005b254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:14.321309 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2242576640}
I20260709 12:23:14.326654 20809 rpcz_store.cc:276] 0709 12:23:11.656226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.711283 (+ 55057us) service_pool.cc:224] Handling call
0709 12:23:11.711603 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.248493 (+2536890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.325709 (+ 77216us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:14.325964 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5822603136}
I20260709 12:23:14.337086 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.338862 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.339748 21435 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:14.334584 21432 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:23:14.341476 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.342310 21435 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:14.315932 25339 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.253677 25348 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.347141 20700 leader_election.cc:400] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:14.363916 20700 leader_election.cc:403] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:23:14.209477 25384 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.364696 25210 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:14.364823 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.366827 20803 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:23:14.350601 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63207us
I20260709 12:23:14.368552 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.289289 25430 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.377761 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.363407 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.334651 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.357609 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.381009 20804 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:14.357481 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.382336 20809 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:23:14.359632 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.359342 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.385102 20810 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:14.350425 20807 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:14.358641 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:14.379695 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.373976 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.363406 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1144) took 1229 ms. Trace:
I20260709 12:23:14.375939 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.397502 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:14.384292 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49735.3us
I20260709 12:23:14.352674 20802 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.400749 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.286052 25217 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:14.403028 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43483.7us
I20260709 12:23:14.403842 20805 raft_consensus.cc:2368] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:23:14.409977 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:14.406955 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.391460 21427 rpcz_store.cc:276] 0709 12:23:13.142647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.149996 (+  7349us) service_pool.cc:224] Handling call
0709 12:23:13.150426 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:13.527699 (+377273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:13.527769 (+    70us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:13.577001 (+ 49232us) log.cc:844] Serialized 52 byte log entry
0709 12:23:13.889985 (+312984us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:13.890030 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.890095 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:14.363131 (+473036us) raft_consensus.cc:1685] finished
0709 12:23:14.363153 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:14.363337 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34739,"raft.queue_time_us":161661,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":17004,"thread_start_us":634678,"threads_started":2,"wal-append.queue_time_us":394219}
W20260709 12:23:14.377454 25376 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.732s	user 0.006s	sys 0.005s
I20260709 12:23:14.418108 20728 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.352559 25190 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:14.414883 25220 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:14.425596 25220 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.426075 25220 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.429011 25384 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.414912 25327 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:23:14.432406 25327 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:14.432828 25327 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:14.433393 25230 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.001s
I20260709 12:23:14.433952 25327 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.435446 25439 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:14.437645 25399 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.587s	user 0.004s	sys 0.004s
I20260709 12:23:14.438133 25399 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.403362 25182 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:14.350517 25330 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:14.441013 25330 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.402347 25311 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:14.402536 25404 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.445367 25330 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.447400 25399 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:14.405452 25188 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.648s	user 0.008s	sys 0.003s
I20260709 12:23:14.448781 25188 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:14.448873 25343 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.285863 25345 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.423300 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:14.415114 25430 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:23:14.452189 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54774.9us
I20260709 12:23:14.422477 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:14.496147 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72945.9us
W20260709 12:23:14.499001 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76643us
I20260709 12:23:14.500259 25385 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:14.500566 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.500804 25385 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:14.501730 25385 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.502242 25434 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:14.502844 25434 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:14.505394 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:14.508371 25247 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:14.508829 25247 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.509222 25247 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:14.450767 25246 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:14.498517 25399 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:14.512887 25399 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:14.513870 25399 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.450188 25314 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.450592 25379 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.524039 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:14.535734 20876 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:14.517048 21427 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:14.541215 25327 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.495471 25188 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.526367 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.361403 25234 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.562143 24963 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.563719 20884 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:23:14.527138 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:14.528401 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:14.529506 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.565816 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.534135 25430 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:14.543674 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:14.350471 25407 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.506395 25330 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:14.558552 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2170) took 2121 ms. Trace:
I20260709 12:23:14.572870 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:14.559255 20880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2255) took 2329 ms (client timeout 2232 ms). Trace:
I20260709 12:23:14.505095 25434 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.496121 25427 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:23:14.576773 20883 rpcz_store.cc:276] 0709 12:23:12.437197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.536006 (+ 98809us) service_pool.cc:224] Handling call
0709 12:23:14.321288 (+1785282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.321903 (+   615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.558265 (+236362us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b540005b250. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:14.558466 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4105009536}
W20260709 12:23:14.585290 20773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.012s	sys 0.020s
W20260709 12:23:14.585672 20773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.012s	sys 0.020s
I20260709 12:23:14.568349 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
W20260709 12:23:14.578086 20880 rpcz_store.cc:269] 0709 12:23:12.229543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.309296 (+ 79753us) service_pool.cc:224] Handling call
0709 12:23:14.558956 (+2249660us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400267854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:14.559134 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2660682624}
W20260709 12:23:14.586728 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.516081 25404 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 12:23:14.595090 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.451694 25376 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.505867 25259 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:14.631405 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.631249 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 297163us
W20260709 12:23:14.633987 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.634845 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.576216 25379 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:14.573400 25210 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:14.644227 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:14.603017 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.617811 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.572667 25389 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:14.576411 25382 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:14.556952 25337 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:23:14.592971 25264 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:14.596936 25174 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.645852 20810 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 8: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:14.660104 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.660772 20810 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:14.597075 25217 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.649552 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.594688 25408 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:14.602046 25311 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:14.686247 25311 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:14.646114 20809 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:14.576665 25330 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:14.652927 21433 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:14.631330 25254 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.653746 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.642957 25247 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.630995 25190 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.638833 25402 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:14.643985 25379 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.666457 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.640367 25117 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.738826 25117 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.741308 25117 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.742038 25408 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:14.746181 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 79822.4us
I20260709 12:23:14.746773 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:14.747418 21429 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:14.749152 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:14.749385 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.750212 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.750932 21431 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:14.752094 21431 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:14.752560 20878 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:14.753038 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.749799 21427 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.691010 25330 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.668016 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.630688 25246 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:23:14.687652 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.566707 25188 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.597148 25230 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.660229 25382 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:14.638695 25326 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.639744 25313 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.639991 25324 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.691043 25217 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.705725 25418 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.707849 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2184) took 1426 ms. Trace:
I20260709 12:23:14.773731 25417 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:14.708635 25220 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.824345 25220 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.714210 25439 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.825594 20801 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:14.715296 25234 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.716341 24963 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.719058 25451 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.685010 25339 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.826823 25451 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.828504 25339 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:14.829237 25451 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.708742 25407 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.729476 20732 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.831549 21427 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:14.733635 25329 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:14.734817 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1163) took 1119 ms. Trace:
I20260709 12:23:14.736902 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2171) took 2299 ms. Trace:
I20260709 12:23:14.695465 25254 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.763000 21428 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:23:14.838865 21428 pending_rounds.cc:85] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 9
I20260709 12:23:14.839221 21428 pending_rounds.cc:107] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:23:14.839486 21428 raft_consensus.cc:2889] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:14.765427 20881 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:14.664342 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:23:14.841197 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 177024us
I20260709 12:23:14.664083 25264 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:14.770718 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102795us
I20260709 12:23:14.670113 25174 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.745121 25379 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.811362 20727 rpcz_store.cc:276] 0709 12:23:13.281709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.290846 (+  9137us) service_pool.cc:224] Handling call
0709 12:23:14.707631 (+1416785us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54000c3fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:14.707773 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162681088}
I20260709 12:23:14.708777 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1161) took 1222 ms. Trace:
I20260709 12:23:14.854048 21434 rpcz_store.cc:276] 0709 12:23:13.486240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.553592 (+ 67352us) service_pool.cc:224] Handling call
0709 12:23:14.708589 (+1154997us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54006333d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:14.708713 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102288768}
I20260709 12:23:14.769426 25239 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:14.803280 25307 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.002s
I20260709 12:23:14.778182 25323 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:14.874394 25307 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.876377 25326 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.876982 25307 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.818661 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:14.699872 25190 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.878645 20808 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:14.879385 25190 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.879496 25324 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.804262 25230 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.881172 20700 leader_election.cc:400] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:23:14.786511 25370 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:14.881517 20700 leader_election.cc:403] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:23:14.881567 25174 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.881780 25370 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.834380 25329 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:14.882200 25370 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:14.883001 25329 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.884361 25329 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:14.884616 25433 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.775s	user 0.003s	sys 0.000s
I20260709 12:23:14.885113 25313 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.689879 25311 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.885479 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.886230 20726 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:14.887104 25456 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:23:14.887461 25210 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:14.888213 25188 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:14.888084 25210 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.835281 20803 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:14.838065 20806 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:14.838327 21429 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:14.834518 21433 rpcz_store.cc:276] 0709 12:23:13.615951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.648485 (+ 32534us) service_pool.cc:224] Handling call
0709 12:23:13.648934 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.689742 (+1040808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.734542 (+ 44800us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54005f4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:14.734752 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308456832}
I20260709 12:23:14.837471 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:14.847743 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:14.892428 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:14.893158 20735 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:23:14.856041 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.859730 25399 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:14.864033 25443 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 12:23:14.868845 25432 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.760s	user 0.004s	sys 0.001s
W20260709 12:23:14.949443 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49319.2us
W20260709 12:23:14.871461 25447 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
I20260709 12:23:14.873198 25264 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.874522 25323 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.815310 25376 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.877385 25259 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:14.763883 25402 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.885136 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:14.987025 21428 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.887846 25382 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.835436 20884 rpcz_store.cc:276] 0709 12:23:12.437472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.538337 (+100865us) service_pool.cc:224] Handling call
0709 12:23:14.563389 (+2025052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:14.564276 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.736673 (+172397us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400267850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:14.736834 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4656368896}
I20260709 12:23:14.837117 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:14.802536 25325 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.855298 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2442) took 1234 ms. Trace:
I20260709 12:23:14.993049 20732 rpcz_store.cc:276] 0709 12:23:13.620905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.623291 (+  2386us) service_pool.cc:224] Handling call
0709 12:23:13.623622 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.729081 (+1105459us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b58005684d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:23:14.854023 (+124942us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:14.854084 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:14.854162 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:14.854227 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.854956 (+   729us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:14.855216 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3039,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":455102848}
I20260709 12:23:14.995518 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.947481 25174 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:14.896622 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:14.897651 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49983.9us
I20260709 12:23:14.899255 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.009366 20735 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:14.901057 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.898375 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:14.907629 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.898144 25396 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:14.898494 25406 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:14.924482 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.881567 25230 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:14.935750 25324 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.936596 25217 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:14.939276 25190 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:14.939913 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83958.6us
I20260709 12:23:15.037010 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2192) took 1354 ms. Trace:
I20260709 12:23:15.038542 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:14.938642 25311 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.940645 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:14.925320 25390 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:14.896638 25239 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.946372 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:14.895608 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:15.048995 20804 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.784093 25389 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.852638 24963 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:14.953636 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:14.955932 25399 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.951682 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:14.957715 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:15.036104 25429 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.933s	user 0.000s	sys 0.003s
I20260709 12:23:14.831084 25434 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:14.843781 25417 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:14.985498 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.096285 21430 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:14.948923 25337 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:14.985258 25458 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.135s	user 0.000s	sys 0.002s
I20260709 12:23:14.997004 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.895968 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:14.955777 25381 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.005239 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.015004 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.017076 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.018011 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:15.023552 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 125244us
I20260709 12:23:15.029559 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.032150 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:15.033881 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.035020 20808 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:23:15.036087 25464 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:23:15.037482 20730 rpcz_store.cc:276] 0709 12:23:13.682564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.805345 (+122781us) service_pool.cc:224] Handling call
0709 12:23:13.805742 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:13.952920 (+147178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:13.952967 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:13.960853 (+  7886us) log.cc:844] Serialized 26 byte log entry
0709 12:23:14.515695 (+554842us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:14.515736 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.515822 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:15.036751 (+520929us) raft_consensus.cc:1685] finished
0709 12:23:15.036781 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:15.036944 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83673,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":6583,"thread_start_us":594334,"threads_started":2,"wal-append.queue_time_us":660665}
I20260709 12:23:15.043578 20803 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:15.045652 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2191) took 1363 ms. Trace:
I20260709 12:23:15.045722 20881 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:15.136001 20734 rpcz_store.cc:276] 0709 12:23:13.681850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.770518 (+ 88668us) service_pool.cc:224] Handling call
0709 12:23:13.770871 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:13.772088 (+  1217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:13.772120 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:13.790777 (+ 18657us) log.cc:844] Serialized 26 byte log entry
0709 12:23:13.919516 (+128739us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:13.919560 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.919637 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:14.868156 (+948519us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:15.031009 (+162853us) raft_consensus.cc:1685] finished
0709 12:23:15.031033 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:15.045588 (+ 14555us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20833,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5122,"spinlock_wait_cycles":78444416,"thread_start_us":84895,"threads_started":1,"wal-append.queue_time_us":259701}
I20260709 12:23:15.138684 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.106248 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:15.140101 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:15.046347 20735 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:15.047224 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.141147 20803 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:15.141373 20884 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:15.051265 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:15.051403 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.142719 20731 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:15.143492 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.143895 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.144769 20731 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:15.145165 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.052642 20875 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:15.053745 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:14.978258 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.148350 21434 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:15.054677 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 159158us
I20260709 12:23:15.148624 20879 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:15.149646 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:15.054735 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 103137us
I20260709 12:23:15.150199 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.059478 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.152339 20728 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:15.059887 25217 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:15.060786 25239 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.061352 25311 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.063319 25247 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.064706 25402 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:15.046283 25324 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.059494 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.067219 25254 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.067410 24963 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:15.071306 20804 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:15.072655 25399 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.049041 25264 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:15.043889 25390 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:15.077431 25406 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:23:15.023211 25325 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.176920 25325 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.031056 25407 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:15.161368 25326 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:15.160116 25376 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.078586 21427 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:15.178624 25376 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.180912 25376 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.049409 25190 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.080559 25323 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:15.083218 25210 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:15.085629 25460 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:23:15.087556 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.089149 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:15.153321 20727 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:15.153854 20883 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:15.216233 21435 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:15.218014 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.160782 25329 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:15.220500 20727 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:15.220886 25324 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.222304 20727 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:15.065982 25379 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.184582 25337 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:15.165385 25335 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:15.225759 25335 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.152949 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.227805 20877 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:15.228452 25335 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.229698 25379 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.160077 25381 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.231060 21433 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:15.232748 20804 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:15.232993 25326 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:15.234087 25326 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:15.234426 25326 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:15.237509 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:15.158354 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:15.158404 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 126340us
I20260709 12:23:15.238938 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:15.160683 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:15.162741 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111589us
I20260709 12:23:15.156280 20882 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:15.239893 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.160789 20729 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.240178 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:15.163174 21429 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:15.240720 21435 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:23:15.168076 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.169234 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:15.166046 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:15.173282 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:15.183697 20879 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:15.185225 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:15.186127 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:15.249601 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63561.5us
I20260709 12:23:15.252193 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:15.157827 25313 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:15.171129 25174 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:15.285925 25404 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:15.286525 25408 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:15.286613 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.287374 20804 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:15.287298 25408 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.288362 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.288942 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:15.289718 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.290452 21434 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:15.232790 25381 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:15.184273 25217 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:15.148375 20732 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:15.239136 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.162453 20731 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:23:15.162320 21428 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:15.160296 25435 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 12:23:15.095793 25330 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.271390 20875 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.238412 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:15.311786 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73432.2us
I20260709 12:23:15.278189 20728 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:15.279553 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 113613us
I20260709 12:23:15.317255 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:15.280580 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107447us
I20260709 12:23:15.318259 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.171785 24963 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.255151 20803 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:15.286032 25174 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.292991 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:15.218851 20883 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:15.182894 25190 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:15.335664 20883 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:15.256338 21433 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:15.300055 25415 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:15.300005 21427 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:15.308189 25314 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:15.241091 25329 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.282202 25401 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:15.298171 25387 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:15.291405 24746 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:15.322008 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:15.323809 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.329177 25174 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:15.299300 25408 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.333688 20881 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:15.302716 25313 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:15.202869 25399 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.345906 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.345438 20878 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:15.364316 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.368117 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.355115 20883 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:15.366844 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.361632 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:15.361764 20803 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:15.379714 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.380470 20803 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:15.380744 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.381521 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.326296 25435 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:23:15.389119 20884 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:15.355245 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.336259 25210 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:15.355962 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:15.363652 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:15.362771 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:15.355476 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62597.8us
I20260709 12:23:15.391923 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:15.392872 20883 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:15.394011 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.395053 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:15.357986 20880 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:15.320420 25404 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:15.325577 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.356204 25329 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.386823 20731 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:15.285429 25326 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.402243 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.361927 25314 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:15.358651 25190 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.416077 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:15.386018 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:15.404372 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.417647 20728 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:15.418114 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:15.367316 24746 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:15.364236 25401 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:15.426151 25468 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:15.426609 25401 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.391618 25399 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.428026 24746 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.430483 25399 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.360765 25014 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:23:15.423516 25326 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.409228 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.409530 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.434624 20804 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:15.362432 25415 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.442366 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.412683 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.373039 25307 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.365957 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.357146 25197 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:15.356348 25234 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.406567 20882 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:23:15.420516 25407 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.420748 25405 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:15.454177 25405 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.423650 25434 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.391032 25370 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.423458 25313 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.423426 25348 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.424885 25314 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.367527 25217 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.435619 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:15.411355 25117 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:23:15.369305 20875 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:15.440855 20729 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:15.440881 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:15.408393 25335 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:15.471522 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:15.484699 25335 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:15.444139 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:15.408890 25220 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:15.422803 25417 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.455523 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:15.471907 25339 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:23:15.483850 20882 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:15.407346 25404 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.485635 20804 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
W20260709 12:23:15.488507 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47742us
I20260709 12:23:15.497430 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:15.445086 25408 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:15.446496 25311 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:23:15.449527 25399 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:15.495407 25084 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:15.489313 25471 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:23:15.498497 25404 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.501282 25408 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.491081 25348 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.501544 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:15.441845 25331 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:23:15.477334 25415 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.488128 25323 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:15.521441 25399 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:15.521905 25399 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.522269 25399 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.515152 25408 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.492657 25325 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:15.486722 25407 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.505728 25335 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.507920 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:15.436856 25190 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.452867 25307 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.528882 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.497680 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:15.456305 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:15.535792 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.567s (SENT) [suppressed 21 similar messages]
I20260709 12:23:15.492800 25264 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:23:15.517810 25220 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:15.528769 25325 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:15.554610 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.555222 20876 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:15.559851 20883 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:15.561868 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.562778 20883 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:15.529487 25084 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:15.540417 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:15.567581 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.571067 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.571712 20729 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.576323 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.550209 25220 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:15.556926 25325 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.603762 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.604488 20728 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:15.566466 25084 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.613963 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:15.611073 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 204911us
I20260709 12:23:15.610333 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:15.619349 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 181092us
I20260709 12:23:15.626192 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:15.626925 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:15.601399 21427 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 12:23:15.575423 25339 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:15.628039 25339 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.559545 25387 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:15.628512 25339 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.603084 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.629545 20880 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12.
I20260709 12:23:15.630772 25417 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:15.627202 20882 raft_consensus.cc:2368] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:23:15.636550 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:15.616613 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:15.639816 25387 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:15.640188 25387 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.640518 25387 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.641270 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:15.648590 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.649693 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:15.621994 20883 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:15.593719 20729 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:15.633484 25264 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.631325 25417 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:15.611620 25084 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:15.674266 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 138510us
I20260709 12:23:15.674551 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.674975 20880 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:15.677846 20883 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:15.684262 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.688344 20880 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:15.707362 20883 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:15.680644 25065 raft_consensus.cc:2764] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:23:15.668824 25264 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:15.711019 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 209527us
I20260709 12:23:15.716239 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.675609 25434 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:23:15.717507 20880 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:15.692724 25370 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:15.712108 25475 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 12:23:15.723671 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87237.6us
I20260709 12:23:15.711792 25405 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.693040 25174 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.684908 25220 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:15.741047 25474 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:23:15.741176 25190 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.729094 25348 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:15.742583 25481 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:15.742807 25264 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:15.743312 25473 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:23:15.744638 25307 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.744047 25339 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:15.745546 25479 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:15.730773 25407 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.748343 25493 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:15.749955 25493 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:15.744984 25477 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:23:15.756356 25459 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.910s	user 0.004s	sys 0.003s
I20260709 12:23:15.757442 25408 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.762639 25330 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.826902 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2434) took 1161 ms. Trace:
I20260709 12:23:15.860916 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.880894 20802 rpcz_store.cc:276] 0709 12:23:14.665149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.022655 (+357506us) service_pool.cc:224] Handling call
0709 12:23:15.031683 (+  9028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.031925 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.031952 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.032414 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.291641 (+259227us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:15.291682 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.291744 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:15.825903 (+534159us) raft_consensus.cc:1685] finished
0709 12:23:15.825985 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:15.826465 (+   480us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":258921,"threads_started":1,"wal-append.queue_time_us":503425}
I20260709 12:23:15.883558 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.884552 25339 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.861717 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:15.885812 20878 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:15.723249 25417 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.836503 20880 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:15.836691 20810 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:15.845582 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:15.846136 21435 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:23:15.846611 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:15.727849 25065 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 12:23:15.852304 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:15.855122 20882 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:23:15.856719 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.858892 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.860719 25246 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.911542 21434 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [8.8-11.9]
I20260709 12:23:15.868189 20805 raft_consensus.cc:2368] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10.
I20260709 12:23:15.867539 25325 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:15.907792 25327 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.085s	user 0.007s	sys 0.021s
I20260709 12:23:15.915350 25376 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.916546 25325 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.916441 25376 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.765311 25234 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:15.861963 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:15.918931 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.920356 20733 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:15.873878 25314 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.763988 25323 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:15.928181 25379 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:15.929670 25405 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:15.776093 25399 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.934172 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:15.765637 25493 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:15.918941 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:15.763197 25472 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 12:23:15.832760 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.941072 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:15.941493 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.942509 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:15.773525 25404 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.955945 25376 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.956127 25065 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:15.956643 25065 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:15.958187 25065 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.963174 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.965790 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.858779 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 966) took 1086 ms. Trace:
I20260709 12:23:15.860081 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:15.886088 21431 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
W20260709 12:23:15.915722 25327 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.008s	sys 0.021s
I20260709 12:23:15.916646 25331 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:16.004240 21434 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:23:15.870137 25174 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:15.923643 25434 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.006868 25434 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.009936 25434 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.924636 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.925186 25314 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.012941 25314 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.929391 25379 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.934361 25399 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:15.876188 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.014357 21428 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:15.766686 25408 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:15.937112 21427 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
W20260709 12:23:16.015576 25500 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:23:15.945174 20884 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.016204 25379 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.018051 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:15.994920 20877 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:16.019260 21433 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:23:15.995275 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:16.021112 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.021819 20881 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
W20260709 12:23:15.999598 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54985.8us
I20260709 12:23:16.023437 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:16.024195 21431 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 7.
I20260709 12:23:16.024884 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:16.025060 25084 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:15.999856 25483 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:16.026981 25234 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:16.027976 25483 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:15.999636 25503 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.003813 25331 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:15.872095 25220 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:16.027266 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.041525 20880 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11.
W20260709 12:23:16.000450 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 266833us
I20260709 12:23:16.001539 20808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47850
I20260709 12:23:16.035131 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.036885 20884 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
W20260709 12:23:16.036917 24963 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.666s	user 0.004s	sys 0.013s
I20260709 12:23:16.050024 25503 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.050992 20880 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:16.052801 25503 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.050025 24963 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.056757 24963 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.057478 20806 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:15.972122 21426 rpcz_store.cc:276] 0709 12:23:14.772376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.772554 (+   178us) service_pool.cc:224] Handling call
0709 12:23:14.772907 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:14.778635 (+  5728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:14.778665 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.295493 (+516828us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:23:15.300541 (+  5048us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.537254 (+236713us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:15.537297 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.537362 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:15.858010 (+320648us) raft_consensus.cc:1685] finished
0709 12:23:15.858050 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:15.858490 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6199,"raft.queue_time_us":11893,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":12570,"spinlock_wait_cycles":4124032,"thread_start_us":219418,"threads_started":1,"wal-append.queue_time_us":365892}
I20260709 12:23:16.062337 25331 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.062844 20731 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:23:16.063303 20731 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:23:16.063666 20731 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:16.065604 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.040199 25065 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:16.072459 20808 raft_consensus.cc:606] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Received request to transfer leadership
W20260709 12:23:16.070712 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 43579.4us
I20260709 12:23:16.076061 21428 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:16.073004 20808 raft_consensus.cc:614] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:16.049088 25174 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:16.086058 25406 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:23:16.086486 24963 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:16.086966 24963 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.087371 24963 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:16.092432 21435 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:16.093545 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2205) took 1035 ms. Trace:
W20260709 12:23:16.094146 25329 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.004s
I20260709 12:23:16.094092 20879 rpcz_store.cc:276] 0709 12:23:15.057769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.149903 (+ 92134us) service_pool.cc:224] Handling call
0709 12:23:16.093067 (+943164us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b5400305a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:16.093325 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74863488}
I20260709 12:23:16.085474 25174 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:16.105466 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.105252 25174 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.107510 25348 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:23:16.108031 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59219.3us
W20260709 12:23:16.086531 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.048877 25434 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.115111 21430 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:23:16.115695 21430 pending_rounds.cc:85] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 9
I20260709 12:23:16.115954 21430 pending_rounds.cc:107] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:23:16.116238 21430 raft_consensus.cc:2889] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:16.114444 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.107272 25174 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:16.123656 21429 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 12:23:16.123159 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.094800 25329 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:16.133471 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 100451us
W20260709 12:23:16.136086 20801 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.008s	sys 0.004s
I20260709 12:23:16.125828 25247 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:23:16.142549 20801 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.008s	sys 0.004s
I20260709 12:23:16.141981 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:16.143342 20729 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:16.148976 25434 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.151301 20883 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:16.152299 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.151935 25385 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:16.154539 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:16.154208 25390 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.160352 20776 leader_election.cc:400] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:23:16.160812 20776 leader_election.cc:403] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:23:16.159649 25387 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.007s
I20260709 12:23:16.164479 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:16.164819 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:16.164896 25387 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.165936 20807 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:16.167289 25197 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:16.167786 25197 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:16.168251 25197 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:16.170362 20734 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:16.171312 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.171885 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:16.172402 25217 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:23:16.173025 25434 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.172010 20730 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:16.147384 25246 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:16.141748 25247 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:16.176930 25404 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.174582 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:16.176751 25503 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:16.197237 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:16.179761 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:16.183336 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:16.198520 20807 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.183020 20808 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:16.158315 25348 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:23:16.194156 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:16.196174 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:16.203755 20880 raft_consensus.cc:606] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:23:16.197337 25503 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.204273 20880 raft_consensus.cc:614] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:16.204867 25503 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:16.208593 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 77926.7us
I20260709 12:23:16.209048 25329 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.209781 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:16.210379 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56011.4us
I20260709 12:23:16.210829 25322 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:23:16.212723 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2195) took 1312 ms. Trace:
I20260709 12:23:16.212631 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:16.213160 20806 rpcz_store.cc:276] 0709 12:23:14.900610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.060955 (+160345us) service_pool.cc:224] Handling call
0709 12:23:15.061281 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:16.058352 (+997071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.212436 (+154084us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b5400127d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:16.212650 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2288107264}
I20260709 12:23:16.217099 25390 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:16.192338 25247 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.218684 25247 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.219697 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.220511 20880 raft_consensus.cc:2376] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10. Re-sending same reply.
I20260709 12:23:16.220675 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:16.212484 25322 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:16.221556 25322 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:16.222837 25322 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:16.224426 25478 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.497s	user 0.003s	sys 0.000s
I20260709 12:23:16.224994 25337 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.225692 25337 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.190915 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.227483 25435 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:16.227909 25435 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:16.229153 25435 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.230675 25435 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:16.232040 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.233422 25337 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:16.177130 25084 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:16.245425 25514 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:23:16.252219 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.252290 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:16.256016 25318 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:16.256304 25486 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:16.255626 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:16.257382 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2198) took 1356 ms. Trace:
I20260709 12:23:16.257705 25486 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:16.258711 25486 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.259083 25486 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.259444 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:16.258236 20801 rpcz_store.cc:276] 0709 12:23:14.901226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.209495 (+308269us) service_pool.cc:224] Handling call
0709 12:23:15.209808 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:15.210014 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:15.210312 (+   298us) log.cc:844] Serialized 14 byte log entry
0709 12:23:16.178944 (+968632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:16.255484 (+ 76540us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:16.255610 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.256871 (+  1261us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.257278 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1037,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":11672,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3574,"thread_start_us":932232,"threads_started":2,"wal-append.queue_time_us":413207}
I20260709 12:23:16.211222 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:16.204193 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:16.265791 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:16.266789 25396 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:16.267539 21399 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11.
I20260709 12:23:16.268563 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:16.295521 20803 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.298980 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:16.322558 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.323490 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:16.324222 20732 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.317651 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.326133 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.328460 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.333166 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.318380 25396 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:16.343056 20808 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.332412 25220 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:16.347262 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:16.347321 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 202669us
I20260709 12:23:16.350037 20734 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:16.317873 24963 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:16.361037 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 149910us
I20260709 12:23:16.362423 25509 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:16.363456 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:16.364164 20804 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.365401 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.366653 20876 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:16.349622 25220 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:16.368044 25220 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:16.371791 25389 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:16.372551 25389 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.375075 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.375758 25396 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.345090 25084 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:23:16.377739 25389 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.379029 25389 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:16.379125 25498 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:16.375980 25493 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:16.379746 25389 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.384263 25331 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:16.385192 25331 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.387718 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.389154 20876 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:16.395017 25387 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:16.399823 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1168) took 2022 ms (client timeout 1882 ms). Trace:
I20260709 12:23:16.402530 20802 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:16.362968 25509 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:16.406127 20803 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:16.405997 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:16.400396 21432 rpcz_store.cc:269] 0709 12:23:14.377288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.377449 (+   161us) service_pool.cc:224] Handling call
0709 12:23:16.399392 (+2021943us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54001e5d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:16.399588 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4516305664}
I20260709 12:23:16.412945 20775 leader_election.cc:400] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
W20260709 12:23:16.420274 25516 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:23:16.398622 25331 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.421344 25331 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:16.421932 25331 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.423236 24963 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.427919 25331 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.429504 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:16.430358 21435 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:16.410082 25509 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.452945 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.453833 21428 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:16.448127 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:16.414992 25399 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:16.490604 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 265675us
W20260709 12:23:16.499040 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 86197us
I20260709 12:23:16.502421 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2262) took 1001 ms. Trace:
I20260709 12:23:16.502877 20875 rpcz_store.cc:276] 0709 12:23:15.456595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.491324 (+ 34729us) service_pool.cc:224] Handling call
0709 12:23:16.095362 (+604038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:16.177694 (+ 82332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:16.177727 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:16.177791 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:16.177825 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.177902 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.455755 (+277853us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400305a50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:16.455859 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4916,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":83675,"spinlock_wait_cycles":1388177152}
I20260709 12:23:16.507350 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.508136 21435 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.498373 25247 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.515905 25247 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.518165 25247 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:16.523967 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142286us
I20260709 12:23:16.467695 25331 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:16.528326 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.528517 25331 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.529596 20849 leader_election.cc:400] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10.
I20260709 12:23:16.532651 25324 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:16.533098 25324 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.533469 25324 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:16.536520 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.530876 25331 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.545045 25220 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.548477 20806 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:23:16.464078 25399 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:16.530994 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.560956 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:16.566807 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.487725 25387 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:16.573458 20732 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:16.461373 25197 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:16.579420 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48638.5us
I20260709 12:23:16.555229 25399 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.590205 21434 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.601815 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:16.603539 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.594923 25404 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:16.616901 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.587555 25379 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.620306 20875 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:23:16.600271 25389 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:16.623714 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45096.7us
I20260709 12:23:16.591639 25387 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.620857 25404 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.639986 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:16.595896 25435 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:23:16.640699 20807 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:16.648599 21431 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:16.650920 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.651265 25401 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.651978 25401 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.655701 25220 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:16.657051 25518 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.483s	user 0.002s	sys 0.002s
I20260709 12:23:16.659096 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:16.659720 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.659981 20876 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:16.660684 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:16.661173 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:16.661862 20877 raft_consensus.cc:606] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:16.661933 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.662348 20877 raft_consensus.cc:614] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:16.662559 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.663231 20729 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:16.663285 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.663568 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:16.664312 25515 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.442s	user 0.000s	sys 0.003s
I20260709 12:23:16.664451 20734 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:16.664913 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.665467 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.661432 20884 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:16.666196 20735 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:16.656870 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:16.665755 20730 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:16.667480 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.667922 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:16.668283 21427 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.668856 20727 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:16.679800 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:16.682264 20802 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:16.688788 25197 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.690616 25379 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:16.691999 25525 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:23:16.693738 25379 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.641153 25404 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.702627 25385 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:16.703325 25385 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.706032 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.706598 25385 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:16.706818 20881 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:16.659277 25313 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:16.708539 25174 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.000s
I20260709 12:23:16.721451 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.722180 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:16.723570 25376 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:16.724382 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.725474 25376 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:16.726024 25376 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:16.727242 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:16.727035 25376 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.727967 25381 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:16.728796 25503 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.000s
I20260709 12:23:16.729314 25246 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:16.729377 25503 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.730124 25246 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.731089 20805 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:16.733413 25313 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.736693 25313 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.745836 25387 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.747292 21428 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:16.748701 20808 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:16.754432 21435 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:16.767818 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:16.770381 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47454.9us
I20260709 12:23:16.770673 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:16.771430 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:16.734117 25503 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.715487 25435 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:16.774375 21435 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:16.774878 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:16.772539 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47176.1us
I20260709 12:23:16.716794 25401 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:16.793336 21434 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:16.798415 21431 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:16.803201 20732 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:16.809391 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:16.791713 25435 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:16.817562 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.818945 21429 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:16.815794 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:16.811349 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 348618us
I20260709 12:23:16.820770 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:16.822664 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:16.823735 20804 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11.
I20260709 12:23:16.830834 20881 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11.
I20260709 12:23:16.834234 20807 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:23:16.837704 25313 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:16.838379 25313 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:16.840893 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.842311 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:16.843461 25498 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:16.844107 25498 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:16.843761 25451 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:23:16.845173 25498 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.847072 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.848727 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.851910 24963 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:16.852366 24963 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.852465 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.852758 24963 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:16.855082 21400 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11.
I20260709 12:23:16.856216 21400 leader_election.cc:403] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:16.856393 25220 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:16.856910 25220 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:16.857586 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:16.857824 25220 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.858575 25497 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11.
I20260709 12:23:16.859290 25497 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:16.859486 25314 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:16.848629 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:16.860455 25314 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:16.867558 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:16.878683 25217 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:16.879488 25217 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:16.880425 25217 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.881780 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:16.883061 21435 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:16.859704 25497 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:16.791443 25493 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:23:16.803973 25401 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:16.894256 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76631.7us
I20260709 12:23:16.898064 21434 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:16.861824 25314 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.942636 20730 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:16.896260 25497 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.809584 25509 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.900017 25401 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.816646 25324 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.829970 25399 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:16.957353 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:16.941344 25387 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:16.994664 20801 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:16.860396 25313 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:16.969513 25401 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.003684 21430 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:16.946827 25396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.030s	sys 0.039s
W20260709 12:23:17.003834 20699 leader_election.cc:336] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.834s (SENT)
I20260709 12:23:16.997400 25327 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:17.011439 25396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.032s	sys 0.039s
I20260709 12:23:16.975550 25331 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:23:17.011245 25174 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:17.072423 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
W20260709 12:23:17.078258 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66102us
W20260709 12:23:17.081724 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 132137us
I20260709 12:23:17.081777 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.089984 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:23:17.107280 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.121474 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.064019 25387 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:17.125299 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 468576us
I20260709 12:23:17.084260 25323 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:17.164530 25323 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.127451 20810 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:23:17.124687 25530 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:17.129724 25065 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:17.130779 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:17.135594 25329 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.136152 25417 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:23:17.180518 25197 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:17.140060 25532 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:23:17.149356 25324 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:17.093782 25337 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:17.155741 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 151935us
W20260709 12:23:17.156836 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48287.2us
W20260709 12:23:17.159055 25486 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.899s	user 0.013s	sys 0.000s
I20260709 12:23:17.059712 25327 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.160239 25247 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:17.162688 25509 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:17.172564 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.101027 25396 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.128262 25385 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:17.175540 20850 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:17.181659 25197 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:17.197531 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:17.121222 25246 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:17.201225 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.173388 25065 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:17.176256 25389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.014s	sys 0.025s
W20260709 12:23:17.206619 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 77853.6us
I20260709 12:23:17.215953 25399 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:17.211638 25396 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.218331 25396 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:17.221568 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:17.222673 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:17.223389 25246 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:17.224545 25246 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:17.224953 25246 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.225363 25387 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.225572 25324 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:17.227034 25324 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:17.227597 25323 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.189494 25379 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:17.228212 25324 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.229084 25379 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.229710 25324 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:17.230280 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.232132 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:17.225365 25246 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:17.233841 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:17.235893 25065 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.196144 25486 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.238027 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.240097 20807 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:17.241530 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:17.253110 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:17.254004 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66516.7us
I20260709 12:23:17.253590 21426 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:23:17.255564 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:17.251140 25376 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
W20260709 12:23:17.265693 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.268625 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.189973 25493 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 12:23:17.251019 25532 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:23:17.250677 25174 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:17.253144 25337 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:17.306500 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:17.244956 25389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.014s	sys 0.026s
I20260709 12:23:17.261924 25247 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:17.261529 25322 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:17.317389 20808 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:23:17.203351 25329 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:17.275015 25530 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:23:17.311875 25404 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.004s
I20260709 12:23:17.278470 25323 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:17.324544 25503 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:17.325106 25535 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:23:17.326376 25382 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.317683 25197 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.240155 25385 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.348793 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.349260 25246 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.353355 25376 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:17.354431 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.308368 25174 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:17.361161 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 123359us
I20260709 12:23:17.361852 25327 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:17.362504 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:17.383795 20801 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:17.364030 25379 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.365067 25486 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:17.365476 24963 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.345294 25322 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.345743 25337 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:17.344622 25509 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:17.405154 20804 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:17.310460 25387 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:17.385366 25174 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.417296 20807 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:23:17.386600 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 301842us
W20260709 12:23:17.385900 21429 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.029s	sys 0.057s
W20260709 12:23:17.420346 21429 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.030s	sys 0.057s
I20260709 12:23:17.400521 20808 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:17.376165 25404 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.403306 25197 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:17.390654 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:17.406599 25322 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.423736 25217 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.431162 20806 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:23:17.408941 25542 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.432783 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:17.441177 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92562.3us
I20260709 12:23:17.442530 20628 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 7 to 9, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 9 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.448629 25246 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.449512 25322 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.456395 25493 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:17.456697 25337 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.456911 20810 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:17.457355 25493 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.458483 25493 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.459115 25498 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:17.389353 25382 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.463128 25382 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.432382 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:17.465044 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:17.465124 20810 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:17.466737 25498 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:17.468667 25435 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.470583 25435 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.472636 25322 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:17.476924 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75858.6us
I20260709 12:23:17.477854 20699 leader_election.cc:400] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12.
I20260709 12:23:17.478143 20699 leader_election.cc:403] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:17.480113 25406 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12.
I20260709 12:23:17.452817 25327 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:17.483850 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.412873 25509 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:17.492722 25327 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:17.487936 20804 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:23:17.493484 25509 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.495244 25417 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:17.496003 25382 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.496585 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.496937 25387 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:17.497762 25387 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:17.498108 25387 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:17.498155 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.496081 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.498818 21432 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:17.509104 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.512141 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.512837 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.514644 20801 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:17.525118 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.529994 20735 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:23:17.487931 25385 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.551819 25406 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:23:17.562608 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:17.544543 25387 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.564255 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.565549 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:17.566000 25497 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.001s
I20260709 12:23:17.566262 20806 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:17.566427 25497 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.564385 25387 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.499614 25217 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.575033 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:17.511667 25493 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.576370 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.609627 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:17.610069 20801 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:17.610311 20732 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.523392 25401 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.613350 25401 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:17.615262 25544 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.395s	user 0.000s	sys 0.002s
I20260709 12:23:17.576885 25406 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:17.615972 25406 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.616442 25327 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:17.618301 25406 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.618494 25331 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:17.577235 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54177us
I20260709 12:23:17.618997 25331 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.619757 25331 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.620007 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:17.581075 20633 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 197) took 37111 ms (client timeout 15000 ms). Trace:
I20260709 12:23:17.581128 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:17.623881 25329 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:17.624243 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79595.2us
I20260709 12:23:17.625123 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.581239 25538 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 12:23:17.582031 20628 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 7 to 9, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 9 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.628118 25348 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:17.629046 25348 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:17.628695 20628 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.629825 24963 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:17.630395 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.631229 25385 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:17.590852 25543 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.373s	user 0.003s	sys 0.000s
I20260709 12:23:17.496668 25382 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.633396 25382 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:17.630559 20628 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.593616 25217 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:17.594033 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77800.8us
I20260709 12:23:17.634627 20628 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 6 to 7, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 7 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.502928 25542 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.595606 25174 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.636082 25382 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:17.636926 25382 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:17.637223 25542 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:17.637310 25382 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:17.637506 25174 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:17.621066 20633 rpcz_store.cc:269] 0709 12:22:40.535212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.535429 (+   217us) service_pool.cc:224] Handling call
0709 12:23:17.580854 (+37045425us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:13.129500 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:13.130170 (+   670us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:13.130213 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:23:13.130230 (+    17us) tablet.cc:662] Decoding operations
0709 12:23:13.164352 (+ 34122us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:13.174217 (+  9865us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:13.187587 (+ 13370us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:23:13.188620 (+  1033us) tablet.cc:701] Row locks acquired
0709 12:23:13.188643 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:23:13.188835 (+   192us) write_op.cc:270] Start()
0709 12:23:13.188995 (+   160us) write_op.cc:276] Timestamp: P: 1783599793188778 usec, L: 0
0709 12:23:13.189021 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:23:13.189909 (+   888us) log.cc:844] Serialized 3664 byte log entry
0709 12:23:14.201056 (+1011147us) op_driver.cc:464] REPLICATION: finished
0709 12:23:14.670590 (+469534us) write_op.cc:301] APPLY: starting
0709 12:23:14.683376 (+ 12786us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:15.304205 (+620829us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:15.320962 (+ 16757us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:15.522815 (+201853us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b180034c960. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:15.757483 (+234668us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:15.757752 (+   269us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 12:23:15.757796 (+    44us) write_op.cc:312] APPLY: finished
0709 12:23:15.758068 (+   272us) log.cc:844] Serialized 89 byte log entry
0709 12:23:15.789686 (+ 31618us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:15.789898 (+   212us) write_op.cc:454] Released schema lock
0709 12:23:15.789974 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:15.790609 (+   635us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:15.790651 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":886324,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":442272,"apply.run_cpu_time_us":24582,"apply.run_wall_time_us":1132934,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41747,"num_ops":9,"prepare.queue_time_us":582182,"prepare.run_cpu_time_us":18408,"prepare.run_wall_time_us":362729,"raft.queue_time_us":200618,"raft.run_cpu_time_us":22154,"raft.run_wall_time_us":410227,"replication_time_us":983521,"spinlock_wait_cycles":21272320,"thread_start_us":1000161,"threads_started":5,"wal-append.queue_time_us":435770,"wal-append.run_cpu_time_us":10142,"wal-append.run_wall_time_us":2410994}]]}
W20260709 12:23:17.600106 25545 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:23:17.638327 25382 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.608291 21426 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:23:17.639789 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:17.654242 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.639289 20628 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 9 to 12, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 12 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:17.576747 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.582592 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:17.584156 25379 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:17.585139 25389 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:17.572166 25246 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:17.668713 20628 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 6 to 9, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 9 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.638329 20804 raft_consensus.cc:2368] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13.
I20260709 12:23:17.595860 21399 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11.
I20260709 12:23:17.629748 25348 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:23:17.597934 25497 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:17.644213 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.644624 25327 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:17.644834 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.605988 25404 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.610896 25324 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.645790 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.637035 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:17.668184 20729 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:23:17.593155 25541 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:23:17.671119 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75402.9us
W20260709 12:23:17.673688 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.667802 25379 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:17.695983 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:17.696652 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2463) took 1063 ms. Trace:
I20260709 12:23:17.707710 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:17.700613 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.702219 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:17.668519 25385 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:17.705164 20801 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:17.707897 20803 rpcz_store.cc:276] 0709 12:23:16.633409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.633593 (+   184us) service_pool.cc:224] Handling call
0709 12:23:16.709115 (+ 75522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:16.731529 (+ 22414us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:17.173289 (+441760us) log.cc:844] Serialized 14 byte log entry
0709 12:23:17.696198 (+522909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:17.696270 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:17.696334 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.696429 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.696581 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40295,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":41244,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":50458,"thread_start_us":923276,"threads_started":3,"wal-append.queue_time_us":363528}
I20260709 12:23:17.669983 25389 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:17.690587 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.699265 20732 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:17.708547 21434 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:17.672037 25318 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:17.708814 20883 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:17.711853 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.711918 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.719131 20804 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:17.719604 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.720151 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.721194 20801 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:17.703919 25247 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:17.693413 25323 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:17.723953 20879 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:17.724387 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.695324 25498 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.668188 25331 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.731472 20628 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.704475 25541 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:17.730195 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:17.731915 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:17.724992 20803 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
W20260709 12:23:17.736311 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 99383.3us
I20260709 12:23:17.743930 20801 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.709708 25493 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:17.739037 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:17.747900 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.748605 21429 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:17.695528 25387 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:23:17.672199 25065 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.728489 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.740140 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.722965 25382 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.752694 21430 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:17.764215 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:17.756220 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.751894 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:17.761999 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.747074 25174 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:23:17.767465 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.732411 25390 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:17.735653 25220 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.707110 25324 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.765187 20875 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:17.778146 25247 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.779018 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:17.781306 25406 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:17.781837 25406 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:17.781428 25217 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Committing config change with OpId 7.8: config changed from index 5 to 8, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.784484 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.785876 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:17.786492 20876 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
W20260709 12:23:17.787729 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142033us
I20260709 12:23:17.781842 20879 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:17.778286 20628 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 9 to 10, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 10 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.789824 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.792347 20884 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:17.754065 25498 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.794836 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.795486 20882 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.796424 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:17.799366 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:17.801570 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.802143 20879 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.802477 20802 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:17.757764 25404 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:17.807027 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.815450 20883 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.802840 20628 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 7 to 9, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 9 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.818076 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.768086 25318 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:17.786747 25324 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.854318 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:17.867614 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.773725 25417 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:17.782683 25406 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.868988 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:17.861024 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55600.6us
I20260709 12:23:17.789737 21429 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Committing config change with OpId 7.8: config changed from index 5 to 8, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.868152 20876 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:17.809360 25376 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:17.903847 20628 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 8, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 8 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.853017 21434 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:17.856199 25318 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:17.922175 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.801260 25327 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 12:23:17.926997 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:17.928462 20804 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:17.897037 24963 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.817411 25486 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 12:23:17.920851 25406 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:17.937342 20628 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 9 to 11, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 11 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.807919 25220 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.928431 25318 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.785873 25379 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:17.947486 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 9 to 10, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 10 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:17.979894 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111031us
I20260709 12:23:17.980100 20628 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 7 to 9, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 9 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.983588 20806 raft_consensus.cc:1217] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 7.8->[9.9-12.10]   Dedup: 9.9->[12.10-12.10]
I20260709 12:23:17.984165 20806 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: [9.9-12.10]
I20260709 12:23:17.982329 20628 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:17.929260 25542 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:17.975176 25573 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:23:17.980813 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:17.973483 20801 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:17.988183 20803 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:17.992439 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:17.903043 25541 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:17.973245 25404 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:17.986558 25323 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.005091 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:18.005380 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:18.005546 25220 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:17.939163 24963 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.006402 20879 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:18.007831 24963 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:18.009250 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:18.015062 20628 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 7 to 10, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 10 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:18.029055 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 96674.9us
I20260709 12:23:17.972790 25348 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:23:18.036406 20628 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 5 to 11, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 11 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:18.066839 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.068854 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:18.076196 25536 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:18.079923 20729 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:18.034301 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:18.081679 20732 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:18.075233 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 260856us
I20260709 12:23:18.069821 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:17.995630 25542 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:18.082446 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101754us
W20260709 12:23:18.091589 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.099256 25542 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:18.101087 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:18.101751 20731 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:23:18.102679 25435 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.631s	user 0.008s	sys 0.001s
I20260709 12:23:18.079022 25065 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:18.083151 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:18.033389 25434 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:23:18.107234 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:18.107422 20730 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:18.020439 25390 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:23:18.108439 20734 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:23:18.112334 20734 pending_rounds.cc:85] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:23:18.112634 20734 pending_rounds.cc:107] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:23:18.112926 20734 raft_consensus.cc:2889] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:18.113314 20729 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:18.113395 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:18.114212 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 191416us
I20260709 12:23:18.114447 20801 raft_consensus.cc:2368] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13.
I20260709 12:23:18.118166 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.119105 20807 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
W20260709 12:23:18.125358 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 116184us
I20260709 12:23:18.128312 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:18.105365 25535 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:18.103658 25417 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:18.112771 25542 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:18.128197 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:18.141508 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.129947 20628 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.112001 25536 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:18.142175 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:18.149144 20732 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
W20260709 12:23:18.151476 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44287us
I20260709 12:23:18.125168 25331 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:18.157677 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:18.131688 25535 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:18.136319 25434 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:18.157848 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:18.142405 25435 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:18.243005 20882 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:18.236699 25594 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.253448 20883 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:18.231931 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:18.260752 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103006us
I20260709 12:23:18.267515 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.253759 25594 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:18.278111 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.278146 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:18.156220 25509 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:18.221357 25558 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:23:18.274641 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.275745 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:18.278970 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.198474 25542 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:18.293596 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:18.279259 21430 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:18.299901 20730 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:18.295097 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:18.306365 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 239636us
I20260709 12:23:18.308202 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:18.239642 25313 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:18.259508 25571 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:18.239578 25535 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:18.255311 25435 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:18.267517 25559 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:23:18.328428 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:18.342981 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 213175us
I20260709 12:23:18.363862 25435 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.365399 25435 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:18.300547 25390 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:23:18.368670 25590 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:18.368791 25390 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:18.370007 25590 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:18.301797 25587 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:18.314899 25220 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:18.322973 25569 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 12:23:18.289449 25314 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:18.375618 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:18.313426 25331 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:18.377987 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:18.235626 25417 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.416579 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:18.357692 25434 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:18.423552 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52867.4us
I20260709 12:23:18.232643 25536 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.328408 25313 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:18.431932 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 136916us
I20260709 12:23:18.409515 25587 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:18.409693 25596 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:18.422505 25434 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:18.422052 25314 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.469080 25434 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:18.462158 25557 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:18.472586 25557 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:18.473675 25557 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.461869 25603 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.476053 25603 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.478525 25603 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:18.415709 25331 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.481336 25331 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:18.460711 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:18.484649 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52551.8us
I20260709 12:23:18.485407 25560 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:18.485653 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 757) took 1016 ms. Trace:
I20260709 12:23:18.485946 25560 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:18.486033 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:18.456135 25576 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 12:23:18.486093 20810 rpcz_store.cc:276] 0709 12:23:17.469054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.469201 (+   147us) service_pool.cc:224] Handling call
0709 12:23:17.469527 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:17.469930 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.469965 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:17.470554 (+   589us) log.cc:844] Serialized 52 byte log entry
0709 12:23:17.671741 (+201187us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:17.671796 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.671879 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:18.485408 (+813529us) raft_consensus.cc:1685] finished
0709 12:23:18.485432 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.485588 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6700,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":50,"thread_start_us":200710,"threads_started":1,"wal-append.queue_time_us":426447}
I20260709 12:23:18.491700 21426 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:18.493427 25323 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:18.494743 25318 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.002s
I20260709 12:23:18.496562 25313 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:18.464780 25582 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 12:23:18.468502 25587 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.486286 25247 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:23:18.468335 25575 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
W20260709 12:23:18.467736 25577 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 12:23:18.431478 25376 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:18.502547 20731 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:18.460436 25594 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:18.508319 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 181776us
W20260709 12:23:18.510229 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 216689us
I20260709 12:23:18.450891 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.515091 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:18.461583 25574 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:18.519147 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 752) took 1167 ms. Trace:
I20260709 12:23:18.523576 21426 raft_consensus.cc:2368] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:18.524874 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:18.525597 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 75098.9us
I20260709 12:23:18.528635 20701 leader_election.cc:400] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:18.528568 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:18.529011 20701 leader_election.cc:403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:18.509534 25220 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:23:18.502246 25318 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.431807 25542 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.530263 25602 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:23:18.558012 25564 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:18.529759 25594 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:18.560029 25594 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.563875 25561 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:23:18.564173 25589 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.116s	user 0.004s	sys 0.000s
I20260709 12:23:18.564373 25561 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:18.565330 25594 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:18.565264 25561 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.520534 20805 rpcz_store.cc:276] 0709 12:23:17.352063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.355121 (+  3058us) service_pool.cc:224] Handling call
0709 12:23:17.355622 (+   501us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:17.511444 (+155822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.511482 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:17.627451 (+115969us) log.cc:844] Serialized 52 byte log entry
0709 12:23:18.031994 (+404543us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:18.032054 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.032149 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:18.518852 (+486703us) raft_consensus.cc:1685] finished
0709 12:23:18.518882 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.519066 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52187,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4886,"thread_start_us":544453,"threads_started":2,"wal-append.queue_time_us":497748}
I20260709 12:23:18.552023 20809 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:23:18.554658 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:18.576009 21434 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.576160 25561 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:18.542027 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:18.583634 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:18.555083 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.530802 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:18.587733 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57040.6us
I20260709 12:23:18.540940 25596 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:23:18.567167 25509 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:18.526850 25376 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:18.487010 25560 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.571796 25603 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:18.541714 25318 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:18.602195 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.594300 25601 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:18.596351 25590 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.611245 25498 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:18.612263 25217 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
I20260709 12:23:18.542191 25606 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 12:23:18.593942 25385 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:18.593053 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:18.596415 25313 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:18.621661 20805 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:18.630172 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 103277us
I20260709 12:23:18.596057 25564 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:23:18.569684 25434 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.637531 25220 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:18.541236 25535 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.620294 25601 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:18.621171 25509 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:18.640980 21434 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
W20260709 12:23:18.642266 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49323.6us
W20260709 12:23:18.644783 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 336640us
W20260709 12:23:18.645298 25406 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.001s
I20260709 12:23:18.646809 25379 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:18.646917 25606 consensus_queue.cc:1243] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:18.643843 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:18.628777 25385 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:18.662371 21430 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:18.665203 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:18.638417 25220 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:18.703763 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53316.9us
I20260709 12:23:18.642499 25601 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:18.626868 25590 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.736635 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:18.771308 25605 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:18.772063 25246 raft_consensus.cc:2764] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:23:18.773496 25535 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:18.773336 25509 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.772104 25552 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:18.780973 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.725026 25406 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:18.795521 25542 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:18.796258 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.797413 25542 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:18.799161 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:18.799975 20726 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:18.800663 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.801473 20806 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:18.784366 25493 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.809850 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:18.808398 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:18.809501 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2227) took 1099 ms. Trace:
I20260709 12:23:18.811601 25602 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:18.787245 25605 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:18.812569 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 168829us
I20260709 12:23:18.813556 20880 rpcz_store.cc:276] 0709 12:23:17.705699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.705859 (+   160us) service_pool.cc:224] Handling call
0709 12:23:18.804983 (+1099124us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:18.805128 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628667648}
I20260709 12:23:18.806210 25552 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:18.820827 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.814687 25313 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.793305 25389 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.733930 25376 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.833739 20726 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:23:18.835899 20726 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:23:18.836292 20726 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:23:18.836659 20726 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:18.818240 25536 raft_consensus.cc:2764] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:23:18.850446 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:18.853156 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.863376 20806 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:18.816169 25493 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.868901 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:18.821517 25602 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.849763 25314 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:18.860293 25435 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:18.867833 25389 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:18.886694 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:18.852180 25406 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:18.888445 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:18.889360 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.848735 25313 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.897830 20806 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:18.874030 25603 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:18.908187 25493 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:18.911725 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:18.884748 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1243902989999697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
I20260709 12:23:18.821334 25220 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.884310 25536 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:18.896628 25314 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.899556 25497 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:18.926460 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:18.929736 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.930971 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:18.899300 25561 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:18.961230 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:18.962554 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:18.962803 25561 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.963717 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.964763 20808 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:23:18.965305 25561 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:18.918927 25594 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.927248 25536 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:18.938192 25590 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:18.939499 25434 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:18.940531 25406 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.942600 25596 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:18.850988 25605 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:18.958546 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.919063 25404 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.962612 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.965962 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.928025 25337 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:23:18.978807 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 704) took 1424 ms. Trace:
W20260709 12:23:18.979924 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49117.8us
I20260709 12:23:18.982398 25379 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.982481 25536 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:18.985709 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.986747 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:18.924763 25246 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 12:23:19.019025 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.971683 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:19.030894 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59363.4us
I20260709 12:23:19.025085 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:19.023132 25404 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:18.985206 25542 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:19.016113 25217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47850 (request call id 0) took 5782 ms. Trace:
I20260709 12:23:18.982698 25603 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:19.028342 25611 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:23:19.022899 20734 rpcz_store.cc:276] 0709 12:23:17.544394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.544926 (+   532us) service_pool.cc:224] Handling call
0709 12:23:17.571528 (+ 26602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:18.126551 (+555023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.958323 (+831772us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b5400226ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:18.968989 (+ 10666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1225771776}
I20260709 12:23:19.072252 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:18.947944 25314 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.027140 20808 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:23:18.985507 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.905320 25552 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.015740 25590 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.950531 25497 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.053148 25598 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:19.053040 25542 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.021533 25601 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.072997 25337 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:19.030004 25379 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.094866 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.095767 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:19.107095 20802 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:19.096633 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.077044 25376 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.072402 25381 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:19.076440 25621 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.985507 25594 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.100992 20807 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
W20260709 12:23:19.104911 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81303.7us
I20260709 12:23:19.115258 25404 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.116159 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.116572 25560 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:19.116809 25626 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:23:19.118224 25619 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:23:19.123546 25313 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.125686 25601 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:19.126531 25578 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.734s	user 0.000s	sys 0.003s
I20260709 12:23:19.127485 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 788) took 1388 ms. Trace:
I20260709 12:23:19.127516 25246 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:19.127944 20884 rpcz_store.cc:276] 0709 12:23:17.739027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.753093 (+ 14066us) service_pool.cc:224] Handling call
0709 12:23:17.774011 (+ 20918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.792717 (+ 18706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:19.127211 (+1334494us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b54000a2dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:19.127396 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29477632}
I20260709 12:23:19.088617 25598 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.088094 25542 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.118062 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.134352 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.075455 25603 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.139757 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.087728 25590 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.153337 25382 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:19.153666 25590 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.153842 25382 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:19.154062 25594 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.155462 25379 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.127687 25497 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:19.100060 25628 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:19.116169 25618 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:23:19.188028 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:19.190696 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.167913 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.136238 25598 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:19.191460 20727 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:19.136137 25509 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:19.135942 25404 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:19.154712 25382 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.135990 25602 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.204857 20809 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:19.209607 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.213191 20732 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:19.189847 25594 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.165588 25571 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:19.211133 20731 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:23:19.161374 25337 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:19.218953 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.129829 25381 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.222270 25509 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.191877 25628 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.235095 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.235891 20805 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
W20260709 12:23:19.246004 25588 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.009s	user 0.002s	sys 0.002s
I20260709 12:23:19.255339 25586 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:19.255924 25586 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.256292 25586 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:19.235772 25594 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.129655 25621 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.268189 20802 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:19.071535 25217 rpcz_store.cc:276] 0709 12:23:13.199318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.199476 (+   158us) service_pool.cc:224] Handling call
0709 12:23:14.313837 (+1114361us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b5400186620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1c520 00007f9e1eca5b42 00007f9e1eca4800 00007f9e1ec9d9fa 00007f9e23c654c9 00007f9e1d1e1c54 00007f9e1d1e1be2 00007f9e1d1e1b11 00007f9e1d1e1a8c 00007f9e1d1e0d42
0709 12:23:14.717737 (+403900us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b4c00032880. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1cf6ba20 00007f9e1cf7ca29 00007f9e1cf42644 00007f9e1cf69f2d 00007f9e1cf69b93 00007f9e1ebf81af 00007f9e1ebf7f3e 00007f9e1ec7d888 00007f9e1ec9a75c 00007f9e1ec9b76a 00007f9e1ec9a91b
0709 12:23:14.718570 (+   833us) log.cc:844] Serialized 501 byte log entry
0709 12:23:18.981993 (+4263423us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":3094,"mutex_wait_us":34990,"raft.queue_time_us":27931,"raft.run_cpu_time_us":1622,"raft.run_wall_time_us":151425,"spinlock_wait_cycles":228167296,"thread_start_us":88781,"threads_started":1,"wal-append.queue_time_us":331006,"wal-append.run_cpu_time_us":2384,"wal-append.run_wall_time_us":1175820}
I20260709 12:23:19.269572 20808 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:19.247524 25382 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.249492 25628 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.279553 25314 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.281796 25399 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:19.282639 25399 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.283962 25376 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:19.285867 25509 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.297632 25621 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:19.322310 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.284034 25603 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.324795 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:19.328311 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.329361 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:19.329761 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:19.330592 20734 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:19.331682 20732 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.332912 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:19.333024 20735 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.334172 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.334990 20730 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:19.336397 25581 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.104s	user 0.001s	sys 0.001s
I20260709 12:23:19.339511 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.342171 20729 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:19.343118 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.344142 21430 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:23:19.346172 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:19.347926 21429 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:23:19.348613 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.349433 20728 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [10.11-12.12]
I20260709 12:23:19.350239 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:19.349120 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:19.351907 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.325129 25390 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.353075 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.354903 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.356045 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.356642 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.357321 20809 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:19.357445 20730 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:23:19.358763 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.358291 20802 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:19.359381 20735 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:19.359365 20727 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:19.354014 20729 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:19.361407 20802 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:19.364207 20809 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.363283 21434 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:19.349810 25493 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.367735 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.368306 25561 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:23:19.369257 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.353020 25404 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:19.363950 25382 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:23:19.379386 25404 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.382931 25404 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:19.379532 25586 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.349377 25542 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.378974 25561 consensus_queue.cc:1243] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:23:19.366842 25390 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.387917 25382 consensus_queue.cc:1243] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:19.404718 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:19.405618 20735 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.378072 25493 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.440706 20730 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:19.433516 20700 leader_election.cc:400] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13.
I20260709 12:23:19.443008 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7ccae903e8484f8d8c3af847e2c4c3d4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:19.444730 20700 leader_election.cc:403] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:19.445969 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.447237 20884 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:23:19.449321 20732 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:19.449915 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.431691 25561 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:19.457698 20727 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:19.568424 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.571466 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:19.572199 20882 raft_consensus.cc:2368] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
I20260709 12:23:19.572170 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.573167 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:19.566603 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.569941 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.585625 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.395989 25536 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:19.589248 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 155934us
I20260709 12:23:19.593542 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.604470 20882 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:19.594223 25643 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.229s
I20260709 12:23:19.594890 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:19.599604 25644 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.171s	user 0.000s	sys 0.002s
I20260709 12:23:19.600047 20733 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:19.600353 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.621085 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.621871 20884 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:19.604360 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.610502 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:19.613855 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.622898 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:19.820456 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 216204us
I20260709 12:23:19.822252 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.444077 25597 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.143s
I20260709 12:23:19.624308 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.454908 25586 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.624898 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:19.625185 20734 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:19.567350 25535 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13.
I20260709 12:23:19.579213 25557 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.636706 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.825876 25557 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.601949 25246 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:19.827534 25246 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.827870 25603 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:19.830008 25246 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.646890 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.830891 20733 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:19.832335 20733 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.833634 20733 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.834493 25594 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.606226 25561 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.837031 25561 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.635887 24963 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:19.636723 25621 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:19.649533 25385 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.652062 25643 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:19.637337 25538 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:19.655761 25493 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.660939 25536 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.664417 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.670485 25390 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.618340 25583 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:19.847591 25583 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.849661 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:19.849879 25583 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.612846 25587 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:19.671553 25542 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:19.632869 25645 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:23:19.632786 25337 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.871685 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:19.681754 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 115554us
I20260709 12:23:19.682050 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
W20260709 12:23:19.692447 25648 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:23:19.693084 25381 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.692757 25404 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.685662 25065 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:23:19.702536 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:19.703423 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.705533 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:19.851567 25603 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:19.710201 20809 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:19.890811 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.887877 20728 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:19.713047 20699 leader_election.cc:400] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 7 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 7. Current term is 8.
W20260709 12:23:19.713441 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.685307 25174 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:19.723712 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.690677 25564 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:23:19.896297 20729 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:23:19.721295 25598 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.507s	user 0.004s	sys 0.007s
I20260709 12:23:19.735119 25574 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:19.736862 21434 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:23:19.899483 21434 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 8
I20260709 12:23:19.877435 25605 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:19.743485 21430 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.842094 24963 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.747349 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:19.748142 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.905287 20804 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:19.749233 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.755353 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:19.906104 20802 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:19.757149 21427 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.758749 21433 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:19.760537 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:19.760684 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.764166 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:19.910159 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.910720 20726 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:23:19.911098 20880 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:19.911811 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.768428 21428 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:19.775233 25541 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:19.777832 25399 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.869690 25538 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.779817 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:19.780609 20808 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:19.757150 25555 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:19.816041 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.838786 20878 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13.
I20260709 12:23:19.814633 25653 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.846673 25382 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:19.851662 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:19.850976 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:19.907634 25318 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.870644 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.876381 20733 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:19.880162 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:19.706182 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.891636 20877 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:19.899856 21434 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:23:19.967427 21434 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:19.967792 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:19.739724 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.898926 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:19.967443 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 258384us
I20260709 12:23:19.968477 20875 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.968593 21435 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:19.913870 20801 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.908726 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:19.851938 25621 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.916548 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.851691 25643 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.911274 20726 pending_rounds.cc:85] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 10
I20260709 12:23:19.919534 21432 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:19.973748 20726 pending_rounds.cc:107] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:23:19.779708 25197 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:19.924536 21430 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:23:19.924983 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 145239us
I20260709 12:23:19.926966 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.930140 20802 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:19.975219 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:19.903703 25574 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:19.980121 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.932293 20876 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:19.862556 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.980822 20879 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 8.
I20260709 12:23:19.907596 25587 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.931609 25399 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:19.986340 25399 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.986871 25399 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:19.946776 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 101037us
I20260709 12:23:20.025076 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.883530 25337 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.950316 25318 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.960106 20882 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:19.963160 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:19.917253 25623 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:19.933004 25555 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.971372 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.974192 20726 raft_consensus.cc:2889] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:19.898339 25348 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:19.983846 25385 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.969062 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:19.981830 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.986589 25574 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:19.987466 25643 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:19.987928 25583 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:20.055994 20735 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:23:20.056519 20735 pending_rounds.cc:85] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:23:20.056404 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:20.056990 20735 pending_rounds.cc:107] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:23:20.057407 20735 raft_consensus.cc:2889] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:20.057696 20882 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:23:20.058610 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89617.6us
I20260709 12:23:19.945912 20883 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.059551 25574 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.060803 25555 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:19.988869 25627 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:20.056227 25583 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.991511 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:20.062311 25583 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.062626 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:19.992069 25595 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:20.063977 25623 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:19.994099 25246 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:20.064445 25623 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.051093 25348 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.064934 25623 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:20.065045 25246 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:20.065555 25246 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:20.066373 25246 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:19.994599 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:19.996054 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.997500 25597 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.999939 25561 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:20.000861 25642 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:19.979697 25656 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:20.008121 25587 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.011348 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:20.012638 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:19.984464 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:19.959492 25603 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:20.028106 25382 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:20.032318 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69245.1us
I20260709 12:23:20.050719 25406 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.038764 21434 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:19.909322 25598 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:20.039746 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.929177 25636 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:23:20.049885 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:20.050287 20879 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:20.051931 20884 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:20.064019 25318 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:19.988062 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:20.059554 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:20.073120 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:20.083024 20883 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:20.099622 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:20.098646 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 104125us
I20260709 12:23:19.981981 25542 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:23:20.098927 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:20.037825 25065 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:20.033881 25571 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:23:20.160068 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100599us
I20260709 12:23:20.162508 20879 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:23:20.118836 21430 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:20.122061 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:20.026732 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:20.125895 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.127945 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:20.166328 25382 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:20.168279 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 141646us
I20260709 12:23:20.169395 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:20.171829 25660 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:23:20.173447 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:20.174244 20883 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:19.955852 25653 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.179138 20882 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:20.062700 25621 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.134807 20730 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:20.142189 20801 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:20.144299 20876 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:20.067028 25348 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:20.141114 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:20.149042 20734 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:20.060703 25536 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:20.116308 25595 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:20.178376 21426 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:20.171095 25535 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.109721 25597 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.181514 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:20.182610 20879 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:20.204727 20876 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:20.204771 20881 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:20.134950 25603 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:20.199370 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58367us
I20260709 12:23:20.132299 25246 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:20.201995 20884 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:20.152669 25557 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.208948 24963 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.210395 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:20.200327 25536 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:20.211777 25536 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.211866 25376 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.212462 25376 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.212908 25587 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.203557 25493 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:20.214570 25376 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.214756 25493 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:20.209015 25561 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:20.216461 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55515.8us
I20260709 12:23:20.135263 25406 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.180647 25382 raft_consensus.cc:2764] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:23:20.153607 25555 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:23:20.178606 25404 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:20.235560 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 326921us
W20260709 12:23:20.247546 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 209212us
I20260709 12:23:20.208186 25595 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.219903 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:20.121574 25656 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.264729 20729 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:20.197116 25571 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:23:20.268384 20881 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:23:20.269551 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 342016us
W20260709 12:23:20.273075 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 374244us
I20260709 12:23:20.274847 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:20.255012 25595 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:20.286793 20881 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:20.288031 20882 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:23:20.288548 20882 pending_rounds.cc:85] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 9
I20260709 12:23:20.288817 20882 pending_rounds.cc:107] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:23:20.289089 20882 raft_consensus.cc:2889] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:20.290823 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71806.6us
I20260709 12:23:20.291002 20878 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:20.292919 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:20.293757 21426 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:20.225950 25197 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:23:20.296053 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:20.265828 24963 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:23:20.265587 25557 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:23:20.321250 20881 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:23:20.323566 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:20.326377 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:20.328925 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:20.331938 20876 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:20.334658 25065 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:20.335803 25557 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:23:20.336207 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:20.350075 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 228105us
I20260709 12:23:20.337256 25536 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:20.304243 25337 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:20.310626 25376 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.314522 25621 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:20.359740 20875 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:20.359345 25385 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.345847 25065 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.351370 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:20.353391 20731 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:23:20.370620 20731 pending_rounds.cc:85] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:23:20.370975 20731 pending_rounds.cc:107] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:23:20.372073 20731 raft_consensus.cc:2889] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:20.294724 25493 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.375624 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:20.304554 25399 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:20.392704 25661 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.198s	user 0.001s	sys 0.001s
I20260709 12:23:20.401896 21429 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:20.372632 25065 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:20.403525 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 232911us
I20260709 12:23:20.377166 25337 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.405086 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:20.374003 25598 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.323237 25603 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:20.400954 25390 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.401726 25536 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.373471 25322 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:23:20.417058 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:20.406335 25385 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:20.415845 25571 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:20.428992 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:20.419261 25642 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:20.466404 25564 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:23:20.488871 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.491032 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:20.492026 21426 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
W20260709 12:23:20.493884 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 196888us
I20260709 12:23:20.502611 20883 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:23:20.503917 20883 pending_rounds.cc:85] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 6
I20260709 12:23:20.504276 20883 pending_rounds.cc:107] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:23:20.504637 20883 raft_consensus.cc:2889] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:20.495000 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:20.467128 25564 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:20.513448 25564 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.515735 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:20.422657 25653 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:20.526255 25674 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:20.526932 25337 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:20.524184 25406 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:20.543424 25385 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:20.555933 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 61060.8us
I20260709 12:23:20.556927 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:20.546113 25605 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.001s
I20260709 12:23:20.442548 25597 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:20.547785 25664 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:23:20.542922 25536 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.545272 25538 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:20.564563 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:20.524405 25642 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:20.575747 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 355879us
I20260709 12:23:20.550625 25535 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:20.588908 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:20.554606 25674 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.600960 20880 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:23:20.601538 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 294018us
W20260709 12:23:20.561146 25662 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:23:20.560339 25627 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:20.577512 25541 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.653s	user 0.006s	sys 0.006s
I20260709 12:23:20.600284 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:20.604902 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:20.581990 25557 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:20.599898 25535 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.601414 25564 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.610309 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:20.601701 25318 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:20.602986 25656 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:20.602883 25597 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:20.555471 25671 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:23:20.612985 20877 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:20.597095 25642 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:20.560372 25385 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.602146 25674 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.613075 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:20.622972 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:20.623759 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:20.599236 25536 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:20.627606 20807 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:20.631428 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:20.632275 20801 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:20.614310 25598 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:20.615144 25535 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.612461 25557 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:20.619978 25627 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:20.653254 25627 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:20.654248 25627 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:20.655943 25623 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.000s
I20260709 12:23:20.656386 25623 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.656752 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2237) took 1563 ms. Trace:
I20260709 12:23:20.657195 20806 rpcz_store.cc:276] 0709 12:23:19.093557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.097094 (+  3537us) service_pool.cc:224] Handling call
0709 12:23:19.097448 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:19.841229 (+743781us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b18004cb400. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1ec97bfb 00007f9e1eca264b
0709 12:23:19.844680 (+  3451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:19.844706 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:19.846212 (+  1506us) log.cc:844] Serialized 26 byte log entry
0709 12:23:20.194887 (+348675us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:20.254435 (+ 59548us) log.cc:844] Serialized 14 byte log entry
0709 12:23:20.271731 (+ 17296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:20.271797 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:20.485804 (+214007us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:23:20.656526 (+170722us) raft_consensus.cc:1685] finished
0709 12:23:20.656549 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:20.656695 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85884,"prepare.queue_time_us":66688,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":82870,"raft.run_cpu_time_us":3853,"raft.run_wall_time_us":242731,"spinlock_wait_cycles":17094400,"thread_start_us":456098,"threads_started":2,"wal-append.queue_time_us":500048}
I20260709 12:23:20.610831 25174 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:23:20.621991 25587 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:20.672078 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2240) took 1002 ms. Trace:
I20260709 12:23:20.643038 25399 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:20.652181 25670 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:20.644726 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:20.693426 21427 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:20.621371 25597 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:20.668457 20801 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
W20260709 12:23:20.669070 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 352555us
I20260709 12:23:20.628113 25655 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.672659 20878 rpcz_store.cc:276] 0709 12:23:19.669377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.226549 (+557172us) service_pool.cc:224] Handling call
0709 12:23:20.287413 (+ 60864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:20.291640 (+  4227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:20.671796 (+380156us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b54000965d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:20.671991 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57614592}
I20260709 12:23:20.639794 25598 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:20.692189 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.712088 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:20.660185 25557 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:20.646667 25656 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:20.732556 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46267.8us
W20260709 12:23:20.736024 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 307114us
I20260709 12:23:20.723101 25538 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:20.672209 25587 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:20.752249 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 423392us
I20260709 12:23:20.758993 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:20.763633 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:20.759459 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 243821us
I20260709 12:23:20.713495 25597 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:20.712812 25246 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.728776 25561 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.723006 25598 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:20.773376 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:20.763690 20883 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:20.751636 25605 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:20.769219 20699 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13.
I20260709 12:23:20.804405 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:20.729846 25542 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:20.729591 25656 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:20.709812 24963 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:23:20.784914 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:20.796255 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.797416 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:20.779398 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 166383us
I20260709 12:23:20.798005 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:20.798775 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:20.774643 25583 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.712s	user 0.009s	sys 0.004s
I20260709 12:23:20.821204 20729 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:20.751405 25673 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.874519 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:20.802850 20878 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 7.
I20260709 12:23:20.774046 25587 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:20.810984 20879 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
W20260709 12:23:20.815920 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 197526us
W20260709 12:23:20.829346 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.860229 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:20.867028 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:20.868333 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:20.869025 20802 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:20.870149 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:20.796999 25555 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:20.878026 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:20.887622 20883 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:20.804447 25337 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.880791 20804 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:20.792079 25681 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:20.891830 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:20.761758 25399 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:23:20.774765 25538 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:20.807611 25542 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:20.933612 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:20.927487 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:20.938673 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:20.972414 20809 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:20.820842 25318 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:20.941556 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:20.943483 20806 raft_consensus.cc:1217] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Deduplicated request from leader. Original: 11.8->[12.9-12.9]   Dedup: 12.9->[]
I20260709 12:23:20.783628 25643 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
W20260709 12:23:20.925416 25595 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.002s
I20260709 12:23:20.986877 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:20.949839 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 804) took 1129 ms. Trace:
I20260709 12:23:21.030898 20880 rpcz_store.cc:276] 0709 12:23:19.820523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.987702 (+167179us) service_pool.cc:224] Handling call
0709 12:23:20.938610 (+950908us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:20.949772 (+ 11162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059180800}
I20260709 12:23:20.729161 25655 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:21.035894 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62913.8us
I20260709 12:23:20.876986 25691 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.039628 25691 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.042734 25691 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:20.922624 25555 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.821797 25605 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.935482 25653 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.808197 25561 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.965744 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:20.967635 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:20.968947 20726 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
W20260709 12:23:21.081063 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 276733us
I20260709 12:23:20.877797 25486 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:23:20.943573 25382 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:20.972363 20729 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:20.940626 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62242.7us
I20260709 12:23:20.878453 25583 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.975991 20806 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:20.972203 20732 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:20.979796 25065 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.003s
I20260709 12:23:20.983464 25700 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:20.984123 25174 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:20.962249 25571 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:20.959794 25694 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:20.960469 25670 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:20.792829 25246 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.955582 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:20.969847 25681 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.035936 20809 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:20.885785 25673 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.139688 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:20.961205 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:21.055426 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 322032us
I20260709 12:23:20.979053 25385 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:21.067256 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.081233 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:20.913159 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53046.2us
I20260709 12:23:21.082376 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:20.985783 25680 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:21.049086 25555 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:21.091526 21429 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:21.092775 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:21.093415 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:21.088510 25703 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:21.093106 25065 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.091668 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:21.047788 25701 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:21.088708 25382 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.111429 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:21.115590 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:21.121155 20883 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:21.043905 25595 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:20.975322 25656 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.132382 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:21.052129 25671 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:23:21.137480 20729 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:20.946098 25337 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.224960 20880 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:21.224736 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:21.094245 20727 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:21.098800 25700 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.362753 20732 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:21.139685 25670 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:21.365921 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 141294us
I20260709 12:23:21.366300 25700 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.366633 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:21.126648 25571 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:21.368018 25636 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:21.368460 25636 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:21.368891 25636 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.368278 25571 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.372804 25670 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.373495 25673 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:21.352633 25538 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:21.354027 25623 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.226435 20881 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:21.225807 20878 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:23:21.352867 20681 auto_rebalancer.cc:893] Removing replace marker failed: Replica 657bc62d20374e539b23641715f22745 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:21.242017 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:21.379755 20681 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:23:21.267287 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:21.312418 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:21.352185 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:21.383828 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.326057 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 722) took 2096 ms. Trace:
I20260709 12:23:21.329603 20726 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:21.263626 20733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:21.386691 20733 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:21.227016 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.387147 20733 raft_consensus.cc:614] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:21.388417 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.389072 21430 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:21.385134 20731 rpcz_store.cc:276] 0709 12:23:19.229728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.229942 (+   214us) service_pool.cc:224] Handling call
0709 12:23:19.240770 (+ 10828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:20.372704 (+1131934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.259969 (+887265us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003acfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:21.325972 (+ 66003us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2557890432}
I20260709 12:23:21.399773 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:21.400203 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:21.401521 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 805) took 1549 ms. Trace:
I20260709 12:23:21.401259 20801 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
W20260709 12:23:21.380224 20681 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
W20260709 12:23:21.229496 25675 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.446s	user 0.000s	sys 0.001s
I20260709 12:23:21.402376 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:21.353478 25697 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
W20260709 12:23:21.350778 25711 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 12:23:21.367118 25700 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:21.428162 25695 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.051s	user 0.001s	sys 0.004s
I20260709 12:23:21.458526 20729 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:21.327770 24963 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:21.242220 25348 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:21.225358 25717 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:21.442898 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 360566us
W20260709 12:23:21.448784 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118046us
I20260709 12:23:21.248968 25385 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:21.425613 25557 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.001s
I20260709 12:23:21.232270 25703 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:21.131659 25621 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.306s
I20260709 12:23:21.447517 25322 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:21.442452 25715 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:21.344457 25583 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:21.391098 25699 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.484766 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 811) took 1589 ms. Trace:
I20260709 12:23:21.337520 25246 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.342025 25536 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:21.454773 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.486708 20881 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:21.101617 25653 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.116483 25666 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.329s
I20260709 12:23:21.413640 25318 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:21.465828 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:21.128008 25694 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.497438 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:21.465055 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:21.390827 25574 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:23:21.140465 25542 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.308207 25390 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:21.454181 25700 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:21.140178 25681 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:21.485338 20884 rpcz_store.cc:276] 0709 12:23:19.895440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.320328 (+424888us) service_pool.cc:224] Handling call
0709 12:23:20.320699 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:20.338222 (+ 17523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:20.338254 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:20.338738 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:23:20.688411 (+349673us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:20.789035 (+100624us) log.cc:844] Serialized 14 byte log entry
0709 12:23:20.824761 (+ 35726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:20.824875 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:21.356549 (+531674us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b180041db80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:21.478859 (+122310us) raft_consensus.cc:1685] finished
0709 12:23:21.484503 (+  5644us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.484687 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4965,"prepare.queue_time_us":201401,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":82638,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":66023,"spinlock_wait_cycles":6374656,"thread_start_us":397518,"threads_started":3,"wal-append.queue_time_us":598022}
I20260709 12:23:21.377609 25655 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:21.498370 20881 raft_consensus.cc:2376] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11. Re-sending same reply.
W20260709 12:23:21.476428 25688 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:23:21.469238 25406 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.660810 25689 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.661806 25689 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:21.508527 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 104671us
I20260709 12:23:21.506126 25381 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:21.477545 25382 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:21.516723 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51742.2us
I20260709 12:23:21.469060 25718 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:21.478623 25703 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:21.408429 25701 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.428658 25680 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.521539 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:21.735213 20884 raft_consensus.cc:2376] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16. Re-sending same reply.
I20260709 12:23:21.736337 20884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:21.737080 20884 raft_consensus.cc:606] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:21.737514 20884 raft_consensus.cc:614] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:21.499221 25322 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:21.498776 25673 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:21.503403 25671 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.472463 25385 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.563313 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:21.478259 25557 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.563930 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:21.804698 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 240965us
W20260709 12:23:21.576639 25598 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.003s
I20260709 12:23:21.577723 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.580766 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:21.581130 25656 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:21.582139 25642 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.942s	user 0.007s	sys 0.001s
W20260709 12:23:21.585861 25674 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.926s	user 0.009s	sys 0.000s
I20260709 12:23:21.588445 25653 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:21.592953 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.823460 20807 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:21.509166 25318 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.586529 25681 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.520080 25700 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:21.830343 25557 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.831441 25681 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:21.831725 25725 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:23:21.832237 20804 raft_consensus.cc:2368] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
I20260709 12:23:21.832124 25586 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.832923 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 732) took 1578 ms. Trace:
W20260709 12:23:21.833117 25683 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 12:23:21.794533 25692 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.354s
I20260709 12:23:21.833336 20734 rpcz_store.cc:276] 0709 12:23:20.254943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.255237 (+   294us) service_pool.cc:224] Handling call
0709 12:23:20.255617 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:20.255965 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:20.442021 (+186056us) log.cc:844] Serialized 14 byte log entry
0709 12:23:20.774628 (+332607us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:20.779248 (+  4620us) log.cc:844] Serialized 26 byte log entry
0709 12:23:20.779409 (+   161us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:20.956009 (+176600us) log.cc:844] Serialized 14 byte log entry
0709 12:23:20.976916 (+ 20907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:20.977005 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:21.832692 (+855687us) raft_consensus.cc:1685] finished
0709 12:23:21.832715 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.832864 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21868,"prepare.queue_time_us":400020,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"raft.queue_time_us":315611,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":76901,"spinlock_wait_cycles":54815744,"thread_start_us":570601,"threads_started":3,"wal-append.queue_time_us":594291}
W20260709 12:23:21.835549 25723 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:23:21.481842 25348 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.522047 25605 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.472018 24963 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.838116 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 733) took 1581 ms. Trace:
W20260709 12:23:21.562384 25535 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.896s	user 0.013s	sys 0.000s
I20260709 12:23:21.840520 25535 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.841958 25486 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:21.842483 25486 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.842840 25535 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.486650 25717 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.634997 20883 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:21.653264 20881 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:21.654016 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.497524 25699 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.520778 25390 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.700796 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:21.612632 25561 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.705600 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:21.497785 25715 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.720434 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:21.724143 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:21.696938 25404 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:21.730014 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 122744us
I20260709 12:23:21.838521 20730 rpcz_store.cc:276] 0709 12:23:20.256414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.256931 (+   517us) service_pool.cc:224] Handling call
0709 12:23:20.270861 (+ 13930us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:20.272160 (+  1299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:20.272191 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:23:20.287710 (+ 15519us) log.cc:844] Serialized 78 byte log entry
0709 12:23:20.711152 (+423442us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:20.773205 (+ 62053us) log.cc:844] Serialized 14 byte log entry
0709 12:23:20.783809 (+ 10604us) log.cc:844] Serialized 14 byte log entry
0709 12:23:20.783982 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:20.784060 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:20.937028 (+152968us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:23:21.349273 (+412245us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b18004b3700. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:21.837894 (+488621us) raft_consensus.cc:1685] finished
0709 12:23:21.837917 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.838059 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72801,"prepare.queue_time_us":185407,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":260131,"raft.run_cpu_time_us":4296,"raft.run_wall_time_us":681750,"spinlock_wait_cycles":37344256,"thread_start_us":372975,"threads_started":1,"wal-append.queue_time_us":1162443}
W20260709 12:23:21.732803 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2245) took 2062 ms (client timeout 2084 ms). Trace:
I20260709 12:23:21.712695 25713 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:21.727882 25603 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:21.741928 20884 raft_consensus.cc:2955] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Committing config change with OpId 7.8: config changed from index 5 to 8, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:21.726432 25595 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:21.685554 25406 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:21.760783 25564 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.138s	user 0.012s	sys 0.000s
W20260709 12:23:21.803296 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 240044us
W20260709 12:23:21.885154 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 161102us
I20260709 12:23:21.885921 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:21.734705 25381 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:21.811548 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:21.865180 25693 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.350s	user 0.000s	sys 0.001s
I20260709 12:23:21.826421 20879 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:21.855594 25605 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:21.824060 25642 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.831538 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2256) took 1046 ms. Trace:
I20260709 12:23:21.868227 25574 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:21.832729 25586 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.839057 25668 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:21.818866 25399 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:21.618762 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:21.619029 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:21.914808 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:21.915172 20801 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:21.604737 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 2.084s (SENT) [suppressed 17 similar messages]
I20260709 12:23:21.843876 21433 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:21.843958 21432 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:23:21.858047 20876 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:21.794664 25337 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:21.862006 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 141658us
W20260709 12:23:21.868076 20877 rpcz_store.cc:269] 0709 12:23:19.670750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.971080 (+300330us) service_pool.cc:224] Handling call
0709 12:23:21.732591 (+1761511us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b5400366d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:21.732728 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329518848}
I20260709 12:23:21.806373 25538 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:21.729569 25382 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:21.823827 20878 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:21.828059 25674 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.821085 25583 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:23:21.891302 20730 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:21.826644 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:21.878916 25713 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:21.896981 25730 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:23:21.896711 25381 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.898470 25720 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:21.825294 25636 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.884971 25322 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.759472 25718 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.902293 20803 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:23:21.902242 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 90799.2us
I20260709 12:23:22.004292 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:21.822295 25598 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.007890 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.013000 20875 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:22.016641 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.017576 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:22.030810 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:22.034525 20730 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:21.924166 25741 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:21.990091 25678 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.865025 25065 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:21.805833 25671 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:21.908744 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.908340 20735 rpcz_store.cc:276] 0709 12:23:20.784726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.785008 (+   282us) service_pool.cc:224] Handling call
0709 12:23:20.785380 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.056037 (+270657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.056076 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:21.084930 (+ 28854us) log.cc:844] Serialized 26 byte log entry
0709 12:23:21.490574 (+405644us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:21.490631 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.490729 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:21.831287 (+340558us) raft_consensus.cc:1685] finished
0709 12:23:21.831323 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.831483 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142276,"raft.queue_time_us":272005,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":9623,"thread_start_us":492348,"threads_started":2,"wal-append.queue_time_us":572853}
I20260709 12:23:21.917415 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.051693 20879 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:22.052050 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.052762 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.921610 25605 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.900097 25694 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.899214 25535 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:21.922258 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:21.894179 25681 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:21.938462 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.949513 20801 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:21.872830 25404 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:21.898770 25699 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:21.901820 25385 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:21.900200 25701 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.987509 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:22.062523 21432 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.914287 25668 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:22.084241 25737 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 12:23:22.086376 25734 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:23:22.090379 25636 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.046623 21430 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:23:22.092658 25586 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:22.095813 25535 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:22.096508 25535 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.986676 25729 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:22.060204 25348 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.100920 25598 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.102823 25674 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:22.104156 25718 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:22.048482 25587 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.119s	user 0.012s	sys 0.001s
I20260709 12:23:21.999903 25486 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:22.129343 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102784us
I20260709 12:23:22.055596 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.130913 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.131711 20735 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:22.123260 25385 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.129648 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2257) took 1344 ms. Trace:
I20260709 12:23:22.002707 25689 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.004184 24963 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.068549 25642 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:22.069309 25404 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:22.071842 25671 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:21.401985 20882 rpcz_store.cc:276] 0709 12:23:19.852427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.305293 (+452866us) service_pool.cc:224] Handling call
0709 12:23:20.305483 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:20.305615 (+   132us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:20.660718 (+355103us) log.cc:844] Serialized 14 byte log entry
0709 12:23:21.378421 (+717703us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b1800356a40. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a
0709 12:23:21.401113 (+ 22692us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:21.401178 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:21.401240 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.401317 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.401450 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22469,"prepare.queue_time_us":481180,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":25053,"spinlock_wait_cycles":48188800,"thread_start_us":533365,"threads_started":2,"wal-append.queue_time_us":303483}
I20260709 12:23:22.072775 20879 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:22.074033 25681 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.075268 25337 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:22.076053 25738 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:22.077093 25735 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:23:22.077497 25739 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
W20260709 12:23:22.078532 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 156349us
I20260709 12:23:22.136303 25390 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.079681 25680 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:21.988159 25382 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.049703 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:21.822679 25686 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:23:22.115046 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:22.122609 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:22.049280 25555 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.003s
I20260709 12:23:22.126443 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:22.128113 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 523359us
I20260709 12:23:22.134598 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:21.922492 25538 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.132598 20732 rpcz_store.cc:276] 0709 12:23:20.785062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.363570 (+578508us) service_pool.cc:224] Handling call
0709 12:23:21.363973 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.428365 (+ 64392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.428400 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:21.428898 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:23:21.561298 (+132400us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:21.561419 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.561558 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:22.129427 (+567869us) raft_consensus.cc:1685] finished
0709 12:23:22.129449 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.129587 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63970,"raft.queue_time_us":73432,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":11640,"thread_start_us":90433,"threads_started":1,"wal-append.queue_time_us":248929}
I20260709 12:23:22.145525 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.146216 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.149462 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.150133 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.152911 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.147698 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.108224 25406 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.107127 25713 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.133543 25587 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.108593 25715 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.162403 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.132261 25729 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:22.164757 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:22.079077 25701 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:22.173215 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 308728us
I20260709 12:23:22.178274 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.206050 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2265) took 1080 ms. Trace:
I20260709 12:23:22.066220 25678 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:22.212715 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48051.8us
I20260709 12:23:22.215427 25404 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:22.184564 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:22.187611 20806 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:22.188699 25538 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.141619 25385 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.155949 25738 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.188135 20810 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:22.129343 25574 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:22.191481 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:22.187175 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:22.203698 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.242782 20882 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:22.245585 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:22.246908 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.209445 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.210434 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.084280 25668 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.188081 20735 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a48bec319402467881543ac686fb8f0c"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:22.219439 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.206480 20884 rpcz_store.cc:276] 0709 12:23:21.125498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.741114 (+615616us) service_pool.cc:224] Handling call
0709 12:23:21.741516 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.741777 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:21.903213 (+161436us) log.cc:844] Serialized 14 byte log entry
0709 12:23:22.205704 (+302491us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:22.205757 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:22.205820 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.205882 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.205992 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":302037,"threads_started":1}
I20260709 12:23:22.252632 20735 raft_consensus.cc:606] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
W20260709 12:23:22.252758 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65700.8us
I20260709 12:23:22.253024 20735 raft_consensus.cc:614] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:22.222309 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.253083 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.219471 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:22.228600 21435 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:22.188881 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.159605 25337 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.232736 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.247433 20882 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:22.194371 25318 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.249004 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.218307 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:22.257609 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.257903 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:22.258142 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:23:22.239763 25597 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.447s	user 0.010s	sys 0.005s
I20260709 12:23:22.254300 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:22.255292 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.258756 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.259436 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:22.261145 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.212769 25564 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:23:22.190198 25555 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.239248 25404 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.212636 25692 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:23:22.204773 25680 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:23:22.265563 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:22.249368 25574 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.369715 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.374153 20809 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:23:22.372238 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.379863 25318 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:22.239173 25717 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.380205 25322 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.386008 25322 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.388669 25322 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.391060 25574 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.382387 25597 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.392161 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2267) took 1266 ms. Trace:
I20260709 12:23:22.392489 21428 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:22.393251 21435 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:22.382617 25404 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.395336 25597 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.396143 25404 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:22.396761 20882 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:22.397644 21433 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:22.229461 25707 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.141s
I20260709 12:23:22.400075 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:22.370759 25557 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:22.400763 25557 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.401422 25557 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.401707 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.404711 20809 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:23:22.393275 25703 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.598s	user 0.005s	sys 0.004s
W20260709 12:23:22.405692 25743 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.256s	user 0.000s	sys 0.003s
I20260709 12:23:22.392766 20881 rpcz_store.cc:276] 0709 12:23:21.125897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.866037 (+740140us) service_pool.cc:224] Handling call
0709 12:23:21.866455 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.869249 (+  2794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.869279 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:21.876381 (+  7102us) log.cc:844] Serialized 26 byte log entry
0709 12:23:22.151483 (+275102us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:22.151539 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.151628 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:22.391786 (+240158us) raft_consensus.cc:1685] finished
0709 12:23:22.391917 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.392096 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12301,"raft.queue_time_us":2810,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":4576,"thread_start_us":228587,"threads_started":1,"wal-append.queue_time_us":294656}
I20260709 12:23:22.406953 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2258) took 1621 ms. Trace:
I20260709 12:23:22.407495 25318 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:22.407450 20726 rpcz_store.cc:276] 0709 12:23:20.785380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.407658 (+622278us) service_pool.cc:224] Handling call
0709 12:23:21.423801 (+ 16143us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.727234 (+303433us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:21.727384 (+   150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:21.729776 (+  2392us) log.cc:844] Serialized 26 byte log entry
0709 12:23:21.824838 (+ 95062us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:21.824878 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.824944 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:22.405088 (+580144us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:22.406624 (+  1536us) raft_consensus.cc:1685] finished
0709 12:23:22.406648 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.406887 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42741,"raft.queue_time_us":47878,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":6260,"spinlock_wait_cycles":125990656,"thread_start_us":273826,"threads_started":2,"wal-append.queue_time_us":248282}
I20260709 12:23:22.408551 25318 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.409606 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.393285 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 735) took 1587 ms. Trace:
I20260709 12:23:22.412458 21431 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:23:22.256685 20810 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:22.266304 25642 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.263818 25741 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:22.406968 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.263947 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:22.415709 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:22.376909 21434 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:23:22.378822 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 113007us
I20260709 12:23:22.416708 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:22.417307 20881 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:22.417362 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:22.264618 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:22.381304 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 123479us
I20260709 12:23:22.400902 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2266) took 1275 ms. Trace:
I20260709 12:23:22.415362 20726 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:22.405948 25703 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.420869 20734 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:22.420969 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2490) took 1469 ms. Trace:
I20260709 12:23:22.266261 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:22.380136 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 125876us
W20260709 12:23:22.426191 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 162290us
I20260709 12:23:22.413857 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:22.426959 20735 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:22.411909 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2257) took 1288 ms. Trace:
I20260709 12:23:22.400701 20801 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:22.427817 20883 rpcz_store.cc:276] 0709 12:23:21.123421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.614770 (+491349us) service_pool.cc:224] Handling call
0709 12:23:21.634585 (+ 19815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.856646 (+222061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.411717 (+555071us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b540019bed0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:22.411846 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44540800}
I20260709 12:23:22.429097 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:22.429807 20883 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:22.243342 25738 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:22.445709 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.431244 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:22.264736 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.431771 21432 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:22.432214 20876 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.424587 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.493906 20734 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:22.418296 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:22.418306 20880 rpcz_store.cc:276] 0709 12:23:21.125726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.838013 (+712287us) service_pool.cc:224] Handling call
0709 12:23:21.838358 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:21.868542 (+ 30184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.868574 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:21.880258 (+ 11684us) log.cc:844] Serialized 52 byte log entry
0709 12:23:22.202573 (+322315us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:22.202618 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.202694 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:22.400643 (+197949us) raft_consensus.cc:1685] finished
0709 12:23:22.400666 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.400833 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22704,"raft.queue_time_us":22914,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":11417,"thread_start_us":282397,"threads_started":1,"wal-append.queue_time_us":335632}
I20260709 12:23:22.424577 20804 rpcz_store.cc:276] 0709 12:23:20.951545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.954919 (+  3374us) service_pool.cc:224] Handling call
0709 12:23:22.420712 (+1465793us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b540019aad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:22.420870 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1621251840}
I20260709 12:23:22.411973 25668 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:22.413516 20727 rpcz_store.cc:276] 0709 12:23:20.805637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.714806 (+909169us) service_pool.cc:224] Handling call
0709 12:23:21.730940 (+ 16134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.805605 (+ 74665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.805638 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:21.812688 (+  7050us) log.cc:844] Serialized 26 byte log entry
0709 12:23:22.191865 (+379177us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:22.191937 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.192018 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:22.393072 (+201054us) raft_consensus.cc:1685] finished
0709 12:23:22.393092 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.393226 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8011,"raft.queue_time_us":145869,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":28735,"thread_start_us":411320,"threads_started":2,"wal-append.queue_time_us":434282}
I20260709 12:23:22.434108 25741 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.264689 25583 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:22.491874 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.382388 25571 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:22.486497 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.399408 25555 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:22.496852 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65720.9us
I20260709 12:23:22.382036 25749 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:22.412065 25390 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:22.607756 25390 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:22.608255 25390 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:22.570438 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.505414 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.571915 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:22.590726 25745 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:22.592056 25751 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:23:22.620639 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.593611 25678 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.595810 25538 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.595819 25587 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.598110 25681 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:22.604149 25535 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
W20260709 12:23:22.606129 25700 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.002s
I20260709 12:23:22.545120 20735 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:22.589600 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:22.501343 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56638us
I20260709 12:23:22.556082 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:22.665246 20880 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:22.615269 20883 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:22.627588 20801 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:22.626808 20876 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:22.666708 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.667402 20883 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:22.629372 25745 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:22.635313 20802 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.004s	sys 0.013s
I20260709 12:23:22.485286 25676 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:22.487175 25689 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:22.668733 20802 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.004s	sys 0.013s
W20260709 12:23:22.675709 20730 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.000s
I20260709 12:23:22.676859 20730 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:22.677083 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.678064 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.498728 25598 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:22.504135 25741 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.556627 25605 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.617413 25583 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:22.678267 20735 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:23:22.671559 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82046us
I20260709 12:23:22.679136 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.635421 25680 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.498273 25642 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:22.496075 25404 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.166s
I20260709 12:23:22.683892 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:22.686133 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:22.690615 20883 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:23:22.650141 25322 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:22.648535 25690 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.681355 25587 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:22.697834 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.670262 25740 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.602468 25749 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.704237 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:22.674152 25676 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:22.684700 25586 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.486114 25729 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.707499 20727 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:22.689780 25583 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.670315 25535 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.729807 25745 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:22.378432 25666 consensus_queue.cc:1175] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 13 status { last_received { term: 12 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:22.733273 25538 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.694043 25701 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.676396 25700 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.708140 25690 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.724495 25740 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.746702 20735 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:23:22.783676 20735 pending_rounds.cc:85] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 4
I20260709 12:23:22.783998 20735 pending_rounds.cc:107] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:23:22.750993 20881 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:22.684065 25689 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.745082 25586 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.766412 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1012) took 1674 ms. Trace:
I20260709 12:23:22.770555 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.714581 25653 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:22.699079 25680 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.743337 25571 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:22.784435 20735 raft_consensus.cc:2889] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:22.785100 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.726344 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:22.806563 20881 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:22.786917 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.807615 21429 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:22.786038 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.809031 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.809525 21435 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:22.752115 20883 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:22.740432 25676 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:22.794713 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.698770 25642 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.797878 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 740) took 1555 ms. Trace:
I20260709 12:23:22.793890 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.704174 25322 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:22.829878 20728 rpcz_store.cc:276] 0709 12:23:21.242772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.825830 (+583058us) service_pool.cc:224] Handling call
0709 12:23:21.827885 (+  2055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.888314 (+ 60429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.888429 (+   115us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:21.888503 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:21.888572 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.797521 (+908949us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b54003ad4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:22.797591 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.797792 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136070528}
W20260709 12:23:22.797101 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74564.6us
I20260709 12:23:22.842747 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2269) took 1184 ms. Trace:
I20260709 12:23:22.843216 21433 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 8.
I20260709 12:23:22.843200 20883 rpcz_store.cc:276] 0709 12:23:21.657965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.829441 (+1171476us) service_pool.cc:224] Handling call
0709 12:23:22.829788 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:22.842412 (+ 12624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:22.842435 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:22.842479 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:22.842519 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.842579 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.842695 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:22.806409 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:22.845937 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.846529 20883 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:22.790292 21428 rpcz_store.cc:276] 0709 12:23:21.091980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.092120 (+   140us) service_pool.cc:224] Handling call
0709 12:23:21.118634 (+ 26514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:22.392995 (+1274361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.766161 (+373166us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:22.766341 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2929119104}
I20260709 12:23:22.814648 25586 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.815145 25337 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.817921 25557 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.820251 25680 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.820997 20730 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:22.779531 25605 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:22.822171 25754 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:23:22.824294 21435 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:23:22.828285 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101990us
I20260709 12:23:22.866773 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:22.790297 25656 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.004s
I20260709 12:23:22.829921 20727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:22.874045 20727 raft_consensus.cc:606] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:22.874470 20727 raft_consensus.cc:614] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:22.874277 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.784008 25701 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.858530 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:22.877836 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.861426 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:22.855137 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:22.756664 25738 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:22.864495 20728 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:22.798530 25653 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:22.796370 25717 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.852195 25710 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:23:22.872464 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:22.858925 25322 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.861307 25404 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:22.862600 25723 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:23:22.824594 25571 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:22.878499 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.910784 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:22.993480 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2254) took 1872 ms. Trace:
W20260709 12:23:23.046284 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 173878us
I20260709 12:23:23.046473 20875 rpcz_store.cc:276] 0709 12:23:21.121212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.121441 (+   229us) service_pool.cc:224] Handling call
0709 12:23:21.121799 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:22.013647 (+891848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.993099 (+979452us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54001d01d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:22.993384 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2027159424}
I20260709 12:23:22.958935 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:22.915630 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:23.047250 20849 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:23:22.900044 20802 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:23.091859 20849 leader_election.cc:403] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:23.092470 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45394.8us
I20260709 12:23:23.094180 20876 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:23.096154 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2326) took 1193 ms. Trace:
I20260709 12:23:23.103833 20880 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:23.097507 20876 rpcz_store.cc:276] 0709 12:23:21.903220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.805710 (+902490us) service_pool.cc:224] Handling call
0709 12:23:23.096079 (+290369us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:23.110297 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 255246us
I20260709 12:23:22.898463 20810 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:23.109472 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.153560 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2271) took 1438 ms. Trace:
I20260709 12:23:23.155028 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:23.154277 20880 rpcz_store.cc:276] 0709 12:23:21.715053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.152417 (+1437364us) service_pool.cc:224] Handling call
0709 12:23:23.152769 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.153009 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.153054 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:23.153120 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:23.153202 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.153300 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.153469 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:22.875953 25736 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.879410 25643 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:23.035002 25701 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:23.084267 25653 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:23.161636 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67985us
I20260709 12:23:22.875710 25574 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.897985 25671 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.160548 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:22.841811 25318 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.311s
I20260709 12:23:23.083506 25337 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:23.043462 25700 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:22.946614 25642 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:22.947494 25689 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.100459 25571 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:23.208814 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:23.190976 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:23.211779 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:23.212723 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:23.109253 25656 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:23:23.165504 25672 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:23.219285 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:23.038748 25583 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.198742 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:23.201418 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:23.158900 25678 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.877732 25605 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.217985 20726 raft_consensus.cc:2368] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
I20260709 12:23:23.101423 25740 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.171161 25557 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:23.221563 25763 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:23:22.977234 25390 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:23.203749 25723 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:23:23.102273 25717 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.260489 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:23.285032 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:23.205083 25574 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.286815 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:23.289486 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:23.290200 20727 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:23.195519 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2273) took 1477 ms. Trace:
I20260709 12:23:23.208979 25770 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.232460 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2251) took 2360 ms. Trace:
I20260709 12:23:23.170239 25404 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:23.235309 25586 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:23.328069 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 150134us
W20260709 12:23:23.237844 25761 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.376s	user 0.000s	sys 0.001s
I20260709 12:23:23.242810 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 787) took 2613 ms. Trace:
I20260709 12:23:23.243690 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2278) took 1039 ms. Trace:
I20260709 12:23:23.249820 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1017) took 1441 ms. Trace:
I20260709 12:23:23.205556 25701 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:23:23.222074 25766 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:23.251096 25603 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:23.359880 25603 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:23.359783 25766 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.361713 20728 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:23.362205 25766 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.252754 25738 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:23.264199 25732 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.364s	user 0.007s	sys 0.000s
I20260709 12:23:23.264765 25535 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.364996 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2261) took 2578 ms. Trace:
I20260709 12:23:23.365396 20731 rpcz_store.cc:276] 0709 12:23:20.786330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.426218 (+639888us) service_pool.cc:224] Handling call
0709 12:23:21.449963 (+ 23745us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.463586 (+ 13623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.463625 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:21.472263 (+  8638us) log.cc:844] Serialized 26 byte log entry
0709 12:23:21.719496 (+247233us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:21.719552 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.719630 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.263564 (+1543934us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:23.364661 (+101097us) raft_consensus.cc:1685] finished
0709 12:23:23.364682 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.364930 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13189,"raft.queue_time_us":49627,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":9748,"spinlock_wait_cycles":5815040,"thread_start_us":209006,"threads_started":1,"wal-append.queue_time_us":327218}
I20260709 12:23:23.220607 25689 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:23.181988 25671 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.291415 25742 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:23.303128 20875 rpcz_store.cc:276] 0709 12:23:21.717789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.151941 (+1434152us) service_pool.cc:224] Handling call
0709 12:23:23.193980 (+ 42039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.194574 (+   594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.194642 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:23.194723 (+    81us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:23.194843 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.195250 (+   407us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.195457 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:23.319530 20803 rpcz_store.cc:276] 0709 12:23:20.868532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.869367 (+   835us) service_pool.cc:224] Handling call
0709 12:23:20.869657 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:22.003776 (+1134119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.228221 (+1224445us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540027ae54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:23.228504 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2352536832}
I20260709 12:23:23.326094 25739 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:23.332870 20882 rpcz_store.cc:276] 0709 12:23:22.204553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.255032 (+ 50479us) service_pool.cc:224] Handling call
0709 12:23:23.243514 (+988482us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b540005ad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:23.243627 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42953984}
I20260709 12:23:23.332952 21430 rpcz_store.cc:276] 0709 12:23:21.808862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.808994 (+   132us) service_pool.cc:224] Handling call
0709 12:23:21.809347 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.091964 (+282617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.249529 (+1157565us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400088c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:23.249746 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545119488}
I20260709 12:23:23.341202 25574 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:23.332154 20807 rpcz_store.cc:276] 0709 12:23:20.629826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.637289 (+  7463us) service_pool.cc:224] Handling call
0709 12:23:20.637611 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.826348 (+1188737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.242564 (+1416216us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:23.242740 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2726700928}
W20260709 12:23:23.353235 25773 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:23:23.336000 25404 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.364051 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2275) took 1489 ms. Trace:
I20260709 12:23:23.432893 20735 rpcz_store.cc:276] 0709 12:23:21.874372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.729100 (+854728us) service_pool.cc:224] Handling call
0709 12:23:22.746221 (+ 17121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:22.820692 (+ 74471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.363826 (+543134us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b54000e79d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:23.363981 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38429568}
I20260709 12:23:23.228763 25672 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.175283 25736 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.319761 25740 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:23.370368 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2270) took 1503 ms. Trace:
I20260709 12:23:23.385138 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:23.436717 21429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.020s
I20260709 12:23:23.437162 21427 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:23.437170 21429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.007s	sys 0.020s
I20260709 12:23:23.385957 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2519) took 1168 ms. Trace:
I20260709 12:23:23.329203 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:23.371112 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 260020us
W20260709 12:23:23.438354 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 109247us
I20260709 12:23:23.328373 25769 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:23.421876 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:23.430230 20882 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
W20260709 12:23:23.393872 25668 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.855s	user 0.010s	sys 0.002s
I20260709 12:23:23.368328 20727 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:23.384155 25656 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:23:23.439908 20882 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 7
I20260709 12:23:23.448180 20882 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:23:23.328068 25770 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.448503 20882 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:23.439950 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:23.438019 20731 rpcz_store.cc:276] 0709 12:23:22.217257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.385029 (+1167772us) service_pool.cc:224] Handling call
0709 12:23:23.385388 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.385599 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.385625 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:23.385668 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:23.385708 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.385768 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.385898 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:23.366852 25586 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:23.446468 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 326294us
I20260709 12:23:23.436625 20732 rpcz_store.cc:276] 0709 12:23:21.866986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.190796 (+323810us) service_pool.cc:224] Handling call
0709 12:23:22.201416 (+ 10620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.201818 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:22.587398 (+385580us) log.cc:844] Serialized 14 byte log entry
0709 12:23:22.865667 (+278269us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.101581 (+235914us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.101822 (+   241us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:23.101861 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.101933 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.370162 (+268229us) raft_consensus.cc:1685] finished
0709 12:23:23.370185 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.370312 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":427233,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":57936,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":13696,"spinlock_wait_cycles":1834240,"thread_start_us":567250,"threads_started":3,"wal-append.queue_time_us":236379}
I20260709 12:23:23.450147 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:23.453437 20849 leader_election.cc:400] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
I20260709 12:23:23.453819 20849 leader_election.cc:403] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:23.456166 20875 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:23.456265 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:23.371299 25689 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.432143 25739 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:23.436553 25718 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:23.463613 20728 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:23.449222 25574 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:23.446070 25716 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:23.467659 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2525) took 1231 ms. Trace:
I20260709 12:23:23.463838 25246 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.469854 25717 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.469749 25736 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:23.471385 25718 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.446285 25535 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:23.475173 25770 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.476955 25583 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.477610 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2270) took 1762 ms. Trace:
I20260709 12:23:23.481132 25382 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.446099 25740 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.482414 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:23.603920 25574 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:23.583350 25708 consensus_queue.cc:1175] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 12 status { last_received { term: 12 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 12 index: 9 } }
I20260709 12:23:23.585738 25671 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:23.604545 20807 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:23.582805 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2324) took 1692 ms. Trace:
I20260709 12:23:23.605587 20882 rpcz_store.cc:276] 0709 12:23:21.890130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.386597 (+1496467us) service_pool.cc:224] Handling call
0709 12:23:23.386948 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.452914 (+ 65966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.582448 (+129534us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b5400200b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:23.582703 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98684160}
I20260709 12:23:23.606719 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.572224 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:23.607273 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.571663 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:23.605960 20731 rpcz_store.cc:276] 0709 12:23:22.236299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.454958 (+1218659us) service_pool.cc:224] Handling call
0709 12:23:23.466900 (+ 11942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.467153 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.467221 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:23.467281 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:23.467357 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.467433 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.467585 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:23.606508 20883 rpcz_store.cc:276] 0709 12:23:21.714782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.849253 (+1134471us) service_pool.cc:224] Handling call
0709 12:23:22.849558 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.875682 (+ 26124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:22.875717 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.085728 (+210011us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.394301 (+308573us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:23.394348 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.394428 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.477360 (+ 82932us) raft_consensus.cc:1685] finished
0709 12:23:23.477389 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.477543 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23808,"raft.queue_time_us":28335,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":4225,"thread_start_us":213320,"threads_started":1,"wal-append.queue_time_us":323313}
W20260709 12:23:23.607219 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.612227 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:23.505335 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:23.454973 25701 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:23.613307 20883 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:23.608053 20882 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:23.634479 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:23.506424 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.628181 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:23.635211 20882 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:23.611377 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.462612 25727 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:23.636510 20809 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:23:23.614219 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:23.459367 25691 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14.
W20260709 12:23:23.555876 25676 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
W20260709 12:23:23.553748 25774 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:23:23.645733 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.651885 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:23.603145 25586 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:23.585598 25605 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.554886 25390 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.493477 25656 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:23.601236 25781 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 12:23:23.654449 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.834204 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:23.835388 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 183571us
I20260709 12:23:23.613727 25574 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:23.770359 20882 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:23.655133 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2274) took 1937 ms. Trace:
W20260709 12:23:23.836566 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 228198us
I20260709 12:23:23.836905 25673 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:23.855588 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:23.856024 25673 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:23.856909 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:23.857568 20728 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:23.787371 20883 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:23.859277 25714 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:23.859872 25714 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:23.837513 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 223362us
I20260709 12:23:23.861872 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:23.862169 25714 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.556389 25739 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.653764 25690 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.863788 25671 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:23.864542 25574 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.865433 25672 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.586112 25571 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.866688 25574 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:23.645135 25727 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.655665 25603 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.872150 25727 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:23.855465 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:23.858985 25772 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.579s	user 0.005s	sys 0.000s
I20260709 12:23:23.867902 20881 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:23.858453 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:23.868002 20880 rpcz_store.cc:276] 0709 12:23:21.718025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.178601 (+1460576us) service_pool.cc:224] Handling call
0709 12:23:23.179600 (+   999us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.444157 (+264557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.444187 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.451490 (+  7303us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.647366 (+195876us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:23.647404 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.647487 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.654868 (+  7381us) raft_consensus.cc:1685] finished
0709 12:23:23.654888 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.655072 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12064,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":9720,"thread_start_us":433686,"threads_started":2,"wal-append.queue_time_us":182694}
I20260709 12:23:23.703598 25246 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.875828 25672 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:23.854429 25656 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.883126 25672 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.884838 25656 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.706995 25598 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:23.840056 25385 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.885571 25672 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.875962 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:23.867116 25740 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.843607 25691 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:23.887573 25691 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.887996 25691 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:23.879261 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:23.627104 25535 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.751766 25674 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:23.882009 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:23.882578 20880 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:23:23.899016 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2285) took 1496 ms. Trace:
I20260709 12:23:23.899621 20880 rpcz_store.cc:276] 0709 12:23:22.402949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.881964 (+1479015us) service_pool.cc:224] Handling call
0709 12:23:23.882322 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.898722 (+ 16400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.898955 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:23.901633 20880 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:23:23.840324 25781 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:23.904615 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:23.653316 25736 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:23.632431 25716 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.914944 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:23.919984 20880 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:23.921507 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:23.751485 25583 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.138s
I20260709 12:23:23.656754 25676 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.927572 20699 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:23.927919 20699 leader_election.cc:403] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:23:23.885581 25598 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.932204 25598 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:23.932771 25603 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.935271 25535 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.936458 25322 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:23.936884 25322 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:23.937673 25741 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.231s	user 0.010s	sys 0.000s
I20260709 12:23:23.937752 25322 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:23.938309 25741 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.940049 25718 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:23.944198 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2278) took 2034 ms. Trace:
I20260709 12:23:23.944690 20734 rpcz_store.cc:276] 0709 12:23:21.909542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.801263 (+891721us) service_pool.cc:224] Handling call
0709 12:23:22.801715 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.844414 (+ 42699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:22.844448 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:22.862267 (+ 17819us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.215562 (+353295us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:23.215637 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.215744 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.778545 (+562801us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:23.943881 (+165336us) raft_consensus.cc:1685] finished
0709 12:23:23.943905 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.944131 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51374,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"spinlock_wait_cycles":7422720,"thread_start_us":349340,"threads_started":2,"wal-append.queue_time_us":376562}
I20260709 12:23:23.945871 25673 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:23.913682 25744 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.755s	user 0.004s	sys 0.004s
W20260709 12:23:23.951735 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2321) took 2062 ms (client timeout 2016 ms). Trace:
W20260709 12:23:23.952216 20879 rpcz_store.cc:269] 0709 12:23:21.889648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.052479 (+162831us) service_pool.cc:224] Handling call
0709 12:23:23.951557 (+1899078us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b5400305a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:23.951662 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44404480}
I20260709 12:23:23.953567 20876 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:23.959478 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:23.960407 21429 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:23.962620 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:23.964227 20726 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:23.964509 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.926112 25740 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:23.924677 25736 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.912467 25246 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:23.965958 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 104174us
I20260709 12:23:23.963356 21429 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:23.971901 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:23.966550 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 299237us
I20260709 12:23:23.966648 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.966982 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:23.973043 21427 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:23.965531 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2523) took 1715 ms. Trace:
I20260709 12:23:23.970018 20728 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:23.972903 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:23.973886 20730 rpcz_store.cc:276] 0709 12:23:22.235661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.677806 (+442145us) service_pool.cc:224] Handling call
0709 12:23:23.951203 (+1273397us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:23.951306 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326875520}
I20260709 12:23:23.969286 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.975126 20880 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:23.975073 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:23.973229 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:23.956983 25322 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:23.978128 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:23.979729 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:23.980530 21428 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:23.982730 20850 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:23:23.977289 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:23.987123 21400 leader_election.cc:400] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
I20260709 12:23:23.987686 21400 leader_election.cc:403] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:23:23.966365 25718 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:23.988560 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:23.992516 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:23.995724 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:23.998729 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.005968 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:23.990572 25653 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.783s	user 0.009s	sys 0.001s
I20260709 12:23:24.012084 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:23.978505 25583 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:24.035919 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2267) took 2894 ms. Trace:
I20260709 12:23:24.011053 25740 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.037192 25740 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:24.037986 25740 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:24.036469 20729 rpcz_store.cc:276] 0709 12:23:21.141479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.720208 (+578729us) service_pool.cc:224] Handling call
0709 12:23:21.720489 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.720683 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.720707 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:21.784000 (+ 63293us) log.cc:844] Serialized 26 byte log entry
0709 12:23:21.948739 (+164739us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:21.948779 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.948844 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.887642 (+1938798us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:24.011147 (+123505us) raft_consensus.cc:1685] finished
0709 12:23:24.011167 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.035806 (+ 24639us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95124736,"thread_start_us":122609,"threads_started":1,"wal-append.queue_time_us":306385}
I20260709 12:23:24.014734 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.016289 25246 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.008972 25713 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:23.996994 25595 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:24.024237 25741 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.989588 25404 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:24.028229 25736 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:24.030202 25673 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.031949 25676 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:23.996698 25674 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:23:24.033460 25535 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:24.034358 25716 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.034409 25689 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.041139 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.022033 25727 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:23.994920 25794 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:23.994491 25792 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
I20260709 12:23:24.009017 25739 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:24.063521 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1213) took 2220 ms. Trace:
I20260709 12:23:23.995008 25714 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:24.046128 25718 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:24.068467 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.064539 25595 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:24.072424 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.045928 25713 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:24.091991 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:24.093362 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52317.7us
I20260709 12:23:24.093600 21426 rpcz_store.cc:276] 0709 12:23:21.840648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.847380 (+  6732us) service_pool.cc:224] Handling call
0709 12:23:21.860510 (+ 13130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.887827 (+ 27317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:21.887877 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:21.887939 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:21.887986 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.060679 (+2172693us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:24.060749 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.060921 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62187392}
I20260709 12:23:24.113422 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.101159 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.115847 20875 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:24.116267 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:24.042284 25653 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.102241 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:24.103971 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.120810 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:24.101042 25673 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:24.098398 25739 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:24.123517 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:24.122536 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82575.7us
I20260709 12:23:24.123780 21429 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
I20260709 12:23:24.099417 25718 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:24.130617 21429 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:24.133176 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:24.128448 25796 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:23:24.101512 25794 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.125927 25535 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:24.131680 25718 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.129774 25739 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.131281 25741 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:24.132166 25672 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:24.131451 25716 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:24.132375 25720 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:24.142040 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.118072 25689 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:23:24.149232 20879 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:24.135648 25404 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:24.159268 25536 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:24.159981 25720 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:24.149101 25716 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.163538 25653 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.188937 25716 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:24.165343 25794 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:24.169337 25571 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:24.171613 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2265) took 3384 ms. Trace:
I20260709 12:23:24.191510 20733 rpcz_store.cc:276] 0709 12:23:20.787502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.480754 (+693252us) service_pool.cc:224] Handling call
0709 12:23:21.504902 (+ 24148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:22.047553 (+542651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:22.047583 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:22.047622 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:22.047663 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.171377 (+2123714us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b5400249fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:24.171423 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.171544 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247492864}
I20260709 12:23:24.172149 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2286) took 1767 ms. Trace:
I20260709 12:23:24.174361 25740 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.180241 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2279) took 1966 ms. Trace:
I20260709 12:23:24.192651 25720 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.192740 20878 rpcz_store.cc:276] 0709 12:23:22.405070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.886690 (+1481620us) service_pool.cc:224] Handling call
0709 12:23:23.887017 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.887396 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.887426 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.887933 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.994006 (+106073us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:23.994061 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.994149 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.171927 (+177778us) raft_consensus.cc:1685] finished
0709 12:23:24.171947 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.172088 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16417,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":1914,"thread_start_us":105635,"threads_started":1,"wal-append.queue_time_us":176326}
I20260709 12:23:24.193002 20877 rpcz_store.cc:276] 0709 12:23:22.213788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.841104 (+1627316us) service_pool.cc:224] Handling call
0709 12:23:23.841543 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.842042 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.842085 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.842713 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.039141 (+196428us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:24.130725 (+ 91584us) log.cc:844] Serialized 14 byte log entry
0709 12:23:24.133281 (+  2556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.133355 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.180035 (+ 46680us) raft_consensus.cc:1685] finished
0709 12:23:24.180053 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.180179 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2309,"prepare.queue_time_us":103392,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":5555,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":20488,"thread_start_us":200383,"threads_started":2,"wal-append.queue_time_us":213311}
I20260709 12:23:24.181833 25536 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:24.195676 25536 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.195993 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.195899 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.197221 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.198239 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.199105 25727 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:24.199846 25740 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.195755 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.201525 20877 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:24.205650 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:24.206599 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.207417 20878 raft_consensus.cc:2368] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
I20260709 12:23:24.186676 25595 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.208348 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.209477 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.137053 25603 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:23:24.145191 25717 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:24.210510 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.210640 25740 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:24.210819 25717 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.148022 25741 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.211066 20878 raft_consensus.cc:2376] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17. Re-sending same reply.
I20260709 12:23:24.211181 25740 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:24.211448 25741 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.212105 25740 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.213032 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.213280 25740 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:24.214105 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.214802 20879 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:24.215308 25713 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.216428 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:24.218508 25404 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.220567 25404 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.224272 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.224932 25717 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.225510 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.180691 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.179431 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:24.226758 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.231247 21399 leader_election.cc:400] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13.
I20260709 12:23:24.231513 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:24.232086 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.227478 21429 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:24.227700 25703 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.230108 21427 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:24.181389 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.232513 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.233332 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.234659 20877 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 8.
I20260709 12:23:24.235443 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.236750 20879 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:24.236590 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.233362 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.237700 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.238412 20877 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:24.239123 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.239802 20877 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:24.240432 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.241374 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.242314 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.234424 20880 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:24.202636 25691 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.244964 20878 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:24.255991 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.256551 20879 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
I20260709 12:23:24.260000 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:24.260325 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.260787 25791 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:24.260969 21428 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:24.261149 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.261333 25791 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:24.261704 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.262598 25583 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:24.262269 25791 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.262852 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:24.264011 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.257097 20879 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 8: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:24.243228 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:24.272046 20877 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:24.272138 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:24.262193 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:24.223098 25685 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.297572 20878 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:24.258919 25486 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:24.255493 25689 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:24.259121 25686 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.301460 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.306805 25598 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:24.308552 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.293550 25538 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:24.295572 25729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.017s	sys 0.020s
I20260709 12:23:24.305981 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:24.294001 25701 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:24.311982 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.290343 25541 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:23:24.296300 25685 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:24.313820 25583 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:24.314246 25685 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:24.321165 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.321826 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2283) took 1944 ms. Trace:
I20260709 12:23:24.324425 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2272) took 2609 ms. Trace:
I20260709 12:23:24.324831 20876 rpcz_store.cc:276] 0709 12:23:21.717515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.156536 (+1439021us) service_pool.cc:224] Handling call
0709 12:23:23.953118 (+796582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.954099 (+   981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.324228 (+370129us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400305a50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:24.324363 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749303680}
I20260709 12:23:24.257048 25404 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:23:24.322258 20882 rpcz_store.cc:276] 0709 12:23:22.377467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.854395 (+1476928us) service_pool.cc:224] Handling call
0709 12:23:23.854747 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.008773 (+154026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:24.008804 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.016991 (+  8187us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.119455 (+102464us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:24.121887 (+  2432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.321380 (+199493us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:24.321434 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.321490 (+    56us) raft_consensus.cc:1685] finished
0709 12:23:24.321505 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.321759 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11,"raft.queue_time_us":14521,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":22258,"spinlock_wait_cycles":321824640,"thread_start_us":101883,"threads_started":1,"wal-append.queue_time_us":125587}
I20260709 12:23:24.332201 20882 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:24.304723 25689 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:24.311226 25729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.017s	sys 0.020s
I20260709 12:23:24.311717 25727 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:24.327798 25714 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:24.328861 25701 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:24.334528 25699 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:24.351559 20877 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:24.353571 20877 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:23:24.354307 20877 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 9
I20260709 12:23:24.354660 20877 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:23:24.339576 25801 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:24.339761 25689 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.354974 20877 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:24.350318 25701 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:24.351148 25699 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:24.366843 20877 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:24.369526 25716 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.367623 25699 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.357453 25801 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:24.382488 25713 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:24.384236 20877 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:24.377113 25594 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:24.379379 25538 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:24.339326 25486 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:23:24.396433 25538 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:24.409332 25538 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.394665 25594 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:24.412012 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 972545us
W20260709 12:23:24.412721 25808 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:23:24.413868 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:24.415014 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:24.416922 25780 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:24.416710 25741 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.417346 25780 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.417718 25780 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.419132 25668 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:24.419740 25668 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:24.420174 25668 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:24.423074 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.425980 20805 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:24.427907 25694 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:24.428488 25694 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.430557 25694 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.431421 20729 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:23:24.432627 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.435143 25710 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.435642 25710 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.437671 25710 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.439565 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.440599 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:24.441569 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.441861 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.442955 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.443254 20726 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:24.360777 25714 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:23:24.441653 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.448742 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.449925 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:24.450820 20803 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
W20260709 12:23:24.451282 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.439529 25322 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:24.453575 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:24.463825 20882 raft_consensus.cc:2376] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14. Re-sending same reply.
I20260709 12:23:24.451712 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.431011 25694 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:24.473341 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.474263 20735 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:24.475834 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:24.476186 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.477821 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.411111 25538 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:24.469911 20730 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:24.419775 25691 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:24.447599 25636 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:24.451543 25781 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:24.469002 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:24.484498 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.472223 20733 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:24.485903 20880 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:24.451407 25603 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:24.479094 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.468632 25671 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:24.490257 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:24.497793 20809 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
W20260709 12:23:24.490792 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65493us
I20260709 12:23:24.468474 25770 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:24.493728 20730 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:24.493069 20727 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:24.495728 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:24.492111 21432 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:24.506100 20735 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.502949 20733 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:24.500952 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:24.517010 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.518179 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:24.528615 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.529937 25636 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:24.530889 25636 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:24.531735 25636 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:24.532243 25636 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.538125 20804 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:24.541076 20810 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:24.541075 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:24.542454 20802 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:24.543877 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:24.544332 25636 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.544842 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:24.488196 25666 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:24.549211 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:24.461202 25322 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.555357 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:24.555277 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.556456 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2334) took 1128 ms. Trace:
I20260709 12:23:24.556869 20875 rpcz_store.cc:276] 0709 12:23:23.427880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.607855 (+179975us) service_pool.cc:224] Handling call
0709 12:23:24.556259 (+948404us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:24.556400 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038008576}
I20260709 12:23:24.451887 25812 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:23:24.491947 25783 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:24.562824 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:24.491274 25536 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.480108 25557 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
W20260709 12:23:24.553357 25793 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.422s	user 0.000s	sys 0.004s
I20260709 12:23:24.564559 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:24.567091 20730 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:24.519009 25770 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:24.519501 25781 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:24.592298 20728 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:24.564884 20733 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:24.591908 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159359us
I20260709 12:23:24.515548 25692 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:23:24.546569 25696 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:24.588364 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:24.580215 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:24.594158 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65625.4us
I20260709 12:23:24.584159 20802 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:24.570685 25817 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:23:24.492130 25694 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.595757 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:24.623503 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.625066 20735 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:24.626338 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:24.627707 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.630308 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.594333 25816 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:24.600457 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.602553 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:24.482509 25801 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.641533 25586 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.642038 25586 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:24.642432 25586 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:24.634662 25794 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:24.647853 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2518) took 1164 ms. Trace:
I20260709 12:23:24.648276 20801 rpcz_store.cc:276] 0709 12:23:23.483890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.484040 (+   150us) service_pool.cc:224] Handling call
0709 12:23:23.552374 (+ 68334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.426623 (+874249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:24.426657 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.427179 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.563313 (+136134us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:24.563356 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.563422 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.647109 (+ 83687us) raft_consensus.cc:1685] finished
0709 12:23:24.647637 (+   528us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.647787 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":873735,"raft.queue_time_us":882157,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":135693,"threads_started":1,"wal-append.queue_time_us":203086}
I20260709 12:23:24.650666 20801 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:24.652311 25598 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:24.577510 25783 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.653702 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:24.654706 25783 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.655922 25597 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.658152 25694 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.659605 25595 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.660055 25595 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:24.660426 25595 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:24.661190 25322 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.663352 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:24.635740 25810 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 12:23:24.667969 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2268) took 3541 ms. Trace:
I20260709 12:23:24.668345 20881 rpcz_store.cc:276] 0709 12:23:21.126143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.822874 (+1696731us) service_pool.cc:224] Handling call
0709 12:23:22.823881 (+  1007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.879297 (+1055416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.667721 (+788424us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400305f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:24.667909 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2400375680}
I20260709 12:23:24.671725 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.586778 25718 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.591794 25770 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.634428 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.634517 21432 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:24.564329 25666 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.688740 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2284) took 2286 ms. Trace:
I20260709 12:23:24.706306 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.706173 20883 rpcz_store.cc:276] 0709 12:23:22.402078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.859695 (+1457617us) service_pool.cc:224] Handling call
0709 12:23:23.860041 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.860439 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.860472 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.860947 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.065066 (+204119us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:24.065122 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.065218 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.203244 (+138026us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b4800240258. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:23:24.435039 (+231795us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:24.688493 (+253454us) raft_consensus.cc:1685] finished
0709 12:23:24.688517 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.688673 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2534,"raft.run_cpu_time_us":2988,"raft.run_wall_time_us":340444,"spinlock_wait_cycles":19165312,"thread_start_us":203752,"threads_started":1,"wal-append.queue_time_us":244332}
I20260709 12:23:24.690564 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:24.595889 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.656424 25597 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.701633 20735 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:24.595588 25700 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:23:24.602312 25486 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:23:24.709055 20802 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:24.709216 20806 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:24.628669 25696 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:24.710496 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:24.715036 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.702792 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:24.739430 20809 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.723613 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.622335 25717 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.743691 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.745759 25691 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:24.746694 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:24.746877 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151092us
I20260709 12:23:24.747339 21432 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:24.700062 25739 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.004s
I20260709 12:23:24.688090 25680 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:24.687601 25406 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.631523 25685 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.718271 25716 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.737825 25322 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:24.689862 25736 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:24.737506 25816 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:24.746721 25783 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:24.738587 25694 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:23:24.778753 25741 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.793447 20806 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:24.751811 25708 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:23:24.778312 25680 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.821316 25717 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.824759 25666 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.826849 25816 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.831113 25685 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.832464 25636 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:24.833384 25636 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:24.833899 25636 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:24.834388 25636 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:24.792938 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49322.5us
I20260709 12:23:24.795375 21432 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:24.837046 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:24.803386 20734 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:24.839385 25680 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:24.785916 25701 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.844445 25718 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:24.846086 25822 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:23:24.846236 25542 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:24.786934 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63455.2us
I20260709 12:23:24.757665 25781 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:24.857378 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 188082us
I20260709 12:23:24.842850 25701 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:24.850060 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.850303 20733 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:24.859726 21432 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:23:24.863489 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.785070 25739 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.786376 25406 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.879566 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:24.866278 25671 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:24.890112 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:24.850536 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 195757us
I20260709 12:23:24.894457 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:24.903390 25794 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:24.897964 25691 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:24.905386 25831 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:24.907451 25831 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:23:24.908466 25811 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.431s	user 0.000s	sys 0.002s
I20260709 12:23:24.914191 25668 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.876633 24963 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:24.897949 25704 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:23:24.915655 24963 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:24.809662 25736 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.883239 25701 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.869836 25708 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:23:24.847652 25696 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.941969 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:24.943123 20730 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:24.956637 20809 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:24.897212 25322 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:24.963979 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 214994us
I20260709 12:23:24.965521 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:24.968428 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129614us
I20260709 12:23:24.944708 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.944603 25704 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:24.886034 25597 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:24.916642 24963 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:24.982307 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:24.973670 25708 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:24.984146 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:24.995167 25594 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.001217 25691 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:25.001681 25691 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:25.003770 25824 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:23:25.004092 25736 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:24.986210 25671 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:25.005038 25671 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:25.006084 25671 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:25.003141 25826 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:23:25.007934 25833 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:25.010818 25686 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.701s	user 0.009s	sys 0.000s
I20260709 12:23:24.973209 25557 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:23:24.975754 25806 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:23:25.012198 20884 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:25.013180 25680 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:24.984476 25836 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:23:25.017462 25714 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:25.023790 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:25.011394 25322 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.019680 25538 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:25.031116 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:25.032717 25806 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:25.033033 25406 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:25.033347 25779 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:25.033914 25780 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 12:23:25.025935 25708 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:25.057243 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88059.5us
I20260709 12:23:25.036942 25714 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:25.029229 25571 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:25.041082 25691 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.043888 25542 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:25.019394 25749 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.044237 25636 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.051334 25680 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.046919 25704 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:25.055704 25668 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.056308 25739 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:25.041653 25699 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.003s
I20260709 12:23:25.041147 20728 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:25.059927 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:25.059760 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:25.036855 25686 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.026600 25710 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:23:25.003084 25586 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.042214 25806 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:25.139698 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80994.6us
W20260709 12:23:25.157291 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 177529us
W20260709 12:23:25.156881 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111551us
I20260709 12:23:25.160820 25704 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:25.161755 25689 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.801s	user 0.008s	sys 0.003s
I20260709 12:23:25.139169 25542 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.057281 25779 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.167316 25699 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:25.169667 25595 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.002s
I20260709 12:23:25.170100 25714 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.137404 25571 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.101048 25708 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.065261 25780 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:25.175149 25603 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.671s	user 0.016s	sys 0.000s
I20260709 12:23:25.149040 25681 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:23:25.194283 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.048624 25838 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
I20260709 12:23:25.176541 25691 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.206652 21433 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:25.208961 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:25.230719 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.231503 20875 raft_consensus.cc:2368] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
I20260709 12:23:25.209540 25841 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:23:25.221984 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:25.222750 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:25.224951 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:25.241888 25704 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.242369 24963 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:25.243304 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.247040 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:25.251618 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:25.251734 25668 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:23:25.233114 25603 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:25.254101 25780 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.255986 25708 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:25.256835 25542 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:25.258244 25571 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.258395 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:25.258487 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.259071 25318 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:25.259512 25318 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:25.261579 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.262157 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:25.241989 25701 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.276811 25603 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:25.278085 25770 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.001s
I20260709 12:23:25.277822 25603 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.201056 25689 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.280552 20879 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:25.281033 25689 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.259621 25781 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:23:25.281625 25603 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.284634 25699 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.286636 25838 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:25.287137 25838 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:25.287745 25806 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:25.288915 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64059.6us
I20260709 12:23:25.289151 25806 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:25.289927 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:25.269212 25714 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.292145 20726 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:25.229943 25681 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:25.273083 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:25.295040 20876 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:25.278584 25770 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.268102 25686 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:25.304430 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:25.307650 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:25.308224 20735 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:25.308473 20727 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:25.305706 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:25.309551 20733 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:25.207856 25595 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.320721 20876 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:25.327939 20730 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:25.335834 25827 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.419s	user 0.005s	sys 0.000s
I20260709 12:23:25.340166 25779 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:25.343866 20875 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:25.344899 25595 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:25.345574 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46724.4us
I20260709 12:23:25.345664 25318 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.346433 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:25.346968 25318 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:25.347133 20880 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:25.350713 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:25.351490 20733 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:25.352756 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:25.306983 25780 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:25.315961 25841 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:25.316398 25699 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:25.298167 25542 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:23:25.357407 25668 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:25.360039 25404 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:25.361156 25846 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:23:25.327055 25686 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:25.376039 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 158539us
I20260709 12:23:25.358297 25671 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:25.395830 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.358839 25841 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:25.366153 25736 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:25.377178 25699 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:25.345582 25595 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:23:25.402247 25736 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.424214 25686 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.427316 25586 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:25.397382 25538 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:25.365577 25681 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.395908 25668 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:25.366926 25692 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:23:25.449570 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.403955 25841 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:25.452114 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.437443 25770 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.440771 25736 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:25.446170 25691 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.402421 25671 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.517851 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.524147 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.527706 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.529371 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.529994 20805 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:25.532151 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.532910 20806 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:23:25.540237 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:25.546723 20884 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2280) took 3327 ms (client timeout 1616 ms). Trace:
W20260709 12:23:25.547253 20884 rpcz_store.cc:269] 0709 12:23:22.219775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.252858 (+ 33083us) service_pool.cc:224] Handling call
0709 12:23:25.546494 (+3293636us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b54001ff254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:25.546609 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6343304448}
I20260709 12:23:25.547628 25668 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.550981 25594 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:25.553972 25825 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.507s	user 0.000s	sys 0.001s
I20260709 12:23:25.558048 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2532) took 1125 ms. Trace:
I20260709 12:23:25.558478 20729 rpcz_store.cc:276] 0709 12:23:24.432544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.452074 (+ 19530us) service_pool.cc:224] Handling call
0709 12:23:24.452536 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.453042 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:24.570793 (+117751us) log.cc:844] Serialized 14 byte log entry
0709 12:23:24.902169 (+331376us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.902699 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.902865 (+   166us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:24.927542 (+ 24677us) log.cc:844] Serialized 14 byte log entry
0709 12:23:24.929080 (+  1538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.929192 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.014142 (+ 84950us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b180045bfe0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:25.553344 (+539202us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:25.557823 (+  4479us) raft_consensus.cc:1685] finished
0709 12:23:25.557848 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.557986 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":73916,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":2535,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":23678,"spinlock_wait_cycles":100173824,"thread_start_us":388288,"threads_started":2,"wal-append.queue_time_us":400334}
I20260709 12:23:25.560781 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.456413 25538 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:25.564922 25853 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:23:25.565200 25848 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 12:23:25.566215 25538 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:25.566807 25538 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:25.567175 25834 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 12:23:25.568518 20805 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 12:23:25.569221 25851 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 12:23:25.569885 25671 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.573159 21435 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:23:25.514137 25681 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:25.648172 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.659569 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.662829 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.661386 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.672125 20880 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:25.519958 25691 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.673316 20878 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:25.676461 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.677894 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.678529 20882 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:25.577240 25538 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.682418 25538 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:25.683969 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.685339 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.686239 20880 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:25.686569 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:25.688483 20878 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:23:25.690932 25819 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.692670 21435 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:23:25.692703 20774 leader_election.cc:400] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
I20260709 12:23:25.695978 20774 leader_election.cc:403] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:23:25.700196 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:25.705078 20878 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: [13.11-16.12]
W20260709 12:23:25.693660 25856 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:23:25.712996 20884 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:25.661229 25781 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:25.720767 20884 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:25.725823 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:25.734493 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:25.735934 20880 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:25.578955 25699 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.708181 25680 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
I20260709 12:23:25.691594 25819 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.744647 20882 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:25.674180 25603 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:23:25.674283 25689 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:23:25.748111 20884 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:25.750288 20882 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:25.751171 20880 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:25.709067 25860 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:25.752591 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 404516us
W20260709 12:23:25.755544 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.759378 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.760303 20807 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:25.740164 25595 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:25.766652 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.767867 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.770212 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.770768 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.771875 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.772862 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.727514 25781 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.780045 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.797041 20806 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:25.780905 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.801052 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.801815 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.803412 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2360) took 1604 ms. Trace:
I20260709 12:23:25.803871 20877 rpcz_store.cc:276] 0709 12:23:24.199282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.394198 (+194916us) service_pool.cc:224] Handling call
0709 12:23:24.395266 (+  1068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.395831 (+   565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:24.395863 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.396697 (+   834us) log.cc:844] Serialized 26 byte log entry
0709 12:23:25.259801 (+863104us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b18003bad00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:25.354044 (+ 94243us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:25.354112 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.355225 (+  1113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.781995 (+426770us) raft_consensus.cc:1685] finished
0709 12:23:25.782026 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.803341 (+ 21315us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":163,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":4941,"spinlock_wait_cycles":3582848,"thread_start_us":93952,"threads_started":1,"wal-append.queue_time_us":1034500}
I20260709 12:23:25.804056 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.809774 20803 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:25.812450 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.814906 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.816128 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.832013 20804 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:25.799559 25823 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:25.838203 20805 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:23:25.780341 25595 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:25.841001 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 307476us
I20260709 12:23:25.838224 25686 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:25.846403 20809 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9.
I20260709 12:23:25.848376 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:25.849494 20729 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:25.866303 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.837589 25823 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:25.869673 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:25.839758 25571 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.870987 20727 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:23:25.845526 25794 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.018s	sys 0.004s
I20260709 12:23:25.869072 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.874955 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.878695 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.893186 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:25.893399 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.898067 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:25.883478 25794 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.018s	sys 0.004s
I20260709 12:23:25.902172 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.908126 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:25.908797 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.911628 20877 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:25.903932 25819 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.924916 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:25.925647 21427 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:25.889103 25691 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.867735 25686 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.935864 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.938632 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.923421 25701 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.945079 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2522) took 1401 ms. Trace:
I20260709 12:23:25.926035 25704 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.945982 20802 rpcz_store.cc:276] 0709 12:23:24.543327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.543490 (+   163us) service_pool.cc:224] Handling call
0709 12:23:25.944820 (+1401330us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54002b61d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:25.945001 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91220352}
I20260709 12:23:25.948609 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.884985 25571 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.924482 25595 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:25.950311 25819 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:25.955808 25819 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.957574 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.958277 25819 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.958879 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.965041 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:25.966378 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:25.970134 20700 leader_election.cc:336] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.643s (SENT)
I20260709 12:23:25.973767 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.977480 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.994285 25736 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:25.996462 25571 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:25.996621 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2547) took 1442 ms. Trace:
I20260709 12:23:26.014436 20734 rpcz_store.cc:276] 0709 12:23:24.554313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.554994 (+   681us) service_pool.cc:224] Handling call
0709 12:23:25.996402 (+1441408us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54003b4a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:25.996554 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557503360}
I20260709 12:23:25.975323 25708 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.952405 25704 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.952966 25701 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.004616 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.009490 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 772) took 1017 ms. Trace:
I20260709 12:23:26.013224 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:25.934247 25823 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:25.978683 25583 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.018447 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:26.027663 20730 rpcz_store.cc:276] 0709 12:23:24.991910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.992307 (+   397us) service_pool.cc:224] Handling call
0709 12:23:24.992631 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.328398 (+335767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.009076 (+680678us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400226fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:26.009394 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770658688}
I20260709 12:23:26.027554 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:26.029902 20808 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:26.031531 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:26.031906 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:26.032500 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:26.034924 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.035679 20878 raft_consensus.cc:2368] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14.
I20260709 12:23:26.035640 20805 raft_consensus.cc:2376] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 9. Re-sending same reply.
I20260709 12:23:26.038698 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.038702 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:25.981108 25874 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:26.039741 20808 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:26.042414 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.043730 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.044790 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.045272 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.023720 25781 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.051210 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.068660 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.069509 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:26.070382 20730 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:26.071571 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:26.037729 25583 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.076365 25701 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:26.077265 25736 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:26.078171 25583 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.044430 25691 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:26.079500 25668 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:26.079664 25691 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.080472 25668 consensus_queue.cc:1243] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:26.081787 25686 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.082731 25691 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.089381 20730 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:26.064745 25557 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:23:26.039536 25689 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:26.098733 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:26.082552 25879 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:23:26.102977 25843 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:26.078804 25681 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.004s
I20260709 12:23:26.103515 25843 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.104543 25843 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.105446 25704 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.106189 25843 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.051390 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.042749 25833 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:23:26.051482 25873 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:26.107594 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:26.108132 25873 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.109274 20801 raft_consensus.cc:1217] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[8.8-13.10]   Dedup: 8.8->[11.9-13.10]
I20260709 12:23:26.109470 25557 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:26.109937 20801 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: [8.8-13.10]
I20260709 12:23:26.109987 25557 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.110615 25873 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.112354 20810 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:23:26.035271 25860 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:23:26.038765 25749 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:26.087862 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.053721 25874 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.095902 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.097755 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:26.095328 25714 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.790s	user 0.013s	sys 0.000s
I20260709 12:23:26.114964 25557 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.120688 25557 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:26.050962 25864 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:23:26.108145 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.125033 25749 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:26.126186 25749 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.127718 25833 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:26.127914 20808 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:26.128221 25833 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.129369 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.129170 25833 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.131421 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:26.083693 25876 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:26.035100 25783 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 12:23:26.100003 25871 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:23:26.100477 20728 raft_consensus.cc:2368] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
I20260709 12:23:26.029060 25708 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.036002 25603 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:23:26.118237 25689 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:26.132223 25833 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.133958 25874 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:26.135175 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.135782 25860 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:26.137130 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:26.137457 25866 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 12:23:26.204463 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:26.112432 25836 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.002s	sys 0.031s
I20260709 12:23:26.139806 25404 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:26.082544 25686 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:26.143258 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 866) took 1243 ms. Trace:
I20260709 12:23:26.116128 25823 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:26.149731 25801 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:26.229239 25838 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.156147 25770 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:26.231424 25770 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:26.232903 25770 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.233753 20809 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:26.234460 20809 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.161146 25536 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.160816 25636 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:26.078881 25841 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.001s
I20260709 12:23:26.165580 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:26.167069 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:26.168219 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.176831 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.189499 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.197899 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:26.203006 25860 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.209573 21433 rpcz_store.cc:276] 0709 12:23:24.899726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.900348 (+   622us) service_pool.cc:224] Handling call
0709 12:23:24.949016 (+ 48668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.230856 (+281840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.142961 (+912105us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54005f3a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:26.143169 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591828352}
I20260709 12:23:26.212311 25404 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:26.177714 25876 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.255484 20875 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:26.217432 25708 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:26.207983 25836 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.002s	sys 0.031s
I20260709 12:23:26.235944 25689 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.243908 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.248472 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:26.254396 21431 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:26.234588 25770 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:26.261161 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 233189us
I20260709 12:23:26.264796 20879 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:26.265869 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.243942 25832 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:26.212101 25686 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.296119 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.296273 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.298173 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:26.298009 25791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.045s	user 0.006s	sys 0.023s
I20260709 12:23:26.303072 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.303916 21432 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:26.244046 25838 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:26.300385 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.302584 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:26.286844 25832 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.290017 25686 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:26.284646 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:26.314790 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:26.305415 25791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.006s	sys 0.024s
I20260709 12:23:26.257515 25841 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.330945 25876 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:26.334407 25861 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.495s	user 0.003s	sys 0.000s
I20260709 12:23:26.300527 25860 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.336308 25860 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:26.341272 25841 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.162070 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:26.141125 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 764) took 1678 ms. Trace:
I20260709 12:23:26.380862 20726 rpcz_store.cc:276] 0709 12:23:24.463182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.463369 (+   187us) service_pool.cc:224] Handling call
0709 12:23:24.463664 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.296341 (+832677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.140797 (+844456us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b54002490d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:26.141045 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904838912}
I20260709 12:23:26.160794 20730 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:26.384653 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 352007us
W20260709 12:23:26.138898 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88683.8us
I20260709 12:23:26.386489 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:26.170465 20733 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:26.405267 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:26.413139 21429 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:26.132870 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:26.387979 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:26.423007 20801 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:26.388129 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:26.170420 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.116268 20805 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:26.389398 20734 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:26.162432 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:26.396019 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:26.397547 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:26.400091 20803 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:26.240204 20809 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:26.403977 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.388121 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:26.386716 20730 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:26.315538 21434 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:26.398777 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:26.467231 20878 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:23:26.516223 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 128670us
I20260709 12:23:26.526324 25791 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:26.527314 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 129007us
I20260709 12:23:26.527588 25791 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.528532 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:26.529445 25636 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:26.535897 25887 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:26.537453 25887 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.539785 25603 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:26.540581 25603 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:26.549906 25832 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:26.551745 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.552742 25832 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:26.542382 25603 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:26.531916 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 145782us
W20260709 12:23:26.563654 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:26.566995 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 264802us
I20260709 12:23:26.570690 20804 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:26.570577 25873 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.571439 25708 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:26.572000 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 175000us
I20260709 12:23:26.572243 25873 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.573395 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:26.575479 25881 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:26.575922 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:26.577313 25571 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:26.579710 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:26.580866 25873 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:26.582286 25873 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.589617 25843 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:26.590220 25843 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:26.590725 25843 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.595110 20883 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:26.598424 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:26.598946 20883 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:23:26.594139 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:26.602794 25876 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:26.616097 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 431516us
I20260709 12:23:26.619062 25891 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:26.622321 25666 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:26.622781 25536 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:26.623593 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.623564 25536 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.626595 25536 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.627318 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:26.628140 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1236) took 2152 ms. Trace:
I20260709 12:23:26.622815 25666 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.629014 25841 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:26.628621 21429 rpcz_store.cc:276] 0709 12:23:24.475697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.476778 (+  1081us) service_pool.cc:224] Handling call
0709 12:23:24.477072 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.413895 (+1936823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.627781 (+213886us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54005de154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:26.628041 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4450540928}
I20260709 12:23:26.610661 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:26.556705 20809 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:26.630231 25666 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.666581 25666 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.671438 25765 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:26.672501 25765 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:26.676149 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.003s (SENT) [suppressed 23 similar messages]
I20260709 12:23:26.692313 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:26.701713 25806 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.702802 25595 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:26.703820 25595 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.702899 25806 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.708846 25897 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
W20260709 12:23:26.742651 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66572.5us
I20260709 12:23:26.707165 25595 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.749567 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:26.720069 25791 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.761088 20884 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
W20260709 12:23:26.766115 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:26.767539 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 157008us
I20260709 12:23:26.770740 24963 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:26.771252 24963 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.771593 24963 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:26.774576 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:26.776094 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 83821.3us
I20260709 12:23:26.777405 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:26.777679 25801 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.010s
I20260709 12:23:26.779493 25536 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:26.780388 25536 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:26.781127 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:26.781431 25671 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:26.782519 25536 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:26.782964 25536 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.783412 25536 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:26.783406 25836 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:26.809744 25583 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:26.822695 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:26.827312 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 117305us
I20260709 12:23:26.783646 25689 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:26.834362 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 175165us
I20260709 12:23:26.833141 25538 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.837630 25583 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.842357 25538 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.824707 25671 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.866784 20879 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:23:26.869028 20879 pending_rounds.cc:85] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 10
W20260709 12:23:26.875583 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66909.8us
I20260709 12:23:26.874586 20879 pending_rounds.cc:107] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:23:26.884471 20879 raft_consensus.cc:2889] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:26.884423 20774 leader_election.cc:400] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14.
I20260709 12:23:26.889421 20774 leader_election.cc:403] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:26.870667 25404 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.896286 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:26.898387 25736 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:26.902375 25680 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:26.903331 25781 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:23:26.904719 25781 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:26.905113 25781 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:26.906211 25781 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.878551 25671 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:26.911525 25794 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:26.914565 25404 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:26.915520 20878 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:26.915896 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:26.916678 25689 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:26.920126 25895 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.241s	user 0.003s	sys 0.001s
I20260709 12:23:26.924171 25806 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.924953 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:26.927484 25791 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:26.927961 25791 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.928330 25791 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.933718 25843 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:26.936465 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80476.7us
W20260709 12:23:26.937178 25860 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.576s	user 0.016s	sys 0.002s
I20260709 12:23:26.938051 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:26.939121 25636 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:23:26.910908 25671 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.943485 25836 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:26.944623 25680 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:26.944744 25873 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:26.883400 25765 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.943526 25322 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:23:26.946345 25542 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:26.947695 25542 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:26.925187 25889 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:23:26.950390 25322 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:26.950801 25322 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:26.951350 25583 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:26.952567 25542 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.954332 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:26.951948 25322 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:26.955377 20803 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:23:26.902635 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.956768 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:26.968628 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.971463 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:26.971877 20875 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:26.972174 20884 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:26.973973 20876 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:26.970826 25538 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:26.975452 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:26.962440 25770 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.690s	user 0.003s	sys 0.010s
I20260709 12:23:26.962705 25322 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.945143 25836 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:23:26.958060 25404 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
W20260709 12:23:26.976611 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 87967.5us
I20260709 12:23:26.971511 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:26.899178 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.996625 25876 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:26.997673 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:26.998484 25888 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.331s	user 0.002s	sys 0.000s
I20260709 12:23:26.999611 20884 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:23:26.999864 25908 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:23:26.999672 25694 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:27.000906 25910 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:27.001251 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:27.001479 25887 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:27.002521 25892 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:23:27.004143 20631 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:27.004184 25536 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:27.007532 25833 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.808s	user 0.013s	sys 0.005s
I20260709 12:23:27.007804 25404 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:23:27.011274 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 319184us
I20260709 12:23:27.019677 25906 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:27.019273 24963 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.021292 20735 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:27.021517 25583 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:27.029556 25694 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:27.023587 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 198) took 27827 ms (client timeout 15000 ms). Trace:
I20260709 12:23:27.024179 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.026765 25904 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:27.027550 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
W20260709 12:23:27.027844 25838 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.008s
I20260709 12:23:27.028679 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.098354 20884 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:27.029086 25891 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:27.026979 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:26.994234 25689 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:27.057605 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56447.7us
W20260709 12:23:27.021286 25686 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.007s
I20260709 12:23:26.899111 25736 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.022966 20733 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:27.102883 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.063493 25887 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:27.101125 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.097321 25832 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:27.056296 25860 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:27.124109 20728 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.097913 25841 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:27.094201 25910 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:27.126039 20884 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:23:27.105758 25404 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:27.142014 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 836) took 1015 ms. Trace:
I20260709 12:23:27.105842 25680 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.128876 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.124717 20631 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.110040 25887 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.055943 25666 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:23:27.150269 20735 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:27.152570 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45628.4us
I20260709 12:23:27.100004 25583 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.153676 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:27.142653 20809 rpcz_store.cc:276] 0709 12:23:26.126815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.134585 (+  7770us) service_pool.cc:224] Handling call
0709 12:23:26.134889 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.233598 (+ 98709us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b5400186620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca11c0 00007f9e1eca2231 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c
0709 12:23:27.111034 (+877436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.117112 (+  6078us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:27.117183 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:27.117247 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.117376 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.141928 (+ 24552us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9751,"raft.queue_time_us":16482,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":4978,"spinlock_wait_cycles":154213248}
I20260709 12:23:27.162488 20879 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:27.169844 20726 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:27.174247 20733 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.148706 25887 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:27.183135 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 211749us
I20260709 12:23:27.121913 25686 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.206756 20728 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.193655 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.175820 25843 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.228453 20727 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:27.234493 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.235532 20880 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:27.230014 20803 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:27.240025 20732 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:27.248802 21431 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:27.150861 25680 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.255718 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:27.091652 20628 rpcz_store.cc:269] 0709 12:22:59.175495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.175679 (+   184us) service_pool.cc:224] Handling call
0709 12:23:27.003037 (+27827358us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:18.957403 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:18.960870 (+  3467us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:18.960946 (+    76us) write_op.cc:435] Acquired schema lock
0709 12:23:18.960979 (+    33us) tablet.cc:662] Decoding operations
0709 12:23:18.976724 (+ 15745us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:19.012986 (+ 36262us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:19.015538 (+  2552us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:23:19.226824 (+211286us) tablet.cc:701] Row locks acquired
0709 12:23:19.226878 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:23:19.227087 (+   209us) write_op.cc:270] Start()
0709 12:23:19.227301 (+   214us) write_op.cc:276] Timestamp: P: 1783599799227010 usec, L: 0
0709 12:23:19.227357 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 12:23:19.260180 (+ 32823us) log.cc:844] Serialized 7040 byte log entry
0709 12:23:19.941402 (+681222us) op_driver.cc:464] REPLICATION: finished
0709 12:23:20.330021 (+388619us) write_op.cc:301] APPLY: starting
0709 12:23:20.330147 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:22.393809 (+2063662us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:22.393931 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:23.338541 (+944610us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:23.339006 (+   465us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:23:23.339068 (+    62us) write_op.cc:312] APPLY: finished
0709 12:23:23.339533 (+   465us) log.cc:844] Serialized 164 byte log entry
0709 12:23:23.551937 (+212404us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:23.552132 (+   195us) write_op.cc:454] Released schema lock
0709 12:23:23.552207 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:23.553277 (+  1070us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:23.553315 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":505541,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":353443,"apply.run_cpu_time_us":111642,"apply.run_wall_time_us":3240305,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79367,"num_ops":18,"prepare.queue_time_us":402215,"prepare.run_cpu_time_us":51521,"prepare.run_wall_time_us":427812,"raft.queue_time_us":254110,"raft.run_cpu_time_us":16050,"raft.run_wall_time_us":329017,"replication_time_us":704393,"thread_start_us":991085,"threads_started":5,"wal-append.queue_time_us":405140,"wal-append.run_cpu_time_us":19701,"wal-append.run_wall_time_us":2437539}]]}
I20260709 12:23:27.260550 25714 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:27.261502 20731 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:27.261943 25696 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:23:27.262316 25783 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:27.262475 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:27.267517 20631 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 5 to 11, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 11 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:27.174367 25904 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.274065 25583 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.276523 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:27.322427 20805 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:27.276638 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:27.278528 20806 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:27.280220 24963 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.284066 25536 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.283360 25910 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.285929 25736 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.193240 25841 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.250339 25865 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.188s
I20260709 12:23:27.194202 25860 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.292706 25686 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.229400 25692 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.202s
I20260709 12:23:27.295154 20882 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:27.295856 25843 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:27.297113 25832 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:27.299238 25781 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:27.257679 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 870) took 1079 ms. Trace:
I20260709 12:23:27.145908 25906 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.311687 20735 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:23:27.317349 20728 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:27.305308 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:27.318602 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:27.321081 25666 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:27.174301 25765 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
W20260709 12:23:27.324322 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:27.376405 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.233032 25749 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:27.329443 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:27.380393 20876 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:27.327692 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.332525 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:27.336400 20733 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
W20260709 12:23:27.318835 25708 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.747s	user 0.013s	sys 0.003s
I20260709 12:23:27.374569 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:27.386650 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.382576 21435 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:27.384356 20729 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.339380 25583 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:27.371887 25832 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:27.369900 21428 rpcz_store.cc:276] 0709 12:23:26.177686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.177872 (+   186us) service_pool.cc:224] Handling call
0709 12:23:26.178548 (+   676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:26.190949 (+ 12401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.190998 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:26.201039 (+ 10041us) log.cc:844] Serialized 26 byte log entry
0709 12:23:26.506517 (+305478us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:26.515768 (+  9251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.522251 (+  6483us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:27.255633 (+733382us) raft_consensus.cc:1685] finished
0709 12:23:27.255656 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.257445 (+  1789us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7223,"raft.queue_time_us":18572,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":2584,"thread_start_us":290446,"threads_started":1,"wal-append.queue_time_us":543109}
I20260709 12:23:27.392928 20726 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:27.385540 25838 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:27.391374 25909 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
W20260709 12:23:27.405674 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144538us
I20260709 12:23:27.416524 20804 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:27.387089 25708 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:27.391610 25833 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:27.423380 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 115117us
I20260709 12:23:27.436326 21434 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:27.404444 25636 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14.
I20260709 12:23:27.387333 25666 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.380049 25749 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.431041 20631 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 10 to 12, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 12 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.366998 25404 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.448423 20878 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:27.449891 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:27.439131 25823 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.066s	user 0.006s	sys 0.012s
I20260709 12:23:27.411157 25836 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:27.453001 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 251830us
I20260709 12:23:27.419440 25686 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:27.472533 25686 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.475652 25904 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.407485 25583 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.479609 25749 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.480000 25832 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.481175 25832 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:27.480494 20631 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 9 to 15, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 15 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:27.452009 25689 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.482952 25689 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.483572 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.485121 25666 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.484738 25694 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.485330 25689 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.485775 25666 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:27.475538 25913 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.162s	user 0.000s	sys 0.002s
I20260709 12:23:27.436332 25833 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.488569 25841 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.488902 25833 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.489905 25404 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.483523 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.490386 25765 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:27.490861 25765 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:27.499591 20735 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:27.499948 25765 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.501454 25765 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:27.501744 25906 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.462492 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:27.503062 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.504690 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:27.468051 25917 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:23:27.471949 25915 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:23:27.475538 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.485915 25694 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.460001 25770 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:27.461421 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:27.489107 20631 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 7 to 10, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 10 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:27.490882 20727 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:27.492007 25921 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:27.519590 20879 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.492641 25836 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.494298 25680 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.498227 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.498538 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:27.395116 25783 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:27.462747 25916 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:23:27.458606 20803 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:27.509153 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:27.483456 25865 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.510075 25583 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.537665 21427 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.514485 20804 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:27.485229 20775 leader_election.cc:400] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
W20260709 12:23:27.508332 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63151.8us
W20260709 12:23:27.517652 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.538743 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.517737 20801 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:27.517009 20876 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
W20260709 12:23:27.471949 25919 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:23:27.478891 25887 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.474988 25686 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.529757 20728 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:27.515081 25923 raft_consensus.cc:2764] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:23:27.534933 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.529206 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:27.532538 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:27.541873 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:27.541899 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:27.543781 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58734.9us
I20260709 12:23:27.545645 20880 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:27.554430 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:27.557214 20735 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:27.565816 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:27.567098 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:27.553247 25881 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.977s	user 0.007s	sys 0.008s
I20260709 12:23:27.573484 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:27.572940 25680 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:27.544562 25736 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.575542 20730 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:23:27.576236 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49217us
I20260709 12:23:27.570314 25708 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:27.580087 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:27.583107 25708 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:27.587494 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:27.588249 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.583268 25542 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:27.596699 25542 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:27.597074 25887 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.597002 25680 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:27.598980 25925 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:23:27.595252 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.601649 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:27.597229 25542 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:27.578984 25881 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:27.602520 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:27.585723 25791 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.003s
I20260709 12:23:27.545012 25865 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.601773 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:27.603667 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:27.618305 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.585019 25595 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:27.615803 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 111277us
I20260709 12:23:27.570142 25801 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:27.589633 25708 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.618939 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:27.616338 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 168836us
I20260709 12:23:27.622251 20810 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:27.603127 25931 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:27.620093 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:27.620215 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:27.612414 21400 connection.cc:256] RPC call timeout handler was delayed by -1.0097112130001733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260709 12:23:27.624212 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.624506 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.633330 20804 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:27.565267 25921 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.617971 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:27.584599 25823 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:23:27.585443 25736 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.584666 25923 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:23:27.640281 20809 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:27.622884 25595 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.649611 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.638252 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.648638 25680 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.649350 25933 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:27.648432 25699 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:23:27.642074 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.649111 20881 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:27.649551 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:27.649874 25929 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.649158 25924 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:27.650535 25860 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.651588 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:27.666817 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82280.9us
I20260709 12:23:27.632735 25931 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.657420 20880 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:27.662375 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:27.681365 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45670.4us
W20260709 12:23:27.674996 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.618534 25791 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:27.671926 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56934.6us
W20260709 12:23:27.685359 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101446us
I20260709 12:23:27.661051 25595 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:27.700114 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.687443 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:27.666603 20631 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 3 to 8, term changed from 5 to 12, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:27.703030 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51562.3us
I20260709 12:23:27.659929 25881 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:27.700829 21433 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:27.709053 21428 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:27.711696 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.706892 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.716837 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.717425 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.718592 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:27.719087 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.720538 25909 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:23:27.723994 25928 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:23:27.701249 25686 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:23:27.697250 25910 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:27.727104 25910 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:27.727464 25836 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.727941 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:27.728312 25910 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.728989 25583 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:27.732398 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.732869 25926 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:27.733429 25926 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:27.735296 25736 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:27.736526 25921 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.732313 20631 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 7 to 11, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 11 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.737910 20631 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.734532 25926 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.739575 21430 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 12:23:27.744625 20631 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 9 to 12, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 12 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.748680 25865 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.746426 21427 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:27.750320 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.751317 25836 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:27.752556 25836 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:27.753657 25836 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:27.723660 25927 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:27.753549 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:27.692646 25931 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:27.756865 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:27.704366 25671 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:27.750655 20631 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 3 to 9, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 9 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:27.763418 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 143276us
I20260709 12:23:27.766595 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.767263 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.760967 20881 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:27.754731 25680 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:27.790905 20735 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:23:27.791141 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:27.797413 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 110258us
I20260709 12:23:27.797482 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.799104 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.756099 25714 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:27.679262 25929 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.712615 25860 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.802317 20884 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [13.7-15.8]
I20260709 12:23:27.796919 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.791575 20735 pending_rounds.cc:85] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 7
I20260709 12:23:27.762784 25636 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:27.819094 20735 pending_rounds.cc:107] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 13.8
W20260709 12:23:27.819149 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62349.2us
I20260709 12:23:27.819515 20735 raft_consensus.cc:2889] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:27.814013 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94033.6us
I20260709 12:23:27.816378 21434 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:27.799990 20883 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:27.816838 21427 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:27.822492 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:27.801363 25322 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.787055 25680 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:27.790591 25801 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:27.826284 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:27.796877 25783 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:27.830893 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.830749 21432 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
W20260709 12:23:27.833598 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83430us
I20260709 12:23:27.824187 25860 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.835024 25791 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.836443 25860 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:27.834496 25935 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:23:27.845595 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.846405 20876 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
W20260709 12:23:27.846730 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56681.7us
I20260709 12:23:27.847733 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:27.849097 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:27.849934 25686 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:27.850844 25938 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:27.851486 25938 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.854574 25938 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.822252 25636 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:27.824239 25671 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:27.855820 25680 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.856117 25636 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.857182 25680 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:27.856899 25671 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.857509 25636 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:27.829988 25914 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.401s	user 0.003s	sys 0.001s
I20260709 12:23:27.817492 25833 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:27.861325 20881 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:27.860767 20631 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.862489 20877 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:27.863500 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66139.5us
I20260709 12:23:27.824121 25921 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:27.863253 25736 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:23:27.866957 25942 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:23:27.832381 25692 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.865720 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:27.909415 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 43773.3us
I20260709 12:23:27.829928 25699 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:27.817211 25841 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:23:27.868594 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.869040 25860 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:27.871309 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:27.914870 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.872769 25929 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:27.873303 25932 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:27.860055 25322 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.879559 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:27.891853 25921 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.880766 25806 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:27.882387 25542 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.883009 25934 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:27.884163 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:27.893110 20879 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:27.897991 20881 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:23:27.923439 25806 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:27.927409 20879 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:27.928009 25322 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.880133 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:27.929253 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49231.5us
W20260709 12:23:27.929548 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61054.7us
I20260709 12:23:27.914018 25696 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:27.922560 25931 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.898986 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:27.919569 20631 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 9 to 12, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 12 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:27.907899 25692 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:27.921037 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:27.873469 25708 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:27.940747 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.925863 25801 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.926093 25921 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:27.950378 20631 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.925387 25934 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:27.960033 20882 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:27.960378 21426 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:27.960604 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 76532.5us
W20260709 12:23:27.961760 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.956910 20631 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.937418 25636 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:27.944149 25404 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:27.943780 25942 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:27.962886 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:27.937537 25603 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:27.965737 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.944370 25696 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:27.975979 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109360us
I20260709 12:23:27.972924 20631 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 9 to 14, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 14 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:27.958055 25583 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:27.978996 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:27.980279 20805 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:27.981653 25696 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:27.987753 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:27.996982 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:27.997941 25692 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:27.972114 25942 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:27.999716 25942 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:27.999954 25918 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.679s	user 0.002s	sys 0.002s
I20260709 12:23:28.002897 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 867) took 1008 ms. Trace:
W20260709 12:23:28.003823 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52239.5us
I20260709 12:23:28.004047 25542 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.004334 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:28.004647 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.004863 25636 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:28.005304 25636 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.005679 25636 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:28.005625 25934 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.006194 25833 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:28.007237 25934 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:28.007378 25783 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:28.007908 25783 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.008265 25783 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.009393 25714 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:28.011075 25932 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:28.011677 25932 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:28.011569 25923 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:28.012739 25932 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.013705 25823 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:28.015484 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:28.015971 20735 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:28.016687 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.016948 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:27.967130 25404 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:28.006309 25765 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:27.978181 25671 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.049656 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1255) took 1095 ms. Trace:
I20260709 12:23:28.050045 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.027531 20631 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 4 to 8, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 8 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:28.033277 25931 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:28.035449 25708 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:28.035841 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.052940 20631 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:28.039884 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:28.040957 25890 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:27.985123 25322 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:23:28.041465 25694 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:27.976753 25806 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.043684 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:28.046200 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:28.003314 20875 rpcz_store.cc:276] 0709 12:23:26.994826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.995031 (+   205us) service_pool.cc:224] Handling call
0709 12:23:27.016601 (+ 21570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.017171 (+   570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.017205 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:27.017828 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.110745 (+ 92917us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:27.110894 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.111123 (+   229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:27.999389 (+888266us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:28.002572 (+  3183us) raft_consensus.cc:1685] finished
0709 12:23:28.002593 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.002828 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9405,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":102,"spinlock_wait_cycles":5444864,"thread_start_us":84747,"threads_started":1,"wal-append.queue_time_us":157387}
I20260709 12:23:28.072605 20727 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:28.055735 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:28.073360 20807 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:28.044654 25823 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:28.076464 25671 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:28.077361 25823 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.047370 25765 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:28.072875 25910 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 12:23:28.079676 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43923us
I20260709 12:23:28.079521 25765 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.080719 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.081389 25765 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.061722 25890 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:28.056593 25680 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.083858 25938 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:28.083751 25890 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.057229 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:28.084638 25938 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:28.085107 25890 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:28.036541 20734 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:28.050153 21430 rpcz_store.cc:276] 0709 12:23:26.953963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.954105 (+   142us) service_pool.cc:224] Handling call
0709 12:23:26.954468 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.740132 (+785664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.044134 (+304002us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b540042ed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:28.049592 (+  5458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739088768}
I20260709 12:23:28.048197 25692 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:28.063864 20883 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:28.047647 25404 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:28.055085 25801 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.071621 25945 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:28.057651 25921 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:28.090581 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.117430 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:28.050613 25542 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:23:28.078758 25322 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:23:28.100636 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.099164 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:28.068612 25694 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:28.121992 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 97529.3us
I20260709 12:23:28.123409 21435 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:28.124358 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:28.132066 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:28.135075 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.063987 25832 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:23:28.136360 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2329) took 1034 ms. Trace:
I20260709 12:23:28.092460 25708 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.099225 25736 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:28.145612 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 105824us
I20260709 12:23:28.137928 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:28.139021 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:28.136791 20878 rpcz_store.cc:276] 0709 12:23:27.102302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.256185 (+153883us) service_pool.cc:224] Handling call
0709 12:23:27.316575 (+ 60390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.448892 (+132317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.124590 (+675698us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001fed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:28.136292 (+ 11702us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284214016}
I20260709 12:23:28.148535 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:28.113035 25692 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:28.152325 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:28.155092 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:28.120999 25806 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:28.131251 25781 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:28.164613 20803 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:23:28.143640 25933 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:28.195817 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:28.197482 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134499us
I20260709 12:23:28.202598 20883 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:28.144402 25832 consensus_queue.cc:1243] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:28.140336 25945 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:28.206851 25946 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:23:28.152835 25931 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:28.166013 25781 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:28.211097 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:28.219710 20809 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:23:28.209105 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:28.200188 25933 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.151872 25783 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.153584 25736 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.232163 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:28.245364 20805 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:28.246495 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 109115us
I20260709 12:23:28.221916 25781 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:28.232885 25933 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.239876 25932 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:28.225694 25841 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:23:28.307888 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:28.248070 25931 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:28.321373 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 122430us
I20260709 12:23:28.319487 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:28.336467 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 98686.8us
I20260709 12:23:28.300572 25932 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:28.314702 25924 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.265012 25694 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.347043 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:28.355073 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:28.356179 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.358355 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:28.349576 20727 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:28.325534 25692 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.335376 25945 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:28.360677 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:28.370453 20802 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:28.366969 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:28.360636 25680 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.371881 25680 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.377375 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:28.380869 25926 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:28.386929 25736 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.389045 25961 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:28.389555 25961 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:28.390416 25961 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.391337 25832 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.391844 25961 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:28.391948 25832 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.395427 25671 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:28.395888 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 276227us
I20260709 12:23:28.395954 25671 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.396770 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:28.398412 25671 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.398583 25801 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:28.399710 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:28.400605 20727 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:28.365535 25924 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:28.403745 25955 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:23:28.404464 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:28.406100 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 253852us
I20260709 12:23:28.408360 21427 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:23:28.409471 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:28.411019 25941 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:28.363309 25603 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:28.411513 25941 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:28.412038 25941 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:28.415628 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.393047 25891 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:28.411664 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.346791 25833 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:28.359242 25951 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:23:28.420037 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:28.352710 25921 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.416523 25881 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:28.426043 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.426913 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.435503 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:28.416811 25909 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:28.418174 25801 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:28.423864 25770 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.435217 20884 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:28.439847 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.440157 25801 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.440707 20729 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:28.433240 25833 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.444989 25833 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:28.445513 25963 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.446128 25963 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.421034 25927 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:28.447371 25927 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:28.447781 25927 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.448089 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:28.439458 25909 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:28.449239 25909 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:28.452077 25956 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:23:28.452348 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102016us
I20260709 12:23:28.453320 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.454802 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.455382 25603 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.455478 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:28.463306 25954 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:23:28.395041 25583 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.438551 25891 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.380940 25699 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.448886 20882 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:28.437455 20807 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:28.465147 25931 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:28.480734 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.476783 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:28.485510 25887 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.485952 25542 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.486361 25921 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.521677 25921 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.523325 25836 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:28.523769 25836 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.524116 25836 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:28.488204 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:28.488590 25962 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:23:28.490196 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:28.491842 25957 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 12:23:28.492796 25967 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:23:28.399052 25671 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:23:28.483703 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:28.533648 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50040.7us
I20260709 12:23:28.448482 25963 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.511436 20883 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:23:28.511907 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:28.512148 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.512670 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:28.538865 21435 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:23:28.513291 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:28.513749 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:28.513796 25595 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.804s	user 0.009s	sys 0.003s
I20260709 12:23:28.514550 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:28.514925 25942 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.001s
I20260709 12:23:28.519534 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:28.410758 25404 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.529740 20810 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:28.534868 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.510974 25806 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:28.535126 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.535812 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:28.541785 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.542610 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.524405 25921 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.577024 20729 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
W20260709 12:23:28.556146 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.544378 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:28.474697 25881 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:28.581007 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45295.9us
W20260709 12:23:28.565992 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 151726us
I20260709 12:23:28.588443 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:28.589536 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:28.598505 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48482us
W20260709 12:23:28.612434 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 157709us
W20260709 12:23:28.549315 25636 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.001s
I20260709 12:23:28.617892 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:28.522804 25932 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.527384 25542 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.536381 25890 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:23:28.601503 25897 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:28.623466 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.627004 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.550186 25583 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:28.632277 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 97470.9us
I20260709 12:23:28.633894 20729 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:28.646102 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.577389 25595 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.584059 25976 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:28.662370 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.580977 25942 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.665488 25832 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:28.654771 25963 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.654107 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:28.622014 25636 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.655313 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.656409 25692 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.697005 25404 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.699079 25542 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:28.703619 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74262.3us
I20260709 12:23:28.704574 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.705178 25909 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.707154 25890 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:28.708879 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53664.6us
I20260709 12:23:28.710425 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.668980 25783 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:28.715929 25973 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:23:28.717540 25823 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:28.718019 25942 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.714174 25791 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.712658 21428 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:28.713317 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.713750 20876 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:28.680778 25975 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:28.743922 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.747431 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:28.753729 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:28.757170 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50551us
I20260709 12:23:28.710940 25671 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:28.712807 25404 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.764034 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.667157 25933 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.732736 25909 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:28.751338 25926 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.767496 25881 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.709961 25692 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.708108 25963 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.790074 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.797653 20729 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:28.794196 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:28.803819 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:28.805371 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:28.813388 25949 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.782s	user 0.003s	sys 0.000s
W20260709 12:23:28.815243 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.817761 25692 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:28.819751 25636 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.820293 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:28.825457 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:28.787890 25890 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.826680 20882 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:28.828841 20884 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:28.835421 20880 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:28.784255 25942 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:28.846165 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:28.848630 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:28.758574 25927 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.858538 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:28.866638 25781 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.000s
I20260709 12:23:28.849503 25907 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.867820 25907 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.868438 25933 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:28.871308 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.785553 25404 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.879954 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:28.880370 20810 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:28.843539 25841 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:28.880832 20801 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:28.843868 25636 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.881915 21432 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:28.882133 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:28.882880 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:28.888115 21430 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 10: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:28.875444 25865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.019s	sys 0.016s
I20260709 12:23:28.860096 25714 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:28.801709 25926 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.894979 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:28.890653 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 305254us
I20260709 12:23:28.903751 20876 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:28.908363 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.909345 20880 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:28.914310 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.867149 25781 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.916249 20734 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:23:28.917462 25781 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.918766 20881 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:28.884778 25909 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.920001 20727 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:28.894345 25791 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:28.879853 25583 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 12:23:28.899478 25865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.019s	sys 0.016s
I20260709 12:23:28.928242 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:28.933806 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 105828us
I20260709 12:23:28.940799 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:28.938920 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:28.943796 20880 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.874115 25942 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.947526 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.954908 20729 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:23:28.880018 25906 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:23:28.960775 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:28.894034 25636 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.975152 20729 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:28.979888 25692 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:28.980671 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.892266 25941 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.985263 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:28.987516 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.988489 20734 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:28.983785 25791 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:28.937536 25832 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:23:28.986773 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:28.940079 25836 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:28.994848 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:28.971736 25542 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:28.998179 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:28.974880 25699 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:28.998795 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:29.000279 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:28.982563 25860 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:29.009965 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.010988 25781 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:29.011379 25770 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:28.982385 25692 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.014297 25692 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.017032 25941 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.017583 25404 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:28.998262 25906 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:29.019253 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:29.019647 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.020211 21427 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:29.021369 25836 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.022007 25636 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.022436 25907 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.005121 20734 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:23:28.986588 25536 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:23:29.034040 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:29.004781 25699 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.038853 20876 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:29.037129 20880 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:29.044183 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:29.044870 21433 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:29.045354 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.048498 20875 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:29.050595 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1266) took 1281 ms (client timeout 1553 ms). Trace:
W20260709 12:23:29.051009 21428 rpcz_store.cc:269] 0709 12:23:27.768682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.770179 (+  1497us) service_pool.cc:224] Handling call
0709 12:23:29.050402 (+1280223us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:29.050517 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049281408}
I20260709 12:23:29.003485 25542 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:29.096077 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.092406 25932 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:29.091493 25781 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:29.100844 25923 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:29.184273 25923 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:29.185488 25923 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.187433 25907 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.188167 25907 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.190776 25907 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.113730 25542 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:28.990090 25791 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.052093 25536 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:29.083639 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:29.216482 20731 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:29.073083 25680 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.053215 25699 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.220254 25770 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:29.220783 25770 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:29.222124 25770 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.052695 25963 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:29.071623 25988 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:23:29.101806 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:29.224831 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:29.086856 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2398) took 1011 ms. Trace:
I20260709 12:23:29.073030 25938 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:29.225643 20883 rpcz_store.cc:276] 0709 12:23:28.075239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.097529 (+ 22290us) service_pool.cc:224] Handling call
0709 12:23:28.114664 (+ 17135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.537892 (+423228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.086604 (+548712us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:29.086792 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95272192}
I20260709 12:23:29.072212 25906 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:29.226675 25980 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.552s	user 0.004s	sys 0.000s
I20260709 12:23:29.100142 25891 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.226004 25770 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:29.109618 25942 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.007081 25860 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.230535 25860 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:29.108531 25961 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.000s
I20260709 12:23:29.227432 25906 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.232764 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.073803 25927 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:29.123796 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.239710 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:29.240783 20883 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 10: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:29.117242 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:29.138962 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.140494 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.142089 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.142760 20801 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:23:29.250769 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 110373us
I20260709 12:23:29.145803 25924 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.145893 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.163632 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.264906 20806 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:29.166735 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.167915 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:29.069705 20727 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:29.191421 25932 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.272152 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.192414 25781 consensus_queue.cc:1243] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:23:29.197996 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.112811 25595 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.199148 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.199431 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.200361 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.201522 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.201898 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.202394 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.187099 25923 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:29.208024 25694 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:29.208895 25832 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:29.211681 25865 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:29.226231 25938 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:29.102738 25603 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:29.142820 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:29.267349 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 150189us
I20260709 12:23:29.270401 21433 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:29.270740 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.272203 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.276042 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.265298 25887 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.306777 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:29.308941 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.279424 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:29.278301 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:29.281903 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 143061us
I20260709 12:23:29.278561 20878 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:29.287807 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:29.288192 25961 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.288614 25680 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:29.280678 25963 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.289772 20734 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:29.291329 20733 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:29.300640 20808 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:29.302381 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:29.275821 20729 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:23:29.343848 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71756.9us
I20260709 12:23:29.345702 25887 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.350522 25887 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.351727 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.351938 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.352671 20802 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
W20260709 12:23:29.305619 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 185127us
I20260709 12:23:29.303961 20803 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:29.305341 20806 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:29.358273 25887 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:29.272367 25942 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:29.275573 25536 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:29.321269 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.360198 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.360319 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.289335 25924 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:29.324517 20735 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:29.324987 20726 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:29.304162 25865 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:29.361359 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.363291 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
W20260709 12:23:29.361464 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.360594 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.305565 25860 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:29.365258 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44108.2us
I20260709 12:23:29.366535 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:29.324188 25833 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:29.324422 25692 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:29.377238 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74959.7us
W20260709 12:23:29.391371 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48120us
I20260709 12:23:29.391479 25636 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:29.394831 25636 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:29.396212 25636 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.398319 25636 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:29.326193 25686 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:29.319778 25699 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.325654 25934 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 12:23:29.423156 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 59749.3us
W20260709 12:23:29.422765 20700 connection.cc:256] RPC call timeout handler was delayed by -1.6186491140001635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
I20260709 12:23:29.420939 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:29.362021 25536 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:29.365240 25671 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:29.343303 25876 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:23:29.373373 25942 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:29.381143 25833 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.376989 25823 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:29.457405 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.374636 25836 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:29.458822 25836 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:29.444378 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.459903 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.383577 25926 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.460561 21435 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:29.405834 25891 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:29.347312 25963 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.404945 25692 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:29.347380 25961 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.460423 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:29.463243 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.485174 21431 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:29.464607 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.484287 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.470693 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.486375 21427 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:29.487692 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.488554 20806 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:23:29.425384 25907 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:29.469192 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.459363 25836 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.425549 25404 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:29.325698 25832 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.379401 25860 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.500679 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.511770 20801 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:29.494719 20809 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:29.486368 21433 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:29.454946 25833 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:29.436569 25977 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:29.472601 25957 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:29.469275 25542 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.448113 25536 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:29.513839 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 135780us
I20260709 12:23:29.493392 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.512036 20806 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:29.420318 25938 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.453851 25781 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:29.496913 25404 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.496392 25680 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.502640 25865 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.520598 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:29.496776 25836 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.524416 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.525285 20807 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:29.507915 25931 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:29.527843 25692 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.529553 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:29.565529 20726 raft_consensus.cc:606] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:29.565944 20726 raft_consensus.cc:614] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:29.531540 25932 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.554147 25860 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.535257 25945 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:29.536041 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.541847 25876 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:23:29.558861 25865 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:29.569685 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.524284 25781 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:29.571215 25781 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:29.543421 25942 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.537123 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.573980 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:29.552453 25963 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:29.557498 25832 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.575548 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.537858 25536 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.538766 26001 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:29.491657 25907 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:29.540755 20878 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:29.560781 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.553574 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.539158 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:29.590543 20876 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:29.557518 25542 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:29.590910 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51830.3us
I20260709 12:23:29.588855 25932 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:29.591710 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.593550 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.594777 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.557687 25404 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.587693 26009 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:29.567622 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.568009 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:29.607529 20806 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:29.544551 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.554879 25961 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:29.588327 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.594970 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:29.620110 20700 leader_election.cc:336] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.816s (SENT)
I20260709 12:23:29.556128 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.620621 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.556284 25699 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:29.539444 25933 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:29.603091 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:29.604194 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.562072 25696 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:29.559031 26020 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:29.609076 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64614.1us
I20260709 12:23:29.628048 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:29.628762 25945 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:29.628880 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:29.629160 25945 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:29.629565 25945 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:29.630400 25929 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:29.630477 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.630944 25929 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.631392 25929 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:29.633704 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:29.634434 20731 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:29.636821 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:29.601612 25963 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:29.523515 25926 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:29.610585 20806 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:29.526337 25806 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:23:29.577049 25791 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:29.658340 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:29.552615 25977 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.567319 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:29.621726 20732 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:29.591827 25932 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:29.637574 20735 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.609565 26001 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:29.640257 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:29.628063 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 254107us
I20260709 12:23:29.557829 25680 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.605983 25860 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:29.629664 25583 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:29.642503 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.659504 21429 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:29.624727 26022 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:29.754720 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 114567us
I20260709 12:23:29.769742 20806 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:29.646740 25961 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:29.775615 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 133236us
I20260709 12:23:29.645496 26016 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:29.839879 20882 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:29.838289 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 241993us
I20260709 12:23:29.652549 26021 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.841499 26021 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:29.799853 20879 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:29.801080 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:29.839829 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64819.5us
I20260709 12:23:29.868911 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:29.768173 25699 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:29.878834 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:29.756215 25932 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:29.749015 25603 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.870043 20878 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:29.940392 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:29.754104 25832 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.133s
W20260709 12:23:29.913867 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 293422us
I20260709 12:23:29.982935 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:29.983846 20879 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:29.648525 25933 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.767401 25897 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.768599 25860 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:29.754525 26029 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:29.762554 25876 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:29.647344 25696 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:29.808565 25906 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.032s	sys 0.025s
I20260709 12:23:29.767915 25963 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:29.839773 25736 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.913146 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:30.015678 25736 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:30.016036 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 124302us
I20260709 12:23:30.017786 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:30.019439 25890 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:30.020123 25890 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.020586 25890 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:29.916731 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:29.941169 20807 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:29.845141 25636 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.797972 26022 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:29.917591 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.025148 26022 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:29.846439 25973 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:23:30.025127 25636 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:29.844815 25967 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:23:29.916880 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.011s	sys 0.002s
W20260709 12:23:30.027402 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.012s	sys 0.003s
I20260709 12:23:30.027459 25942 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:30.028462 25942 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.917591 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:29.949435 20808 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:30.028934 25942 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:30.030512 20808 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:23:30.030980 20808 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 7
I20260709 12:23:30.031417 20808 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:23:30.031757 20808 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:30.033452 20808 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:23:30.033739 20808 pending_rounds.cc:85] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:23:30.033974 20808 pending_rounds.cc:107] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:23:30.034184 20808 raft_consensus.cc:2889] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:29.872049 25583 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:29.866964 25907 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.035763 25992 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.036027 25907 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:30.036541 25992 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.952088 25945 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:29.960000 26018 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:23:29.940326 25941 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:30.041329 25945 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:29.961956 21430 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:23:29.963739 25680 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.043076 20804 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:29.964162 20803 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:23:29.973268 25770 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.745s	user 0.009s	sys 0.000s
I20260709 12:23:30.044349 25583 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.045529 25770 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:30.046077 25770 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.046521 25770 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.941061 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:29.975025 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.051811 20734 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:29.870697 25542 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:29.993953 20881 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
W20260709 12:23:30.007963 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 378228us
I20260709 12:23:30.017992 25736 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:30.055408 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:30.031024 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 114436us
I20260709 12:23:29.748332 25823 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:29.980070 20735 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:30.072338 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:30.072825 20730 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:29.798162 25961 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.115010 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:30.116238 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60932.7us
W20260709 12:23:30.121685 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49480.6us
I20260709 12:23:29.918599 25791 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:29.954677 25929 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.127804 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:30.133966 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 539229us
I20260709 12:23:30.143877 20808 pending_rounds.cc:85] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 11
I20260709 12:23:30.144305 20808 pending_rounds.cc:107] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:23:30.144572 20808 raft_consensus.cc:2889] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:30.143728 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:30.056150 25906 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.036s	sys 0.028s
I20260709 12:23:30.160106 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:30.007438 25860 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:29.652833 26020 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:29.988284 25603 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.127569 25961 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:30.142350 25736 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:30.130796 25926 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:30.189090 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.011906 25897 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.170485 26029 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:30.174438 25699 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:30.126137 25921 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:23:30.115630 25876 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.207646 25926 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:30.234884 20728 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.057s	sys 0.109s
I20260709 12:23:30.191653 25671 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:30.230919 25692 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:30.242714 20728 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.058s	sys 0.109s
I20260709 12:23:30.158416 25791 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.206988 25736 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:30.192523 25536 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.228248 25699 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:30.240180 26029 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:30.242332 25926 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:30.243541 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 215130us
I20260709 12:23:30.287892 25736 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:30.318867 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 299874us
I20260709 12:23:30.289104 25699 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:30.325490 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:30.325835 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:30.329509 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:30.259387 25671 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.271591 25692 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.254819 25823 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.327888 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:30.341290 20882 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:23:30.347963 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.362282 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.350975 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.310637 25963 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:30.364675 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.365123 25963 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.365806 25963 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:30.328816 25694 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.023s	user 0.008s	sys 0.004s
I20260709 12:23:30.368255 25694 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:30.368666 25694 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:30.369097 25694 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:30.329283 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.000s	sys 0.008s
I20260709 12:23:30.308338 26024 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:30.333386 25781 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.762s	user 0.008s	sys 0.004s
W20260709 12:23:30.371403 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.000s	sys 0.009s
W20260709 12:23:30.371783 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 605879us
I20260709 12:23:30.371742 25781 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.374066 25781 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.367444 20806 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:30.377123 20882 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:30.384157 25963 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.387620 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.388372 25933 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.392700 25686 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:30.392995 26032 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.394235 26029 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.396188 25897 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:30.396488 25671 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:30.399308 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71535.3us
I20260709 12:23:30.354900 26021 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:30.401861 25404 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.001s
I20260709 12:23:30.330269 26009 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:30.426026 20805 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:30.408991 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:30.474045 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65154us
I20260709 12:23:30.447710 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2580) took 1026 ms. Trace:
W20260709 12:23:30.403632 25865 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.834s	user 0.010s	sys 0.000s
I20260709 12:23:30.312436 25536 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:23:30.457625 21431 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:30.438470 26044 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:30.481535 20810 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:23:30.476171 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:30.466650 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:30.483593 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:30.484266 20882 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:30.391635 25696 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.468815 20728 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 12:23:30.474994 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.477416 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:30.486549 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 327045us
I20260709 12:23:30.444516 25929 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.476058 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:30.474916 20804 rpcz_store.cc:276] 0709 12:23:29.421343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.960013 (+538670us) service_pool.cc:224] Handling call
0709 12:23:30.042654 (+ 82641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.043645 (+   991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.447473 (+403828us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b540019afd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:30.447647 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188920192}
I20260709 12:23:30.492071 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:30.496762 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:30.496878 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:30.498378 20876 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:23:30.482642 25923 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.191s	user 0.012s	sys 0.000s
W20260709 12:23:30.478873 25833 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.935s	user 0.010s	sys 0.000s
I20260709 12:23:30.476655 26038 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.444214 25686 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:30.479549 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.500373 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:30.507931 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108630us
I20260709 12:23:30.485327 26044 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.492023 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:30.511145 20735 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:30.488216 25841 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:30.472057 25781 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:23:30.521871 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:30.454468 26032 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.512890 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:30.478226 25404 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.506569 25696 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:30.551430 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59503.9us
I20260709 12:23:30.552546 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:30.502611 25603 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.506337 25927 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:30.562044 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.562788 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:30.586771 26041 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.234s	user 0.002s	sys 0.000s
W20260709 12:23:30.578138 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65334.4us
I20260709 12:23:30.546731 25841 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.530612 26044 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:30.507969 25833 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.565305 20876 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:30.602820 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:30.579870 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:30.563737 25696 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.559712 25897 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.628062 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:30.581072 25927 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:30.600854 25921 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.607738 25841 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:30.560042 26033 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.564992 25671 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:30.634925 25927 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:30.638996 25671 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.640173 25833 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.640823 25833 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:30.627694 25696 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:30.641824 25671 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.641872 25636 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.596282 25603 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:30.624732 26045 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:30.638641 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.563019 25876 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:30.604562 25890 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.006s
W20260709 12:23:30.602133 25770 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
I20260709 12:23:30.510711 26009 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:23:30.561070 25923 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.630180 26020 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.637130 25938 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:30.660355 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2329) took 1245 ms. Trace:
I20260709 12:23:30.660749 20806 rpcz_store.cc:276] 0709 12:23:29.415037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.836510 (+421473us) service_pool.cc:224] Handling call
0709 12:23:29.836942 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.375591 (+538649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.660108 (+284517us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54003d6dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:30.660295 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219466624}
I20260709 12:23:30.643968 25934 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 12:23:30.646777 25932 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.002s
I20260709 12:23:30.512321 26038 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.647316 25865 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:30.644683 25836 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.062s	user 0.012s	sys 0.000s
I20260709 12:23:30.663149 25932 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.663679 25836 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.664223 25404 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:30.665650 25836 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.666327 25932 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:30.646090 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66299.4us
I20260709 12:23:30.606572 26053 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:30.651754 25770 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.655908 26032 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.636063 26048 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:23:30.656553 26022 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.001s
W20260709 12:23:30.657665 26028 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.376s	user 0.001s	sys 0.000s
I20260709 12:23:30.652937 25890 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:30.650962 25942 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.004s
I20260709 12:23:30.649673 25887 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 12:23:30.662887 26051 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:23:30.668484 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.669183 26053 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.669355 25929 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:30.667385 25692 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.634363 25897 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.670481 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:30.678124 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:30.708348 21433 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:30.677712 26026 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.475s	user 0.000s	sys 0.002s
W20260709 12:23:30.715384 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44983.4us
I20260709 12:23:30.716497 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:30.677842 25931 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.101s	user 0.009s	sys 0.002s
I20260709 12:23:30.667846 25781 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.726061 25781 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.692565 25923 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.645390 25921 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:30.729049 25860 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 12:23:30.729456 25860 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.694106 21432 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:30.638505 26033 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.731487 25860 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.732342 26033 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:30.695796 26053 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:30.689298 26022 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.735608 21429 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:23:30.676505 25699 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.735939 20801 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:30.737007 26022 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.737488 25699 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.700143 25929 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.675319 25770 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.723528 20803 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:30.724748 25931 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:30.721650 25692 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:30.725283 25897 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:30.727430 26039 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.368s	user 0.002s	sys 0.000s
I20260709 12:23:30.715411 26009 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:30.696298 25942 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.740996 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:30.742640 21433 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
W20260709 12:23:30.738210 26052 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:23:30.674379 25961 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.744071 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:30.675415 25736 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.751295 25926 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.734599 25841 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:30.755564 20810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2324) took 1330 ms (client timeout 1566 ms). Trace:
I20260709 12:23:30.728359 25781 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.728101 25927 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.763680 21435 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
W20260709 12:23:30.773828 20810 rpcz_store.cc:269] 0709 12:23:29.413713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.457080 (+ 43367us) service_pool.cc:224] Handling call
0709 12:23:30.744096 (+1287016us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:30.744286 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2352709376}
I20260709 12:23:30.765659 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:30.772339 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:30.761126 25832 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.009s	sys 0.013s
I20260709 12:23:30.793439 25865 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:30.782239 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:30.796793 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:30.758947 26009 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:30.795430 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2611) took 1142 ms. Trace:
I20260709 12:23:30.801218 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.773350 26022 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.802531 21428 raft_consensus.cc:2368] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13.
I20260709 12:23:30.798228 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:30.776772 25823 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:30.799256 20729 rpcz_store.cc:276] 0709 12:23:29.632770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.647608 (+ 14838us) service_pool.cc:224] Handling call
0709 12:23:29.648047 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.658616 (+ 10569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.658664 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.755056 (+ 96392us) log.cc:844] Serialized 26 byte log entry
0709 12:23:29.993055 (+237999us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:29.993109 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.993193 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.707202 (+714009us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:30.766508 (+ 59306us) raft_consensus.cc:1685] finished
0709 12:23:30.766543 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.775014 (+  8471us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9371,"raft.queue_time_us":283,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":96580,"spinlock_wait_cycles":193981824,"thread_start_us":123641,"threads_started":1,"wal-append.queue_time_us":495764}
I20260709 12:23:30.770304 25694 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.805346 21427 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:30.808730 25926 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.809168 26053 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.820348 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.821077 20730 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:30.773299 25692 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:30.794708 25770 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:30.794773 25923 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:30.791739 25832 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.012s	sys 0.026s
I20260709 12:23:30.817545 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:30.797680 25865 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:30.824733 25942 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.811661 25836 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:30.775594 25860 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:23:30.842814 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:30.810468 25907 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.774s	user 0.009s	sys 0.005s
I20260709 12:23:30.829401 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2581) took 1172 ms. Trace:
I20260709 12:23:30.832171 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.832312 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:30.832686 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.814306 26022 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.833442 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.846652 20735 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:30.833894 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.846976 20734 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:23:30.834312 26053 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.804035 25781 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:30.845402 26045 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:23:30.849335 25781 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.851895 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:30.852566 20726 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:30.845997 20807 rpcz_store.cc:276] 0709 12:23:29.656702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.312751 (+656049us) service_pool.cc:224] Handling call
0709 12:23:30.313251 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:30.313722 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.313764 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:30.314450 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.481557 (+167107us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:30.481602 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.481671 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.829199 (+347528us) raft_consensus.cc:1685] finished
0709 12:23:30.829228 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.829346 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58577,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":166610,"threads_started":1,"wal-append.queue_time_us":316307}
I20260709 12:23:30.837842 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.853552 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:30.822108 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.847664 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:30.854497 20735 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:30.828425 25692 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:30.855973 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.856201 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.847927 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:30.840586 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.828944 25770 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:30.856905 20731 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
I20260709 12:23:30.828280 25841 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.857502 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:30.854416 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:30.862762 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:30.863350 20804 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:30.863993 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2579) took 1494 ms. Trace:
I20260709 12:23:30.857788 21430 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:23:30.848526 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.860775 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:30.844825 25823 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:30.882740 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:30.881762 20735 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:30.866703 25736 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.845817 25536 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:30.867313 25929 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.867755 25961 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.852758 25921 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.868157 26022 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.856819 25976 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.891044 20726 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:30.866729 20881 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:30.870365 25865 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:30.862262 26029 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:30.902590 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:30.902662 26029 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.902887 25865 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.903599 20884 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:30.871894 26053 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:30.904150 26053 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:30.904544 26053 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:30.904628 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:30.905088 26029 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.905304 25865 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.872690 25699 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.902432 25976 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.907439 25699 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.909272 25976 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.909430 25699 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.909997 25806 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:30.911031 20883 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:23:30.872953 25770 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.873739 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:30.874284 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:30.876937 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:30.866258 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48797.4us
I20260709 12:23:30.912904 26056 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:30.910980 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:30.887949 20878 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:30.907080 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:30.888875 20804 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:30.915113 26056 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:30.946244 20726 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:30.891736 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:30.946760 20735 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:30.862985 25694 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.831177 26009 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.898975 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:30.903507 25945 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.905388 20881 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:23:30.859782 25942 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:30.864384 20802 rpcz_store.cc:276] 0709 12:23:29.362311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.619327 (+257016us) service_pool.cc:224] Handling call
0709 12:23:29.638300 (+ 18973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.145558 (+507258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.145595 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.145644 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:30.145695 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.856122 (+710427us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54003b6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:30.856171 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.856280 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165991680}
I20260709 12:23:30.883414 20810 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
I20260709 12:23:30.950524 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.950829 20810 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:23:30.888702 25926 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:30.943833 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:30.952924 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:30.894788 25907 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:30.916668 20807 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:30.918218 20732 raft_consensus.cc:2368] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12.
I20260709 12:23:30.859705 25692 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:30.947548 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.954835 25926 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.955215 26056 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.956211 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:30.952001 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2578) took 1639 ms. Trace:
I20260709 12:23:30.892043 25823 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.963236 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:30.963522 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:30.962721 20809 rpcz_store.cc:276] 0709 12:23:29.312778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.570609 (+257831us) service_pool.cc:224] Handling call
0709 12:23:29.570901 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.127441 (+556540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.127481 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.127524 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:30.127555 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.951791 (+824236us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b5400126e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:30.951838 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.951950 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279411072}
I20260709 12:23:30.964702 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.949177 25938 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:30.965823 26038 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.967403 26009 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:30.954123 25942 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:30.952606 25945 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:30.952013 26065 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:30.953881 25806 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:23:30.952950 25781 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:30.969776 26056 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:30.978669 20735 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:30.957159 25907 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:30.957989 25536 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:23:30.984570 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.988955 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:30.984472 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.958328 25921 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:30.992408 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:30.993049 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:30.995154 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:30.995343 20879 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:30.971769 25887 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:23:31.005344 25887 consensus_queue.cc:1243] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:31.006014 25887 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:31.006361 25887 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:30.981688 26065 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.009759 25536 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:31.010406 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2337) took 1474 ms. Trace:
I20260709 12:23:31.010862 20803 rpcz_store.cc:276] 0709 12:23:29.536288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.044406 (+508118us) service_pool.cc:224] Handling call
0709 12:23:30.044834 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.739484 (+694650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.010139 (+270655us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54003d5ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:31.010343 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1560348032}
I20260709 12:23:31.011739 25536 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:30.996619 25938 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:31.013777 25938 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.015003 25536 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:31.016018 25909 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:31.016420 25909 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:31.017297 25909 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:31.012344 20805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2317) took 2703 ms (client timeout 1816 ms). Trace:
I20260709 12:23:31.018715 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 874) took 1595 ms. Trace:
W20260709 12:23:31.018573 20805 rpcz_store.cc:269] 0709 12:23:28.309359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.344293 (+ 34934us) service_pool.cc:224] Handling call
0709 12:23:31.012139 (+2667846us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400127354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:31.012253 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4635549440}
I20260709 12:23:31.019217 20801 rpcz_store.cc:276] 0709 12:23:29.423671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.961265 (+537594us) service_pool.cc:224] Handling call
0709 12:23:29.961611 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.736422 (+774811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.018489 (+282067us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b540019a0d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:31.018658 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780212096}
I20260709 12:23:31.021483 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:30.985025 25696 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.024127 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.024879 20880 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:31.025707 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.026875 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.027482 20877 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:31.028316 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.029284 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.005252 25942 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:31.035696 20881 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:31.030539 20805 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.036612 20806 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.031661 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.032315 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.037928 20878 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:31.032462 20810 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:31.039359 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.039987 20880 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.018467 25909 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:31.035072 20879 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:31.025942 25696 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.036016 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.030197 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.033592 20801 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:31.036062 25832 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.048645 20804 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:31.042585 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:30.977757 25927 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.049304 20805 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:31.043612 20810 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:23:31.044337 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.050688 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.050873 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.051635 20805 raft_consensus.cc:2410] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:23:31.052152 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.053596 20884 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:31.058089 20880 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:31.052819 25921 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:23:31.059376 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.059892 25841 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:31.060401 25841 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.060210 25832 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.060523 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.060791 25841 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.061167 20875 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:31.062430 25832 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.059973 20809 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:31.048377 26036 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.053275 25961 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:31.064261 26036 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.064628 25961 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.065588 25890 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.066512 26036 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.066757 25961 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:30.971490 26044 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.048122 25865 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:31.067713 25977 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.067830 21433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26ced453445642b09bda798a2963748b"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:43408
I20260709 12:23:31.068493 21433 raft_consensus.cc:606] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:23:31.068892 21433 raft_consensus.cc:614] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:31.069962 26031 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.853s	user 0.003s	sys 0.000s
I20260709 12:23:31.067739 25865 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.069272 26044 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:31.072654 26066 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:23:31.072852 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2338) took 1536 ms. Trace:
I20260709 12:23:31.073215 26055 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:31.073413 25865 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.075567 20804 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:31.070930 25692 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.074931 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.079118 25945 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.060056 20810 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:31.037251 26001 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:31.053551 26056 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:31.083773 26001 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:31.065640 26064 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:23:31.080374 21433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4415902e5b30489691f67f4a635305b4"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:43408
I20260709 12:23:31.086930 21433 raft_consensus.cc:606] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
W20260709 12:23:31.089056 26062 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.283s	user 0.002s	sys 0.000s
I20260709 12:23:31.088830 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:31.090873 25976 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:31.091221 25881 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:31.091403 25976 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.092689 25975 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:31.093621 25976 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.093918 25404 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:31.094322 25404 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:31.094461 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:31.095149 25404 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.095961 25876 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:31.096544 25876 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:31.096587 25881 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.097214 25881 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:31.095724 26061 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:23:31.049558 25887 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.100482 25823 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.100768 25992 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:31.101907 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.097602 25876 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.096256 25404 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:31.108172 25841 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.105901 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:31.111039 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.114506 20878 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:31.128397 20807 raft_consensus.cc:2368] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10.
I20260709 12:23:31.152004 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.145308 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:31.154531 20805 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.073276 20808 rpcz_store.cc:276] 0709 12:23:29.536542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.056997 (+520455us) service_pool.cc:224] Handling call
0709 12:23:30.143538 (+ 86541us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:30.436093 (+292555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.436140 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:30.436923 (+   783us) log.cc:844] Serialized 52 byte log entry
0709 12:23:30.606712 (+169789us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:30.773300 (+166588us) log.cc:844] Serialized 14 byte log entry
0709 12:23:30.773764 (+   464us) log.cc:844] Serialized 14 byte log entry
0709 12:23:31.072399 (+298635us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b5400181620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1a733 00007f9e1ec9fdbf 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c
0709 12:23:31.072451 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.072543 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:31.072660 (+   117us) raft_consensus.cc:1685] finished
0709 12:23:31.072674 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.072797 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25495,"prepare.queue_time_us":369273,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4549,"raft.queue_time_us":191224,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":40932,"spinlock_wait_cycles":8013568,"thread_start_us":454422,"threads_started":3,"wal-append.queue_time_us":207071}
I20260709 12:23:31.147029 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:31.155328 20877 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:31.139720 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:31.142714 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 90608.8us
I20260709 12:23:31.139233 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:31.117434 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:31.158567 21435 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 11: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:31.157740 20729 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.160467 20877 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:31.157011 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.161139 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.162380 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.162752 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.163005 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:31.166790 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.167248 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.167512 20877 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:31.168411 20775 leader_election.cc:400] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12.
I20260709 12:23:31.168738 20775 leader_election.cc:403] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:31.167501 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.169332 20805 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:31.169596 21427 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:31.171028 20877 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 12:23:31.174690 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.175551 20810 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:31.161926 25961 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:31.174125 25933 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:31.184902 25933 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:31.186031 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:31.185075 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.176882 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.058313 25536 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:31.186772 20726 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.142948 25890 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:31.194093 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.197016 25781 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:31.198845 25897 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:31.157840 25976 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.187252 25865 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:31.199494 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.212634 21435 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:31.199946 25770 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.188393 20732 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:31.200338 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:31.215154 21427 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.200891 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.174191 25921 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:23:31.185873 25933 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.202440 25961 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:31.226148 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.186545 26036 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.203263 25907 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.204219 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.226197 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.205104 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.206576 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.227283 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.227696 20878 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:31.228076 20801 raft_consensus.cc:2368] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13.
I20260709 12:23:31.139452 25881 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.207262 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.207477 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.227555 21428 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:31.207827 20731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:31.208475 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.129205 25887 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:31.209723 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:31.138906 25841 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.215503 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.214609 20802 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:23:31.213894 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.220583 20726 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:31.213479 25897 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:31.221231 20729 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:31.221390 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.241036 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.241469 20880 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.240597 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.222797 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.098001 26053 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.227331 25971 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:31.255300 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.237128 25890 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:31.224675 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.255806 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.214609 25865 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:31.229552 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.206735 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.230268 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.209326 25909 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:23:31.233783 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:31.239142 20883 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.239964 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.238119 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.245069 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.269253 20732 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:31.269845 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.270205 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.245335 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.271157 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:31.246168 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.264422 20808 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:31.272851 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.273578 20805 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:31.245076 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.241663 20884 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:31.248265 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:31.214615 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:31.276543 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62017.5us
I20260709 12:23:31.277174 21427 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:31.249106 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:31.250626 20880 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:31.244534 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.279340 20726 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:31.256507 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:31.096287 25926 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.259891 20879 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:31.231846 26044 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:31.284675 26044 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:31.230557 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.285888 20803 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.268074 20730 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:23:31.286855 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.268013 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.287493 20801 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.222159 26009 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:31.293874 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.294545 20884 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.280740 20729 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.296880 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.255288 25929 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:31.282255 21434 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:31.281792 25890 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.282794 25833 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:31.282682 26050 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:23:31.227576 25945 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:23:31.283236 25536 raft_consensus.cc:2764] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:23:31.269025 21435 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:31.236606 25961 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:31.240645 25832 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:31.291800 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:31.309363 26025 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.907s	user 0.024s	sys 0.049s
I20260709 12:23:31.225876 25770 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.271003 20732 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:31.329838 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.302995 25929 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.299980 25927 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:31.300135 26009 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:31.300738 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.263909 25806 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:31.255815 25971 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.303802 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:31.304360 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:31.365342 21434 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:31.307507 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.302548 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.302583 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.299310 26001 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:31.238546 25942 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:23:31.288879 25736 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.288592 25823 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:31.314461 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.308594 26027 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:23:31.317760 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.393671 20808 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:23:31.286948 25963 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:31.321897 20803 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.221993 25976 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.254812 26024 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:23:31.285800 26044 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.341279 20805 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:31.399816 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.354575 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.401675 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.401865 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.338507 25927 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:31.405968 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.372615 20730 raft_consensus.cc:2368] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:23:31.321394 25833 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:31.374042 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:31.376444 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73966.8us
I20260709 12:23:31.380908 20733 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:31.311992 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.374866 25971 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.420312 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.381126 25921 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:31.340013 25929 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:31.324784 26025 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.025s	sys 0.049s
I20260709 12:23:31.357692 25923 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:31.352962 26009 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:31.300832 25897 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.410786 25841 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:31.408802 20729 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:31.450739 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:31.374369 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:31.451915 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77686.8us
I20260709 12:23:31.409118 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.453469 20808 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:23:31.410315 20805 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:31.377245 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73530.6us
I20260709 12:23:31.398828 25806 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:31.430372 25680 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:31.431679 20882 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:31.429235 20807 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:31.355688 25865 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.397734 25963 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.394824 25736 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.402065 25536 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:31.407418 25927 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.444547 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.275031 26058 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:23:31.446276 20879 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:23:31.474673 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 205802us
I20260709 12:23:31.475696 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.479594 20882 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:31.422890 25696 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:31.455897 25791 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.422955 25921 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.454908 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:31.408146 25823 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:31.421276 20734 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:31.428391 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.429888 26065 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.459542 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:31.405658 25781 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:23:31.433264 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:31.457425 25841 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.483748 20849 leader_election.cc:400] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13.
I20260709 12:23:31.371528 20801 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:31.512790 20849 leader_election.cc:403] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:31.461788 25832 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:31.458593 25680 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:31.465086 25963 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:31.500124 20730 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:31.411476 25961 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.473917 25897 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:31.465862 25881 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:31.506942 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47501.8us
I20260709 12:23:31.508214 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:31.525767 20734 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.526015 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:31.513705 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.456140 25923 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.518914 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:31.515228 25832 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.486804 25942 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:31.506884 25696 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:31.532537 20733 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:31.500522 25921 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.494768 26053 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.471659 25536 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.517724 25841 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:31.538987 25860 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:31.541086 26037 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.214s	user 0.004s	sys 0.000s
W20260709 12:23:31.541889 26070 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:23:31.517998 25694 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:31.516598 25909 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:31.535148 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 286967us
I20260709 12:23:31.522172 25977 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13.
W20260709 12:23:31.593192 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 191367us
I20260709 12:23:31.597899 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:31.535118 25832 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.487972 25833 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.532974 25806 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:31.550060 25881 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:31.603382 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 107361us
I20260709 12:23:31.605579 20733 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:31.495628 26009 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.620337 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.521341 25791 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
W20260709 12:23:31.659592 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 251876us
W20260709 12:23:31.660581 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62829.3us
I20260709 12:23:31.662309 21435 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:31.668972 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:31.603248 25694 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:31.676672 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.677415 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:31.626843 26082 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.685333 20884 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:31.687376 20849 leader_election.cc:400] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:23:31.691682 20849 leader_election.cc:403] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:31.695122 26072 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:23:31.709547 20733 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:31.655741 25909 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:31.714207 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:31.668545 25801 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:31.725715 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2612) took 2092 ms. Trace:
I20260709 12:23:31.721374 20733 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:31.727715 20733 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:31.680068 25694 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.688055 25881 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:31.675901 25976 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.718878 21434 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:31.726166 20727 rpcz_store.cc:276] 0709 12:23:29.633047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.633177 (+   130us) service_pool.cc:224] Handling call
0709 12:23:29.633523 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.639094 (+  5571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.639123 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.641525 (+  2402us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.050942 (+409417us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:30.050986 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.051055 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:31.441744 (+1390689us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:31.716799 (+275055us) raft_consensus.cc:1685] finished
0709 12:23:31.716821 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.725622 (+  8801us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5176,"raft.queue_time_us":5518,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":10581,"spinlock_wait_cycles":252659072,"thread_start_us":409091,"threads_started":1,"wal-append.queue_time_us":642698}
I20260709 12:23:31.688719 25977 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:31.740346 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:31.739389 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 119152us
I20260709 12:23:31.603935 25680 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.712707 26009 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:31.715221 26079 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 10. Current term is 11.
I20260709 12:23:31.786041 25736 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.789436 25781 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:31.790021 25781 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:31.691887 26045 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.797462 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:31.602787 25823 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.754251 20733 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:23:31.682375 25696 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.749325 25942 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:31.764846 25909 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:31.767781 26075 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:23:31.712507 26082 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.890587 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.779917 26084 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:31.783964 25680 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:31.852794 26044 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.809209 25536 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.809890 25923 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.898388 20730 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:31.751356 25881 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:31.802209 25929 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.791764 25806 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.790884 25781 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.826622 26083 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:31.831979 26024 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:31.807278 26045 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.842540 21433 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:23:31.843477 25686 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:31.857431 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 445811us
I20260709 12:23:31.915470 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.915645 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.883471 20733 pending_rounds.cc:85] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 6
I20260709 12:23:31.916766 20733 pending_rounds.cc:107] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 9.7
W20260709 12:23:31.876854 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.541s (SENT) [suppressed 2 similar messages]
I20260709 12:23:31.917177 20733 raft_consensus.cc:2889] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:31.845265 26036 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
I20260709 12:23:31.892037 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:31.751493 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:31.803632 25887 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.005s
I20260709 12:23:31.849869 25963 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.883464 25891 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:31.822590 25841 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.900319 25881 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.943742 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:31.946367 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:31.949461 26055 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:31.854557 25927 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.973030 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:31.973740 20879 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:23:31.921185 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 170066us
I20260709 12:23:31.975039 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:31.867995 26022 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.923396 20726 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:31.861500 25694 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.926580 26045 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:31.932408 25841 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.923584 25961 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:23:31.986928 26082 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:31.933195 25971 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 12:23:31.935667 20807 raft_consensus.cc:2368] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:23:31.930944 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:31.955466 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.989152 20878 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:31.988611 25971 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.956447 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.990175 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:31.948912 25891 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.993032 25891 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:31.994866 25971 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:31.984699 26032 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:31.984851 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.996554 20883 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:31.995033 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.997793 20731 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:31.998234 20731 raft_consensus.cc:614] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:31.978545 26020 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:31.918869 26024 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:31.999712 26103 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:32.000603 26103 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:31.999655 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.001879 26024 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.002838 26103 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.001104 20883 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:32.005931 26009 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.006174 20729 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:32.006520 26009 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.008008 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:32.010214 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.009857 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:32.012404 20884 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:32.012877 20804 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:31.982420 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:32.014722 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:32.015322 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.017002 20879 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:31.942965 25770 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.019451 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:32.020612 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:32.022094 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 145264us
I20260709 12:23:32.021109 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.019121 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:32.027245 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:32.028111 20804 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:32.029249 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:31.954021 26083 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:32.034562 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:32.035462 20876 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:31.923851 26036 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12.
I20260709 12:23:32.034221 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:32.038470 20801 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:32.066601 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:32.070006 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:32.070374 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.073009 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.073366 20806 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:32.068382 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:32.075297 20806 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:32.072801 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:32.079478 20850 leader_election.cc:400] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13.
I20260709 12:23:32.033874 26032 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:32.026039 26020 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:32.082361 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:32.083525 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:32.023561 26024 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:31.995301 26022 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.098984 21435 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:23:32.101421 26022 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.102141 26088 raft_consensus.cc:993] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:23:32.102746 26088 raft_consensus.cc:1081] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:23:32.071221 25907 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:32.091568 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:32.104348 21434 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:32.082639 26095 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:32.105075 26095 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.105481 26095 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:32.087666 26078 raft_consensus.cc:2764] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:23:32.108531 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:32.096936 26020 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:32.109489 26020 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:32.111173 25927 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.112200 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:31.979218 25832 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.114934 25907 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:32.115554 25907 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.115969 25907 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:32.116122 20805 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:32.118570 25832 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.119997 26087 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:32.121114 25694 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.121644 26050 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:32.121872 26045 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:32.122129 26032 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:32.122273 26050 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:32.069376 25770 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:23:32.123363 26032 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.094201 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:32.123589 26050 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.071354 26055 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:32.097834 26082 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.125121 20804 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:23:32.125571 20804 pending_rounds.cc:85] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 8
I20260709 12:23:32.098224 25404 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:23:32.037366 25961 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:32.076411 25929 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:32.090186 25841 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:32.066133 25865 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:23:32.070973 26083 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:32.162263 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:32.125880 20804 pending_rounds.cc:107] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:23:32.104262 26009 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.138581 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:32.143666 20806 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:32.041973 25931 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.183326 25931 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.144794 20810 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:32.085641 26081 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.192559 26081 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.193033 25583 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:32.193732 25583 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:32.077816 25923 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.195405 25923 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.196318 25926 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:32.196869 25926 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:32.197870 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 299467us
I20260709 12:23:32.198181 25923 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.195093 26081 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.201803 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:32.202541 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:32.203387 21435 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:32.205519 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:32.206248 21427 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:32.125322 26082 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.177970 20878 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:32.178845 20804 raft_consensus.cc:2889] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:32.180560 25404 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:32.210448 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:32.210812 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.165879 20879 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:32.218896 20805 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.220031 21434 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 12. Ops: []
W20260709 12:23:32.225534 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.164671 25841 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:32.230934 20730 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:23:32.226940 20806 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:32.231884 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.232457 20734 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:32.235936 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.223727 25770 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.238982 20805 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:32.241444 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:32.242274 20880 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:32.244458 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:32.244874 20734 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:32.245434 20805 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:32.219962 25933 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:23:32.249253 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.249426 25770 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.250789 20727 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:32.159538 25929 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:23:32.252082 25907 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.253070 26083 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.256119 25841 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.195063 25938 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:23:32.220777 25881 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.261931 25881 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.267541 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:32.272069 25963 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.194875 25583 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.274914 25583 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:32.279762 25907 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.283893 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:32.286309 20877 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:32.209156 26053 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 12:23:32.228824 25921 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.631s	user 0.013s	sys 0.000s
I20260709 12:23:32.290087 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.300343 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:32.302588 20849 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:23:32.302999 20849 leader_election.cc:403] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:32.292688 20881 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:32.290820 20880 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:32.304721 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:32.309484 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:32.310614 20805 raft_consensus.cc:2368] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16.
I20260709 12:23:32.316375 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:32.321837 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.322691 20727 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:32.324595 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:32.321867 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:32.325915 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:32.218304 26020 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.252460 25770 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.333101 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:32.338059 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:32.339876 20805 raft_consensus.cc:2368] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10.
I20260709 12:23:32.310544 25909 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:32.295173 26053 consensus_queue.cc:1243] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:32.341707 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:32.343927 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:32.351792 20805 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:32.332356 25945 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:32.333360 25931 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.339533 25865 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:32.311857 25921 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.329826 26080 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:23:32.341922 25404 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:32.345125 25933 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.322041 25907 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:23:32.345134 25929 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:32.344823 26097 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:32.351657 25975 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:32.341465 25971 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:23:32.359683 25909 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:32.385874 20805 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:32.407958 20728 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:32.410843 20805 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:23:32.385926 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60126.9us
I20260709 12:23:32.412523 20880 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:32.412734 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:32.417991 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.418830 20727 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:32.420899 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.384371 26082 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:32.421911 20733 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:23:32.386394 25404 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:32.421499 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.411880 20805 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4415902e5b30489691f67f4a635305b4"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:43456
I20260709 12:23:32.385921 25929 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.429138 20805 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:32.384548 25865 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:32.429224 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.414891 26097 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:32.431278 20805 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.385912 25933 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.434396 26097 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.435457 25933 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.420027 25945 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:32.437021 25945 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.439941 26020 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.443234 25692 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.445973 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:32.446692 20809 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:32.447860 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2627) took 1175 ms. Trace:
I20260709 12:23:32.448284 20726 rpcz_store.cc:276] 0709 12:23:31.272493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.280136 (+  7643us) service_pool.cc:224] Handling call
0709 12:23:32.447615 (+1167479us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54000e79d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:32.447767 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311102208}
W20260709 12:23:32.449013 26113 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:23:32.450412 26110 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:23:32.451081 25921 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.451624 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:32.451867 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.452821 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.453487 20730 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:32.453373 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.421332 26081 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:32.454734 20735 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:32.464773 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.448773 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:32.448309 20809 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
W20260709 12:23:32.451936 26116 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:32.458060 20803 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:32.508368 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.508235 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.509189 20735 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.510190 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:32.512073 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 168264us
W20260709 12:23:32.511881 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63191.3us
I20260709 12:23:32.512600 20734 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:32.513155 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:32.433159 25929 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:32.515005 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.423310 26053 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:23:32.519937 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:32.515220 25923 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:32.527897 20726 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:32.528924 21400 leader_election.cc:400] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16.
I20260709 12:23:32.547058 20730 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:32.549535 21400 leader_election.cc:403] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:23:32.549048 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.538107 25909 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.539278 25975 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:32.540072 25692 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:32.541297 25977 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:32.464140 26081 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.526010 26083 raft_consensus.cc:2764] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:23:32.531991 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:32.545321 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:32.534469 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.564986 20734 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:32.537660 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:32.567824 25770 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:32.568434 25770 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:32.568588 26118 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:23:32.568975 25770 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.566778 20726 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:32.571415 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.557524 26056 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.573189 26056 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.574645 20735 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:32.537096 20805 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.559448 26088 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:32.558089 25692 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.547148 25921 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:32.549559 26082 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.636118 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.559847 26081 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:32.689076 20734 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:32.700345 26056 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.724406 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 809) took 1367 ms. Trace:
I20260709 12:23:32.724884 20729 rpcz_store.cc:276] 0709 12:23:31.357129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.450412 (+ 93283us) service_pool.cc:224] Handling call
0709 12:23:32.720667 (+1270255us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54001d7754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:32.724341 (+  3674us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258771200}
I20260709 12:23:32.727491 21400 leader_election.cc:400] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 9 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10.
I20260709 12:23:32.729766 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:32.728060 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 182850us
I20260709 12:23:32.728391 20734 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:32.731177 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:32.731365 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:32.728891 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.654690 25921 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.738226 20882 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:32.740219 20805 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.736436 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46798 (request call id 34) took 1528 ms. Trace:
I20260709 12:23:32.742776 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.742965 20881 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:32.743250 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:32.741036 20731 rpcz_store.cc:276] 0709 12:23:31.200615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.207709 (+  7094us) service_pool.cc:224] Handling call
0709 12:23:32.720306 (+1512597us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54001d7754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec99853 00007f9e23c6760a 00007f9e1d1ed7d4 00007f9e1d1ed762 00007f9e1d1ed691 00007f9e1d1ed60c 00007f9e1d1ec8c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:32.728508 (+  8202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757211008}
I20260709 12:23:32.739475 20734 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:32.743640 20884 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:32.744923 20734 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:32.732360 26032 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:32.748306 26131 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:32.747802 26032 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.619352 25404 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.650205 26082 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.754562 20803 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:32.755368 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:32.756114 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:32.619163 25865 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.743136 26058 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:32.757815 20731 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.749583 25887 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:32.749925 26133 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:32.742470 25923 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:32.752732 25945 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:23:32.771749 26032 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.772053 25945 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:32.772763 25945 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:32.763887 26058 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.775616 26058 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.779312 25938 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:32.773949 25945 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.763376 26097 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:32.783664 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2628) took 1499 ms. Trace:
I20260709 12:23:32.783778 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.784061 20728 rpcz_store.cc:276] 0709 12:23:31.284132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.709982 (+425850us) service_pool.cc:224] Handling call
0709 12:23:31.710241 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.409399 (+699158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.783420 (+374021us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54002981d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:32.783596 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604090624}
I20260709 12:23:32.789309 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.746620 25921 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.793236 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:32.797396 20728 pending_rounds.cc:85] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:23:32.797816 20728 pending_rounds.cc:107] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:23:32.798125 20728 raft_consensus.cc:2889] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:32.806771 26127 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:23:32.788232 25923 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.808158 26088 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16.
I20260709 12:23:32.809473 25923 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:32.811128 25686 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.816565 26082 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.785801 26056 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:32.788331 26133 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:32.818702 25921 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:32.819782 26133 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:32.821524 20802 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2592) took 1590 ms (client timeout 1860 ms). Trace:
W20260709 12:23:32.821977 20802 rpcz_store.cc:269] 0709 12:23:31.231478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.244835 (+ 13357us) service_pool.cc:224] Handling call
0709 12:23:32.821286 (+1576451us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:32.821460 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466472832}
I20260709 12:23:32.823376 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:32.796247 25945 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:32.840047 25929 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.841393 26081 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.841221 25770 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.846151 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:32.857793 20802 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
W20260709 12:23:32.833534 26095 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.728s	user 0.013s	sys 0.000s
W20260709 12:23:32.837360 25583 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 12:23:32.907418 20729 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:32.916397 26032 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.917090 26032 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:32.856599 25887 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.918471 25887 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:32.919349 25841 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:32.920011 25841 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:32.920754 26125 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.377s	user 0.000s	sys 0.002s
I20260709 12:23:33.091485 20731 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:32.968801 26139 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:33.082569 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 544978us
I20260709 12:23:33.098316 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:32.929527 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.953200 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:33.102190 20731 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
W20260709 12:23:33.085494 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 342314us
W20260709 12:23:33.112267 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 159128us
W20260709 12:23:33.073771 20835 heartbeater.cc:646] Failed to heartbeat to 127.19.36.254:46039 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.36.254:46039 timed out after 15.000s (SENT)
I20260709 12:23:33.146744 20849 leader_election.cc:400] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:23:33.147212 20849 leader_election.cc:403] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:33.149200 20731 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:23:33.152271 20731 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:33.054108 26123 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:33.154942 20731 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:33.103232 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:32.920782 25887 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:33.157557 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54484us
I20260709 12:23:32.956964 25583 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.083292 25841 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.161581 20731 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:32.886806 26095 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.166882 20803 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 12. Ops: []
I20260709 12:23:33.167891 20731 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:23:33.169298 20731 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:33.172195 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.156011 25806 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:33.165277 25583 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.177604 25806 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.178051 25806 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:33.180914 20732 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
W20260709 12:23:33.182817 20732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 804) took 1853 ms (client timeout 1923 ms). Trace:
W20260709 12:23:33.191403 20732 rpcz_store.cc:269] 0709 12:23:31.329212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.329556 (+   344us) service_pool.cc:224] Handling call
0709 12:23:33.182623 (+1853067us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540028d054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:33.182748 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4255344000}
I20260709 12:23:33.173622 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:33.203761 20802 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:33.205178 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:33.205950 20802 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:33.208405 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1075) took 2258 ms (client timeout 2190 ms). Trace:
W20260709 12:23:33.209000 21432 rpcz_store.cc:269] 0709 12:23:30.950069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.950292 (+   223us) service_pool.cc:224] Handling call
0709 12:23:33.208092 (+2257800us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54005f4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:33.208282 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855082880}
I20260709 12:23:33.215575 25770 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:33.156912 26123 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:33.217093 25897 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:33.217052 26123 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.217792 25897 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.218596 26123 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:33.220121 21433 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:33.221374 25929 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.223865 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:33.224568 21431 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:33.217681 26088 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:33.103566 26139 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.187764 25891 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.171366 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:33.188715 26050 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:33.189234 26032 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.193418 25926 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.168426 26083 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:23:33.199609 21426 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:33.226696 26088 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.174598 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:33.187896 25806 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.240161 20805 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:23:33.239919 20810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2344) took 2028 ms (client timeout 1966 ms). Trace:
I20260709 12:23:33.284857 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:33.242974 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:33.284844 20810 rpcz_store.cc:269] 0709 12:23:31.211921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.213121 (+  1200us) service_pool.cc:224] Handling call
0709 12:23:33.239604 (+2026483us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b540025d5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:33.239815 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897150848}
I20260709 12:23:33.220136 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 892) took 1353 ms. Trace:
I20260709 12:23:33.233595 21434 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:33.279445 25583 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.280864 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:33.281378 25770 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.281620 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:33.281929 25923 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.281378 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:33.299206 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:33.288267 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:33.286921 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:33.288499 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.287025 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.291514 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.303853 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.305330 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.295636 20878 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:23:33.296922 21428 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:23:33.296185 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:33.286613 20801 rpcz_store.cc:276] 0709 12:23:31.866830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.867044 (+   214us) service_pool.cc:224] Handling call
0709 12:23:31.867380 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.068095 (+200715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.219877 (+1151782us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54002a99d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:33.220068 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392895744}
I20260709 12:23:33.296326 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:33.274925 20807 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:33.309779 21435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.006s	sys 0.024s
W20260709 12:23:33.310243 21435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.006s	sys 0.024s
W20260709 12:23:33.311370 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48846.4us
I20260709 12:23:33.312305 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:33.312662 20875 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:33.311079 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:33.308429 20809 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:33.353972 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.355899 20881 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:33.191403 25699 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:33.357254 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:33.362035 21433 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:33.362401 21433 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:33.284368 26133 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:23:33.227387 26137 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.228590 26081 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.353070 25770 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.359642 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:33.284159 25931 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:33.295388 25977 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.295735 25933 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.120s
I20260709 12:23:33.284366 26020 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 12:23:33.227115 26058 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.299104 25891 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.413688 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:33.711921 26144 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260709 12:23:33.659925 20850 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.032s	sys 0.046s
W20260709 12:23:33.712582 20850 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.032s	sys 0.049s
W20260709 12:23:33.712846 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 665086us
I20260709 12:23:33.714775 25931 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.716758 25945 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:33.717362 25945 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.721124 21427 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:33.724277 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:33.444280 20880 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:23:33.732107 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:33.735668 20802 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.000s
I20260709 12:23:33.736167 20802 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
W20260709 12:23:33.738216 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 378704us
I20260709 12:23:33.739243 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:33.361106 25926 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:33.420310 20807 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:33.719885 25921 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:33.361334 25603 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:33.357671 25836 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.361696 25841 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.354871 26032 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:33.361634 25699 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:33.411150 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46258.5us
I20260709 12:23:33.772938 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:33.367501 26147 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.777271 21427 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:33.521189 21433 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.778194 21433 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.778402 21432 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:33.762837 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:33.779063 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.008s	sys 0.016s
I20260709 12:23:33.627573 26133 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.412057 26089 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:33.780936 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.009s	sys 0.016s
I20260709 12:23:33.781304 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:33.770181 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:33.555482 26027 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.744064 26029 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:33.773133 20735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.006s	sys 0.010s
W20260709 12:23:33.784318 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 637681us
W20260709 12:23:33.807368 20735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.008s	sys 0.012s
I20260709 12:23:33.806373 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:33.809064 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.810133 20876 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:23:33.505102 25671 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:33.750322 26130 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:33.752547 25945 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.750288 25921 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:33.752123 26024 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:33.357277 25923 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.821768 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:33.772513 25963 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:33.837904 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2605) took 1966 ms. Trace:
I20260709 12:23:33.772852 25975 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:33.803507 25696 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.012s	sys 0.012s
I20260709 12:23:33.777392 25836 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.834928 25942 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:33.795840 25603 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:33.835651 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 901) took 1754 ms (client timeout 2253 ms). Trace:
I20260709 12:23:33.760730 26137 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.837591 25923 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.411808 26148 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.403s
I20260709 12:23:33.838438 20804 rpcz_store.cc:276] 0709 12:23:31.871146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.039917 (+168771us) service_pool.cc:224] Handling call
0709 12:23:32.040370 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.221541 (+181171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.837651 (+1616110us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54002bff54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:33.837831 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193105792}
I20260709 12:23:33.797657 26032 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:33.839887 21430 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.000s
W20260709 12:23:33.837831 26152 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:23:33.803507 26089 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:33.810506 26132 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.006s	sys 0.014s
W20260709 12:23:33.860956 20806 rpcz_store.cc:269] 0709 12:23:32.081230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.082105 (+   875us) service_pool.cc:224] Handling call
0709 12:23:33.835476 (+1753371us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b540027ae54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:33.835589 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124939520}
W20260709 12:23:33.875996 26132 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.007s	sys 0.014s
I20260709 12:23:33.836550 25770 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:33.823033 25832 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:33.876837 26132 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.877485 25832 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:33.877362 26132 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.824816 25583 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.878499 25583 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.875418 26089 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:33.880751 25583 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:33.863541 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.019s	sys 0.010s
I20260709 12:23:33.828914 25921 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:33.883790 26132 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:33.885877 26132 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:33.887640 25832 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:33.888208 25832 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.888780 25931 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.890477 25832 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:33.860637 25963 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:33.734274 25977 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.827953 26024 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:33.781967 25841 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.823539 25671 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.870486 21430 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:33.875144 25603 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:33.875561 26145 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:23:33.891667 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.019s	sys 0.010s
I20260709 12:23:33.867134 21433 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:33.813290 26133 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.817942 26027 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.817719 26029 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.906059 21427 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:33.871301 25806 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:33.827608 26130 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:33.907564 20808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.024s	sys 0.013s
I20260709 12:23:33.853509 25975 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:33.869035 25696 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.012s	sys 0.012s
W20260709 12:23:33.912932 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.003s	sys 0.014s
I20260709 12:23:33.865513 25942 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:33.916056 20808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.024s	sys 0.013s
W20260709 12:23:33.953269 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.004s	sys 0.014s
I20260709 12:23:33.900779 26024 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:33.941722 20876 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
W20260709 12:23:33.908912 26079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.002s	sys 0.016s
I20260709 12:23:33.906065 25881 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:33.904145 26139 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:33.807489 26147 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:33.953728 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 641476us
W20260709 12:23:33.956863 20875 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.006s	sys 0.021s
W20260709 12:23:33.960263 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.032s	sys 0.023s
W20260709 12:23:33.981997 20875 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.006s	sys 0.021s
W20260709 12:23:33.982125 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.032s	sys 0.023s
W20260709 12:23:33.964516 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.003s	sys 0.024s
I20260709 12:23:33.982388 25603 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:23:33.982909 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.003s	sys 0.024s
W20260709 12:23:33.983256 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 672238us
I20260709 12:23:33.984567 20732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:33.984706 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:33.985117 20732 raft_consensus.cc:606] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:33.985496 20732 raft_consensus.cc:614] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:33.985688 20727 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 12:23:33.986052 20727 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 7
I20260709 12:23:33.986255 20727 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 12:23:33.986474 20727 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:33.986517 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:33.987278 25923 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.987922 25923 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.989998 26148 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:33.990674 26148 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:33.991601 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:33.992177 25603 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:33.992671 25603 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.993534 26148 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:33.994239 26148 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:33.990872 25923 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:33.892845 25699 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:33.966418 26088 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:34.017625 20879 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.013s	sys 0.033s
W20260709 12:23:34.020139 20879 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.013s	sys 0.033s
I20260709 12:23:34.039137 26058 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:34.039691 26155 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 12:23:34.039323 26157 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:23:34.040310 26032 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.041128 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2383) took 1579 ms. Trace:
I20260709 12:23:34.041759 25975 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:34.041524 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1092) took 1513 ms. Trace:
I20260709 12:23:34.043708 21435 rpcz_store.cc:276] 0709 12:23:32.528375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.532422 (+  4047us) service_pool.cc:224] Handling call
0709 12:23:32.538919 (+  6497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:32.728975 (+190056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:32.729013 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:32.737797 (+  8784us) log.cc:844] Serialized 52 byte log entry
0709 12:23:33.310598 (+572801us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:33.366989 (+ 56391us) log.cc:844] Serialized 14 byte log entry
0709 12:23:33.439486 (+ 72497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.439565 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:34.041331 (+601766us) raft_consensus.cc:1685] finished
0709 12:23:34.041351 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.041470 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61061,"prepare.queue_time_us":56173,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":12142,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":72954,"spinlock_wait_cycles":815872,"thread_start_us":733040,"threads_started":2,"wal-append.queue_time_us":1030403}
I20260709 12:23:34.046705 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:34.048485 26147 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:34.025249 26146 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:23:34.049844 26133 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.012759 26161 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.050863 26027 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.037130 25836 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.040370 25671 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.041502 20728 rpcz_store.cc:276] 0709 12:23:32.461335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.796643 (+335308us) service_pool.cc:224] Handling call
0709 12:23:32.797113 (+   470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:32.799458 (+  2345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:32.799507 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:32.801893 (+  2386us) log.cc:844] Serialized 52 byte log entry
0709 12:23:33.165645 (+363752us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:33.165691 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.165762 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:34.040882 (+875120us) raft_consensus.cc:1685] finished
0709 12:23:34.040905 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.041068 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2810,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":16562,"thread_start_us":311227,"threads_started":1,"wal-append.queue_time_us":1007395}
I20260709 12:23:34.105540 21435 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:33.994761 25603 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:34.199409 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 418177us
W20260709 12:23:33.960202 26079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.002s	sys 0.017s
W20260709 12:23:34.186174 21429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.019s	sys 0.022s
I20260709 12:23:34.186260 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 921) took 2026 ms. Trace:
W20260709 12:23:34.254894 21429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.019s	sys 0.023s
I20260709 12:23:34.254947 20883 rpcz_store.cc:276] 0709 12:23:32.032239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.032403 (+   164us) service_pool.cc:224] Handling call
0709 12:23:32.032795 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:32.033255 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:32.033298 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:32.067580 (+ 34282us) log.cc:844] Serialized 52 byte log entry
0709 12:23:33.242211 (+1174631us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b1800455440. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:33.903168 (+660957us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:33.903246 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.903436 (+   190us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:34.057938 (+154502us) raft_consensus.cc:1685] finished
0709 12:23:34.057957 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.058150 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3112,"raft.queue_time_us":98,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":39101,"spinlock_wait_cycles":2845440,"thread_start_us":660528,"threads_started":1,"wal-append.queue_time_us":1969972}
I20260709 12:23:34.257202 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:34.198782 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 391143us
W20260709 12:23:34.185515 26123 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.966s	user 0.009s	sys 0.004s
I20260709 12:23:34.180528 25887 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:34.193970 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 895) took 2318 ms. Trace:
I20260709 12:23:34.058599 25699 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:34.186499 25686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.006s	sys 0.029s
I20260709 12:23:34.182552 25975 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:34.246289 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54464.7us
I20260709 12:23:34.185755 26083 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.259522 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:34.285175 21426 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1089) took 2193 ms (client timeout 1917 ms). Trace:
I20260709 12:23:34.262001 20808 rpcz_store.cc:276] 0709 12:23:31.869776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.870462 (+   686us) service_pool.cc:224] Handling call
0709 12:23:31.870733 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:31.883477 (+ 12744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.883528 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:33.217484 (+1333956us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:23:33.217898 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 12:23:33.953588 (+735690us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:33.953633 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.953729 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:34.183103 (+229374us) raft_consensus.cc:1685] finished
0709 12:23:34.183127 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.188081 (+  4954us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12253,"raft.queue_time_us":14486,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2352,"spinlock_wait_cycles":4329856,"thread_start_us":735301,"threads_started":1,"wal-append.queue_time_us":962257}
W20260709 12:23:34.300477 21426 rpcz_store.cc:269] 0709 12:23:32.091480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.091603 (+   123us) service_pool.cc:224] Handling call
0709 12:23:34.284963 (+2193360us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400088c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:34.285092 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2471845120}
I20260709 12:23:34.284411 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.305889 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2418) took 1013 ms. Trace:
I20260709 12:23:34.308197 20883 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:34.307094 20880 rpcz_store.cc:276] 0709 12:23:33.292184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.292491 (+   307us) service_pool.cc:224] Handling call
0709 12:23:33.292745 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.730521 (+437776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.305656 (+575135us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:34.305831 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347585408}
I20260709 12:23:34.311291 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.312108 20880 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:34.313310 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.182657 25921 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.314664 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.244529 26133 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.317704 20880 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:34.320436 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.321066 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:34.260340 25832 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:34.317215 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:34.322197 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.188885 26089 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.323652 21426 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:23:34.324016 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.007s	sys 0.030s
W20260709 12:23:34.307036 25686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.008s	sys 0.035s
W20260709 12:23:34.325955 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.282718 26137 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.260794 25836 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:34.330958 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:34.337888 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:34.331382 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.007s	sys 0.030s
I20260709 12:23:34.345032 20733 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:34.310081 26139 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:34.193372 26161 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.366312 25942 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:34.319422 25977 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:34.368058 20849 connection.cc:256] RPC call timeout handler was delayed by -2.0273096189998796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:23:34.284593 25887 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:34.368708 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 598482us
I20260709 12:23:34.371209 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:34.379552 20732 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 16 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 12:23:34.281596 25671 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:23:34.370183 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:34.380254 20727 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:34.380321 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:34.305123 26083 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.384006 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:34.386323 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.388669 20883 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:34.424080 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.366911 25696 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.396128 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:34.316717 25975 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.400501 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:34.401598 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:34.435175 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:34.327539 26078 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:23:34.405102 20727 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:34.328797 25832 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.393708 26029 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.394831 25977 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:34.374634 26171 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:34.415884 20808 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [13.8-14.9]
I20260709 12:23:34.420521 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.481730 20807 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
W20260709 12:23:34.391644 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 402464us
I20260709 12:23:34.382371 26020 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:23:34.369665 25836 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.436712 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:34.484208 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47592us
I20260709 12:23:34.426019 20883 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:34.446281 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44490.9us
I20260709 12:23:34.457538 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.406165 25933 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.418497 25963 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:34.407845 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2443) took 1110 ms. Trace:
I20260709 12:23:34.478982 20809 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:34.486429 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:34.486624 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.497991 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.505435 20808 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 8.
I20260709 12:23:34.501500 25603 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:34.507319 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:34.507645 25603 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.508203 25603 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:34.460103 25696 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.517735 25696 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.522449 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.406926 26032 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:23:34.503593 20875 rpcz_store.cc:276] 0709 12:23:33.297833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.353783 (+ 55950us) service_pool.cc:224] Handling call
0709 12:23:33.354060 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.354279 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:33.982479 (+628200us) log.cc:844] Serialized 14 byte log entry
0709 12:23:34.406466 (+423987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:34.406538 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:34.406618 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.406731 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.407743 (+  1012us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42697,"prepare.queue_time_us":86135,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":21567,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":42541,"spinlock_wait_cycles":18816,"thread_start_us":989441,"threads_started":2}
I20260709 12:23:34.528967 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:34.532537 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.514941 25897 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:34.496781 25977 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.469620 26024 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:34.655269 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 126357us
I20260709 12:23:34.528235 26177 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:34.463800 26170 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:34.600907 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2419) took 1307 ms. Trace:
W20260709 12:23:34.608040 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 275053us
I20260709 12:23:34.326095 26133 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:34.540905 20849 connection.cc:256] RPC call timeout handler was delayed by -1.3357362780002404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:23:34.613984 25841 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:34.692384 20849 connection.cc:256] RPC call timeout handler was delayed by -1.2801041180005086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 12:23:34.460389 26174 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:34.529884 26147 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:34.700609 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:34.702378 26160 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.694s	user 0.009s	sys 0.031s
I20260709 12:23:34.705212 25992 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:34.705664 25992 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:34.706055 25992 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.698642 26147 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.706401 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1317) took 1344 ms. Trace:
I20260709 12:23:34.708968 26147 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.696566 26174 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.709904 26174 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:34.502281 25671 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:34.686829 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:34.651525 26186 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:34.617972 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:34.689260 20875 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:23:34.615991 26182 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:23:34.612282 26165 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
W20260709 12:23:34.741665 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.537412 25923 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.719475 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:34.682555 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:34.521876 26079 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.537690 26148 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.539529 26027 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:34.724594 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.725194 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.725323 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.750852 20807 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:34.605087 25686 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.123s
I20260709 12:23:34.615758 26083 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.614353 25832 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.655423 26089 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.640394 25921 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:34.778216 26184 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:23:34.762908 26079 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:34.780686 26181 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:23:34.777294 25692 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.762773 26027 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.795902 26080 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:34.796644 26080 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.797962 26027 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.776062 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 931) took 1061 ms. Trace:
I20260709 12:23:34.802385 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2638) took 2341 ms. Trace:
I20260709 12:23:34.780036 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:34.625100 26178 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 12:23:34.686286 20879 rpcz_store.cc:276] 0709 12:23:33.293105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.293227 (+   122us) service_pool.cc:224] Handling call
0709 12:23:33.293473 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.293709 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:34.022389 (+728680us) log.cc:844] Serialized 14 byte log entry
0709 12:23:34.599550 (+577161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:34.600530 (+   980us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:34.600609 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.600688 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.600829 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31855,"prepare.queue_time_us":118236,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":44040,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":8232,"thread_start_us":1273047,"threads_started":3,"wal-append.queue_time_us":595173}
I20260709 12:23:34.616808 25975 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.520084 25933 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.732342 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.686332 26170 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.528893 26172 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.169s
I20260709 12:23:34.581921 25696 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.148s
I20260709 12:23:34.688517 26161 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:34.790894 25926 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.022s	user 0.010s	sys 0.000s
I20260709 12:23:34.706923 21427 rpcz_store.cc:276] 0709 12:23:33.361633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.778194 (+416561us) service_pool.cc:224] Handling call
0709 12:23:33.902822 (+124628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.911872 (+  9050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.706110 (+794238us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b5400453150. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:34.706307 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197694208}
I20260709 12:23:34.754498 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:34.730266 25992 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.750562 20809 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:34.696483 25887 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.505018 26171 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.697021 26177 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:34.730481 25841 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:34.757190 26137 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.931581 26137 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.762616 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:34.698769 26078 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:23:34.895820 26020 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:34.758520 25927 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:34.784045 26185 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:23:34.799973 20876 rpcz_store.cc:276] 0709 12:23:33.714854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.914232 (+199378us) service_pool.cc:224] Handling call
0709 12:23:33.914546 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.956861 (+ 42315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.775720 (+818859us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:34.775979 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61275904}
I20260709 12:23:34.862286 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 918) took 2852 ms. Trace:
I20260709 12:23:34.762989 26148 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.735595 25671 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.750564 20801 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:34.807945 20883 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:34.762513 25836 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.802951 20734 rpcz_store.cc:276] 0709 12:23:32.492803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.745633 (+252830us) service_pool.cc:224] Handling call
0709 12:23:32.750362 (+  4729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.750583 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:32.957993 (+207410us) log.cc:844] Serialized 14 byte log entry
0709 12:23:34.774850 (+1816857us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b18003ca780. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a
0709 12:23:34.775121 (+   271us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:34.801960 (+ 26839us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:34.802043 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.802152 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.802304 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":548123,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":23604,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":62313856,"thread_start_us":647595,"threads_started":2,"wal-append.queue_time_us":915863}
I20260709 12:23:34.941895 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:34.944701 20876 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:34.935104 26137 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.795146 25692 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:34.808512 25963 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:34.782610 26133 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:34.896200 20804 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:34.786851 26130 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.787417 25897 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.768409 25832 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.896697 21431 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
W20260709 12:23:34.968569 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 424222us
I20260709 12:23:34.891738 26161 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:34.913834 26171 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.914899 26177 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.862251 26170 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:34.984720 25671 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:34.923414 26150 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.258s	user 0.013s	sys 0.009s
I20260709 12:23:34.924306 26024 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:34.926450 25992 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:34.737571 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:34.929885 25603 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.002642 26161 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.988495 25897 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:34.937156 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 190921us
I20260709 12:23:35.010460 25897 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:35.011114 25977 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:34.959044 25692 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:34.808265 25781 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:34.939672 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 220308us
I20260709 12:23:34.939973 20877 rpcz_store.cc:276] 0709 12:23:31.955177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.956983 (+  1806us) service_pool.cc:224] Handling call
0709 12:23:31.957186 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.288336 (+331150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.807458 (+2519122us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540022f6d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:34.807705 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756439424}
I20260709 12:23:34.759722 25923 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.862792 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.025823 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.026479 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:35.026383 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:34.898856 25841 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:34.950233 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 226333us
W20260709 12:23:34.950973 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.955610 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:34.781318 26167 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.399s	user 0.000s	sys 0.002s
I20260709 12:23:34.940725 25927 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.969354 25595 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.973863 25963 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:34.973891 25832 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.075901 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2413) took 1790 ms. Trace:
I20260709 12:23:35.077277 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:35.040463 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:34.994155 26174 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:34.994611 26020 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:35.002908 20803 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:35.015424 26186 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:35.005399 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 322955us
I20260709 12:23:35.007906 25933 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.010998 26161 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:34.795053 26079 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:35.015000 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.122710 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.123328 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:35.123829 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.125162 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.126302 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.120601 25696 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:23:35.129173 25696 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:35.017637 20810 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:35.019411 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:34.868014 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2379) took 2406 ms. Trace:
I20260709 12:23:35.027613 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:35.027978 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.089859 20807 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:35.017693 25692 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:35.032219 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.136272 21427 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:35.136377 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:34.869999 26080 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:35.078565 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:35.003664 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:35.080390 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:35.085245 26050 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.789s	user 0.011s	sys 0.004s
I20260709 12:23:35.152264 20876 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:35.076397 20884 rpcz_store.cc:276] 0709 12:23:33.285254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.287247 (+  1993us) service_pool.cc:224] Handling call
0709 12:23:33.353623 (+ 66376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:33.915820 (+562197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:33.915854 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:33.951464 (+ 35610us) log.cc:844] Serialized 26 byte log entry
0709 12:23:34.381679 (+430215us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:34.381721 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.046997 (+665276us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:35.047049 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:35.047093 (+    44us) raft_consensus.cc:1685] finished
0709 12:23:35.047105 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.075824 (+ 28719us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4699,"raft.queue_time_us":87246,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2972,"spinlock_wait_cycles":1283690880,"thread_start_us":400998,"threads_started":1,"wal-append.queue_time_us":582192}
I20260709 12:23:35.084743 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.017077 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.126906 20875 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:35.134009 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:35.135165 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.154896 20801 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
W20260709 12:23:35.082469 25699 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.798s	user 0.013s	sys 0.000s
I20260709 12:23:35.081570 26078 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:35.132972 20735 rpcz_store.cc:276] 0709 12:23:32.459011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.577881 (+118870us) service_pool.cc:224] Handling call
0709 12:23:32.729060 (+151179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:32.824637 (+ 95577us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:33.145538 (+320901us) log.cc:844] Serialized 14 byte log entry
0709 12:23:33.807845 (+662307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:33.808334 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:23:33.808468 (+   134us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:33.808498 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.701655 (+893157us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:34.709234 (+  7579us) spinlock_profiling.cc:243] Waited 6.95 ms on lock 0x7b1800393220. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:23:34.709820 (+   586us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b54000c3fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:34.709877 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:34.864683 (+154806us) raft_consensus.cc:1685] finished
0709 12:23:34.864704 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.864859 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":681580,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":36853,"raft.queue_time_us":214,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":16827,"spinlock_wait_cycles":240712704,"thread_start_us":903348,"threads_started":2,"wal-append.queue_time_us":810899}
I20260709 12:23:35.023214 26024 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:23:35.018653 26195 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:23:35.181011 26032 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:35.183903 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2391) took 1199 ms. Trace:
I20260709 12:23:35.184325 20733 rpcz_store.cc:276] 0709 12:23:33.984697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.986255 (+  1558us) service_pool.cc:224] Handling call
0709 12:23:35.183688 (+1197433us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54003b4554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:35.183841 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822664960}
I20260709 12:23:35.176044 25907 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.137487 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.185174 25907 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.187026 20728 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:35.185518 20879 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:35.137564 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:35.151487 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71183.3us
I20260709 12:23:35.031879 25841 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:35.032848 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.155294 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.156239 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.023684 26200 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
I20260709 12:23:35.083446 25603 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.083966 26177 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.174265 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.122236 25887 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.176615 20806 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:35.176687 20878 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:35.081835 26020 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:35.189090 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:35.189340 21433 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:35.193331 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.175402 26078 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.196111 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.197178 21426 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:35.198396 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.198218 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.194615 20877 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:35.214144 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:35.207309 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:35.214715 20877 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:35.211792 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.189199 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.173357 26080 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:35.185602 26148 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.201629 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:35.207545 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.217010 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1307) took 1963 ms. Trace:
I20260709 12:23:35.211582 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.222221 20810 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:35.079394 26130 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 12:23:35.181741 26032 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.217844 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:35.185472 26178 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:23:35.226166 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.220906 21429 rpcz_store.cc:276] 0709 12:23:33.253272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.253404 (+   132us) service_pool.cc:224] Handling call
0709 12:23:33.253797 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:33.264057 (+ 10260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:33.264108 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:33.353078 (+ 88970us) log.cc:844] Serialized 52 byte log entry
0709 12:23:34.255466 (+902388us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:34.255507 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.255575 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:34.780569 (+524994us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:35.209264 (+428695us) raft_consensus.cc:1685] finished
0709 12:23:35.209287 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.216949 (+  7662us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9767,"raft.queue_time_us":10496,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":17522,"spinlock_wait_cycles":32570240,"thread_start_us":898129,"threads_started":1,"wal-append.queue_time_us":952579}
I20260709 12:23:35.209844 26095 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.195940 25865 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:23:35.216665 26078 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:35.229820 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:35.228770 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:35.078199 25595 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.227022 20876 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:35.225809 25977 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:35.219502 26080 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:35.249866 20879 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:35.229384 25603 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:35.202561 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78736.4us
I20260709 12:23:35.209496 26177 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.216029 26141 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:23:35.250967 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:35.264895 25907 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.234755 26192 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:35.265666 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61151.4us
I20260709 12:23:35.212716 25887 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:35.267048 21400 connection.cc:256] RPC call timeout handler was delayed by -1.1310474189999695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
I20260709 12:23:35.271368 20810 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:35.249996 20877 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:35.273795 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.274601 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.275174 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.280855 20735 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:35.281414 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.280886 20734 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:35.281975 21430 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:35.283759 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2631) took 2829 ms. Trace:
I20260709 12:23:35.284176 20726 rpcz_store.cc:276] 0709 12:23:32.454734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.573579 (+118845us) service_pool.cc:224] Handling call
0709 12:23:32.573896 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.636206 (+ 62310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:32.636236 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:32.636281 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:32.636316 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.283547 (+2647231us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b54001d8154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:35.283594 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.283705 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141817600}
I20260709 12:23:35.285187 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.252113 25977 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.251593 26148 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.276024 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.299873 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.300724 20730 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:35.290375 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:35.287735 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:35.257902 25603 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:35.289625 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:35.272132 26192 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:35.283035 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.255237 26080 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:35.268560 26130 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:35.290347 26205 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:35.296759 21427 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:35.296660 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.297314 25923 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.297927 26177 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.298681 25832 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.298821 26171 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.294562 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:35.308184 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:35.328655 20806 raft_consensus.cc:2376] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12. Re-sending same reply.
I20260709 12:23:35.329633 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.292388 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.331060 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.309659 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:35.314630 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.317240 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.318482 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.334029 20729 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:35.334872 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.319772 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:35.278152 26187 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:35.335811 20731 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:35.332801 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.337960 21430 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:35.273345 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:35.324411 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.321151 26130 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.333498 20732 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:35.354087 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:35.327453 21435 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:35.306772 26192 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:35.306885 26174 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:35.348074 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53575us
I20260709 12:23:35.356834 20809 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:35.356460 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.348703 26171 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:35.349790 25923 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:35.300926 25977 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:35.299491 25921 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.291828 25770 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.358546 20732 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:35.359381 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.354887 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:35.306967 25603 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:35.352272 26205 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:35.257170 26186 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.368711 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:35.369340 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.359691 26130 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:35.313649 26079 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.374107 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 100847us
I20260709 12:23:35.379707 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.308468 26148 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:35.385015 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:35.371071 20731 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:35.384029 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.393101 20809 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 10.
I20260709 12:23:35.394198 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.394629 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:35.371670 25832 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:35.397987 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.398878 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.357983 25907 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:23:35.379711 25921 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.368729 26050 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:23:35.411500 25927 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.415091 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2640) took 1084 ms. Trace:
I20260709 12:23:35.415601 20728 rpcz_store.cc:276] 0709 12:23:34.330239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.330474 (+   235us) service_pool.cc:224] Handling call
0709 12:23:35.186846 (+856372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.187469 (+   623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.414894 (+227425us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54003b4550. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:35.415024 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934510464}
I20260709 12:23:35.417459 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.419274 26032 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.379496 25770 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.422168 20809 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:35.387785 26095 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:35.424041 25770 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.424525 26161 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:35.425169 25921 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.425555 26133 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:35.427413 25907 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:35.429404 25992 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:35.431391 20809 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:35.434723 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.435168 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.438378 20729 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:35.439082 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.439284 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.439924 20729 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:35.440768 20731 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:23:35.442194 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.405243 25909 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:35.429636 25975 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:35.400007 25832 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:35.428678 26188 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:35.464931 20734 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:35.430447 25923 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:35.467590 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.433930 26095 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.396044 26079 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:35.487758 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 130704us
I20260709 12:23:35.489064 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:35.491806 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:35.492624 20728 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:35.487701 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61580.4us
I20260709 12:23:35.444305 25907 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:35.463066 26050 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:35.466885 25832 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:35.463584 25909 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:35.468544 25671 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.490322 26095 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:35.515421 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 117321us
I20260709 12:23:35.518671 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:35.475704 26161 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:35.506425 25909 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:35.466048 26133 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:35.584017 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95077.2us
I20260709 12:23:35.618521 26133 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:35.584123 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56862.8us
I20260709 12:23:35.622431 26161 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:35.623303 25921 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:35.617302 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:35.623879 25921 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:35.625658 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:35.626329 26170 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.011s
I20260709 12:23:35.626772 26170 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:35.627391 25770 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:35.589354 26211 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:23:35.590245 26212 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:23:35.607547 25887 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.641562 25887 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:35.608114 26216 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:35.609556 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:35.614123 20728 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:35.510309 25927 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.487249 25923 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.662814 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:35.615439 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:35.626863 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:35.507504 26050 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:35.686447 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1310) took 2432 ms (client timeout 1904 ms). Trace:
I20260709 12:23:35.676205 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.689344 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.690418 20728 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:35.677320 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:35.677790 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:35.679745 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 161234us
I20260709 12:23:35.691902 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:35.693133 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:35.694345 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:35.694747 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2414) took 2394 ms (client timeout 2172 ms). Trace:
W20260709 12:23:35.687125 21434 rpcz_store.cc:269] 0709 12:23:33.254177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.287930 (+ 33753us) service_pool.cc:224] Handling call
0709 12:23:35.679141 (+2391211us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:35.686341 (+  7200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2255163264}
I20260709 12:23:35.696805 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.693028 26226 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:35.697578 21434 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:35.698449 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.511418 25671 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:35.695221 20881 rpcz_store.cc:269] 0709 12:23:33.285587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.287544 (+  1957us) service_pool.cc:224] Handling call
0709 12:23:35.679436 (+2391892us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:35.679565 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2810112}
I20260709 12:23:35.699012 21434 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:35.703892 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.706925 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:35.708659 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:35.709690 20728 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:35.525189 25841 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.532588 26137 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.164s
I20260709 12:23:35.674086 26171 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.664227 26170 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.496747 26205 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.672874 26216 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.700577 26133 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:35.717546 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.737015 20733 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 16 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 12:23:35.721793 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 921) took 2436 ms (client timeout 2158 ms). Trace:
I20260709 12:23:35.680346 25699 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:35.722149 21426 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:35.699102 25975 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:35.737629 21432 rpcz_store.cc:269] 0709 12:23:33.285634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.295954 (+ 10320us) service_pool.cc:224] Handling call
0709 12:23:35.720320 (+2424366us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:35.721185 (+   865us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060752768}
I20260709 12:23:35.740412 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.742316 25891 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.701404 26139 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:35.692530 25770 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:35.751859 20881 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:35.756994 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:35.682300 26188 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:23:35.672832 26220 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:23:35.642951 25907 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.760993 20733 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:35.762996 20808 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:35.763502 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 136715us
I20260709 12:23:35.681061 25927 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.769059 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:35.767993 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.735671 25887 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:35.770745 20733 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:35.746357 26133 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.761152 20728 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:35.754763 25975 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.703569 26078 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.743089 26216 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:35.770637 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:35.786219 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.798682 20733 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:35.800209 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.801050 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:35.802220 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:35.804930 26176 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:35.805549 26176 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:35.807960 25923 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.815704 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1313) took 2560 ms. Trace:
W20260709 12:23:35.816507 26020 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.002s
I20260709 12:23:35.816085 25770 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:35.816138 21428 rpcz_store.cc:276] 0709 12:23:33.254970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.398427 (+143457us) service_pool.cc:224] Handling call
0709 12:23:33.554715 (+156288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.716179 (+161464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:33.716214 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:33.716266 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:33.716301 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.815439 (+2099138us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b540029ca54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:35.815496 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.815643 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370792832}
I20260709 12:23:35.817829 26050 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.818279 25921 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:35.814617 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.820987 20881 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:35.780112 25891 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:35.780938 26133 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:35.822113 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.824862 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:35.826870 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:35.787035 25975 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:35.827703 20881 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:35.844141 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:35.816964 26020 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:35.910660 21431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 920) took 2685 ms (client timeout 3224 ms). Trace:
I20260709 12:23:35.897418 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2440) took 2600 ms. Trace:
I20260709 12:23:35.965109 20884 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:35.914842 20881 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:23:35.893263 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:35.909022 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2644) took 1525 ms. Trace:
I20260709 12:23:35.899529 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:35.858549 20803 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:35.903093 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:35.913668 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 143089us
I20260709 12:23:36.011885 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2612) took 1594 ms. Trace:
I20260709 12:23:36.009157 20801 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:36.008641 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2621) took 1301 ms. Trace:
I20260709 12:23:35.988778 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.017421 20804 rpcz_store.cc:276] 0709 12:23:34.417442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.417593 (+   151us) service_pool.cc:224] Handling call
0709 12:23:34.417918 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.964230 (+546312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.011570 (+1047340us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54002b5cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:36.011803 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099157888}
I20260709 12:23:35.893182 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2421) took 1636 ms. Trace:
I20260709 12:23:36.017865 20808 rpcz_store.cc:276] 0709 12:23:34.707587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.731979 (+ 24392us) service_pool.cc:224] Handling call
0709 12:23:36.008339 (+1276360us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b540027b854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:36.008561 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2066273664}
I20260709 12:23:36.045598 20808 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:23:35.986531 21431 rpcz_store.cc:269] 0709 12:23:33.225707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.242682 (+ 16975us) service_pool.cc:224] Handling call
0709 12:23:35.910405 (+2667723us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54004d9c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:35.910543 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3703149312}
I20260709 12:23:36.020411 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:35.902070 26222 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:23:35.954779 26079 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:35.858755 25692 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.004s
I20260709 12:23:35.847983 25923 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:35.887476 26130 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:35.901986 25595 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:35.857792 25927 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:36.007256 26208 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.801s	user 0.002s	sys 0.000s
W20260709 12:23:35.898838 26187 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.531s	user 0.005s	sys 0.002s
I20260709 12:23:35.898818 25963 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.891727 26173 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.459s	user 0.007s	sys 0.004s
I20260709 12:23:35.895653 25881 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:36.024786 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:35.898360 26159 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:35.903887 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:36.026538 25603 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.000s
I20260709 12:23:36.026901 20732 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:36.027907 26020 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.028314 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.082121 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:36.029428 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:36.028142 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:36.086158 20730 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
W20260709 12:23:35.902792 26217 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:23:35.900372 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:35.986703 20878 rpcz_store.cc:276] 0709 12:23:33.297061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.306975 (+  9914us) service_pool.cc:224] Handling call
0709 12:23:33.307264 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.208171 (+1900907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.897139 (+688968us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54003f98d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:35.897348 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4298410880}
I20260709 12:23:36.050220 20883 rpcz_store.cc:276] 0709 12:23:34.259370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.531051 (+271681us) service_pool.cc:224] Handling call
0709 12:23:34.531972 (+   921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.943493 (+411521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.892933 (+949440us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540005b254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:35.893118 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618061696}
W20260709 12:23:35.904596 26228 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:23:35.878834 26050 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.070307 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:36.116650 20802 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
W20260709 12:23:36.064919 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 171766us
I20260709 12:23:36.118180 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:35.903779 25841 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.001868 26194 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:36.120707 26194 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.019333 25933 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:36.123401 25832 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.001s
I20260709 12:23:36.123494 25933 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.123951 25832 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.125706 25933 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.126183 25832 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:36.126353 25977 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.746s	user 0.007s	sys 0.002s
I20260709 12:23:35.859747 26186 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:35.858175 25781 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.126919 26186 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:36.127027 25781 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.127422 26186 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.981745 25891 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:36.123059 26194 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.085691 25603 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.129864 20735 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:36.130851 25603 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:36.077330 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 381338us
I20260709 12:23:36.131493 26050 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.132186 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:36.132503 26192 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.758s	user 0.003s	sys 0.009s
I20260709 12:23:36.132911 26192 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.072469 25770 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:36.030779 20727 rpcz_store.cc:276] 0709 12:23:34.383180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.383579 (+   399us) service_pool.cc:224] Handling call
0709 12:23:35.908832 (+1525253us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b540028cb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:35.908957 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45659904}
I20260709 12:23:36.133724 26050 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.134346 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:36.086297 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.097522 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:36.090446 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:36.089016 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:36.077054 25671 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:36.101930 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73672.2us
W20260709 12:23:36.082005 26080 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.002s
I20260709 12:23:36.083425 26187 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:23:36.086931 26095 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.001s
I20260709 12:23:36.072150 25692 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.083402 26159 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:36.083058 25881 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:36.064296 25926 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:23:36.083280 25963 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.119771 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.128311 26078 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:35.891314 25699 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.181s
I20260709 12:23:36.133123 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:36.082607 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.143782 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.170032 20732 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.171204 26233 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:36.172699 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:36.170678 20883 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 18 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:23:36.105278 25923 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.135859 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.173866 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.174031 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:36.136657 20727 raft_consensus.cc:2376] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15. Re-sending same reply.
W20260709 12:23:36.178624 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83070.1us
I20260709 12:23:36.184862 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.145520 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:36.177228 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 59173.3us
I20260709 12:23:36.172624 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.177897 26236 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:36.189064 26078 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.189291 26236 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.192188 26236 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.192688 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2460) took 1222 ms. Trace:
W20260709 12:23:36.177837 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 277557us
I20260709 12:23:36.193236 20883 rpcz_store.cc:276] 0709 12:23:34.969925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.181459 (+1211534us) service_pool.cc:224] Handling call
0709 12:23:36.192222 (+ 10763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.192399 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.192423 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:36.192462 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:36.192499 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.192546 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:36.192636 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:36.200332 25921 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.201293 26230 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:36.182085 25595 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:23:36.181941 25891 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.203879 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.204334 25891 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.195034 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:36.205278 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.188306 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.205879 21428 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:36.207261 25897 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:36.207675 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.207861 25897 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.209225 25897 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.211290 26188 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:36.211892 26188 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:36.215734 26148 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.179713 26231 raft_consensus.cc:2764] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:23:36.216926 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2444) took 2817 ms. Trace:
I20260709 12:23:36.177472 25927 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:23:36.217334 20882 rpcz_store.cc:276] 0709 12:23:33.399130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.399420 (+   290us) service_pool.cc:224] Handling call
0709 12:23:35.679580 (+2280160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.763491 (+ 83911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:35.763524 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:35.763572 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:35.763608 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.215941 (+452333us) spinlock_profiling.cc:243] Waited 83.7 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:36.216002 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:36.216754 (+   752us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b5400306950. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:36.216870 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5435251968}
I20260709 12:23:36.213066 26188 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.186081 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.221923 26176 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.218899 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:36.232163 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.231282 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.233687 20882 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14.
I20260709 12:23:36.235220 20774 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14.
I20260709 12:23:36.235627 20774 leader_election.cc:403] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:23:36.235962 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63391.4us
I20260709 12:23:36.236431 26137 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:36.236735 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.237102 25907 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:36.237377 25865 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:36.237977 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.239569 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:36.238857 26050 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:36.241261 26050 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:36.241570 26137 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:36.177819 26233 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:36.241991 26137 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:36.242326 26233 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.242467 26137 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.194999 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.246150 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.246455 20883 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:36.246841 21431 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:36.247454 20881 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:36.247609 20731 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:36.249173 20727 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 12:23:36.249326 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.134985 26192 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:36.251699 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2616) took 1790 ms. Trace:
I20260709 12:23:36.253320 20805 rpcz_store.cc:276] 0709 12:23:34.460884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.461408 (+   524us) service_pool.cc:224] Handling call
0709 12:23:34.461717 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.540222 (+ 78505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:34.540258 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:34.540299 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:34.540357 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.251459 (+1711102us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b54003c7154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:36.251513 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:36.251645 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180038528}
I20260709 12:23:36.251861 20806 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:36.134789 26079 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:36.254086 21431 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:36.255105 20882 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:36.244949 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:36.259279 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.260098 20880 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.260453 20809 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:36.257601 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:36.262876 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.263226 20727 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:36.264452 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76238.4us
I20260709 12:23:36.265249 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:36.267535 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.133770 25770 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:36.274863 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:36.276249 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:36.277582 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.278486 20882 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:23:36.262612 25909 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.003s
I20260709 12:23:36.279160 20727 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:36.244611 26176 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.156006 26020 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 12:23:36.252314 26172 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.248556 25891 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.290755 20727 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:36.295643 25963 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.295732 20727 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:36.202940 25781 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.298103 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:36.261289 26079 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:36.271008 26050 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:36.264000 26123 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:36.273677 26231 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:36.280792 25770 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:36.296460 25942 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:36.314203 25942 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:36.314589 25942 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:36.285385 26194 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:36.180575 26095 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.315256 26194 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.316958 26095 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.296541 25699 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.317356 26194 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.317747 21427 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:36.303680 26079 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.317812 25699 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.320037 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.307946 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:36.285009 26078 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:36.322232 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:36.143239 25977 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.289674 26032 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.310401 25975 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.284816 26235 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:23:36.311578 20882 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:36.297112 25909 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.331718 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2465) took 1360 ms. Trace:
I20260709 12:23:36.332100 20882 rpcz_store.cc:276] 0709 12:23:34.971410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.291996 (+1320586us) service_pool.cc:224] Handling call
0709 12:23:36.311385 (+ 19389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.331434 (+ 20049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.331632 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:36.333112 25909 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:36.333216 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.334107 20882 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:36.335152 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.312631 25781 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.312920 25692 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.180910 26080 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.303669 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.309412 26050 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:36.325646 20727 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:36.347900 26050 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:36.326571 25404 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:36.327042 25671 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:36.330122 20880 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:36.297889 26159 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.312026 26231 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:36.361510 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1320) took 1608 ms. Trace:
I20260709 12:23:36.340495 26235 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:36.328449 26132 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:36.334811 25897 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.324815 26123 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:36.338920 26078 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:36.370646 20727 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:36.371522 20877 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:36.351603 20876 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:36.351775 21432 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:36.353068 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 915) took 1350 ms. Trace:
W20260709 12:23:36.339170 26216 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.004s
I20260709 12:23:36.309662 20883 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:36.357889 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:36.358749 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 937) took 1615 ms. Trace:
I20260709 12:23:36.358042 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.361937 21433 rpcz_store.cc:276] 0709 12:23:34.753407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.779706 (+ 26299us) service_pool.cc:224] Handling call
0709 12:23:36.361316 (+1581610us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400750154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:36.361449 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882183424}
I20260709 12:23:36.369891 20732 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:36.373301 20807 rpcz_store.cc:276] 0709 12:23:35.002153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.004193 (+  2040us) service_pool.cc:224] Handling call
0709 12:23:35.004486 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.133728 (+129242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.352818 (+1219090us) spinlock_profiling.cc:243] Waited 84.9 ms on lock 0x7b540027ae54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:36.353000 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195300096}
I20260709 12:23:36.353557 25404 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:36.374193 21431 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:36.375092 20879 rpcz_store.cc:276] 0709 12:23:34.743694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.229919 (+486225us) service_pool.cc:224] Handling call
0709 12:23:35.249686 (+ 19767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.252782 (+  3096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.358552 (+1105770us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b54001fde50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:36.358693 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44779776}
W20260709 12:23:36.347931 26224 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.536s	user 0.004s	sys 0.000s
I20260709 12:23:36.304183 26172 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.368427 26132 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.412441 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:36.423602 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:36.369217 26235 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:36.387637 20880 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:23:36.449132 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2469) took 1375 ms. Trace:
W20260709 12:23:36.394162 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82814us
I20260709 12:23:36.374548 25671 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:36.449571 20880 rpcz_store.cc:276] 0709 12:23:35.074076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.384899 (+1310823us) service_pool.cc:224] Handling call
0709 12:23:36.386686 (+  1787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:36.448863 (+ 62177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.449077 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:36.443926 21435 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:23:36.444526 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:36.451278 21435 pending_rounds.cc:85] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 8
I20260709 12:23:36.413693 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:36.373891 26159 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.451085 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:36.428400 20879 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:36.451622 21435 pending_rounds.cc:107] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:23:36.454698 21435 raft_consensus.cc:2889] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:36.424674 25404 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:36.442253 26132 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.385980 26199 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.455348 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.456719 21434 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:36.393543 26080 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.388365 25897 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.460074 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:36.384091 25692 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:36.405380 25699 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.462622 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2428) took 1385 ms. Trace:
I20260709 12:23:36.449839 26032 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:36.378620 26233 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.459298 26020 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:36.391829 26078 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.384835 26123 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.460711 25671 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:36.501880 20879 rpcz_store.cc:276] 0709 12:23:35.077731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.427839 (+1350108us) service_pool.cc:224] Handling call
0709 12:23:36.428175 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.461730 (+ 33555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.462567 (+   837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:36.492192 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2401) took 1045 ms. Trace:
I20260709 12:23:36.378995 26216 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.509882 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.509651 20729 rpcz_store.cc:276] 0709 12:23:35.446312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.449280 (+  2968us) service_pool.cc:224] Handling call
0709 12:23:35.449756 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:35.464542 (+ 14786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:35.464575 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:35.478153 (+ 13578us) log.cc:844] Serialized 26 byte log entry
0709 12:23:35.583578 (+105425us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:35.583644 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.583784 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:36.347344 (+763560us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:36.491840 (+144496us) raft_consensus.cc:1685] finished
0709 12:23:36.491870 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:36.492105 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14310,"raft.queue_time_us":139056,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"spinlock_wait_cycles":23701376,"thread_start_us":76988,"threads_started":1,"wal-append.queue_time_us":303528}
I20260709 12:23:36.517920 20879 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: [13.7-17.8]
I20260709 12:23:36.518920 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2432) took 1411 ms. Trace:
W20260709 12:23:36.492758 26241 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:23:36.519399 20879 rpcz_store.cc:276] 0709 12:23:35.107336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.517071 (+1409735us) service_pool.cc:224] Handling call
0709 12:23:36.517562 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:36.518537 (+   975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.518839 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:36.531020 25832 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:36.532378 25832 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:36.532823 25832 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:36.493920 26192 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:36.503525 26020 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.471799 26218 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.464555 25926 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:23:36.390915 25881 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.537328 25933 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
W20260709 12:23:36.542651 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 74539.8us
I20260709 12:23:36.470391 25942 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.499401 26080 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:23:36.500168 25770 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.509172 25671 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:36.513306 25692 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:36.496424 26231 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:23:36.499955 26186 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.507169 25841 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:23:36.534552 25832 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.491614 26246 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:23:36.537298 26192 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:36.551517 25933 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:36.541352 26020 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.564162 25942 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:36.547600 25975 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.480078 26199 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.508854 26235 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.569406 26216 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.569530 26192 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.571072 26192 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.573017 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.575301 26199 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.553877 25671 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.559096 26032 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.556629 25977 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.565291 25933 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:36.588353 26141 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:36.593993 25832 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:36.629249 26221 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.825s	user 0.004s	sys 0.000s
I20260709 12:23:36.606762 26141 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:36.631037 26244 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:23:36.631409 26141 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.633029 26233 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:36.633749 26141 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:36.636269 26172 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.638151 26186 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.643324 26079 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.646198 26079 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:36.646009 26020 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.647691 25865 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14.
I20260709 12:23:36.648427 26137 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.649531 26132 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.594278 26032 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.651561 26137 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.651870 26132 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:36.652424 26133 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.784s	user 0.010s	sys 0.002s
I20260709 12:23:36.653250 26032 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.653074 26133 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.655429 25933 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.656265 26133 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:36.656395 26050 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.656822 25933 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:36.658667 25897 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:36.659091 25897 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:36.660410 26050 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:36.660017 25897 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.661291 25897 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:36.665345 26020 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:36.700618 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 505412us
I20260709 12:23:36.704310 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.709962 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:36.551635 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2400) took 1114 ms. Trace:
I20260709 12:23:36.806983 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:36.808522 25921 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:23:36.811685 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 101946us
I20260709 12:23:36.812157 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.813613 21435 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:36.814731 25603 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:23:36.713097 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 539115us
W20260709 12:23:36.714174 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 527252us
I20260709 12:23:36.715059 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2407) took 1266 ms. Trace:
I20260709 12:23:36.719879 20804 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:36.727883 20809 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [13.8-14.9]
I20260709 12:23:36.731612 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:36.746088 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.748431 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2435) took 1601 ms. Trace:
I20260709 12:23:36.748464 20882 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:36.748520 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2425) took 2424 ms. Trace:
I20260709 12:23:36.749313 20878 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:36.767683 21428 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:36.768168 25891 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:36.779667 25686 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:36.780305 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:36.817795 25603 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:23:36.808598 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:36.852854 25770 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:36.611318 25692 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:36.858114 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.858920 25891 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:36.806613 20731 rpcz_store.cc:276] 0709 12:23:35.437717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.441764 (+  4047us) service_pool.cc:224] Handling call
0709 12:23:36.247436 (+805672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.247975 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.551413 (+303438us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54003b4050. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:36.551567 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1846001536}
I20260709 12:23:36.860679 25891 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.858968 26174 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:36.863787 26174 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:36.824723 20875 rpcz_store.cc:276] 0709 12:23:34.324346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.227864 (+903518us) service_pool.cc:224] Handling call
0709 12:23:35.228207 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:35.228619 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:35.228657 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:35.229122 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:23:36.361949 (+1132827us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b1800381460. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:36.459257 (+ 97308us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:36.459297 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.459366 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:36.747750 (+288384us) raft_consensus.cc:1685] finished
0709 12:23:36.747801 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:36.748309 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2858,"raft.queue_time_us":2076,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6967,"spinlock_wait_cycles":6709120,"thread_start_us":97120,"threads_started":1,"wal-append.queue_time_us":1277241}
I20260709 12:23:36.865080 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.866678 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.867156 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:36.865090 26174 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:36.855309 21426 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:36.870034 26174 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:36.871563 25865 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:36.871563 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:36.873039 25865 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:36.873499 25865 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.873939 25865 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.823348 20733 rpcz_store.cc:276] 0709 12:23:35.448030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.192321 (+744291us) service_pool.cc:224] Handling call
0709 12:23:36.192566 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.192755 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:36.193069 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 12:23:36.713703 (+520634us) log.cc:844] Serialized 14 byte log entry
0709 12:23:36.714088 (+   385us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:36.714172 (+    84us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:36.714269 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.714425 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:36.714708 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366934,"prepare.queue_time_us":60483,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":584894,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":256,"thread_start_us":152090,"threads_started":1,"wal-append.queue_time_us":194369}
W20260709 12:23:36.876766 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.878954 26216 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:36.880616 20805 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:36.882133 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:36.883482 26237 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:36.885098 26148 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:36.885587 26148 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:36.892566 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:36.894901 26229 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.896363 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.897619 20875 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:36.898197 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.896605 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:36.915972 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:36.911777 26216 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:36.824239 20879 rpcz_store.cc:276] 0709 12:23:35.146907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.526321 (+1379414us) service_pool.cc:224] Handling call
0709 12:23:36.526705 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:36.527070 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.527114 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:36.527799 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 12:23:36.543297 (+ 15498us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:36.543330 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.543400 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:36.747768 (+204368us) raft_consensus.cc:1685] finished
0709 12:23:36.747805 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:36.748159 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36191,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":13601,"thread_start_us":15130,"threads_started":1,"wal-append.queue_time_us":30177}
I20260709 12:23:36.886927 26148 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:36.956717 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:36.959244 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2368) took 1767 ms. Trace:
I20260709 12:23:36.959743 20803 rpcz_store.cc:276] 0709 12:23:35.191261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.191382 (+   121us) service_pool.cc:224] Handling call
0709 12:23:35.191668 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.028921 (+837253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.958762 (+929841us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54003d68d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:36.959037 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533199232}
I20260709 12:23:36.961119 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:36.962045 20803 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:36.963861 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:36.964870 20810 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:36.966228 20809 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:23:36.968398 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 938) took 2224 ms. Trace:
I20260709 12:23:36.968966 20884 rpcz_store.cc:276] 0709 12:23:34.743974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.236624 (+492650us) service_pool.cc:224] Handling call
0709 12:23:35.236942 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.986980 (+750038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.967746 (+980766us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b540005ad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:36.968156 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673890560}
I20260709 12:23:36.970201 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:36.971201 20807 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:36.946306 26229 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.976518 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.977614 20884 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.979347 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.980315 20879 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:36.981048 25927 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:36.981488 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:36.983090 20879 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.986833 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:36.995543 20881 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:36.997044 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:36.999141 25770 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:37.001255 25770 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.001677 25770 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.003800 20880 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:37.000758 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2476) took 1338 ms. Trace:
I20260709 12:23:36.981488 25927 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:37.006142 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2429) took 1925 ms. Trace:
I20260709 12:23:37.006865 25927 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.009773 25927 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.000799 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:37.006593 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:37.012982 26237 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.013427 26237 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.013592 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2638) took 1690 ms. Trace:
I20260709 12:23:37.013868 26237 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.014022 20806 rpcz_store.cc:276] 0709 12:23:35.322610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.450739 (+128129us) service_pool.cc:224] Handling call
0709 12:23:35.451137 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.258345 (+807208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.013113 (+754768us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54002b70d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:37.013423 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1841001728}
I20260709 12:23:36.986044 26216 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.005821 20878 rpcz_store.cc:276] 0709 12:23:35.662628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.119404 (+456776us) service_pool.cc:224] Handling call
0709 12:23:37.000504 (+881100us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54001fed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:37.000685 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1319789824}
I20260709 12:23:37.016978 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:37.017998 20878 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:23:37.017902 26230 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.018635 26230 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:37.019753 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 455278us
I20260709 12:23:37.021099 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.006573 20880 rpcz_store.cc:276] 0709 12:23:35.080259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.450622 (+1370363us) service_pool.cc:224] Handling call
0709 12:23:37.003385 (+552763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.004385 (+  1000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.005901 (+  1516us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54001fed50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:37.006065 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1270207872}
I20260709 12:23:37.022768 26250 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:37.025190 26130 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.025367 26187 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:37.026899 26187 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:37.027139 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 928) took 1122 ms. Trace:
I20260709 12:23:37.027365 26187 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:37.027498 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:37.027644 20804 rpcz_store.cc:276] 0709 12:23:35.910784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.060092 (+149308us) service_pool.cc:224] Handling call
0709 12:23:36.060523 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.834229 (+773706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.026851 (+192622us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54003b7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:37.027058 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514909440}
I20260709 12:23:37.029580 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:37.021894 26188 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.021011 26229 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:37.051837 20875 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:37.061415 26229 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.064637 26229 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:37.071358 25404 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.009s
I20260709 12:23:37.072046 25404 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.073416 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 945) took 2136 ms. Trace:
I20260709 12:23:37.073819 20877 rpcz_store.cc:276] 0709 12:23:34.937192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.286119 (+348927us) service_pool.cc:224] Handling call
0709 12:23:35.286429 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.371949 (+1085520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.072936 (+700987us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400171e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:37.073224 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2495010176}
I20260709 12:23:37.075560 20880 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:37.083086 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.089849 20805 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:37.084797 25404 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.092110 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:37.096060 25881 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.092128 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 944) took 2175 ms. Trace:
I20260709 12:23:37.113278 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:37.114838 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2438) took 1892 ms. Trace:
W20260709 12:23:37.115309 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 407691us
I20260709 12:23:37.146742 20880 rpcz_store.cc:276] 0709 12:23:35.222183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.025877 (+1803694us) service_pool.cc:224] Handling call
0709 12:23:37.026717 (+   840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.077729 (+ 51012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.114468 (+ 36739us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:37.114713 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111729024}
I20260709 12:23:37.116783 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.028554 26187 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:37.122737 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 226403us
I20260709 12:23:37.118786 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:37.141068 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.145404 20884 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:37.159477 20726 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:37.161656 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 924) took 1265 ms. Trace:
I20260709 12:23:37.162312 20801 rpcz_store.cc:276] 0709 12:23:35.896134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.899169 (+  3035us) service_pool.cc:224] Handling call
0709 12:23:36.008959 (+109790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.019366 (+ 10407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.161125 (+1141759us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540027b850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:37.161420 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251618176}
I20260709 12:23:37.163929 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.165674 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:37.167207 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.168160 20802 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 12: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:37.169032 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:37.173161 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.173022 21428 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:37.174741 25897 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:37.175552 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:37.178205 25836 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:37.179097 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:37.180404 20809 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:37.181427 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.182746 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.183476 20804 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:37.185019 26195 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:37.185480 26195 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.185814 26195 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:37.187443 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60808.3us
I20260709 12:23:37.188589 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:37.189123 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2473) took 1754 ms. Trace:
I20260709 12:23:37.189538 20882 rpcz_store.cc:276] 0709 12:23:35.434762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.188002 (+1753240us) service_pool.cc:224] Handling call
0709 12:23:37.188361 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.188797 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.188824 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.188864 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:37.188891 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.188944 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.189046 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.190367 25699 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:37.190965 25699 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:37.191481 25699 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:37.165825 26236 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:37.146487 20876 rpcz_store.cc:276] 0709 12:23:34.916653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.253652 (+336999us) service_pool.cc:224] Handling call
0709 12:23:35.253971 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.372434 (+1118463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.091670 (+719236us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540005b754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:37.091948 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523789696}
I20260709 12:23:37.197753 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.185689 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.202142 20877 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:37.203696 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:37.174866 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:37.197611 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.079651 25770 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.218858 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2440) took 1846 ms. Trace:
I20260709 12:23:37.223842 20877 rpcz_store.cc:276] 0709 12:23:35.372208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.074839 (+1702631us) service_pool.cc:224] Handling call
0709 12:23:37.076140 (+  1301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.218592 (+142452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.218797 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.228268 20807 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
W20260709 12:23:37.234398 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59715.2us
I20260709 12:23:37.235847 20803 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:37.196218 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.587s (SENT) [suppressed 13 similar messages]
I20260709 12:23:37.239952 20875 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:37.241293 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2439) took 1927 ms. Trace:
I20260709 12:23:37.242206 20875 rpcz_store.cc:276] 0709 12:23:35.313840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.058593 (+1744753us) service_pool.cc:224] Handling call
0709 12:23:37.128830 (+ 70237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.240663 (+111833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.241202 (+   539us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.224774 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2475) took 1679 ms. Trace:
I20260709 12:23:37.243566 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:37.248899 20882 rpcz_store.cc:276] 0709 12:23:35.525768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.190517 (+1664749us) service_pool.cc:224] Handling call
0709 12:23:37.190942 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.203414 (+ 12472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.203458 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.203677 (+   219us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:37.204135 (+   458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.204427 (+   292us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.204940 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:37.263870 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50778.3us
I20260709 12:23:37.200551 26236 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:37.267359 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.266914 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.270491 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:37.287545 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 124326us
I20260709 12:23:37.288169 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:37.288883 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.173076 26235 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:37.197185 25865 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.294301 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
W20260709 12:23:37.298233 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.263782 25603 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:37.275466 25897 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:37.299801 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 77585.4us
I20260709 12:23:37.269616 25595 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:23:37.306770 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2405) took 1688 ms. Trace:
I20260709 12:23:37.307132 20876 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:37.311820 26123 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:37.312337 26123 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.312681 26123 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:37.313359 25865 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.316452 21431 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:37.318747 20805 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:37.295135 25836 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:23:37.320202 25836 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:37.320542 25836 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:37.321698 20881 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:37.321616 25836 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.287220 25909 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:37.323414 25909 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:37.323694 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 65422.9us
I20260709 12:23:37.324481 25909 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.325736 25909 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:37.326035 26218 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:37.327078 26218 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:37.327131 26257 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:37.327462 25603 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.328106 25404 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:37.328320 26257 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:37.328763 25404 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.328997 26257 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.329993 25603 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.330819 25404 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:37.331378 25404 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:23:37.331959 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.333300 26020 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:37.335314 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:37.335579 26095 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:37.335961 21431 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.336139 26095 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:37.338104 26148 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.338464 20806 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:23:37.327662 26218 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:37.339883 20806 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:23:37.331682 25404 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:37.346815 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:37.348021 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75885.8us
I20260709 12:23:37.341321 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.345170 20733 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:37.351735 20802 raft_consensus.cc:2368] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:37.347077 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2443) took 1135 ms. Trace:
I20260709 12:23:37.300902 26187 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.356544 20876 rpcz_store.cc:276] 0709 12:23:36.211828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.234864 (+1023036us) service_pool.cc:224] Handling call
0709 12:23:37.284858 (+ 49994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.310067 (+ 25209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.347021 (+ 36954us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.356945 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:37.361114 20809 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:37.359158 25404 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.363950 25404 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.364684 25865 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:37.365633 25865 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:37.366532 25865 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:37.366871 25865 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:37.367300 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.335656 25897 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:37.368105 26200 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:37.368528 26200 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:37.368364 25897 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.323087 25836 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:37.371070 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.374311 21431 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:37.382027 20808 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:37.382777 20804 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:37.370232 26200 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:37.385589 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.389614 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.396663 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:37.396986 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.384028 26236 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.400267 20882 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:37.395874 26200 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:37.401055 26236 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:37.308750 20730 rpcz_store.cc:276] 0709 12:23:35.447496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.180732 (+733236us) service_pool.cc:224] Handling call
0709 12:23:36.195893 (+ 15161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:36.715590 (+519697us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:36.716173 (+   583us) log.cc:844] Serialized 14 byte log entry
0709 12:23:36.910616 (+194443us) log.cc:844] Serialized 14 byte log entry
0709 12:23:37.088678 (+178062us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.089264 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.089422 (+   158us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:37.089455 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.089558 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:37.125324 (+ 35766us) raft_consensus.cc:1685] finished
0709 12:23:37.125343 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.135957 (+ 10614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":588680,"prepare.queue_time_us":100267,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":92,"raft.queue_time_us":706993,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":19118,"spinlock_wait_cycles":5376,"thread_start_us":300513,"threads_started":2,"wal-append.queue_time_us":180589}
I20260709 12:23:37.343762 26257 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:37.402546 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.354755 26218 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.401289 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 954) took 1121 ms. Trace:
I20260709 12:23:37.412359 20882 rpcz_store.cc:276] 0709 12:23:36.280159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.281130 (+1000971us) service_pool.cc:224] Handling call
0709 12:23:37.353563 (+ 72433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.401037 (+ 47474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.401235 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.401192 25770 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:37.458633 25933 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.698s	user 0.007s	sys 0.014s
I20260709 12:23:37.461557 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:37.463717 26218 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:37.465127 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2404) took 2017 ms. Trace:
I20260709 12:23:37.468209 26237 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.470677 26237 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:37.471978 20880 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:23:37.472709 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2452) took 1152 ms. Trace:
I20260709 12:23:37.473026 20880 rpcz_store.cc:276] 0709 12:23:36.320508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.411967 (+1091459us) service_pool.cc:224] Handling call
0709 12:23:37.471660 (+ 59693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.472452 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.472656 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.474531 20880 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:23:37.474680 20875 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:37.474879 20880 pending_rounds.cc:85] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 8
I20260709 12:23:37.475971 25865 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.477465 25865 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:37.479278 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.465524 20728 rpcz_store.cc:276] 0709 12:23:35.447226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.819795 (+372569us) service_pool.cc:224] Handling call
0709 12:23:35.908967 (+ 89172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:35.909403 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:36.909836 (+1000433us) spinlock_profiling.cc:243] Waited 7.58 ms on lock 0x7b18003acde0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:23:37.075335 (+165499us) log.cc:844] Serialized 14 byte log entry
0709 12:23:37.291784 (+216449us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.405274 (+113490us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.405484 (+   210us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:37.405529 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.424024 (+ 18495us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:37.463763 (+ 39739us) raft_consensus.cc:1685] finished
0709 12:23:37.463887 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.464784 (+   897us) spinlock_profiling.cc:243] Waited 88.8 ms on lock 0x7b540028cb50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:37.464928 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15722,"prepare.queue_time_us":101865,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":27375,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":10871,"spinlock_wait_cycles":221708928,"thread_start_us":337328,"threads_started":2,"wal-append.queue_time_us":270458}
I20260709 12:23:37.473042 20881 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:37.352643 26095 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.487097 25923 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.491582 25923 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:37.491976 25923 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.492760 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.493356 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.493956 20806 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.494079 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.454514 25921 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:37.495415 26032 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:23:37.495981 20805 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:37.407323 26216 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:37.498121 25671 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.898s	user 0.009s	sys 0.005s
I20260709 12:23:37.498414 26095 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:37.498579 25671 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.501319 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.502424 20806 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:37.502748 25404 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:37.506656 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:37.507742 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 234654us
W20260709 12:23:37.488077 26264 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:23:37.488263 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63614.7us
I20260709 12:23:37.508937 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.509366 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.488528 26020 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:23:37.510341 26172 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:37.510788 26172 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:37.511854 26172 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:37.489339 26272 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 12:23:37.516131 20883 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:37.507931 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.517688 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.518625 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 947) took 2330 ms. Trace:
I20260709 12:23:37.487216 25595 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.519906 25595 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:37.522176 26230 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
I20260709 12:23:37.507925 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.523162 20803 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:37.524008 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.525084 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.513453 26172 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.491043 25907 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:37.458650 25692 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:37.527410 25907 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.527724 26195 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.528040 25907 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:37.528074 25692 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:37.529242 25692 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.530053 26195 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:37.488849 26263 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 12:23:37.531312 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2454) took 1079 ms. Trace:
W20260709 12:23:37.532392 26188 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 12:23:37.542204 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.532171 20875 rpcz_store.cc:276] 0709 12:23:36.401179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.419429 (+1018250us) service_pool.cc:224] Handling call
0709 12:23:37.474413 (+ 54984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.475097 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.480072 (+  4975us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:37.550154 20879 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.003s
I20260709 12:23:37.551234 20808 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:37.518986 20883 rpcz_store.cc:276] 0709 12:23:35.187990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.574083 (+1386093us) service_pool.cc:224] Handling call
0709 12:23:36.574385 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.516594 (+942209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.518353 (+  1759us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:37.518558 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164870528}
I20260709 12:23:37.565356 20879 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:37.482790 20880 pending_rounds.cc:107] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:23:37.575161 20880 raft_consensus.cc:2889] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:37.579031 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:37.544211 20804 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:37.585111 20878 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:37.585376 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44884.4us
W20260709 12:23:37.586041 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.586390 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.605619 20804 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:23:37.522742 26230 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.608922 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.551116 26216 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:37.625243 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 116475us
I20260709 12:23:37.626340 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:37.627532 25686 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:37.635277 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:37.637028 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:37.631824 20883 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.638473 26229 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:37.643741 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 134454us
I20260709 12:23:37.643842 26229 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.645010 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.645812 26279 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:23:37.646579 25891 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:23:37.638763 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 135598us
I20260709 12:23:37.647853 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:37.650171 20700 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.646150 26229 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.651268 26237 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:37.651955 26237 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:37.654699 26229 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:37.655905 26229 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:37.656246 26229 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:37.657043 26229 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.658288 26229 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.658082 26123 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.608337 25770 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:37.663823 26216 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:37.665158 25927 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.655s	user 0.008s	sys 0.006s
I20260709 12:23:37.664826 25923 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.667253 26230 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.629180 25881 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:37.668511 26192 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.669124 26192 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.669566 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:37.643141 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.687769 20806 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:23:37.673584 20809 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:37.614688 26188 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.701423 26188 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.681556 20801 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
W20260709 12:23:37.714618 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.714622 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 253118us
I20260709 12:23:37.691592 25671 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.695045 26250 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:37.692406 25881 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:37.722945 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75188.2us
I20260709 12:23:37.694621 25770 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:37.740826 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.696746 26230 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:23:37.720877 25686 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:37.762794 20805 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
W20260709 12:23:37.763247 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.781237 20810 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:23:37.781286 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 110215us
W20260709 12:23:37.787758 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 306062us
I20260709 12:23:37.746997 26192 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:37.796437 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:37.798964 25923 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:37.823457 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.836203 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:37.826886 25692 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.807914 26218 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:23:37.838919 25692 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.843317 25692 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:37.851891 26278 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:37.852576 20629 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 4 to 12. New cstate: current_term: 12 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:37.853688 26188 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:37.856935 25595 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:37.840965 25881 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:37.862200 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:37.863045 20875 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:37.866063 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:37.837673 26187 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:23:37.868155 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:37.869352 25832 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:37.869390 20875 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:23:37.869818 25832 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:37.829041 26123 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:37.870828 25832 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.884395 25832 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:37.871134 25595 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:37.885572 25595 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:37.872903 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.873808 26280 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:37.876993 25907 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.878468 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:37.905640 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.816665 25686 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:37.902448 20883 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
W20260709 12:23:37.911927 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.914503 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:37.916728 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.917162 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.917382 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:37.920902 20802 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 11: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:37.925565 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 292601us
I20260709 12:23:37.925639 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:37.928637 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:37.934567 20803 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:37.947889 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.947319 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:37.950915 21428 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
I20260709 12:23:37.955241 21428 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:23:37.926862 25923 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:37.923533 26236 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.961805 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.970903 20807 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:37.968617 20808 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
W20260709 12:23:37.969074 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.950464 25595 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:37.970978 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:37.976209 20805 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:37.977834 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2437) took 2766 ms. Trace:
I20260709 12:23:37.952647 26216 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:37.978241 20878 rpcz_store.cc:276] 0709 12:23:35.211517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.018782 (+1807265us) service_pool.cc:224] Handling call
0709 12:23:37.574724 (+555942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.602813 (+ 28089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.977648 (+374835us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540021cad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:37.977776 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241721088}
W20260709 12:23:37.980904 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52356.8us
I20260709 12:23:37.956548 26192 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:37.954244 26280 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:37.977407 25595 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:37.988798 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:37.989384 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:37.990247 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:37.992132 20878 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:37.997587 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.991648 20875 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:38.009982 20726 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:37.950384 26172 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:23:37.971710 25923 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:38.011967 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 139194us
W20260709 12:23:38.012893 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.030140 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:38.030465 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:38.036715 25595 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.037894 20875 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:23:38.068235 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.038300 20727 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:38.043498 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:38.005292 25686 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:38.048254 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2391) took 1079 ms. Trace:
W20260709 12:23:38.052053 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.052963 20878 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:38.010094 26123 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:38.055687 26282 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:23:38.059929 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:38.065577 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:38.071573 20810 rpcz_store.cc:276] 0709 12:23:36.968375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.234953 (+266578us) service_pool.cc:224] Handling call
0709 12:23:38.048003 (+813050us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:38.048185 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153976704}
I20260709 12:23:38.077417 20726 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:37.979239 26236 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.029273 25923 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.079983 20728 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 12:23:38.081458 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71337.3us
I20260709 12:23:38.010226 26216 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:38.088313 20734 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:38.090072 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2459) took 1219 ms. Trace:
I20260709 12:23:38.085058 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:38.091691 20730 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:38.094192 20735 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:38.093482 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:38.099401 20731 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:38.084105 25671 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:38.088768 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2460) took 1103 ms. Trace:
I20260709 12:23:38.100018 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:38.105988 26020 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:38.106338 26267 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 12:23:38.107580 26283 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:23:38.108491 26265 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.404s	user 0.000s	sys 0.003s
I20260709 12:23:38.099453 26188 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:38.109262 26188 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:38.109941 20732 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:38.110613 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2483) took 1211 ms. Trace:
I20260709 12:23:38.110270 26188 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.106189 26280 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:38.070982 26250 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:23:38.115458 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:38.112254 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:38.196913 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 97021.7us
I20260709 12:23:38.197003 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 953) took 1917 ms. Trace:
I20260709 12:23:38.200198 20729 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:23:38.206013 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 101325us
I20260709 12:23:38.111157 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:38.209529 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.207026 20732 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:38.210566 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:38.212534 20878 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 12:23:38.106657 26020 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:38.214951 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.197598 20877 rpcz_store.cc:276] 0709 12:23:36.279788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.281063 (+1001275us) service_pool.cc:224] Handling call
0709 12:23:37.376925 (+ 95862us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:37.504867 (+127942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.504895 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:37.505475 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 12:23:37.676819 (+171344us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:37.676868 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.676944 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:38.112969 (+436025us) raft_consensus.cc:1685] finished
0709 12:23:38.112996 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.196915 (+ 83919us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22679,"raft.queue_time_us":23960,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":170648,"threads_started":1,"wal-append.queue_time_us":422786}
I20260709 12:23:38.216259 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:38.111066 20876 rpcz_store.cc:276] 0709 12:23:36.898991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.469058 (+570067us) service_pool.cc:224] Handling call
0709 12:23:37.469390 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.503444 (+ 34054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.503483 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.504005 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.760492 (+256487us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:37.760549 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.760646 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:38.054854 (+294208us) spinlock_profiling.cc:243] Waited 9.35 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:38.110293 (+ 55439us) raft_consensus.cc:1685] finished
0709 12:23:38.110320 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.110548 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33450,"raft.queue_time_us":33727,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":21498624,"thread_start_us":256048,"threads_started":1,"wal-append.queue_time_us":404795}
I20260709 12:23:38.112354 25595 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:38.220389 21428 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:38.227963 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:38.229576 26281 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:23:38.219774 25686 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:38.232225 25686 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:38.233449 25686 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.216820 26218 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.235469 26218 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:38.236907 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 150356us
I20260709 12:23:38.237879 26218 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:38.238595 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:38.240164 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:38.240671 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:38.241984 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:38.242794 21434 raft_consensus.cc:2368] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:38.243036 25770 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:38.244356 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:38.245026 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:38.246003 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2442) took 2034 ms. Trace:
I20260709 12:23:38.246371 20884 rpcz_store.cc:276] 0709 12:23:36.211551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.234009 (+1022458us) service_pool.cc:224] Handling call
0709 12:23:37.343287 (+109278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.537802 (+194515us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:37.791403 (+253601us) log.cc:844] Serialized 14 byte log entry
0709 12:23:38.086503 (+295100us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:38.104870 (+ 18367us) log.cc:844] Serialized 26 byte log entry
0709 12:23:38.105384 (+   514us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:38.105428 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.105528 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:38.245802 (+140274us) raft_consensus.cc:1685] finished
0709 12:23:38.245821 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.245950 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10792,"prepare.queue_time_us":423304,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":43700,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":27095,"thread_start_us":450325,"threads_started":2,"wal-append.queue_time_us":429052}
I20260709 12:23:38.248369 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:38.249123 20884 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:38.249960 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.254837 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.255647 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:38.256361 20879 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.272971 20879 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:23:38.101091 20880 rpcz_store.cc:276] 0709 12:23:36.978632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.667705 (+689073us) service_pool.cc:224] Handling call
0709 12:23:37.668140 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.668433 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:37.813143 (+144710us) log.cc:844] Serialized 14 byte log entry
0709 12:23:38.078951 (+265808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:38.079078 (+   127us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:38.079154 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.079275 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.081623 (+  2348us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":414006,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":158,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4161,"thread_start_us":401542,"threads_started":2}
I20260709 12:23:38.098244 26123 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.280289 26148 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:38.280957 26148 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.281651 26123 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:38.283588 26148 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:38.285110 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71447.3us
I20260709 12:23:38.286427 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:38.277535 20880 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:38.302416 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:38.307168 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68662.7us
I20260709 12:23:38.215405 25836 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:23:38.310202 20809 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
I20260709 12:23:38.313588 20883 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:38.306854 25897 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:38.314792 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 72408.7us
I20260709 12:23:38.224402 25907 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:38.315403 25897 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:38.316026 25907 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:38.316443 25897 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.219111 26230 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 12:23:38.288672 25692 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.329500 20883 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:23:38.330480 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.330796 25671 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:38.359889 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.360689 20809 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:23:38.306977 25909 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:38.390509 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 94380.7us
I20260709 12:23:38.383083 26123 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:38.446173 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.426143 20730 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
W20260709 12:23:38.448916 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56335.6us
I20260709 12:23:38.452380 25909 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:38.453544 25909 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.331284 25671 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.338076 25595 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:38.457171 25595 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.336738 26230 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 12:23:38.460171 25595 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:38.338142 26148 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.123s
W20260709 12:23:38.420673 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.464355 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.396710 20775 leader_election.cc:336] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.974s (SENT)
I20260709 12:23:38.466611 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:38.397908 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:38.450524 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:38.471982 20883 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:38.333930 25699 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.075s	user 0.010s	sys 0.000s
I20260709 12:23:38.399586 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:38.481304 21433 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:38.331054 25692 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:38.493544 26284 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.494437 26284 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.497579 26284 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:38.506034 20883 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:38.397969 26123 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.515501 25770 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:38.516600 20727 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:38.518357 26123 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:38.335937 26285 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:38.484652 25595 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.338524 26235 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:38.335515 26187 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.150s
I20260709 12:23:38.602425 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:38.485558 25699 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.520570 26032 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:38.524065 26020 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:38.529177 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:38.628805 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:38.629657 21427 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:38.629657 20883 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:38.090979 20882 rpcz_store.cc:276] 0709 12:23:36.870299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.482195 (+611896us) service_pool.cc:224] Handling call
0709 12:23:37.539420 (+ 57225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.539662 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:37.839430 (+299768us) log.cc:844] Serialized 14 byte log entry
0709 12:23:38.089724 (+250294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:38.089778 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:38.089834 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.089907 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.090019 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62774,"prepare.queue_time_us":486191,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":4297,"raft.queue_time_us":1794,"thread_start_us":427823,"threads_started":2}
I20260709 12:23:38.449015 26229 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.313349 26216 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.635653 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:38.636824 20878 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:23:38.591555 25770 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:38.535533 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:38.643538 20733 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:38.542961 25832 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.645100 25832 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.545868 26192 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:38.647704 26192 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:38.649041 26192 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.650722 26192 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:38.551585 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.651842 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.553484 20631 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 301) took 19631 ms (client timeout 15000 ms). Trace:
I20260709 12:23:38.568739 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:38.573781 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:38.662334 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88689.5us
I20260709 12:23:38.595475 25595 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.391599 26172 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.578531 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:38.680562 20875 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:38.593932 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 307678us
I20260709 12:23:38.594561 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:38.681617 21434 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:38.588541 26279 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.119s	sys 0.283s
W20260709 12:23:38.683132 26279 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.119s	sys 0.284s
I20260709 12:23:38.335366 26195 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.241s
I20260709 12:23:38.684207 25595 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:38.684564 26195 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:38.685310 20877 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:38.625098 21426 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 12:23:38.626649 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.688952 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.661303 20876 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:38.672492 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:38.593997 26285 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:38.584404 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 186576us
I20260709 12:23:38.682569 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:38.683727 25923 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.000s
I20260709 12:23:38.663586 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:38.705207 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:38.706522 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.685019 26195 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:38.709985 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:38.680341 26288 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.667s	user 0.000s	sys 0.005s
W20260709 12:23:38.712574 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.713783 20700 leader_election.cc:400] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:38.714250 20700 leader_election.cc:403] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:38.716202 21428 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:38.668295 25770 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:38.703387 26283 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:23:38.695680 26123 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:38.698807 26236 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:38.639655 26229 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.708434 26218 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:23:38.736719 21434 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:38.713577 26235 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:38.715842 25595 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:38.748111 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:38.748945 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:38.703737 26172 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:38.747174 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64712.8us
I20260709 12:23:38.751152 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:38.710820 25923 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:38.369009 26250 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.745199 26148 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:23:38.764809 21434 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:38.748883 25595 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.764281 26195 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.769207 25923 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:38.777969 21427 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:23:38.746452 26123 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:38.652681 20631 rpcz_store.cc:269] 0709 12:23:18.922297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.982802 (+ 60505us) service_pool.cc:224] Handling call
0709 12:23:38.553181 (+19570379us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:28.853266 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:28.854178 (+   912us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:28.854248 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:23:28.854276 (+    28us) tablet.cc:662] Decoding operations
0709 12:23:28.969117 (+114841us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:28.981388 (+ 12271us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:28.983227 (+  1839us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:23:29.075610 (+ 92383us) tablet.cc:701] Row locks acquired
0709 12:23:29.075661 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:23:29.075853 (+   192us) write_op.cc:270] Start()
0709 12:23:29.076013 (+   160us) write_op.cc:276] Timestamp: P: 1783599809075783 usec, L: 0
0709 12:23:29.076039 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:23:29.076935 (+   896us) log.cc:844] Serialized 7215 byte log entry
0709 12:23:30.293546 (+1216611us) op_driver.cc:464] REPLICATION: finished
0709 12:23:30.970589 (+677043us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1bb950ac 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e226d5ef0
0709 12:23:31.494686 (+524097us) write_op.cc:301] APPLY: starting
0709 12:23:31.515319 (+ 20633us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:31.991632 (+476313us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:31.991675 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:32.282501 (+290826us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:32.282932 (+   431us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:23:32.282982 (+    50us) write_op.cc:312] APPLY: finished
0709 12:23:32.354779 (+ 71797us) log.cc:844] Serialized 164 byte log entry
0709 12:23:32.417632 (+ 62853us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:32.417841 (+   209us) write_op.cc:454] Released schema lock
0709 12:23:32.417922 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:32.418385 (+   463us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:32.418416 (+    31us) write_op.cc:454] Released schema lock
0709 12:23:32.920198 (+501782us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
Metrics: {"mutex_wait_us":38307,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":1141928,"apply.run_cpu_time_us":39165,"apply.run_wall_time_us":947153,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79367,"num_ops":18,"prepare.queue_time_us":412262,"prepare.run_cpu_time_us":30423,"prepare.run_wall_time_us":439273,"raft.queue_time_us":721085,"raft.run_cpu_time_us":75484,"raft.run_wall_time_us":1203304,"replication_time_us":1176648,"spinlock_wait_cycles":127116288,"thread_start_us":2042295,"threads_started":5,"wal-append.queue_time_us":322282,"wal-append.run_cpu_time_us":115586,"wal-append.run_wall_time_us":3542245}]]}
I20260709 12:23:38.786857 21434 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:38.763859 26235 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:38.785441 25595 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.798642 25832 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:38.802129 26229 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:38.792780 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2487) took 1372 ms. Trace:
I20260709 12:23:38.808694 20875 rpcz_store.cc:276] 0709 12:23:37.420573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.076514 (+655941us) service_pool.cc:224] Handling call
0709 12:23:38.087670 (+ 11156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:38.681152 (+593482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.792538 (+111386us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b540019cdd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:38.792721 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334318848}
I20260709 12:23:38.809326 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:38.810398 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 346642us
W20260709 12:23:38.831138 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.847621 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 450907us
I20260709 12:23:38.784412 25671 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.805847 25836 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:38.808533 26195 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:38.808864 25770 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:38.847695 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43585.1us
W20260709 12:23:38.891362 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.788383 26285 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:38.867136 26188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.003s	sys 0.017s
W20260709 12:23:38.906548 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.912626 26296 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.327s	user 0.002s	sys 0.000s
I20260709 12:23:38.913735 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:38.894586 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.894451 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:38.865401 25897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.006s	sys 0.011s
W20260709 12:23:38.844825 26280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.001s	sys 0.011s
I20260709 12:23:38.905037 20803 raft_consensus.cc:1217] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[8.8-15.11]   Dedup: 8.8->[11.9-15.11]
W20260709 12:23:38.844512 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.003s	sys 0.009s
W20260709 12:23:38.937247 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.004s	sys 0.009s
I20260709 12:23:38.937453 26020 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:38.937564 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 599752us
I20260709 12:23:38.932717 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:38.915144 26188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.003s	sys 0.018s
I20260709 12:23:38.865818 25699 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:38.849129 26229 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:38.917106 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:38.941845 20810 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:38.867903 25923 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.847651 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:38.919503 20801 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.920670 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:38.920655 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:38.946171 20808 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:38.924485 21433 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:38.785740 26250 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:38.927902 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:38.931723 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:38.952183 21426 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:38.921445 25897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.007s	sys 0.011s
I20260709 12:23:38.935631 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:38.938957 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:38.938436 26280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.001s	sys 0.011s
I20260709 12:23:38.948869 21427 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:23:38.941569 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 369126us
I20260709 12:23:38.984462 20806 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:38.936859 20803 pending_rounds.cc:85] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 8
I20260709 12:23:38.993182 20803 pending_rounds.cc:107] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:23:38.997308 20803 raft_consensus.cc:2889] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:39.004649 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51873.6us
I20260709 12:23:38.937997 26020 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:39.005620 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:38.991315 26308 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.027041 21432 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:39.011466 25686 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.029090 25686 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.032416 26250 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:39.035383 26148 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
I20260709 12:23:39.035545 26195 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:39.036204 26148 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:39.036713 26148 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.006646 25923 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.040446 26308 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.041755 26229 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.042277 26229 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.042860 26308 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:39.044924 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2493) took 1370 ms. Trace:
I20260709 12:23:39.032240 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:39.050994 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.933406 26123 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:39.066112 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60544.9us
I20260709 12:23:38.996712 26236 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.071669 21431 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:39.058633 25686 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:39.074009 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:39.047758 26188 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:39.057663 20801 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:23:39.048691 20881 rpcz_store.cc:276] 0709 12:23:37.674251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.286842 (+612591us) service_pool.cc:224] Handling call
0709 12:23:38.287326 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.314093 (+ 26767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:38.314143 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:38.370016 (+ 55873us) log.cc:844] Serialized 26 byte log entry
0709 12:23:38.843863 (+473847us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:38.843921 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.843996 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.044719 (+200723us) raft_consensus.cc:1685] finished
0709 12:23:39.044738 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.044866 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6236,"raft.queue_time_us":26430,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"thread_start_us":374846,"threads_started":1,"wal-append.queue_time_us":564472}
I20260709 12:23:39.081998 21431 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:39.088843 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 161050us
W20260709 12:23:39.089811 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.082002 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:39.089529 21431 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.086553 26148 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.092358 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:39.092980 21433 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:39.094070 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.068848 26020 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.095538 25686 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.080715 26310 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.096320 25686 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.102352 21428 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 11.
I20260709 12:23:39.067783 25671 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:39.088362 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:39.070732 26309 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:39.071116 26306 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:23:39.123034 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 179360us
I20260709 12:23:39.003860 25836 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.149008 20733 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:39.149052 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 241185us
I20260709 12:23:39.151154 20728 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:39.027812 26235 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:39.160238 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 237408us
I20260709 12:23:39.160918 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.160984 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.133641 26309 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:39.163117 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 155652us
I20260709 12:23:39.105315 26236 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.169246 20875 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:23:39.170149 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:39.170394 26236 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:39.170668 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.171129 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.171738 20882 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:39.172194 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.172737 20883 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:39.136418 25699 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:39.179344 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:39.188516 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.189337 20728 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:39.191260 20733 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:39.194331 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:39.195003 21428 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:39.194460 26236 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:39.144327 26308 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:23:39.208374 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.209156 20883 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:39.210032 20731 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:39.206938 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:39.175459 26309 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:39.213938 20882 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:39.180194 25699 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.220224 20733 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 12:23:39.223636 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.204957 21400 connection.cc:256] RPC call timeout handler was delayed by -1.4770617110002604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 12:23:39.224292 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.228295 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.137578 26310 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:39.230950 21400 connection.cc:256] RPC call timeout handler was delayed by -1.3089876260000892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20850000000000002s for a response
W20260709 12:23:39.232095 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.196075 26172 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:39.269565 21400 connection.cc:256] RPC call timeout handler was delayed by -1.2300330820003182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
W20260709 12:23:39.269986 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 101941us
W20260709 12:23:39.270267 21400 connection.cc:256] RPC call timeout handler was delayed by -1.0785577210003794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 12:23:39.270815 21400 connection.cc:256] RPC call timeout handler was delayed by -1.0626042630001393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30510000000000004s for a response
I20260709 12:23:39.272444 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.273075 20726 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:39.205572 26306 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.235898 20879 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:39.226953 26195 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:39.230216 25699 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:39.287091 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:39.289101 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.296195 20883 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:39.292419 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.206097 26187 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:39.299988 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.300678 20726 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:39.294195 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:39.303362 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:39.308310 20882 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:39.308408 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.290059 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.309507 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.312204 20879 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:23:39.314034 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.314795 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.315382 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.316283 20726 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
W20260709 12:23:39.292461 26314 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:23:39.300021 26313 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.308444 26216 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:39.323590 26313 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.323863 26216 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.325997 26313 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.326182 26216 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.327855 20728 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:39.322929 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.226508 26032 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:23:39.333019 26032 consensus_queue.cc:1243] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:39.322418 26306 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.338056 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:39.338865 21433 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:39.340479 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2496) took 1447 ms. Trace:
I20260709 12:23:39.341017 20880 rpcz_store.cc:276] 0709 12:23:37.892879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.308607 (+415728us) service_pool.cc:224] Handling call
0709 12:23:38.308888 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.696207 (+387319us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:38.722613 (+ 26406us) log.cc:844] Serialized 14 byte log entry
0709 12:23:39.070786 (+348173us) log.cc:844] Serialized 14 byte log entry
0709 12:23:39.071025 (+   239us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.071523 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.071653 (+   130us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:39.071685 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.071751 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.340209 (+268458us) raft_consensus.cc:1685] finished
0709 12:23:39.340237 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.340412 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15994,"prepare.queue_time_us":18241,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":68,"raft.queue_time_us":41729,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":6704,"spinlock_wait_cycles":16675840,"thread_start_us":657016,"threads_started":2,"wal-append.queue_time_us":529015}
I20260709 12:23:39.294584 26195 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.349407 20876 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
W20260709 12:23:39.350064 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66319.6us
W20260709 12:23:39.350553 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.370510 20733 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:39.313935 26230 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:39.372270 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.373183 20883 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:39.374692 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.237567 26323 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.133s
I20260709 12:23:39.391106 20726 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:39.393541 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.393543 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:39.326575 26313 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:39.394311 20726 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:39.354658 26195 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:39.375360 25595 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
I20260709 12:23:39.401629 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:39.317036 26187 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.353626 26306 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:39.362447 26321 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:23:39.407876 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.392123 25686 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:39.416887 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 96349.1us
I20260709 12:23:39.415757 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.375733 26230 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:39.402904 26192 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.725s	user 0.008s	sys 0.004s
I20260709 12:23:39.401762 26308 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.422042 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:39.426491 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:39.427132 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.422062 20733 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:39.422039 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.429805 20727 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:23:39.430902 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:39.431910 20727 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: [15.15-15.15]
I20260709 12:23:39.435669 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.438755 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.439586 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.440215 20730 raft_consensus.cc:2376] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13. Re-sending same reply.
I20260709 12:23:39.446380 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:39.453308 20730 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:39.458106 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.460070 20730 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:39.462134 26321 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:39.464890 26321 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.466886 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.405718 26313 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:39.421996 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 118717us
I20260709 12:23:39.406370 26280 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:39.473093 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.406193 26200 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:39.487766 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.477639 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:39.517421 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.421856 26230 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:39.533341 20849 connection.cc:256] RPC call timeout handler was delayed by -1.4971996019999096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
W20260709 12:23:39.567173 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.576560 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 221375us
I20260709 12:23:39.592413 25832 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.593281 25832 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:39.612381 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.614264 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:39.627087 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.630578 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.631453 20883 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:39.628111 20732 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:39.472651 26236 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:39.409678 25595 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.481640 26313 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:39.491214 26285 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:39.483665 26192 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:39.645711 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 267240us
I20260709 12:23:39.481837 26280 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:39.643162 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.692057 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.693475 20734 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:23:39.617264 25832 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:39.652369 26316 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 12:23:39.690117 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:39.700387 20729 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:39.701268 21434 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:39.482395 26308 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.643766 26236 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:39.724522 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:39.575116 25770 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.130s
I20260709 12:23:39.696215 26188 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:39.695472 25881 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:39.696026 26280 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:39.689692 26192 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:39.756856 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.757445 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.758275 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:39.698518 26172 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:39.739024 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.687579 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73438.4us
W20260709 12:23:39.773785 26327 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.291s	user 0.000s	sys 0.000s
W20260709 12:23:39.776592 26328 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 12:23:39.728086 26236 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:39.726095 26339 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.806447 20732 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:23:39.808514 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159233us
I20260709 12:23:39.761327 26192 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.812496 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.813383 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:39.815632 20729 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:39.816855 25595 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.827487 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:39.803699 26316 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:23:39.822943 20727 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:39.763084 26172 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:39.763816 26188 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:39.834445 20726 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:39.835785 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:39.836086 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.838943 20730 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:39.845086 21435 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:39.846747 20734 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:39.852128 26188 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.809610 26310 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:39.885686 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142903us
W20260709 12:23:39.862279 26330 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:23:39.865096 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.866863 26200 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.817476 26322 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:39.892376 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.842675 26317 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
W20260709 12:23:39.904556 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.844241 26172 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.845515 26307 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.912235 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:39.915925 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.914805 21431 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.916635 20878 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:39.917388 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:39.922520 21435 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:39.924070 20730 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:39.926026 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:39.855144 26187 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.808827 26339 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.944098 20879 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:39.947163 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:39.940608 21431 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:23:39.951467 26310 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.952070 26310 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:39.955488 26095 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:23:39.955519 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 964) took 1594 ms. Trace:
I20260709 12:23:39.957050 20884 rpcz_store.cc:276] 0709 12:23:38.361050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.539813 (+178763us) service_pool.cc:224] Handling call
0709 12:23:38.540238 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.540773 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:38.586830 (+ 46057us) log.cc:844] Serialized 14 byte log entry
0709 12:23:38.797372 (+210542us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b18002f9380. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:23:39.080846 (+283474us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.091131 (+ 10285us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.091486 (+   355us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:39.091520 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.091804 (+   284us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.948647 (+856843us) raft_consensus.cc:1685] finished
0709 12:23:39.955297 (+  6650us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.955459 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1392,"prepare.queue_time_us":45792,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2177,"raft.queue_time_us":4012,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":20821,"spinlock_wait_cycles":18965760,"thread_start_us":281701,"threads_started":1,"wal-append.queue_time_us":561692}
I20260709 12:23:39.958796 26229 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.947482 26187 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:39.960145 26329 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:23:39.960013 26187 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.926165 26307 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.964600 26307 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.965199 26307 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:39.966966 26334 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:39.950989 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:39.927520 26322 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:39.970850 26322 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:39.970728 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:39.971547 26337 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:39.975757 26310 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:39.988657 20880 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.003s	sys 0.040s
I20260709 12:23:39.965561 26307 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:39.996683 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1356) took 2148 ms. Trace:
W20260709 12:23:40.000110 20880 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.003s	sys 0.041s
I20260709 12:23:40.001207 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.000234 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:39.913110 26200 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:39.937570 26338 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.020604 21431 rpcz_store.cc:276] 0709 12:23:37.801522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.801701 (+   179us) service_pool.cc:224] Handling call
0709 12:23:37.814203 (+ 12502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:39.071582 (+1257379us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54001e71a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca11c0 00007f9e1eca2231 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c
0709 12:23:39.828793 (+757211us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b58005fc3d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:23:39.949125 (+120332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.949396 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077707520}
I20260709 12:23:40.030100 26310 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.030478 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 181796us
W20260709 12:23:40.030675 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64902.5us
I20260709 12:23:40.030851 26310 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.033243 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.041059 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.060688 20883 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:40.061928 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60829.4us
I20260709 12:23:40.034762 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.068415 26352 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:40.068858 26352 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.069240 26352 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.071354 25671 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.072811 26313 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:40.073673 26309 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.804s	user 0.010s	sys 0.008s
I20260709 12:23:40.074151 20731 raft_consensus.cc:2368] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14.
I20260709 12:23:40.074141 26309 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.078001 25671 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:40.013600 26307 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.080291 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:40.069268 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:40.085167 21430 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.094856 26352 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.099344 21430 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:40.100183 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 949) took 1018 ms. Trace:
I20260709 12:23:40.100867 26352 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.100596 21430 rpcz_store.cc:276] 0709 12:23:39.081778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.082683 (+   905us) service_pool.cc:224] Handling call
0709 12:23:40.069132 (+986449us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aeee 000000000036eb39 00007f9e23bee1e0 00007f9e23cb70c4 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:23:40.100119 (+ 30987us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13383936}
I20260709 12:23:40.102705 26320 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:40.033653 26310 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.010810 26123 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:23:40.109709 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:40.026753 26336 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:40.110993 21435 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.055810 26337 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:40.040444 26308 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:40.042129 26200 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:40.129089 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48884.9us
W20260709 12:23:40.112023 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 640787us
I20260709 12:23:40.105050 26313 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:40.108090 26320 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:40.130467 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:40.152247 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.113682 26336 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:40.127351 26235 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:40.034159 26338 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.129110 26310 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.109496 26352 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:23:40.036744 26280 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.114256 26236 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:40.159729 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 117256us
I20260709 12:23:40.152630 26229 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:40.161587 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:40.135115 26331 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:40.090622 26353 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:23:40.175261 26338 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.108587 26172 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.159070 26336 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.174705 25881 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:40.172310 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:40.162524 26308 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.154757 26306 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.180176 21427 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:40.175292 26331 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.176335 26353 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:40.183828 26172 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.185312 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:40.186080 25699 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.870s	user 0.013s	sys 0.007s
I20260709 12:23:40.185979 25933 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:40.195971 26309 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:40.240881 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:40.241758 20884 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.196906 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.197613 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:40.197692 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:40.198531 26357 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:23:40.200217 26230 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.675s	user 0.008s	sys 0.004s
I20260709 12:23:40.204089 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:40.136566 26192 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.207276 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:40.220810 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.170115 26313 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:40.259131 26187 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.223603 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.220940 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.228403 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:40.171358 26235 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:40.263578 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2468) took 1063 ms. Trace:
I20260709 12:23:40.264259 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:40.264147 26235 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.265777 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.266630 20883 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:40.231896 20878 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:40.268710 20878 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:40.237093 20879 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:40.269166 26339 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:40.271152 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 86010.8us
I20260709 12:23:40.272063 20735 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:40.272415 20735 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:40.272822 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.273154 20735 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.273686 26285 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:40.274361 20735 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.274559 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.275415 26187 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.275633 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 968) took 1136 ms. Trace:
I20260709 12:23:40.275852 26285 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:40.276021 20875 rpcz_store.cc:276] 0709 12:23:39.138866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.143490 (+  4624us) service_pool.cc:224] Handling call
0709 12:23:40.275454 (+1131964us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:40.275578 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11632000}
I20260709 12:23:40.277567 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.278461 26200 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:40.279809 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1126) took 1475 ms (client timeout 1599 ms). Trace:
W20260709 12:23:40.280301 21434 rpcz_store.cc:269] 0709 12:23:38.804861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.805128 (+   267us) service_pool.cc:224] Handling call
0709 12:23:40.279528 (+1474400us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b540050c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:40.279707 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048789888}
I20260709 12:23:40.286105 21426 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:40.287424 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 943) took 1279 ms. Trace:
I20260709 12:23:40.287887 21426 rpcz_store.cc:276] 0709 12:23:39.007619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.008918 (+  1299us) service_pool.cc:224] Handling call
0709 12:23:40.285865 (+1276947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:40.286584 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.287182 (+   598us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540050c8d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:40.287346 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2844541312}
I20260709 12:23:40.278944 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:40.289450 25923 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:40.263993 20881 rpcz_store.cc:276] 0709 12:23:39.200067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.211332 (+ 11265us) service_pool.cc:224] Handling call
0709 12:23:39.284060 (+ 72728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:39.284550 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:39.284583 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.292407 (+  7824us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.375910 (+ 83503us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:39.375963 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.376046 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:40.009378 (+633332us) raft_consensus.cc:1685] finished
0709 12:23:40.009410 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:40.263329 (+253919us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b54001fe850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:40.263523 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34977,"raft.queue_time_us":6316,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":14975,"spinlock_wait_cycles":36514560,"thread_start_us":82240,"threads_started":1,"wal-append.queue_time_us":190584}
I20260709 12:23:40.295378 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:40.296979 26349 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:40.297541 26349 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:40.221247 26337 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.225220 26352 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:23:40.179767 26310 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.299686 26200 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:40.300215 26200 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:40.301349 26310 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.301093 26200 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.246429 26320 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.302814 26320 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:40.303953 26352 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:40.242929 20882 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:40.304561 26352 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:40.236305 26353 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.305786 26352 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.244331 25699 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.251170 26192 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:40.254237 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:40.225903 26195 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.000s
I20260709 12:23:40.255157 26123 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:40.255716 25933 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:40.255815 26230 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.255095 26322 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:40.257542 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:40.258286 26172 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.243162 20877 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.219766 25881 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:40.293663 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89694.5us
I20260709 12:23:40.236882 26332 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:40.316927 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:40.298398 26349 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.178228 26229 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.306439 26336 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.256563 26280 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:40.320749 21435 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:40.322330 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43489.7us
I20260709 12:23:40.319538 25909 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.325362 20884 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:40.327921 26352 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:40.329192 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 147870us
I20260709 12:23:40.331880 20876 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:40.233319 26306 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.333628 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.322455 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:40.326020 26313 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:40.315814 25933 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:40.320093 25881 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:40.315659 26123 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:40.331554 25832 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:40.320785 26195 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.327378 26322 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:40.332341 26332 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:40.364464 20877 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:40.366427 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:40.369159 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.342764 26353 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:40.351353 26313 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.348359 26309 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:40.359365 26310 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:40.359158 26236 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:40.342792 26172 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:23:40.385638 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:40.386607 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:40.379009 26313 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:40.393180 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 195635us
I20260709 12:23:40.380388 26200 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:40.383630 26192 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.383563 26310 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.359493 25909 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.412549 26307 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.414870 26307 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.415517 26307 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.415983 26307 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.416695 25909 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:40.383575 26309 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.417853 26307 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.419219 26309 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:40.419713 26364 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:23:40.420469 26332 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.401957 26280 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:23:40.387620 25933 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.424796 26322 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.433456 26123 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:40.434679 26348 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.260s	user 0.002s	sys 0.000s
I20260709 12:23:40.437037 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 944) took 1542 ms. Trace:
I20260709 12:23:40.437516 20803 rpcz_store.cc:276] 0709 12:23:38.894377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.897830 (+  3453us) service_pool.cc:224] Handling call
0709 12:23:38.898351 (+   521us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:23:39.044761 (+146410us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:39.157448 (+112687us) log.cc:844] Serialized 14 byte log entry
0709 12:23:40.302047 (+1144599us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b18002c71c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a
0709 12:23:40.302288 (+   241us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:23:40.307915 (+  5627us) log.cc:844] Serialized 78 byte log entry
0709 12:23:40.308157 (+   242us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:40.308402 (+   245us) log.cc:844] Serialized 14 byte log entry
0709 12:23:40.328651 (+ 20249us) log.cc:844] Serialized 14 byte log entry
0709 12:23:40.328859 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.329016 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:40.431093 (+102077us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b18002c71c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:40.436815 (+  5722us) raft_consensus.cc:1685] finished
0709 12:23:40.436837 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:40.436980 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6283,"prepare.queue_time_us":181482,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":126,"raft.queue_time_us":15400,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":299,"spinlock_wait_cycles":19409920,"thread_start_us":189277,"threads_started":2,"wal-append.queue_time_us":211541}
W20260709 12:23:40.443009 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73197.2us
I20260709 12:23:40.444558 20701 leader_election.cc:400] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15.
W20260709 12:23:40.443758 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79168.4us
I20260709 12:23:40.404616 26310 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:40.448490 20735 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:23:40.454766 25770 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:23:40.408087 26236 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.456840 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.404095 26192 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.462522 26320 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.464866 25699 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.478708 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.478787 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.479661 20808 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:40.480603 20803 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:40.480829 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.479115 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.482442 20809 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:40.483623 20733 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:40.483950 20803 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:40.484290 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2688) took 1015 ms. Trace:
I20260709 12:23:40.484683 20733 rpcz_store.cc:276] 0709 12:23:39.469355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.469534 (+   179us) service_pool.cc:224] Handling call
0709 12:23:39.469878 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:40.483457 (+1013579us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b5400331920. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca11c0 00007f9e1eca2231 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c
0709 12:23:40.484042 (+   585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.484237 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9714560}
I20260709 12:23:40.486099 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.487154 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:40.487339 26323 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:40.487802 21432 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:40.488945 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:40.487516 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.479722 20802 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:40.489614 21428 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:23:40.495694 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93763.5us
I20260709 12:23:40.471081 26123 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.496590 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:40.496635 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 162993us
I20260709 12:23:40.498950 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.499783 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.460899 26236 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.497171 26200 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:23:40.510522 26336 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:40.512259 26331 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:40.499161 26337 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:23:40.516479 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.517233 20803 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:40.518733 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.519475 20803 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:40.520857 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.521492 20803 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.523164 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.523972 20808 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:40.527388 26229 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:40.527587 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.528857 26280 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.529345 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.530673 26280 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:40.537438 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:40.538436 20809 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:40.542564 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:40.543594 20735 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.540041 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.572014 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:40.572759 20730 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.576026 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:40.576392 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 76727us
I20260709 12:23:40.577847 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.539310 26337 consensus_queue.cc:1243] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:23:40.583705 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84834.8us
I20260709 12:23:40.584534 26172 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:40.585804 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.586612 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.587181 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.588100 20802 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:40.587672 20810 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:40.543763 26195 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.592404 20730 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
W20260709 12:23:40.594890 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45724.4us
I20260709 12:23:40.588836 25686 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:40.571897 26188 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:40.599157 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.601949 26370 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:40.602826 26369 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:23:40.605173 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.607484 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.608898 20809 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:40.611943 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:40.608151 20802 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
I20260709 12:23:40.622515 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.579562 26235 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:23:40.614284 25699 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:40.614729 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:40.632754 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54992.8us
I20260709 12:23:40.633307 21435 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:40.633987 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:40.615355 20810 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:23:40.616318 26335 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.730s	user 0.001s	sys 0.003s
I20260709 12:23:40.607429 26188 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.581836 25836 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:40.611157 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.589969 26332 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:23:40.543145 26349 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:23:40.628779 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.665261 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.665970 20804 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:40.638260 25836 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:40.674350 25881 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.693522 26310 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:40.700320 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66431.8us
I20260709 12:23:40.701079 20806 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:40.704234 20805 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:40.642066 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.645623 26313 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.632819 25699 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.649863 26332 consensus_queue.cc:1243] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:40.638175 26188 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:40.659770 26351 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:40.635618 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.709466 20808 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:40.720052 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.710207 20802 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:23:40.710462 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.721565 20876 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:40.721675 20809 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
W20260709 12:23:40.710829 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68871.3us
I20260709 12:23:40.722759 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.712165 21435 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:23:40.713909 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.723038 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.724318 20810 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:40.724257 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.726933 20876 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:40.732180 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:40.732522 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.732765 20877 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:40.721233 20802 pending_rounds.cc:85] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 5
I20260709 12:23:40.733247 20802 pending_rounds.cc:107] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:23:40.733490 20802 raft_consensus.cc:2889] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:40.734591 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.735316 20802 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:40.735857 20884 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:40.736074 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.736588 20802 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.738086 26336 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:40.711328 26339 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:40.743602 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.744736 20803 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:40.744807 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.745744 20807 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:23:40.745839 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.746711 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.738511 26336 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:40.747394 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:40.749168 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.751107 20804 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:40.754882 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.742529 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.756945 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:40.757784 20808 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.758394 20877 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:23:40.760432 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:40.761999 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:40.763264 21434 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:23:40.773856 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.775480 26323 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:40.776378 26323 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:40.776700 26323 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:40.776858 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:40.713512 25770 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.777868 26323 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.780368 25933 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:40.783481 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:40.784253 21435 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:40.741609 25923 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:23:40.785413 25923 consensus_queue.cc:1243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:23:40.785962 25923 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:40.786338 25923 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:40.786039 20802 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:40.787343 26375 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:40.787420 26020 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:40.788030 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.776628 21427 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:40.791638 20802 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:23:40.721055 25699 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:40.795228 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.815382 20802 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:40.825989 20802 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:40.826133 20803 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:40.815937 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.715737 26337 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.827668 26337 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.829807 26337 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:40.817938 26331 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:40.819779 26338 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:40.744524 26352 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:40.832012 26352 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:40.832446 26352 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.835515 26338 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.836282 26338 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.827332 20804 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:23:40.822029 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:40.779047 26323 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.753494 26351 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:40.767030 26313 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:40.788743 26375 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.778628 25770 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.739029 26339 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.811573 26229 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.825148 26020 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:40.885198 20808 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:40.853525 26230 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:40.895460 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 259895us
I20260709 12:23:40.863603 26375 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:40.901846 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.918848 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:40.792865 25923 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.932036 20882 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:40.925738 26373 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.920835 26337 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.927942 26020 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:40.945978 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44246.8us
W20260709 12:23:40.946465 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132693us
I20260709 12:23:40.928006 26381 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:40.931171 26310 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:40.936446 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 225821us
I20260709 12:23:40.937435 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:40.898439 26352 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:40.926115 25671 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:40.962160 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:40.949534 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.941825 26200 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.970747 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:40.971820 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:40.996510 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 112210us
I20260709 12:23:40.999949 21434 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:41.053251 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:41.053956 21427 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:41.055644 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:41.056555 21427 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:40.950351 26373 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.968665 26390 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:40.988436 26310 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:40.988180 26378 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:41.093041 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:41.094457 20775 connection.cc:256] RPC call timeout handler was delayed by -1.145198912000069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 12:23:41.099722 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:41.106765 21431 raft_consensus.cc:2376] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12. Re-sending same reply.
I20260709 12:23:40.994390 26187 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:40.999231 26301 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:41.104281 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:41.132848 21427 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:23:41.109912 20775 leader_election.cc:336] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.658s (SENT)
I20260709 12:23:41.110780 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:40.971117 25671 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:40.967666 26337 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.052652 26309 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:41.099728 26382 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 12:23:41.138036 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:41.146476 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:41.005792 25832 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 12:23:40.990007 26200 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.000818 26195 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 12:23:41.131567 26387 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:41.139981 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 177970us
I20260709 12:23:41.105552 26378 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.167150 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.176892 20733 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:41.154028 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:41.127267 26280 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.101361 26285 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:41.161281 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 461425us
I20260709 12:23:41.135000 26301 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:41.131703 26390 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:41.139678 26216 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:41.167271 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:41.212822 20735 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:41.168222 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:40.959519 26351 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:41.184291 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74426.7us
I20260709 12:23:41.193800 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:41.154975 26230 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:41.195329 26378 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:41.202855 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:41.223109 21431 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:41.162351 26310 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:41.201699 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.224822 20808 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:23:41.225276 20808 pending_rounds.cc:85] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 8
I20260709 12:23:41.225144 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:41.225551 20808 pending_rounds.cc:107] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:23:41.225838 20808 raft_consensus.cc:2889] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:41.213480 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:41.217451 21435 raft_consensus.cc:2368] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12.
W20260709 12:23:41.200209 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46248.9us
I20260709 12:23:40.987293 26381 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.155s
I20260709 12:23:41.004855 26370 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 12:23:41.221452 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:41.147801 26309 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.186228 26308 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:41.207556 26301 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:41.203454 25770 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:41.235884 26313 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:41.236778 26337 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.213461 26339 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:41.237718 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:41.240778 20809 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:41.209899 26391 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:41.245864 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:41.248034 21434 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:23:41.251938 26383 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 12:23:41.254932 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:41.205421 26280 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.258018 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:41.259260 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:41.229215 26322 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.229954 26370 consensus_queue.cc:1243] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:41.229718 26230 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:41.210215 26285 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.278761 25770 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:41.284735 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:41.285490 26310 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:41.291237 26322 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.306208 20809 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:41.282112 26352 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.278919 26216 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:23:41.243304 26308 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:41.322146 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.407819 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:41.568434 20877 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:41.470875 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:41.458139 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:41.547798 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:41.451014 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.470546 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.471925 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.472716 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.470546 20732 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:41.471529 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.471176 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.441195 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:41.780213 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:41.786444 21426 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:41.781960 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:41.781960 20629 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 271) took 56252 ms (client timeout 15000 ms). Trace:
I20260709 12:23:41.787400 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:41.408174 20882 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:41.779364 20730 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:41.728736 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:41.786846 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:41.469882 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:41.318498 26310 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:41.480581 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:41.793637 20803 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:41.340077 26216 consensus_queue.cc:1243] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 17.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:41.711341 20728 raft_consensus.cc:2376] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16. Re-sending same reply.
I20260709 12:23:41.791625 20884 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:41.327987 26399 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.408886 26200 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.788800 21431 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:41.418294 26280 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:41.858947 20727 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:41.468657 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:41.420143 25881 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:41.505638 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.507134 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:41.433022 26308 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.415863 26306 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:41.332667 26353 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:41.930867 26353 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.931351 26353 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:41.405494 26250 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:41.935484 26250 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.937736 26250 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.370185 26285 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.812620 20808 pending_rounds.cc:85] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 4
I20260709 12:23:41.942219 20808 pending_rounds.cc:107] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:23:41.942528 20808 raft_consensus.cc:2889] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:41.869683 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82367.6us
I20260709 12:23:41.944182 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:41.946106 26370 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:41.946653 26370 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:41.947576 26370 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:41.406701 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:41.475576 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:41.507294 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 250512us
I20260709 12:23:41.347556 25671 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:41.371140 26309 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.878664 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:41.962668 20807 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
W20260709 12:23:41.957633 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 55522.9us
I20260709 12:23:41.964301 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:41.409121 26373 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:41.423178 26336 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:41.965404 26352 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.965656 26336 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:41.965338 26200 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.966017 26352 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:41.966503 26200 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.880578 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:41.968304 26352 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:41.969285 26200 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.787127 26392 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.162s
I20260709 12:23:41.643960 26320 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.970019 26320 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.971231 26381 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:41.971655 26381 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:41.972168 26320 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:41.787673 26386 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.029s	sys 0.056s
W20260709 12:23:41.972795 26386 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.029s	sys 0.065s
I20260709 12:23:41.972604 26381 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:41.973258 20735 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:41.974030 26381 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:41.974778 20802 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:41.370293 26230 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:41.482430 26188 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.004s
I20260709 12:23:41.422658 26192 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:41.472419 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.328537 26390 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:41.812376 26351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.013s	sys 0.037s
W20260709 12:23:41.477312 26229 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.002s
I20260709 12:23:41.405718 26148 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:23:41.493505 26375 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.001s
W20260709 12:23:41.482174 25699 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.620s	user 0.002s	sys 0.009s
W20260709 12:23:41.501092 26323 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.003s
W20260709 12:23:41.493127 26338 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.000s
I20260709 12:23:41.354193 26378 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.406318 26032 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.476s
I20260709 12:23:41.311740 26187 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:41.779492 20630 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 5 to 11, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 11 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:41.474478 26123 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.969s	user 0.010s	sys 0.002s
I20260709 12:23:41.784082 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:41.475324 25686 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.647753 26313 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:41.896795 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 416269us
I20260709 12:23:41.513854 26403 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:41.900032 20733 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:23:41.664677 20850 leader_election.cc:336] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.536s (SENT)
I20260709 12:23:41.465600 25923 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:41.424402 26236 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:41.901396 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:42.025597 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:41.902911 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:41.530455 26387 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:41.322101 26389 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.898746 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:42.032281 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 133584us
I20260709 12:23:41.383306 26322 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:41.327823 26380 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:23:41.900811 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:41.928751 20727 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:23:41.552649 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 262162us
I20260709 12:23:41.873355 26410 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.883739 26369 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:41.956489 20880 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:41.960376 25770 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:41.958976 21434 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:41.965283 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:41.417284 25933 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.636s	user 0.004s	sys 0.007s
W20260709 12:23:41.976485 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.457273 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:42.054807 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:42.055267 20878 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:41.984200 20726 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:41.887436 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:41.480999 26301 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:41.404609 25836 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:41.994238 20804 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:42.059343 26404 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:41.904628 26172 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:23:41.997288 26148 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:42.004837 26331 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:42.061993 26331 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.009670 26415 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 12:23:42.007781 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 113038us
I20260709 12:23:42.064445 26301 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:42.065239 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:42.066311 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2415) took 1471 ms. Trace:
I20260709 12:23:42.066629 20801 rpcz_store.cc:276] 0709 12:23:40.594534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.594681 (+   147us) service_pool.cc:224] Handling call
0709 12:23:42.066128 (+1471447us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540027bd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:42.066229 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3316086144}
I20260709 12:23:41.985805 26188 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.068089 21426 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:23:42.069813 26188 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:42.023381 26313 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.072713 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:42.073405 26313 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:42.020531 20735 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:23:42.073618 21431 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:42.023759 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:42.075162 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51485.6us
I20260709 12:23:42.011348 20734 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.075964 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:42.077438 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:42.021508 20630 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:42.080585 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:42.080276 20630 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.081881 20878 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:41.897547 26399 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:42.049245 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 384596us
I20260709 12:23:41.998260 26229 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:41.985411 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:42.022033 25686 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.059084 26187 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.062448 26331 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:41.993940 26351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.019s	sys 0.041s
W20260709 12:23:41.789330 20629 rpcz_store.cc:269] 0709 12:22:45.529202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.529495 (+   293us) service_pool.cc:224] Handling call
0709 12:23:41.781858 (+56252363us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:39.161504 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:39.416023 (+254519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:39.425191 (+  9168us) write_op.cc:435] Acquired schema lock
0709 12:23:39.470636 (+ 45445us) tablet.cc:662] Decoding operations
0709 12:23:39.712940 (+242304us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:39.739451 (+ 26511us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:39.751041 (+ 11590us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:39.751567 (+   526us) tablet.cc:701] Row locks acquired
0709 12:23:39.751638 (+    71us) write_op.cc:260] PREPARE: finished
0709 12:23:39.751870 (+   232us) write_op.cc:270] Start()
0709 12:23:39.752118 (+   248us) write_op.cc:276] Timestamp: P: 1783599819751790 usec, L: 0
0709 12:23:39.752174 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 12:23:39.753343 (+  1169us) log.cc:844] Serialized 541 byte log entry
0709 12:23:40.228827 (+475484us) op_driver.cc:464] REPLICATION: finished
0709 12:23:40.370918 (+142091us) write_op.cc:301] APPLY: starting
0709 12:23:40.378421 (+  7503us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:40.607160 (+228739us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:40.617778 (+ 10618us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:40.618655 (+   877us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:40.618919 (+   264us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:23:40.618980 (+    61us) write_op.cc:312] APPLY: finished
0709 12:23:40.619302 (+   322us) log.cc:844] Serialized 25 byte log entry
0709 12:23:40.619517 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:40.619632 (+   115us) write_op.cc:454] Released schema lock
0709 12:23:40.619689 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:40.620225 (+   536us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:40.620258 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1670529,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":126462,"apply.run_cpu_time_us":5658,"apply.run_wall_time_us":261981,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":514827,"prepare.run_cpu_time_us":42158,"prepare.run_wall_time_us":884973,"raft.queue_time_us":151502,"raft.run_cpu_time_us":10176,"raft.run_wall_time_us":164233,"replication_time_us":418461,"thread_start_us":805855,"threads_started":4,"wal-append.queue_time_us":250253,"wal-append.run_cpu_time_us":58422,"wal-append.run_wall_time_us":1627876}]]}
W20260709 12:23:42.112892 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58219.6us
I20260709 12:23:42.085662 25770 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:23:42.055203 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.059091 20875 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:42.082726 20630 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.024682 26280 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:42.095808 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 110497us
I20260709 12:23:42.124796 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:42.128440 21400 connection.cc:256] RPC call timeout handler was delayed by -1.5877538699996876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 12:23:42.131138 21400 connection.cc:256] RPC call timeout handler was delayed by -1.4113538699998571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
I20260709 12:23:42.132912 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:42.099753 20878 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:23:42.106422 20734 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:42.023962 26403 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:42.006052 26375 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.138377 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:42.139461 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.119767 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:42.037351 26405 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:42.070966 26148 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.150825 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:42.113773 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:42.152848 20726 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:42.145601 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:42.154074 20735 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.005621 26419 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:23:42.153748 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:42.010072 25699 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.093566 26323 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:42.046096 26195 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:42.147641 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:42.225257 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.225894 20878 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:42.226231 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:42.096362 26301 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:42.161695 21428 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 9.
W20260709 12:23:42.228458 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108770us
I20260709 12:23:42.136339 26392 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:42.144009 26188 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:42.143349 26401 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:42.241631 20879 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:23:42.009886 26338 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.161324 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:42.140772 26310 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:42.094874 26410 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.083930 26306 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.232198 20726 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: [15.7-17.8]
I20260709 12:23:42.268989 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.232632 20735 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:42.271173 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:42.115078 26322 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
I20260709 12:23:42.236892 20806 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:42.238185 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84538.9us
I20260709 12:23:42.146123 26412 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:42.211895 26236 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:42.246398 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.243171 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.294057 20727 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:42.146572 20630 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 7 to 10, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 10 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:42.211514 26285 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:42.048765 26389 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.228579 26195 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.258692 20876 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:42.235939 21434 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:42.241740 26401 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.237812 26415 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:42.312747 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:42.160794 26405 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:42.287460 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:42.295675 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 331450us
I20260709 12:23:42.257714 26338 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:42.298141 20775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.513s (SENT) [suppressed 30 similar messages]
I20260709 12:23:42.148855 26280 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.290865 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.305503 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.286203 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.327068 20726 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:42.326555 20630 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 7 to 10, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 10 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:42.327566 20735 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:23:42.308790 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 345031us
I20260709 12:23:42.260948 26419 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:42.314590 20877 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:23:42.291457 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:42.315344 20804 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:42.227602 26407 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:23:42.271100 26188 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:42.224963 26353 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.306370 26148 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:42.308161 26195 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:42.333707 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.313695 26401 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:42.324945 25933 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:42.338689 20884 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:23:42.342671 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:42.324487 26236 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:42.358199 20735 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:42.359716 20630 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 11 to 15, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 15 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:42.344372 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:42.344492 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53110.4us
I20260709 12:23:42.345316 20727 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:42.294471 26306 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.336841 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:42.375689 20878 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:42.316291 26405 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:42.355420 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:42.319878 26387 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.307415 26285 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.379364 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:42.381129 20878 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:42.392184 20727 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:23:42.383004 20726 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:42.333335 26216 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:42.390338 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:42.372583 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:42.399235 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:42.408636 25770 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:42.409137 25770 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.409531 25770 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:42.421525 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 335031us
W20260709 12:23:42.424070 20850 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.733s (SENT)
I20260709 12:23:42.354440 26370 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.426625 20630 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 7 to 11, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 11 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:42.427505 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 289216us
I20260709 12:23:42.431085 21428 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:23:42.432613 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1896284259996719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
I20260709 12:23:42.433444 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:42.433710 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:42.356998 25933 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:42.278273 26403 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.439419 25686 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:42.439623 25671 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:42.440517 26419 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:42.441205 26404 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:42.439622 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.352391 26200 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:23:42.446326 20630 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 4 to 6, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 6 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:23:42.447739 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149629us
I20260709 12:23:42.426604 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:42.482961 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 166780us
I20260709 12:23:42.398550 26301 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:42.429564 26373 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:42.565922 20731 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
W20260709 12:23:42.644761 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 211151us
W20260709 12:23:42.566941 26406 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 12:23:42.571290 25699 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:42.577178 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:42.583702 26123 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:42.660080 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:42.660642 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.661458 21431 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
W20260709 12:23:42.588904 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 109019us
I20260709 12:23:42.589804 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:42.662859 20877 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:42.590668 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:42.390956 26310 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.591382 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.599391 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2532) took 1141 ms. Trace:
I20260709 12:23:42.606427 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:42.665248 20726 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:42.665552 20804 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:42.617274 20803 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:23:42.620170 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.429131 26386 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.204s
I20260709 12:23:42.445541 25933 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:42.641314 25770 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.669534 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:42.670331 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.670840 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:42.643514 26323 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:42.647130 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.671696 21429 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:42.672619 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:42.673242 21428 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:42.649179 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:42.649955 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:42.541879 26396 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:42.437716 26370 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.617070 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 953) took 1918 ms. Trace:
I20260709 12:23:42.666518 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.703428 20806 rpcz_store.cc:276] 0709 12:23:40.699036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.732310 (+ 33274us) service_pool.cc:224] Handling call
0709 12:23:42.616834 (+1884524us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003ca854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:42.617007 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3448326912}
I20260709 12:23:42.703948 20726 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:23:42.665158 20884 rpcz_store.cc:276] 0709 12:23:41.458392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.891885 (+433493us) service_pool.cc:224] Handling call
0709 12:23:41.892207 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:42.357112 (+464905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.599076 (+241964us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:42.599323 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999871872}
I20260709 12:23:42.669375 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:42.648582 21400 leader_election.cc:336] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.570s (SENT)
I20260709 12:23:42.674584 21430 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:42.661863 21431 pending_rounds.cc:85] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 11
I20260709 12:23:42.705092 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:42.705531 21431 pending_rounds.cc:107] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:23:42.705838 21431 raft_consensus.cc:2889] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:42.664057 26123 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:42.706758 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:42.651314 26280 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:42.615708 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 976) took 1106 ms. Trace:
I20260709 12:23:42.676093 20630 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:42.678296 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:42.646034 26390 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:42.699935 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:42.700611 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:42.645287 25881 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:42.644631 26373 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.431905 26427 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.237s
I20260709 12:23:42.704614 20879 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:42.704571 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:42.664467 26415 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:42.711711 26431 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.742277 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1133) took 1600 ms. Trace:
I20260709 12:23:42.709468 25770 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:42.723624 26428 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.480s	user 0.001s	sys 0.000s
W20260709 12:23:42.754480 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.655121 26375 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:42.712404 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.722862 20807 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:23:42.712700 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:42.763896 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.764914 20734 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:42.722313 20630 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:42.673995 26386 consensus_queue.cc:1243] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:42.714270 20731 raft_consensus.cc:2376] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17. Re-sending same reply.
I20260709 12:23:42.708055 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.677477 25836 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.727914 20806 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:23:42.728341 26192 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:42.675415 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:42.738360 26462 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:42.793285 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 872) took 1323 ms. Trace:
I20260709 12:23:42.727082 26434 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.732264 21435 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:42.729240 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 80644.3us
W20260709 12:23:42.749763 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.796757 20727 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:42.745168 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:42.740734 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:42.798146 20803 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:42.644645 25671 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:42.545926 26188 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:42.727144 26373 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:42.758282 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:42.813664 21431 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:42.712203 26229 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.707820 26301 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:42.713040 20876 rpcz_store.cc:276] 0709 12:23:41.509159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.883729 (+374570us) service_pool.cc:224] Handling call
0709 12:23:42.615344 (+731615us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54000a2dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:42.615615 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":534540032}
I20260709 12:23:42.767535 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:42.817173 21426 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:42.769095 26440 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:23:42.770149 26433 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:23:42.676802 26032 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:23:42.742736 21427 rpcz_store.cc:276] 0709 12:23:41.141983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.142109 (+   126us) service_pool.cc:224] Handling call
0709 12:23:41.142481 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.151426 (+  8945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.151457 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.192982 (+ 41525us) log.cc:844] Serialized 26 byte log entry
0709 12:23:41.713666 (+520684us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:41.713721 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:41.713789 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:42.742016 (+1028227us) raft_consensus.cc:1685] finished
0709 12:23:42.742049 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:42.742213 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8532,"raft.queue_time_us":19032,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":26872,"thread_start_us":336612,"threads_started":1,"wal-append.queue_time_us":806962}
I20260709 12:23:42.721690 26423 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:42.712679 26378 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:42.789266 20734 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:42.722257 26432 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:42.796838 21400 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.659s (SENT)
I20260709 12:23:42.793885 20730 rpcz_store.cc:276] 0709 12:23:41.470077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.790471 (+320394us) service_pool.cc:224] Handling call
0709 12:23:41.790898 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.860196 (+ 69298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.860242 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.897906 (+ 37664us) log.cc:844] Serialized 26 byte log entry
0709 12:23:42.331074 (+433168us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:42.331133 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.332214 (+  1081us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:42.792935 (+460721us) raft_consensus.cc:1685] finished
0709 12:23:42.792968 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:42.793166 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5813,"raft.queue_time_us":196,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":58176,"thread_start_us":348567,"threads_started":1,"wal-append.queue_time_us":614759}
I20260709 12:23:42.700165 26323 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:42.743314 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80325.4us
I20260709 12:23:42.762257 26369 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:23:42.823923 26187 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:42.858331 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:42.765611 26399 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:42.850682 20630 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 7 to 11, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 11 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:42.871857 20731 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:42.874178 26441 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:23:42.874928 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2439) took 1085 ms. Trace:
I20260709 12:23:42.874857 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:42.879113 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 220787us
I20260709 12:23:42.881685 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
W20260709 12:23:42.880805 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 157559us
I20260709 12:23:42.663609 26236 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.985970 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:42.986667 20806 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.761890 26431 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:42.987509 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.746274 26415 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.977558 20805 rpcz_store.cc:276] 0709 12:23:41.789769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.789916 (+   147us) service_pool.cc:224] Handling call
0709 12:23:41.790235 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.877258 (+ 87023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.877291 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.887058 (+  9767us) log.cc:844] Serialized 26 byte log entry
0709 12:23:42.279327 (+392269us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:42.279363 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.279405 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:42.874639 (+595234us) raft_consensus.cc:1685] finished
0709 12:23:42.874666 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:42.874844 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70058,"raft.queue_time_us":106943,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":45600,"thread_start_us":303595,"threads_started":1,"wal-append.queue_time_us":479272}
I20260709 12:23:42.810124 26410 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:42.810637 26353 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:42.812350 20727 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:42.890919 20726 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:42.890830 20734 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:42.821012 26389 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:42.956374 26435 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.416s	user 0.001s	sys 0.000s
I20260709 12:23:43.009291 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:43.010020 20734 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:42.957274 20630 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 11 to 13, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 13 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:42.958611 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2440) took 1167 ms. Trace:
W20260709 12:23:42.956707 26430 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.482s	user 0.000s	sys 0.001s
I20260709 12:23:42.896919 26432 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:42.896602 26462 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 12:23:42.974574 20803 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:42.979367 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:42.980332 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 183508us
W20260709 12:23:43.020121 21400 leader_election.cc:336] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.702s (SENT)
I20260709 12:23:43.020143 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:42.981621 26403 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:42.982259 26436 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:23:42.899413 26187 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:42.991389 21431 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:42.850149 26229 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:42.967221 26453 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:23:42.822772 26434 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.052608 26229 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.051108 26434 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.060753 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2695) took 1100 ms. Trace:
I20260709 12:23:43.061219 20802 rpcz_store.cc:276] 0709 12:23:41.960451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.978051 (+ 17600us) service_pool.cc:224] Handling call
0709 12:23:41.978416 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.978675 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.978713 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.985142 (+  6429us) log.cc:844] Serialized 26 byte log entry
0709 12:23:42.278996 (+293854us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:42.279034 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.279100 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.060514 (+781414us) raft_consensus.cc:1685] finished
0709 12:23:43.060537 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.060690 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":293483,"threads_started":1,"wal-append.queue_time_us":404307}
W20260709 12:23:42.980919 26461 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:23:43.063901 26370 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:43.066926 26432 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.067420 26250 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.068012 26250 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.069011 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2442) took 1276 ms. Trace:
I20260709 12:23:42.993842 26301 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.072742 26415 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:42.997761 25671 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.077004 20726 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:43.015368 26285 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:42.980895 26460 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:23:43.020404 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:43.018923 20630 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 10 to 11, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 11 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.093420 20805 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:42.880621 26123 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.027971 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:42.993602 26392 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.032054 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:43.044358 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65059.9us
I20260709 12:23:43.062822 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:43.069499 20808 rpcz_store.cc:276] 0709 12:23:41.792264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.792417 (+   153us) service_pool.cc:224] Handling call
0709 12:23:41.812389 (+ 19972us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:41.943182 (+130793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.943243 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:41.968222 (+ 24979us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:23:41.968755 (+   533us) log.cc:844] Serialized 52 byte log entry
0709 12:23:42.212460 (+243705us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:42.402001 (+189541us) log.cc:844] Serialized 14 byte log entry
0709 12:23:42.406395 (+  4394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.406468 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.068612 (+662144us) raft_consensus.cc:1685] finished
0709 12:23:43.068753 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.068938 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3602,"prepare.queue_time_us":432499,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":5378,"raft.queue_time_us":9062,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":11624,"spinlock_wait_cycles":14138240,"thread_start_us":412864,"threads_started":2,"wal-append.queue_time_us":382489}
W20260709 12:23:42.945876 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 521775us
I20260709 12:23:43.087615 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:43.087977 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2441) took 1296 ms. Trace:
I20260709 12:23:43.102835 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:43.066829 26336 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.101s	user 0.010s	sys 0.000s
W20260709 12:23:43.090430 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70349.6us
W20260709 12:23:43.107287 21400 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.748s (SENT)
I20260709 12:23:43.011420 20809 rpcz_store.cc:276] 0709 12:23:41.790821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.790941 (+   120us) service_pool.cc:224] Handling call
0709 12:23:41.791250 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.877509 (+ 86259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.877538 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.905713 (+ 28175us) log.cc:844] Serialized 26 byte log entry
0709 12:23:42.029745 (+124032us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:42.029787 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.029857 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:42.958311 (+928454us) raft_consensus.cc:1685] finished
0709 12:23:42.958344 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:42.958536 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106331,"raft.queue_time_us":107627,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":46989,"thread_start_us":123620,"threads_started":1,"wal-append.queue_time_us":229253}
I20260709 12:23:43.107503 20726 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:43.108460 20809 raft_consensus.cc:1217] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[10.5-11.6]   Dedup: 10.5->[11.6-11.6]
I20260709 12:23:43.040560 26187 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:43.104503 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:43.090354 26404 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:43.106046 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:43.107609 20881 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:23:43.112766 20881 pending_rounds.cc:85] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 5
I20260709 12:23:43.109058 20809 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [10.5-11.6]
I20260709 12:23:43.110481 20802 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:43.104859 20806 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:43.113082 20881 pending_rounds.cc:107] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:23:43.113497 20881 raft_consensus.cc:2889] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:43.089949 26250 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.099864 20810 rpcz_store.cc:276] 0709 12:23:41.791733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.791869 (+   136us) service_pool.cc:224] Handling call
0709 12:23:41.882863 (+ 90994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.929413 (+ 46550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.929449 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.983932 (+ 54483us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:23:41.993443 (+  9511us) log.cc:844] Serialized 26 byte log entry
0709 12:23:42.300522 (+307079us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:42.546268 (+245746us) log.cc:844] Serialized 14 byte log entry
0709 12:23:42.546560 (+   292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.546641 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.078276 (+531635us) raft_consensus.cc:1685] finished
0709 12:23:43.087744 (+  9468us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.087905 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62729,"prepare.queue_time_us":423007,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":301,"raft.queue_time_us":46669,"raft.run_cpu_time_us":616,"raft.run_wall_time_us":95599,"spinlock_wait_cycles":47479040,"thread_start_us":524776,"threads_started":2,"wal-append.queue_time_us":763665}
W20260709 12:23:43.116163 20850 leader_election.cc:336] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.856s (SENT)
I20260709 12:23:43.116781 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:43.117388 20801 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:43.118688 20630 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.113610 20808 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:43.123490 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:43.094110 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:43.127347 20630 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:43.109460 26285 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:43.131181 20809 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:43.129109 20630 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 8 to 10, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 10 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.135911 26323 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.136858 26250 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.137357 26250 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:43.138230 26331 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.990s	user 0.009s	sys 0.001s
I20260709 12:23:43.139573 26250 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.144047 26431 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:43.144770 26411 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:43.145316 26411 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.145752 26411 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:43.111989 26404 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:43.147720 26404 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:43.138664 26331 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.169256 20734 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:43.170374 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:43.116091 26338 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.174747 26338 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:43.176829 26195 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.809s	user 0.010s	sys 0.000s
I20260709 12:23:43.177933 26195 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.182152 26463 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:43.116783 26123 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:43.183899 26460 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:43.184614 26460 consensus_queue.cc:1243] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:43.185073 26460 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:43.186257 26410 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:43.186635 26410 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:43.187417 26389 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:43.187824 26389 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:43.188931 26389 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:43.192279 26456 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:23:43.117648 26378 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.198868 26378 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:43.205636 26381 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.231s	user 0.008s	sys 0.005s
I20260709 12:23:43.206295 26381 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.207221 25933 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.108244 26386 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:43.206779 26381 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.104652 26392 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.236546 20810 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:23:43.377717 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:43.369656 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 145789us
I20260709 12:23:43.369573 20882 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:43.392324 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:43.370699 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 198429us
I20260709 12:23:43.393913 20803 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 9.
I20260709 12:23:43.394649 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:43.282015 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.397897 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 290625us
W20260709 12:23:43.398912 21400 leader_election.cc:336] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.753s (SENT)
I20260709 12:23:43.399531 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:43.327323 20733 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:43.404177 20805 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:43.253818 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 875) took 1779 ms. Trace:
I20260709 12:23:43.396952 26386 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:43.349858 26405 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.931s	user 0.009s	sys 0.000s
I20260709 12:23:43.436990 20731 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:43.440775 26405 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.443154 26405 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.406971 20728 rpcz_store.cc:276] 0709 12:23:41.474247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.895990 (+421743us) service_pool.cc:224] Handling call
0709 12:23:41.930015 (+ 34025us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:42.235019 (+305004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.235072 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:42.297352 (+ 62280us) log.cc:844] Serialized 52 byte log entry
0709 12:23:42.632073 (+334721us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:42.813301 (+181228us) log.cc:844] Serialized 14 byte log entry
0709 12:23:42.887804 (+ 74503us) log.cc:844] Serialized 14 byte log entry
0709 12:23:42.890878 (+  3074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.891005 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.253517 (+362512us) raft_consensus.cc:1685] finished
0709 12:23:43.253542 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.253736 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":872937,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":117,"raft.queue_time_us":85290,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":17123,"thread_start_us":703497,"threads_started":3,"wal-append.queue_time_us":704398}
W20260709 12:23:43.443940 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 327803us
W20260709 12:23:43.376369 26148 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.018s	user 0.012s	sys 0.000s
W20260709 12:23:43.343917 25923 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.274s	user 0.009s	sys 0.005s
I20260709 12:23:43.447141 20732 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:23:43.440616 26386 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.474356 26386 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:43.475749 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81248.6us
I20260709 12:23:43.476649 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:43.477392 26148 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:43.478250 26148 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.416911 26351 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:43.478945 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:43.293798 26336 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:43.468024 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:43.252785 26285 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:43.472225 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73346.3us
I20260709 12:23:43.489527 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:43.400988 26403 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:43.473614 20729 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:23:43.390272 26200 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:23:43.466689 26415 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:43.466884 26399 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:43.436641 26378 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:43.516129 20726 pending_rounds.cc:85] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 6
I20260709 12:23:43.516521 20726 pending_rounds.cc:107] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:23:43.516849 20726 raft_consensus.cc:2889] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:43.465880 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:43.528064 20850 connection.cc:256] RPC call timeout handler was delayed by -1.2753364330001205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 12:23:43.528851 20850 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.110s (SENT)
I20260709 12:23:43.529382 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:43.513578 26216 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.522156 26378 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:43.513154 26353 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:23:43.535161 26399 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:43.539300 26336 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:43.579700 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 97396.8us
W20260709 12:23:43.566726 26401 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.191s	user 0.013s	sys 0.000s
I20260709 12:23:43.586748 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:43.568826 26195 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:43.600883 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 111437us
I20260709 12:23:43.605516 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:43.595918 26439 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
I20260709 12:23:43.583716 26310 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.557511 26216 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.587867 26331 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:43.617304 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.613620 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 182933us
I20260709 12:23:43.601734 26378 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:43.621835 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:43.624181 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 105664us
I20260709 12:23:43.609508 26482 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:43.628965 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:43.597592 26401 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:43.635419 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.636122 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.606264 26399 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:23:43.637707 26482 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.644557 26390 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:43.645157 26390 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:43.646569 26482 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:43.646976 26480 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:23:43.648487 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2711) took 1226 ms. Trace:
I20260709 12:23:43.628646 26381 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:43.648897 20730 rpcz_store.cc:276] 0709 12:23:42.421773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.889685 (+467912us) service_pool.cc:224] Handling call
0709 12:23:42.890027 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.043940 (+153913us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:23:43.408842 (+364902us) log.cc:844] Serialized 14 byte log entry
0709 12:23:43.514161 (+105319us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:43.514212 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:43.514263 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.648294 (+134031us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54000c49d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:43.648334 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.648431 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38433,"prepare.queue_time_us":253045,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":1973,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3066,"spinlock_wait_cycles":353439104,"thread_start_us":414203,"threads_started":2,"wal-append.queue_time_us":157971}
W20260709 12:23:43.646582 26476 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:23:43.629458 26351 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:43.641819 26392 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.640779 26148 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:43.662299 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:43.640415 26488 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:43.668614 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:43.652588 26490 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:43.675076 26387 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:43.675376 26490 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:43.676131 26490 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:43.678442 26250 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:43.679008 26250 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.681192 26250 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:43.683084 25699 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:43.684748 20730 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:43.687079 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:43.688755 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:43.627744 26310 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.693368 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:43.693998 20880 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:43.642275 26032 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:23:43.695273 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:43.695314 20883 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 9.
I20260709 12:23:43.704944 20878 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:43.695712 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:43.708045 20730 raft_consensus.cc:1217] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Deduplicated request from leader. Original: 8.6->[10.7-11.8]   Dedup: 10.7->[11.8-11.8]
I20260709 12:23:43.708889 20730 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 12:23:43.688201 26392 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:23:43.860030 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 162791us
I20260709 12:23:43.860597 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:43.861433 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:43.862430 20730 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:43.865408 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:43.865800 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:43.866171 20876 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:43.866509 20877 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:43.695822 26336 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.173s
I20260709 12:23:43.683514 25699 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:43.877982 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 192106us
I20260709 12:23:43.672832 26403 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.195s
W20260709 12:23:43.868065 26373 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.003s	user 0.010s	sys 0.002s
W20260709 12:23:43.911972 26188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.007s	sys 0.014s
I20260709 12:23:43.832041 26148 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:43.667117 26489 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.211s
I20260709 12:23:43.867729 26392 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:43.942452 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 299368us
I20260709 12:23:43.871631 26250 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:43.883311 26236 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:43.944187 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:43.956161 20727 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:23:43.943912 26188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.007s	sys 0.015s
W20260709 12:23:43.959084 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 332848us
I20260709 12:23:43.945590 26392 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:43.970525 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:43.975093 25836 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.182s	user 0.006s	sys 0.006s
I20260709 12:23:43.975740 25836 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:43.976114 26310 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.976284 25836 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.976926 26310 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:43.977588 26310 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.937857 26373 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.979274 25832 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.980127 25832 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.981854 26373 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.982242 26310 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:43.982547 26373 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:43.987826 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.869410 26486 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 12:23:43.954284 26353 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:43.978924 25836 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:43.995253 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51167.2us
I20260709 12:23:43.998227 26310 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.998996 26310 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:44.000891 26493 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:44.001550 26493 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:44.001950 26310 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:44.001629 20850 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.599s (SENT)
I20260709 12:23:44.004789 26493 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:44.003696 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:44.007817 26493 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:23:44.015740 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45411.1us
I20260709 12:23:44.017139 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:43.997202 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:43.964797 26285 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:43.988363 26250 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:44.047374 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 79724us
I20260709 12:23:43.993621 26148 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:44.027855 25836 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.014178 26353 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:44.015072 26471 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:43.966027 26236 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:44.020545 26496 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:43.971235 25699 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:43.951675 26351 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:43.983127 26373 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:44.028548 26489 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:23:44.033394 26488 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:44.076020 26353 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:44.077337 26471 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:44.047892 26123 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.865s	user 0.011s	sys 0.000s
I20260709 12:23:44.241061 26216 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:44.094511 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1411) took 1435 ms. Trace:
I20260709 12:23:44.377776 21426 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:23:44.378726 21426 pending_rounds.cc:85] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 10
I20260709 12:23:44.090179 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 960) took 1960 ms. Trace:
I20260709 12:23:44.379410 21426 pending_rounds.cc:107] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:23:44.124158 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:44.380101 21426 raft_consensus.cc:2889] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:44.123694 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:44.380029 20807 rpcz_store.cc:276] 0709 12:23:42.129390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.140185 (+ 10795us) service_pool.cc:224] Handling call
0709 12:23:42.140564 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:42.763282 (+622718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.089278 (+1325996us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400186b20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca2a02 00007f9e1ec9edd6 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:23:44.089929 (+   651us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54003cad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:44.090123 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393479808}
I20260709 12:23:44.381219 26373 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:44.381821 26369 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.098299 20735 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:44.382462 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:44.228662 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:44.273921 26392 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:44.885634 20878 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:44.492893 20732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2708) took 2616 ms (client timeout 1663 ms). Trace:
I20260709 12:23:44.890342 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:44.245530 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 962) took 1623 ms. Trace:
I20260709 12:23:44.228749 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:44.899935 20880 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 9: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:44.368873 25832 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:44.493106 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2537) took 2994 ms. Trace:
I20260709 12:23:44.905752 20882 rpcz_store.cc:276] 0709 12:23:41.482019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.968150 (+486131us) service_pool.cc:224] Handling call
0709 12:23:41.968463 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.392389 (+1423926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.476322 (+1083933us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:44.476534 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3221533440}
I20260709 12:23:44.377590 21433 rpcz_store.cc:276] 0709 12:23:42.659399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.747848 (+ 88449us) service_pool.cc:224] Handling call
0709 12:23:42.748864 (+  1016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:42.895713 (+146849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.895756 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:42.895807 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:42.895852 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.094200 (+1198348us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54006333d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:44.094272 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:44.094436 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337167616}
I20260709 12:23:44.380472 20883 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:44.116636 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2715) took 1692 ms. Trace:
W20260709 12:23:44.104908 26448 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.811s	user 0.011s	sys 0.007s
W20260709 12:23:44.229326 26187 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.092s	user 0.006s	sys 0.006s
W20260709 12:23:44.097504 20733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2438) took 1854 ms (client timeout 2128 ms). Trace:
I20260709 12:23:44.102581 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2697) took 1500 ms. Trace:
I20260709 12:23:44.639743 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2536) took 3158 ms. Trace:
W20260709 12:23:44.889213 20732 rpcz_store.cc:269] 0709 12:23:41.861238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.883833 (+ 22595us) service_pool.cc:224] Handling call
0709 12:23:44.477185 (+2593352us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000e6ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:44.477336 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3219544320}
W20260709 12:23:44.687417 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.015s	sys 0.003s
W20260709 12:23:44.923898 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.016s	sys 0.008s
I20260709 12:23:44.926008 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:44.926638 20807 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:44.302937 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.037550 26403 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.036042 26499 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:44.365669 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 348567us
I20260709 12:23:44.923331 20803 rpcz_store.cc:276] 0709 12:23:42.602168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.019878 (+417710us) service_pool.cc:224] Handling call
0709 12:23:44.102379 (+1082501us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b5400127854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:44.102506 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":857568640}
I20260709 12:23:44.931124 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:44.148078 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.921730 20878 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:44.244946 26320 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:23:44.733554 20729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 870) took 3264 ms (client timeout 1799 ms). Trace:
I20260709 12:23:44.370761 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:44.213982 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 160557us
I20260709 12:23:44.159650 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:44.932880 20730 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
W20260709 12:23:44.932785 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:44.933004 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 562262us
I20260709 12:23:44.495050 26466 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.109679 26389 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:44.908211 20731 rpcz_store.cc:276] 0709 12:23:42.424621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.993012 (+568391us) service_pool.cc:224] Handling call
0709 12:23:43.017477 (+ 24465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.440425 (+422948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.116322 (+675897us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b54002e54d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:44.116549 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964346112}
I20260709 12:23:44.064319 26399 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:44.519896 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.016s	sys 0.004s
W20260709 12:23:44.938880 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.017s	sys 0.005s
W20260709 12:23:44.939021 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 941981us
W20260709 12:23:44.414572 26390 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.769s	user 0.012s	sys 0.007s
W20260709 12:23:44.516579 26331 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.002s	sys 0.014s
W20260709 12:23:44.087932 26487 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.396s	user 0.000s	sys 0.001s
I20260709 12:23:44.095992 26387 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:23:44.097606 26404 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.921s	user 0.004s	sys 0.009s
W20260709 12:23:44.082345 26494 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:23:44.894127 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:44.080646 26236 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:44.476922 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2509) took 1730 ms. Trace:
I20260709 12:23:44.935849 26373 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.923441 20881 rpcz_store.cc:276] 0709 12:23:41.481741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.965690 (+483949us) service_pool.cc:224] Handling call
0709 12:23:41.965970 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.114063 (+1148093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.639436 (+1525373us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400305a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:44.639676 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2624162560}
I20260709 12:23:44.377558 26123 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:44.066988 26401 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:44.094842 25881 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:44.923154 20733 rpcz_store.cc:269] 0709 12:23:42.243130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.246056 (+  2926us) service_pool.cc:224] Handling call
0709 12:23:44.097279 (+1851223us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540028cb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:44.097423 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2376219008}
I20260709 12:23:44.932155 20802 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:23:44.898710 21434 rpcz_store.cc:276] 0709 12:23:42.622042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.624451 (+  2409us) service_pool.cc:224] Handling call
0709 12:23:42.624774 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:42.713988 (+ 89214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.714020 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:42.714059 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:42.714090 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.245264 (+1531174us) spinlock_profiling.cc:243] Waited 89 ms on lock 0x7b54007201d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:44.245321 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:44.245461 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204695296}
W20260709 12:23:44.391311 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:44.645273 26455 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.769s	user 0.012s	sys 0.017s
I20260709 12:23:44.584004 26471 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:44.101468 26306 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:44.617496 26503 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:44.932760 20729 rpcz_store.cc:269] 0709 12:23:41.469527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.469649 (+   122us) service_pool.cc:224] Handling call
0709 12:23:44.722298 (+3252649us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54001d7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:44.733284 (+ 10986us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3696695040}
I20260709 12:23:44.246044 26496 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:44.709321 26506 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.119837 26415 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:44.974211 25881 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:44.942322 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:44.944752 26331 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.009s	sys 0.017s
I20260709 12:23:44.950817 20883 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:44.993803 20803 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
W20260709 12:23:44.096448 26498 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:23:44.091800 26411 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.945s	user 0.011s	sys 0.002s
I20260709 12:23:44.952148 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:44.983670 26401 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:44.998838 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:44.953855 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:44.912537 25699 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:44.911873 26489 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:23:44.576138 26511 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:44.919533 26484 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.009s	sys 0.006s
I20260709 12:23:44.387667 26381 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:44.960963 20727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.020s	sys 0.020s
I20260709 12:23:44.233616 25671 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.712s
I20260709 12:23:44.068361 25836 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:44.067631 26250 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:44.928077 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.007334 26381 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.009383 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.010083 20805 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:23:44.964892 20884 rpcz_store.cc:276] 0709 12:23:42.747011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.748369 (+  1358us) service_pool.cc:224] Handling call
0709 12:23:42.748679 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:42.852079 (+103400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.852123 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:42.877614 (+ 25491us) log.cc:844] Serialized 26 byte log entry
0709 12:23:43.235547 (+357933us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:43.235585 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.476404 (+1240819us) spinlock_profiling.cc:243] Waited 68.5 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:44.476475 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:44.476547 (+    72us) raft_consensus.cc:1685] finished
0709 12:23:44.476566 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:44.476842 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34393,"raft.queue_time_us":34908,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":13921,"spinlock_wait_cycles":157565824,"thread_start_us":276410,"threads_started":1,"wal-append.queue_time_us":278022,"wal-append.run_cpu_time_us":1772,"wal-append.run_wall_time_us":1054095}
I20260709 12:23:44.935249 26320 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:44.936669 26200 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:44.935665 26499 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.907224 26369 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:44.935395 25933 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:44.936218 26505 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:44.088204 26148 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:44.975499 20807 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:23:44.961957 26353 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:44.977490 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2698) took 2375 ms. Trace:
I20260709 12:23:44.944680 26390 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:44.971482 25832 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:45.005085 26511 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.001313 26411 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.043740 25832 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:44.979094 26488 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.045008 26390 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:45.045382 26411 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.045012 26511 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.045848 25832 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.043449 20808 rpcz_store.cc:276] 0709 12:23:42.602536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.098945 (+496409us) service_pool.cc:224] Handling call
0709 12:23:43.113452 (+ 14507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.122409 (+  8957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.977243 (+1854834us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b54003a0f50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:44.977426 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19485056}
I20260709 12:23:45.047449 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.048092 20808 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:44.986204 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.001212 26392 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:44.954943 26460 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 12:23:44.989825 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 232877us
I20260709 12:23:45.076566 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.077425 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:45.078967 26392 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:45.079429 26392 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:45.079839 26392 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:45.084889 26514 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:23:45.087414 26494 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:23:45.089027 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2720) took 1921 ms. Trace:
I20260709 12:23:44.990204 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2445) took 2333 ms. Trace:
I20260709 12:23:45.091722 20801 rpcz_store.cc:276] 0709 12:23:42.656816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.118127 (+461311us) service_pool.cc:224] Handling call
0709 12:23:43.252674 (+134547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:43.342249 (+ 89575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:43.342287 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:43.405367 (+ 63080us) log.cc:844] Serialized 26 byte log entry
0709 12:23:43.622012 (+216645us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:43.961148 (+339136us) log.cc:844] Serialized 14 byte log entry
0709 12:23:43.961351 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.961425 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:44.990005 (+1028580us) raft_consensus.cc:1685] finished
0709 12:23:44.990028 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:44.990151 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91693,"prepare.queue_time_us":474046,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":142913,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":36541,"thread_start_us":481801,"threads_started":2,"wal-append.queue_time_us":206548}
I20260709 12:23:45.089426 20726 rpcz_store.cc:276] 0709 12:23:43.168036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.488333 (+320297us) service_pool.cc:224] Handling call
0709 12:23:43.515914 (+ 27581us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:43.517484 (+  1570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:43.517515 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:23:43.519314 (+  1799us) log.cc:844] Serialized 78 byte log entry
0709 12:23:43.620779 (+101465us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:43.628466 (+  7687us) log.cc:844] Serialized 14 byte log entry
0709 12:23:43.632350 (+  3884us) log.cc:844] Serialized 14 byte log entry
0709 12:23:43.632559 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.632650 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:45.088751 (+1456101us) raft_consensus.cc:1685] finished
0709 12:23:45.088784 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:45.088961 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35523,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"raft.queue_time_us":6711,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":26545,"thread_start_us":59839,"threads_started":1,"wal-append.queue_time_us":157837}
I20260709 12:23:45.094297 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:45.094887 20801 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:45.099856 26320 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.101210 26320 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:45.015288 26499 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:44.678442 26502 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:44.938541 26187 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.103515 26502 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.104875 26187 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.000643 20809 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:23:45.105669 26502 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.106261 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.000765 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:44.966869 26404 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:44.971544 26373 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:45.003052 20727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.020s	sys 0.020s
I20260709 12:23:44.988086 26507 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:45.003710 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:44.974844 26236 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:23:44.991394 26415 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:45.002863 26484 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.010s	sys 0.006s
I20260709 12:23:45.000658 26401 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.122367 26401 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:44.989491 26471 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:23:45.037693 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:45.128010 26516 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:45.128845 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.129086 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.129559 20805 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:45.130574 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:45.038151 26517 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:23:44.989831 26503 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.131310 26507 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.131840 26507 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:45.132884 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.133385 26503 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.048974 20806 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:44.942337 26403 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.134649 26381 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.128057 26499 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.136204 26403 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.058771 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.136886 20808 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
I20260709 12:23:45.137349 20808 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 9: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:45.137378 26499 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.060300 25836 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.033955 26250 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.049268 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:45.007673 25671 consensus_queue.cc:1243] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:45.077458 20807 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:23:45.152668 20807 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:44.948540 26466 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:44.996161 26506 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.051352 26387 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.138629 20808 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:23:45.130735 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.121279 20802 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
W20260709 12:23:45.141290 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63934.1us
I20260709 12:23:45.163489 20809 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:45.162949 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:45.142164 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 208207us
I20260709 12:23:45.013155 26200 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.156251 20805 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:45.064185 26505 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:45.167600 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 236544us
W20260709 12:23:45.160858 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 191806us
I20260709 12:23:45.166639 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:45.212080 20801 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:23:45.167806 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:45.221259 20881 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:45.221931 20801 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:45.222076 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54350.3us
W20260709 12:23:45.203486 20879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.009s	sys 0.010s
W20260709 12:23:45.186276 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.169749 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2732) took 1119 ms. Trace:
W20260709 12:23:45.223984 20879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.009s	sys 0.013s
I20260709 12:23:45.202539 20802 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:23:45.223611 20850 leader_election.cc:336] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.698s (SENT)
W20260709 12:23:45.223663 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.185739 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2723) took 1531 ms. Trace:
I20260709 12:23:45.227135 20728 rpcz_store.cc:276] 0709 12:23:43.654211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.654349 (+   138us) service_pool.cc:224] Handling call
0709 12:23:43.654690 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:43.659665 (+  4975us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:43.659706 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:43.660173 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:23:44.492837 (+832664us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b180048da80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:44.881834 (+388997us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:44.881892 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.881999 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:45.161481 (+279482us) raft_consensus.cc:1685] finished
0709 12:23:45.161500 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:45.185652 (+ 24152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4615,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5884,"spinlock_wait_cycles":6065408,"thread_start_us":388688,"threads_started":1,"wal-append.queue_time_us":1294209}
I20260709 12:23:45.186035 20808 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:45.230901 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.224687 20727 rpcz_store.cc:276] 0709 12:23:44.050082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.050634 (+   552us) service_pool.cc:224] Handling call
0709 12:23:44.051091 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:44.051572 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:44.051619 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:44.065311 (+ 13692us) log.cc:844] Serialized 26 byte log entry
0709 12:23:45.119140 (+1053829us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:45.119201 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.119280 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:45.169421 (+ 50141us) raft_consensus.cc:1685] finished
0709 12:23:45.169451 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:45.169659 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7665,"thread_start_us":1003847,"threads_started":1,"wal-append.queue_time_us":886598}
I20260709 12:23:45.231562 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:45.233291 20880 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:45.159410 26499 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.234215 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:45.160373 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 223814us
W20260709 12:23:45.236918 20701 connection.cc:256] RPC call timeout handler was delayed by -1.1264285970000856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
I20260709 12:23:45.182317 26381 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:45.237278 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.255628 20882 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:23:45.256724 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:45.262521 20880 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:45.262682 20876 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:45.230276 26148 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:45.259446 20775 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.513s (SENT)
I20260709 12:23:45.259889 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.212257 26488 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.257597 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:45.196228 26195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.009s	sys 0.013s
I20260709 12:23:45.266065 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:45.160821 26336 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.269027 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:45.234812 25836 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.202872 25881 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.168263 26387 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.256696 26511 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.315096 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.324468 20802 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:45.321560 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.314052 20877 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:45.325723 21434 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:45.326120 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57776.2us
W20260709 12:23:45.443262 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.313026 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:45.344704 20881 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:45.344939 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:45.478327 20883 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:23:45.478539 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:45.409500 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:45.436367 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 171899us
I20260709 12:23:45.479700 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.479756 21434 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:45.480327 20881 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:23:45.481143 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.482123 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.262190 26381 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.427151 20882 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 15 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
W20260709 12:23:45.269646 26195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.009s	sys 0.015s
I20260709 12:23:45.438306 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:45.315850 26532 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:23:45.267038 26148 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.265707 26522 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:45.278213 20775 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.526s (SENT)
I20260709 12:23:45.496534 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:45.497851 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 219666us
I20260709 12:23:45.266129 26496 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:45.439422 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.499564 26496 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.500527 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.501341 21429 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:45.265394 25686 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:45.502599 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.503238 25686 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.305197 26338 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:45.504135 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.505074 26526 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:45.505612 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.506438 21429 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:45.270087 26399 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:45.306300 26521 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.317685 26471 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:45.198964 26415 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.437306 25836 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.475750 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.262521 26499 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.269901 26489 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.392262 20876 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:45.483489 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:45.484444 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:45.485028 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.489151 21427 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:45.544294 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.343945 26488 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.426616 26336 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:45.516834 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:45.520227 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 288386us
I20260709 12:23:45.476830 26404 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.439802 26507 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.497542 26525 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.538578 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.486778 26381 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:45.439332 26511 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.509049 25686 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.541666 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:45.497421 26148 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:45.502457 26496 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.487790 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:45.535666 20849 connection.cc:256] RPC call timeout handler was delayed by -1.4169096770001488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 12:23:45.745303 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:45.746354 21435 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:23:45.746598 25881 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:45.747462 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.749070 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:45.548215 21431 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:45.610508 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.604260 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:45.753298 20876 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:45.601270 26530 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:23:45.659293 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.548627 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:45.263304 26250 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.233s
W20260709 12:23:45.660353 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 118795us
W20260709 12:23:45.765893 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.603735 20808 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:45.662500 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 155430us
W20260709 12:23:45.663532 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 157273us
I20260709 12:23:45.529146 26373 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.658851 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:45.510839 26410 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.658706 21429 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
W20260709 12:23:45.758023 20849 connection.cc:256] RPC call timeout handler was delayed by -1.2153096770002776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
I20260709 12:23:45.881567 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:45.882041 20849 connection.cc:256] RPC call timeout handler was delayed by -1.187829234000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 12:23:45.882541 21435 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:23:45.603897 26524 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.599849 26415 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:45.763298 21427 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:45.606695 26540 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:45.860558 21433 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:45.860751 20801 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:45.768174 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:45.601435 26200 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.604871 25836 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.600391 26387 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.755808 26285 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:45.603648 26526 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:45.879807 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:45.600654 26529 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 12:23:45.882602 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1856096770002296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 12:23:45.755627 25686 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.884076 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.606153 26506 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:45.546814 26471 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.750787 26466 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.544154 26399 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.608045 26489 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:45.898016 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 129956us
I20260709 12:23:45.898779 21427 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:45.935710 21427 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:45.897495 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 414427us
I20260709 12:23:45.640535 26401 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:45.905923 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:45.907423 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1640096770001946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 12:23:45.937248 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1316096770001423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
I20260709 12:23:45.938035 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:45.907428 21431 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:45.939139 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:45.935773 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.942147 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:45.939581 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:45.944631 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:45.946533 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:45.947602 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 966) took 1848 ms (client timeout 2182 ms). Trace:
W20260709 12:23:45.947932 21428 rpcz_store.cc:269] 0709 12:23:44.099177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.934406 (+1835229us) service_pool.cc:224] Handling call
0709 12:23:45.947521 (+ 13115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:45.948021 21427 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.949156 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:45.950115 20734 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:45.950758 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:45.951368 20733 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:45.952142 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:45.952734 20733 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:45.908900 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.922698 21433 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
W20260709 12:23:45.902704 26535 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.005s	sys 0.020s
I20260709 12:23:45.897336 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.916056 26285 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:45.960027 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:45.954850 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:45.961071 20734 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:45.962458 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:45.911861 26505 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:45.963279 20728 raft_consensus.cc:2376] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14. Re-sending same reply.
I20260709 12:23:45.957204 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.970206 21429 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:45.971374 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:45.930127 26539 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:23:45.949106 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.959499 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.978229 21435 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:45.964819 21434 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:45.979049 21429 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:23:45.886404 26502 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:23:45.982072 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:45.983162 20802 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:23:45.954530 26387 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.994530 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.000401 20734 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:45.980794 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:45.896690 26524 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.917075 25686 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:46.134794 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 475972us
I20260709 12:23:46.093268 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.006090 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:45.543926 26521 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.984798 26523 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:46.147737 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:45.899573 26540 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:45.545583 26522 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.137398 20734 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:45.966337 26535 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.005s	sys 0.021s
I20260709 12:23:46.157802 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.158300 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1418) took 3263 ms. Trace:
I20260709 12:23:46.158705 21426 rpcz_store.cc:276] 0709 12:23:42.894874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.978414 (+ 83540us) service_pool.cc:224] Handling call
0709 12:23:42.978736 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:44.380781 (+1402045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.158037 (+1777256us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540029ca54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:46.158223 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3216462080}
I20260709 12:23:45.919090 26526 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:45.750322 26525 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.146629 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:46.116308 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:46.149508 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.964908 26250 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.183s
W20260709 12:23:46.260289 21430 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.004s	sys 0.017s
I20260709 12:23:46.152745 21428 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:46.156612 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:46.109447 26192 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.996066 26554 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:46.132334 26390 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:23:46.171164 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 165188us
I20260709 12:23:45.986826 26236 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.177s
W20260709 12:23:46.144879 26452 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 2.707s	user 0.015s	sys 0.018s
W20260709 12:23:46.281245 21430 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.004s	sys 0.019s
I20260709 12:23:46.306705 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2502) took 4188 ms. Trace:
I20260709 12:23:46.307133 20875 rpcz_store.cc:276] 0709 12:23:42.118352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.234236 (+115884us) service_pool.cc:224] Handling call
0709 12:23:42.234566 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:42.244534 (+  9968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.244578 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:42.264081 (+ 19503us) log.cc:844] Serialized 26 byte log entry
0709 12:23:42.632590 (+368509us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:42.632666 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.632780 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:45.905680 (+3272900us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:46.306315 (+400635us) raft_consensus.cc:1685] finished
0709 12:23:46.306345 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.306624 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6201,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":15589632,"thread_start_us":312733,"threads_started":1,"wal-append.queue_time_us":865625}
I20260709 12:23:46.282086 20802 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:46.302241 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2717) took 1378 ms. Trace:
I20260709 12:23:46.311398 20804 rpcz_store.cc:276] 0709 12:23:44.923941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.928630 (+  4689us) service_pool.cc:224] Handling call
0709 12:23:44.953076 (+ 24446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:45.203800 (+250724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.203838 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:45.219915 (+ 16077us) log.cc:844] Serialized 26 byte log entry
0709 12:23:45.439556 (+219641us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:45.439609 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.439686 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.302001 (+862315us) raft_consensus.cc:1685] finished
0709 12:23:46.302021 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.302174 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27825,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":97,"thread_start_us":431101,"threads_started":2,"wal-append.queue_time_us":279787}
I20260709 12:23:46.313853 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:46.316929 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.317719 21428 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
I20260709 12:23:46.318285 21428 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:23:46.280874 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.319376 20734 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:46.322631 20804 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:46.111230 26546 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.276s
I20260709 12:23:46.325801 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 968) took 1091 ms. Trace:
I20260709 12:23:46.291383 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2512) took 2169 ms. Trace:
I20260709 12:23:46.244441 26506 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:23:46.247125 26499 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.325058 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.334024 26511 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.334465 26541 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:46.335057 26488 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.336133 26401 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.244441 26285 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.337363 21427 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:46.338138 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 402864us
I20260709 12:23:46.295550 26523 raft_consensus.cc:2764] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:46.111986 26387 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.329793 20804 rpcz_store.cc:276] 0709 12:23:45.234111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.315119 (+1081008us) service_pool.cc:224] Handling call
0709 12:23:46.322372 (+  7253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.325442 (+  3070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.325737 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:46.279986 26549 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.202s
I20260709 12:23:46.367415 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.350943 20802 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 10.
I20260709 12:23:46.359647 20734 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:23:46.354530 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.353497 20728 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:46.328786 26552 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.281467 26466 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 12:23:46.345182 26390 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:46.352412 26507 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:46.399055 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.016s	sys 0.026s
W20260709 12:23:46.429970 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.019s	sys 0.029s
I20260709 12:23:46.398888 20804 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:46.419629 26511 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.319131 26192 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.430845 21427 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:23:46.352634 20879 rpcz_store.cc:276] 0709 12:23:44.122231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.123257 (+  1026us) service_pool.cc:224] Handling call
0709 12:23:44.366793 (+243536us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:44.367426 (+   633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:44.367474 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:44.368296 (+   822us) log.cc:844] Serialized 52 byte log entry
0709 12:23:45.227875 (+859579us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:45.227915 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.227983 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.291066 (+1063083us) raft_consensus.cc:1685] finished
0709 12:23:46.291095 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.291301 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":778,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":30096,"thread_start_us":702845,"threads_started":1,"wal-append.queue_time_us":807008}
W20260709 12:23:46.430384 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 891954us
I20260709 12:23:46.433867 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:46.433269 21427 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 10
I20260709 12:23:46.436430 21427 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:23:46.436794 21427 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:46.438287 20804 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:46.438032 26511 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:46.438789 26392 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.358s	user 0.012s	sys 0.000s
I20260709 12:23:46.440639 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 970) took 1202 ms. Trace:
I20260709 12:23:46.440989 20802 rpcz_store.cc:276] 0709 12:23:45.237991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.006518 (+768527us) service_pool.cc:224] Handling call
0709 12:23:46.440574 (+434056us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:46.366995 26250 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:46.419610 26541 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:46.372562 26369 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.379560 26399 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:46.382928 26499 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:46.441716 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.463660 21430 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
W20260709 12:23:46.444485 26512 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:23:46.458285 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.381659 25686 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.459396 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.366639 26460 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:23:46.436429 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:46.371889 26489 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.465152 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.455852 20733 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:46.444016 26250 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:46.444556 26495 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:46.432718 26552 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:46.480326 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.004s	user 0.023s	sys 0.046s
I20260709 12:23:46.471493 26390 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:46.489338 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.023s	sys 0.046s
W20260709 12:23:46.489609 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53253.9us
W20260709 12:23:46.489831 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.01521e+06us
I20260709 12:23:46.489979 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.490906 21430 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:23:46.492892 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:46.494072 20776 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:23:46.494434 20776 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:23:46.498495 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.499884 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:46.503157 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:46.483068 21426 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:46.444778 26471 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.487891 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.442225 26392 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.512004 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.513407 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.515208 20733 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:46.515138 21427 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:46.516835 21427 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:46.485905 26495 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:46.508181 26505 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:46.531819 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:46.507740 26488 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 9. Current term is 10.
I20260709 12:23:46.549520 20699 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:46.541718 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.514266 26489 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.554881 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.555163 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:46.556437 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.557133 20727 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.529502 26495 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:46.523367 26471 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:46.578492 20728 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:46.541177 20733 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:46.492370 26499 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.523206 26559 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:23:46.563869 26557 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:23:46.566890 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2714) took 4142 ms. Trace:
I20260709 12:23:46.551716 25686 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:46.568528 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2455) took 1642 ms. Trace:
I20260709 12:23:46.589955 20805 rpcz_store.cc:276] 0709 12:23:44.926369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.130317 (+203948us) service_pool.cc:224] Handling call
0709 12:23:46.568346 (+1438029us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:46.568465 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57733888}
I20260709 12:23:46.569739 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.569794 26392 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.544757 26505 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.571628 26556 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:23:46.571771 26381 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.815s	user 0.009s	sys 0.002s
I20260709 12:23:46.522835 26369 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:23:46.591347 20729 raft_consensus.cc:2410] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:23:46.589133 20735 rpcz_store.cc:276] 0709 12:23:42.424272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.986482 (+562210us) service_pool.cc:224] Handling call
0709 12:23:44.098127 (+1111645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:44.895675 (+797548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.566618 (+1670943us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540028cb50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:46.566797 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2555691264}
I20260709 12:23:46.572485 26506 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:46.594833 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.602908 21428 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:46.607421 20729 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:46.609606 20727 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:46.599897 20733 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:23:46.601080 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 259458us
I20260709 12:23:46.572635 26488 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:46.589826 26524 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.552604 26550 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:23:46.614429 20733 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:46.622153 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.010s	sys 0.007s
I20260709 12:23:46.530905 26541 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:46.626291 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.010s	sys 0.008s
I20260709 12:23:46.627429 20728 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:46.562249 26489 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.593425 26526 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.601866 26505 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:46.602694 26559 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:46.554172 26250 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.588966 26471 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.632362 26524 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:46.602140 25686 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:46.620867 26488 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.611218 26540 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.655469 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.610463 26381 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.657204 21430 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:23:46.664647 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.658048 20728 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 12:23:46.656441 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 352515us
I20260709 12:23:46.668890 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.669749 20735 raft_consensus.cc:2376] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13. Re-sending same reply.
I20260709 12:23:46.670701 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.652928 26192 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:46.664232 26148 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.894s	user 0.010s	sys 0.002s
I20260709 12:23:46.653674 26552 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.682935 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.684732 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.686024 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.687744 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:46.689448 20875 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:46.689658 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.690450 21426 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:23:46.656692 26524 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:46.692207 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:46.692348 20880 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:46.695335 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.696262 20728 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:46.700181 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.701853 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.702508 20879 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:46.703639 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.704514 20882 raft_consensus.cc:2410] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:23:46.701860 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.674399 26250 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.693019 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.691797 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1430) took 1192 ms. Trace:
I20260709 12:23:46.693992 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.709044 20881 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:23:46.656909 26488 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.707213 20877 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:46.709918 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.700527 26524 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:46.692960 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 189903us
I20260709 12:23:46.705381 20735 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:46.705837 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.671813 26148 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:46.709321 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 154260us
I20260709 12:23:46.716265 20728 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:23:46.717171 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.718274 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.718425 26336 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:46.721288 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2718) took 1797 ms. Trace:
I20260709 12:23:46.721740 20810 rpcz_store.cc:276] 0709 12:23:44.924220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.133741 (+209521us) service_pool.cc:224] Handling call
0709 12:23:45.134021 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.235509 (+101488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.235539 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:45.235577 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:45.235614 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.721021 (+1485407us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540019afd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:46.721074 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.721216 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232953600}
I20260709 12:23:46.725055 26148 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.709420 26559 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.725911 20726 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:46.707476 21435 rpcz_store.cc:276] 0709 12:23:45.507539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.986636 (+479097us) service_pool.cc:224] Handling call
0709 12:23:45.996306 (+  9670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.996567 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:46.603714 (+607147us) log.cc:844] Serialized 14 byte log entry
0709 12:23:46.691145 (+ 87431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:46.691530 (+   385us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:46.691572 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.691631 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.691736 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8660,"prepare.queue_time_us":530250,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":553,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6873,"thread_start_us":543622,"threads_started":2}
I20260709 12:23:46.727628 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:46.728438 21435 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:46.732183 26381 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.736981 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2728) took 3063 ms. Trace:
I20260709 12:23:46.746556 20730 rpcz_store.cc:276] 0709 12:23:43.673892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.863388 (+189496us) service_pool.cc:224] Handling call
0709 12:23:43.863756 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:44.937637 (+1073881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.736675 (+1799038us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000e7ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:46.736899 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458290560}
W20260709 12:23:46.737727 26415 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.835s	user 0.010s	sys 0.001s
I20260709 12:23:46.737884 26505 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.738937 26471 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:46.740302 26544 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.846s	user 0.000s	sys 0.016s
I20260709 12:23:46.743501 26526 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.743749 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2706) took 4118 ms. Trace:
I20260709 12:23:46.764065 20806 rpcz_store.cc:276] 0709 12:23:42.625137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.113637 (+488500us) service_pool.cc:224] Handling call
0709 12:23:43.113920 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.133993 (+2020073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.743501 (+1609508us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:46.743668 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4449657600}
W20260709 12:23:46.744163 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.756448 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:46.732221 26502 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:23:46.745493 20774 connection.cc:256] RPC call timeout handler was delayed by -1.2133271400002741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 12:23:46.770332 20774 connection.cc:256] RPC call timeout handler was delayed by -1.0765271400000529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 12:23:46.749243 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:46.751706 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.773582 20728 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:46.709295 26569 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.775041 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 988) took 1548 ms. Trace:
I20260709 12:23:46.746234 26488 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:46.778631 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48133.4us
W20260709 12:23:46.783759 21400 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.560s (SENT)
I20260709 12:23:46.783138 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:46.775799 20878 rpcz_store.cc:276] 0709 12:23:45.226671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.227511 (+   840us) service_pool.cc:224] Handling call
0709 12:23:45.227799 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:45.228149 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:23:45.657479 (+429330us) log.cc:844] Serialized 14 byte log entry
0709 12:23:46.464827 (+807348us) log.cc:844] Serialized 14 byte log entry
0709 12:23:46.464996 (+   169us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:46.465461 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:23:46.469009 (+  3548us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:46.469065 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.469158 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.774729 (+305571us) raft_consensus.cc:1685] finished
0709 12:23:46.774763 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.774953 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68999,"prepare.queue_time_us":37114,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":94,"raft.queue_time_us":215409,"raft.run_cpu_time_us":507,"raft.run_wall_time_us":53041,"thread_start_us":952835,"threads_started":3,"wal-append.queue_time_us":669937}
I20260709 12:23:46.790771 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.792127 20878 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:23:46.793879 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.799299 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:46.799505 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:46.800153 26336 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:46.800352 20877 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:46.801393 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.801640 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:46.804184 20701 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 9 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 9. Current term is 10.
I20260709 12:23:46.770052 26502 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:46.809180 20875 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:46.782397 26390 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:46.810015 26390 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:46.810458 26390 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.813114 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:46.781709 26499 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:46.782070 26471 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:46.816980 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2465) took 1873 ms. Trace:
I20260709 12:23:46.805766 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:46.819296 26336 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:46.821435 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:46.805161 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:46.824443 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.817437 20801 rpcz_store.cc:276] 0709 12:23:44.943454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.222905 (+279451us) service_pool.cc:224] Handling call
0709 12:23:45.264295 (+ 41390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.893294 (+628999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.816796 (+923502us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b540019a5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:46.816931 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371193728}
I20260709 12:23:46.825774 20735 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:46.830260 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:46.794684 26525 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.834228 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:46.835875 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:46.797873 26578 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:46.803620 26576 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.707592 25686 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.842613 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:46.833931 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:46.851176 21400 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.628s (SENT)
I20260709 12:23:46.820858 26331 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.780028 26526 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.870407 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:46.878077 20880 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:46.870407 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.865243 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:46.872838 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1424) took 1528 ms (client timeout 1660 ms). Trace:
W20260709 12:23:46.882072 21433 rpcz_store.cc:269] 0709 12:23:45.344098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.344655 (+   557us) service_pool.cc:224] Handling call
0709 12:23:46.872600 (+1527945us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54007b2ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:46.872760 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879522560}
I20260709 12:23:46.875790 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:46.887372 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:46.832094 26381 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:46.894928 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:46.895556 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 142607us
I20260709 12:23:46.895000 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:46.895740 20882 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:46.898391 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:46.900333 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:46.888494 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:46.909241 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.844956 26552 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:46.840765 26578 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.866415 26580 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.853672 26550 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:46.919564 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:46.832176 26338 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:23:46.864410 26489 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.875531 26389 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:46.794245 26569 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.932443 26389 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:46.932938 26389 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:46.934077 26569 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.920507 20802 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:46.867899 26488 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.915140 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:46.921314 26578 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:46.918691 26552 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:46.937233 26552 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.925460 26571 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:46.918095 26401 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:46.895085 26331 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.934911 26569 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:46.928212 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:46.940749 20731 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:46.929016 26553 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:46.941579 26553 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:46.940451 26569 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.942581 26553 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.929270 25699 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:46.931885 26594 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:46.927591 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:46.944991 26594 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:46.864135 26576 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.939787 26571 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:46.936205 20732 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:46.940522 26580 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.854529 26392 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:23:46.894729 26466 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:46.926712 26192 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:46.894629 26525 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:23:46.936414 26575 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:23:46.924999 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:46.958412 20882 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:23:46.959355 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:46.973392 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56086.6us
I20260709 12:23:46.945818 26571 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:46.905988 26499 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.947986 26505 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:46.945422 26594 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:46.913216 26353 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:46.965889 26401 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.977612 26489 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:46.899619 26502 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.972497 26466 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.003436 26526 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:47.004253 26576 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.004962 26580 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.004650 26389 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.005654 26569 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.002111 26571 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.007359 26571 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:47.008494 26524 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.008963 26389 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:47.010706 26524 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:47.011772 26390 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:46.983781 26525 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:23:47.027726 26525 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:47.028146 26525 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:47.029018 26525 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.030714 26594 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:46.993232 26567 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:23:47.035903 26578 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.037498 26571 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:47.037902 26571 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:47.038789 26571 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.041321 26578 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:47.076967 26597 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:23:47.090806 26499 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:47.101372 26558 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.613s	user 0.003s	sys 0.000s
W20260709 12:23:47.130645 26537 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.584s	user 0.012s	sys 0.010s
W20260709 12:23:46.968092 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54402.7us
I20260709 12:23:47.172552 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:46.948232 20732 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:47.176566 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.178458 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2464) took 1241 ms. Trace:
I20260709 12:23:46.970713 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.184118 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.197687 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.230754 20730 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:47.179049 20726 rpcz_store.cc:276] 0709 12:23:45.937112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.939289 (+  2177us) service_pool.cc:224] Handling call
0709 12:23:47.177888 (+1238599us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54003b4f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:47.178219 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782777728}
I20260709 12:23:47.232461 26521 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:46.963169 20775 connection.cc:256] RPC call timeout handler was delayed by -1.4264672869999231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
I20260709 12:23:47.194721 20806 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:47.220595 20880 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:47.214051 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2723) took 2288 ms. Trace:
I20260709 12:23:47.214594 26331 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:47.000710 26489 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:47.323028 20801 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:47.322609 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:47.334122 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68794.9us
I20260709 12:23:47.336056 26525 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.337510 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.337740 26502 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:47.339290 26556 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.339561 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.340821 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.341163 26594 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:47.342177 26524 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:47.342378 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:47.342748 26524 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.343118 20730 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.344880 26526 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:23:47.345441 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.346627 21433 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:47.348060 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.347965 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.350788 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:47.351387 26522 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.351691 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.351933 21426 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.353328 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.354471 26466 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:46.989953 26401 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.359393 26390 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:47.359937 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.224326 26285 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
W20260709 12:23:47.342204 20775 connection.cc:256] RPC call timeout handler was delayed by -1.3913672869998663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
W20260709 12:23:47.363773 20775 connection.cc:256] RPC call timeout handler was delayed by -1.3013672869997208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
W20260709 12:23:47.364432 20775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.924s (SENT) [suppressed 29 similar messages]
W20260709 12:23:47.364858 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.367291 26524 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:47.368012 20730 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:47.368782 20882 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:23:47.369246 20882 pending_rounds.cc:85] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 6
I20260709 12:23:47.369547 20882 pending_rounds.cc:107] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:23:47.369851 20882 raft_consensus.cc:2889] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:47.134625 26552 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.301901 20807 rpcz_store.cc:276] 0709 12:23:44.925404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.154680 (+229276us) service_pool.cc:224] Handling call
0709 12:23:46.575298 (+1420618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.575680 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:46.575721 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:46.575780 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:46.575833 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.575913 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.213485 (+637572us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54003c7650. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:47.213770 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57992,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":7219,"spinlock_wait_cycles":3266379648}
I20260709 12:23:47.384318 20732 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:47.383965 26489 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.371786 26576 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:47.347841 26511 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.426548 26401 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:47.429111 26506 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.808s	user 0.009s	sys 0.007s
I20260709 12:23:47.429971 26506 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:47.439282 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.440745 20805 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:47.441309 26387 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:47.457872 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:47.458758 20879 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:23:47.470355 21426 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:47.496384 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 987) took 2270 ms. Trace:
I20260709 12:23:47.497007 20883 rpcz_store.cc:276] 0709 12:23:45.225919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.226034 (+   115us) service_pool.cc:224] Handling call
0709 12:23:45.478088 (+252054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:45.478735 (+   647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.495834 (+2017099us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54000a41d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:47.496147 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578902272}
I20260709 12:23:47.500625 26576 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:47.480373 26401 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:47.504954 26401 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.508420 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.517590 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.520278 20731 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:47.520665 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:47.532119 26552 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:47.526994 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.518040 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:47.533546 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 169171us
W20260709 12:23:47.535136 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.010s	sys 0.026s
W20260709 12:23:47.539356 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.010s	sys 0.026s
I20260709 12:23:47.539798 20879 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:23:47.502322 26576 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.542773 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:47.548619 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.007s	sys 0.022s
I20260709 12:23:47.549947 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:47.551218 20729 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:47.551764 26525 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:47.552577 26525 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.555371 26511 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:47.556182 26511 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.558799 26511 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:47.559659 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:47.561575 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 967) took 2329 ms (client timeout 2794 ms). Trace:
I20260709 12:23:47.562439 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:47.567685 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.007s	sys 0.022s
W20260709 12:23:47.563259 20808 rpcz_store.cc:269] 0709 12:23:45.232433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.238405 (+  5972us) service_pool.cc:224] Handling call
0709 12:23:47.561108 (+2322703us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54003c9954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:47.561338 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773456128}
I20260709 12:23:47.569427 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:47.573235 20882 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
W20260709 12:23:47.578469 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 757030us
I20260709 12:23:47.580157 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.581390 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.582120 26351 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:47.583091 26351 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:47.584358 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1148) took 2103 ms (client timeout 1540 ms). Trace:
W20260709 12:23:47.584844 21434 rpcz_store.cc:269] 0709 12:23:45.480795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.481889 (+  1094us) service_pool.cc:224] Handling call
0709 12:23:47.583864 (+2101975us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:47.584086 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108406016}
I20260709 12:23:47.584628 26351 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:47.591048 26250 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.881s	user 0.005s	sys 0.016s
I20260709 12:23:47.593202 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:47.595201 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.589635 26525 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.600133 21434 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:47.597788 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.597390 26338 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:47.617074 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 980) took 1234 ms. Trace:
I20260709 12:23:47.611644 26338 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:47.617660 21430 rpcz_store.cc:276] 0709 12:23:46.382470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.523542 (+141072us) service_pool.cc:224] Handling call
0709 12:23:46.523835 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.664781 (+140946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.616537 (+951756us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:47.616865 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299277184}
I20260709 12:23:47.618726 26338 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.582103 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.601605 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1427) took 2083 ms. Trace:
I20260709 12:23:47.583424 26309 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.610685 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.611094 26511 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:47.610153 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69750.8us
W20260709 12:23:47.611393 26495 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.021s	user 0.007s	sys 0.018s
I20260709 12:23:47.622246 26571 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:47.634816 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:47.580963 20801 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:47.602538 26410 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:47.572129 26401 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:47.680699 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98706.5us
I20260709 12:23:47.681098 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:47.672879 26511 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.682753 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:47.683512 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:47.685036 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:47.685529 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:47.686393 26565 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.687254 26565 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.688463 26309 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.689999 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:47.690558 26565 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:47.691438 20850 leader_election.cc:336] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.259s (SENT)
I20260709 12:23:47.691646 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.691991 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:47.692297 21426 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:47.676674 26495 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.696343 26495 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.696426 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.691458 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:47.697681 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.698107 21428 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.699970 26565 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:47.702131 26285 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:47.702332 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:47.702767 26285 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:47.702965 26587 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:47.685075 26511 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:47.703315 26285 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:47.703588 26587 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.685460 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.704103 26587 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.704572 21435 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:47.705262 20802 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:47.705734 26590 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:47.706385 26590 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:47.702160 26415 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.707981 26415 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.710793 26415 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.711138 26594 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.711462 26415 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:47.711867 26415 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:47.711833 26594 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.712442 26415 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:47.716804 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:47.718597 26599 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.718878 26590 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.719841 26526 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:47.721087 26392 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:47.704263 26338 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.690965 26309 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.724797 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:47.725059 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 766328us
I20260709 12:23:47.730897 21434 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:47.731961 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.733067 21431 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:23:47.735147 26488 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:47.735939 26488 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:47.738327 26488 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.739625 26381 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:47.740127 26381 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.740471 26594 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:47.740389 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:47.744340 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:47.744618 26506 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:47.745283 26506 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:47.740566 26381 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:47.771145 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:47.741801 26614 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:23:47.721120 26599 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:47.781471 21428 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:47.782922 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.660640 21432 rpcz_store.cc:276] 0709 12:23:45.458764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.476337 (+ 17573us) service_pool.cc:224] Handling call
0709 12:23:45.488716 (+ 12379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:46.331672 (+842956us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:46.331715 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:46.332165 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 12:23:46.398197 (+ 66032us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b180046d4a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:46.612279 (+214082us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:46.612336 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.384040 (+771704us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b54005f4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:47.384117 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:47.384202 (+    85us) raft_consensus.cc:1685] finished
0709 12:23:47.384216 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.541647 (+157431us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":617,"raft.queue_time_us":31427,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":1574,"spinlock_wait_cycles":1946710016,"thread_start_us":213823,"threads_started":1,"wal-append.queue_time_us":348623}
I20260709 12:23:47.792277 21427 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 12:23:47.794556 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.794801 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.795249 26578 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:23:47.795912 26600 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:47.796651 26578 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:47.796986 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.798062 26600 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:47.798357 26513 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:47.799002 26513 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:47.799777 26513 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:47.721343 26552 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:23:47.797127 26578 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:47.798548 26600 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.782934 20734 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:23:47.812225 26600 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:47.816144 21435 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:23:47.818611 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 881) took 1190 ms. Trace:
I20260709 12:23:47.819490 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.819890 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:47.820549 21432 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:47.825218 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:47.746702 26506 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.827298 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.827718 26320 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:47.828003 20727 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:47.828199 26320 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:47.828603 26320 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.809698 26594 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:47.738888 26587 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.819077 21435 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:47.773216 26392 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:47.781505 21434 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:47.819022 20734 rpcz_store.cc:276] 0709 12:23:46.628303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.812303 (+184000us) service_pool.cc:224] Handling call
0709 12:23:46.812595 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:47.817583 (+1004988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.818327 (+   744us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b54000e92d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:47.818535 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2215989504}
I20260709 12:23:47.819917 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.839299 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:47.839617 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 145949us
I20260709 12:23:47.803604 26599 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:47.846313 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.849278 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:47.849360 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:47.852139 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.854081 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.861586 20883 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:47.863157 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:47.863464 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.865424 21431 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:47.868853 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49120.2us
I20260709 12:23:47.794884 26524 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:47.875748 20801 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:47.882176 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:47.896514 21427 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:47.880378 26526 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:47.881705 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:47.884449 20731 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:47.909395 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:47.921814 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:47.916867 20806 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:47.927415 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 147813us
I20260709 12:23:47.940729 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:47.932139 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:47.910895 26526 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.968587 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:47.969997 20850 connection.cc:256] RPC call timeout handler was delayed by -1.1942894039998464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3178s for a response
I20260709 12:23:47.952955 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:47.953792 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:47.981482 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 116783us
I20260709 12:23:47.985350 21435 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:47.979737 21427 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:48.009572 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:47.954706 26526 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:48.022969 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.029501 20883 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:48.034597 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 105331us
I20260709 12:23:48.037456 26353 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:23:48.038573 26353 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:48.039019 26353 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:48.039388 26501 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:48.040408 26505 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:48.040825 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.040925 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:48.041599 20875 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:48.042225 26501 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:48.042520 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.042730 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:48.039469 20700 leader_election.cc:336] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.677s (SENT)
I20260709 12:23:48.043287 20728 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:23:48.034024 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:48.051775 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:48.058494 21431 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:48.059814 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.049105 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.056159 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:48.068480 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:48.010771 26565 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.076217 20875 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:48.061131 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:48.049928 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:48.078369 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:48.025816 26578 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:48.120349 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 269745us
I20260709 12:23:48.128357 20878 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:48.129165 26612 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:48.129758 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.130597 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:48.133648 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.134595 20880 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:23:48.135620 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:48.137588 20882 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:48.141462 20806 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:48.153276 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.065958 26401 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:48.178059 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 128206us
I20260709 12:23:48.190928 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.192052 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.154421 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:48.202632 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.214915 20877 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:48.228557 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.229317 20880 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:48.011627 26392 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:48.244277 21430 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:48.245925 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:48.000382 26599 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:48.250135 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 271224us
I20260709 12:23:48.177467 26587 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:48.257721 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.040225 26353 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.268828 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:48.270171 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.272359 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:48.275734 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.284704 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:48.293795 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.285686 21430 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:48.286391 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:48.309866 20880 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:48.250623 26634 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:48.316860 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:48.306284 20878 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:48.057147 26501 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.257566 26569 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.273969 26599 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:48.313436 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:48.324525 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50637.3us
I20260709 12:23:48.221292 26401 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.323850 21427 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:48.243484 26488 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:23:48.344410 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.294868 26489 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:48.311100 26502 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:48.359320 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84532.8us
I20260709 12:23:48.353283 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:48.312425 26524 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:48.348019 20882 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:48.361966 26567 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.362221 20875 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:23:48.362958 26626 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:23:48.362597 26236 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:23:48.363320 26580 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:48.364104 20880 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:23:48.364203 26630 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.262s	user 0.000s	sys 0.002s
W20260709 12:23:48.364338 26633 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:23:48.366299 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 967) took 2438 ms. Trace:
I20260709 12:23:48.367331 26495 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:48.371472 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 506098us
I20260709 12:23:48.372081 26569 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:48.385771 26619 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.351s	user 0.002s	sys 0.000s
I20260709 12:23:48.386417 26502 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:48.388736 26624 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.410s	user 0.000s	sys 0.001s
I20260709 12:23:48.341295 26506 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:48.398787 26353 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:48.405632 26636 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:23:48.411983 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 129129us
W20260709 12:23:48.413964 26627 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
W20260709 12:23:48.415006 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.416466 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:48.418743 26489 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:48.422163 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 267839us
W20260709 12:23:48.398458 26513 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.000s
I20260709 12:23:48.274026 26505 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:23:48.464829 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:48.473448 21435 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:23:48.468528 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 112294us
I20260709 12:23:48.485834 20883 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:23:48.500157 20875 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:48.541123 20878 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:23:48.550200 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:48.509378 26506 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:48.511013 26580 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:48.536528 26410 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.856s	user 0.010s	sys 0.004s
I20260709 12:23:48.483438 21429 rpcz_store.cc:276] 0709 12:23:45.928008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.109826 (+181818us) service_pool.cc:224] Handling call
0709 12:23:47.584190 (+1474364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:47.584963 (+   773us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:23:47.585345 (+   382us) log.cc:844] Serialized 14 byte log entry
0709 12:23:47.726490 (+141145us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:47.812621 (+ 86131us) log.cc:844] Serialized 26 byte log entry
0709 12:23:47.812835 (+   214us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:47.812869 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.833477 (+ 20608us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:48.069307 (+235830us) raft_consensus.cc:1685] finished
0709 12:23:48.069330 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:48.365903 (+296573us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54005302d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:48.366077 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":766,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":4858,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":19662,"spinlock_wait_cycles":3389383040,"thread_start_us":138337,"threads_started":1,"wal-append.queue_time_us":160022}
W20260709 12:23:48.623960 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159255us
I20260709 12:23:48.625770 20880 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:48.633177 20878 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:23:48.633739 20878 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 10
I20260709 12:23:48.634003 20878 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:23:48.634214 20878 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:48.640390 20883 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:48.567164 26506 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:48.530720 26495 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.607125 26634 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:23:48.656431 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:48.658043 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:48.661048 26590 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:48.661141 26567 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:48.661749 26590 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:48.654681 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:48.663393 26309 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:48.666431 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:48.667346 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:48.668408 26538 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.669014 26538 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.672175 26634 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:48.647073 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:48.541976 26513 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:48.701757 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:48.703544 26629 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.600s	user 0.001s	sys 0.004s
I20260709 12:23:48.704433 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:48.643635 26524 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:48.680305 26401 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:48.714591 20884 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:23:48.714975 20884 pending_rounds.cc:85] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 7
I20260709 12:23:48.715207 20884 pending_rounds.cc:107] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:23:48.715449 20884 raft_consensus.cc:2889] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:48.720070 26469 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.720662 26469 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:48.721376 26578 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:48.709903 26285 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.006s	user 0.008s	sys 0.007s
W20260709 12:23:48.709721 26381 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.927s	user 0.011s	sys 0.002s
W20260709 12:23:48.736737 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 506932us
I20260709 12:23:48.711558 26488 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:48.711790 26524 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:23:48.748433 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 198378us
I20260709 12:23:48.708331 26538 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:48.725823 26600 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.904s	user 0.005s	sys 0.009s
I20260709 12:23:48.744500 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:48.662871 26590 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:48.757665 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:48.746795 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:48.761651 21431 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:48.741151 26594 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.908s	user 0.009s	sys 0.005s
W20260709 12:23:48.741128 26571 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.057s	user 0.014s	sys 0.000s
I20260709 12:23:48.738425 26410 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:48.764534 26600 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:48.746142 26469 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:48.771013 21427 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:48.772644 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2547) took 3560 ms. Trace:
I20260709 12:23:48.751132 26576 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:48.740238 26638 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:48.776130 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71922.3us
W20260709 12:23:48.734984 26526 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.002s
I20260709 12:23:48.792414 20876 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:48.752791 26524 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:48.797731 20884 rpcz_store.cc:276] 0709 12:23:45.211825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.212003 (+   178us) service_pool.cc:224] Handling call
0709 12:23:45.212306 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:48.715945 (+3503639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.772305 (+ 56360us) spinlock_profiling.cc:243] Waited 3.44 s on lock 0x7b54001d01d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:48.772578 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7915573632}
W20260709 12:23:48.775053 26641 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:23:48.804034 20881 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
W20260709 12:23:48.794945 26320 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.957s	user 0.011s	sys 0.004s
I20260709 12:23:48.783025 26594 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:48.737746 26285 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:48.824898 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:48.741231 26381 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:48.833089 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 278247us
I20260709 12:23:48.795446 26410 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:48.804601 26488 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.852043 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:48.844404 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:48.831167 20773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.043s	sys 0.041s
I20260709 12:23:48.801250 26576 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:48.854069 20773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.043s	sys 0.041s
I20260709 12:23:48.823076 25686 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:48.830327 26567 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:48.882513 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57958us
I20260709 12:23:48.886493 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1448) took 1169 ms. Trace:
I20260709 12:23:48.886739 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:48.886981 21426 rpcz_store.cc:276] 0709 12:23:47.717285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.717441 (+   156us) service_pool.cc:224] Handling call
0709 12:23:47.781342 (+ 63901us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:47.875411 (+ 94069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:47.875465 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:47.926976 (+ 51511us) log.cc:844] Serialized 26 byte log entry
0709 12:23:48.431482 (+504506us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:48.431544 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.431640 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:48.886255 (+454615us) raft_consensus.cc:1685] finished
0709 12:23:48.886279 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:48.886430 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62561,"raft.queue_time_us":55061,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":35614,"thread_start_us":396540,"threads_started":1,"wal-append.queue_time_us":536460}
W20260709 12:23:48.889535 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:48.890002 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.841490 26410 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:48.905748 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 103250us
I20260709 12:23:48.826485 26526 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.881644 26638 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:48.881547 26571 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:48.881546 25686 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.915537 26538 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:48.917039 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:48.834645 26320 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.816536 26580 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:48.920485 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.918185 26638 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.934474 26638 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:48.929050 21432 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.786s	user 0.013s	sys 0.000s
W20260709 12:23:48.929234 26552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.050s	sys 0.093s
W20260709 12:23:48.953720 26552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.051s	sys 0.094s
I20260709 12:23:48.923035 26571 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:48.926563 25686 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:48.889292 26488 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:48.936757 26634 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.888046 26567 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:48.936982 26569 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:48.983888 21401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.016s	sys 0.018s
W20260709 12:23:48.984455 21401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.017s	sys 0.019s
W20260709 12:23:48.984961 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.948395 26513 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:48.946949 21432 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
W20260709 12:23:48.988523 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.991869 26600 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:48.996286 26599 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.657s	user 0.005s	sys 0.006s
W20260709 12:23:48.996771 26643 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:23:48.998814 21431 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
W20260709 12:23:49.000017 20877 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.002s
I20260709 12:23:49.000386 20877 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:49.000921 20880 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:49.001534 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:49.002197 20877 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:49.002818 20880 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:48.963260 26576 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.011353 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:48.971675 26571 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:49.029327 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57955.1us
I20260709 12:23:48.980850 26502 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.033690 20881 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:23:49.038055 20881 pending_rounds.cc:85] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 13
I20260709 12:23:49.038431 20881 pending_rounds.cc:107] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:23:49.038728 20881 raft_consensus.cc:2889] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:49.021735 26511 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:49.042582 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.022257 26649 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:49.028034 26489 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.051357 26538 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:49.053320 26644 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:49.037204 26236 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:49.056049 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.060299 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:49.063982 26526 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.020504 26587 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.065022 26587 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.068866 26488 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.066881 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:49.069962 21427 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:23:49.070091 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:49.074064 26642 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.402s	user 0.000s	sys 0.004s
W20260709 12:23:49.078398 26415 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.365s	user 0.009s	sys 0.004s
I20260709 12:23:49.079763 26285 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.080757 21431 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:49.085657 26495 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:49.035420 26392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.004s	sys 0.042s
W20260709 12:23:49.086688 26392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.005s	sys 0.042s
W20260709 12:23:49.093261 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45020.2us
I20260709 12:23:49.095621 26381 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.099798 20877 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:48.987365 26634 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:49.102892 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 751848us
I20260709 12:23:49.078781 26638 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:49.111603 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 142881us
I20260709 12:23:49.112212 26526 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:49.110378 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.116302 20876 raft_consensus.cc:2410] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:23:49.116750 20882 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.640s	user 0.013s	sys 0.000s
I20260709 12:23:49.117635 20882 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:49.118338 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1007) took 1011 ms. Trace:
I20260709 12:23:49.063596 26410 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.118772 20882 rpcz_store.cc:276] 0709 12:23:48.106482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.190450 (+ 83968us) service_pool.cc:224] Handling call
0709 12:23:49.118261 (+927811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:49.098992 26649 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:49.138523 20882 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:23:49.132203 26338 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.406s	user 0.015s	sys 0.000s
I20260709 12:23:49.140568 26338 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:49.145066 20883 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:49.123991 26392 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:49.078866 26415 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.153600 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:49.156258 21428 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:49.135061 26594 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:49.135254 26495 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.176390 21427 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:23:49.179121 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 471497us
I20260709 12:23:49.122855 26502 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:23:49.150099 26392 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:49.191483 26320 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.178551 26587 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.206692 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:49.206017 26353 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.690s	user 0.002s	sys 0.011s
I20260709 12:23:49.198252 26594 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:49.219919 20875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.013s	sys 0.025s
W20260709 12:23:49.255295 20875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.013s	sys 0.025s
W20260709 12:23:49.255820 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 424663us
W20260709 12:23:49.222837 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.196322 26502 consensus_queue.cc:1243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:23:49.217754 20618 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.015s	sys 0.029s
W20260709 12:23:49.259990 20618 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.015s	sys 0.029s
W20260709 12:23:49.260681 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 325965us
W20260709 12:23:49.286450 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 228762us
I20260709 12:23:49.217450 26638 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:49.258405 26506 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.002s
I20260709 12:23:49.291492 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:49.290434 21426 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:23:49.297077 21426 pending_rounds.cc:85] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 11
I20260709 12:23:49.297366 21426 pending_rounds.cc:107] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:23:49.297154 21428 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.297695 21426 raft_consensus.cc:2889] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:49.287443 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:49.202982 26495 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.295708 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.284057 26236 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:49.216037 26649 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.315356 26495 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.316242 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.316461 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:49.317445 26495 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:49.318013 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:49.299288 26578 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.577s	user 0.014s	sys 0.001s
I20260709 12:23:49.321667 21431 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:49.331841 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 237650us
I20260709 12:23:49.289582 26285 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:23:49.343250 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.323120 26580 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:49.348970 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.331275 26469 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:49.359933 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 247531us
I20260709 12:23:49.304975 26353 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:23:49.302404 26506 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.374596 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.376845 20876 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.377332 20882 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:49.340644 26410 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:49.375988 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:49.349040 26320 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.336915 26587 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:23:49.384969 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:49.327672 26392 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.359215 26495 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.388042 21428 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:49.399847 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.373747 26580 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:49.340399 26501 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 12:23:49.417261 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 129703us
I20260709 12:23:49.345918 26538 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:23:49.415282 21431 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
W20260709 12:23:49.408775 20700 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.505s (SENT)
I20260709 12:23:49.378551 26634 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.366142 26599 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:23:49.446285 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:49.452232 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.452950 20878 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:49.455283 20882 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
W20260709 12:23:49.455695 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75818.2us
I20260709 12:23:49.456988 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.458515 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.444460 26488 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:49.432528 26502 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:49.403792 26320 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.467751 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:49.417941 26495 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:49.473915 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82997.1us
I20260709 12:23:49.473840 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:49.476415 26663 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.381664 26594 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.483772 26655 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:49.483572 26647 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 12:23:49.484251 26655 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:49.485394 26675 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:49.485343 26655 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.483495 26664 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:49.486836 26655 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:49.487111 26661 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:23:49.487437 26664 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:49.501699 26662 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:23:49.502034 26664 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.503422 26663 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.470214 26488 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:49.506633 26488 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.507496 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:49.508031 26488 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.508493 20879 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:49.510644 20879 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:49.512223 26506 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.514019 26511 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.514511 26511 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:49.477355 26502 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:49.523968 26665 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:23:49.524729 26320 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:49.525808 26651 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260709 12:23:49.529253 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 186643us
I20260709 12:23:49.532076 26612 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:49.532428 26410 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.532718 26495 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:49.483536 26646 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:23:49.533501 26495 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:49.533656 26410 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.486055 26675 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.534687 26675 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:49.535989 26410 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:49.488402 26580 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:49.539606 26571 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 12:23:49.540066 26571 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.532500 26672 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:49.474611 26657 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:49.483484 26671 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:23:49.487524 26658 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.540989 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:49.478720 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:49.543324 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:49.582525 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:23:49.498265 25686 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.002s
W20260709 12:23:49.548861 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70314.8us
I20260709 12:23:49.589316 20882 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 10.
I20260709 12:23:49.591427 20884 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:49.506664 26663 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:49.599427 26524 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.759s	user 0.007s	sys 0.005s
I20260709 12:23:49.599961 20879 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:49.599728 26664 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:49.599014 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 131365us
I20260709 12:23:49.602187 26664 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.602332 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:49.603632 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:49.604797 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:49.601437 26401 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:49.605804 26401 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:49.606249 26401 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:49.608995 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:49.610486 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:49.611131 20803 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:23:49.611487 26495 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:49.611584 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:49.612193 26495 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:49.612146 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:49.612949 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 123096us
I20260709 12:23:49.613073 26495 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:49.613517 26565 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:49.613607 26495 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:49.614313 26495 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:49.614837 26489 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:49.599926 26524 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.617229 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:49.617882 20879 raft_consensus.cc:606] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:49.618266 20879 raft_consensus.cc:614] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:49.618854 26524 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.621142 26511 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:49.632704 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.487654 26674 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.679870 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:49.680894 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.590327 26672 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:49.532521 26612 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:49.688850 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73878.6us
W20260709 12:23:49.691256 20700 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.592s (SENT)
I20260709 12:23:49.693905 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:49.694509 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2549) took 1346 ms. Trace:
I20260709 12:23:49.697971 20878 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:49.691287 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:49.638626 26502 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.681504 26663 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:49.688408 26538 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:49.723868 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 250441us
I20260709 12:23:49.694931 20875 rpcz_store.cc:276] 0709 12:23:48.344168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.554039 (+209871us) service_pool.cc:224] Handling call
0709 12:23:48.554592 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:49.255809 (+701217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:49.255839 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:49.292360 (+ 36521us) log.cc:844] Serialized 26 byte log entry
0709 12:23:49.403642 (+111282us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:49.403683 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.403765 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:49.690601 (+286836us) raft_consensus.cc:1685] finished
0709 12:23:49.690621 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.690782 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53214,"raft.queue_time_us":97124,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":695384,"threads_started":2,"wal-append.queue_time_us":181242}
I20260709 12:23:49.598054 25686 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.715940 26578 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.732622 20878 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:23:49.735849 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:49.735078 25686 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.738021 26489 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:49.738307 26652 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.738487 26401 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.739112 26652 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.762269 26652 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.765725 26652 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:49.611025 26675 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.739976 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.696946 26657 raft_consensus.cc:2764] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:23:49.740666 26594 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:49.627635 26658 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.727636 26511 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:49.749956 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:49.728329 26571 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.777411 26663 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:49.793212 26663 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.793687 26663 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:49.755496 26671 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:49.758273 26410 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:49.754871 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:49.768718 20879 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:49.738632 26489 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.775218 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.782684 20882 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 12:23:49.797910 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 106697us
I20260709 12:23:49.810034 20876 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:49.730279 26538 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.826943 26538 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.811385 26513 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:49.828667 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78866.2us
I20260709 12:23:49.829114 26513 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:49.829645 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:49.830133 26513 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.812860 26488 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.813279 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:49.831504 26569 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:49.794263 26495 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.834383 26353 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:49.835602 26353 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:49.835965 26353 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:49.836580 26495 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.797567 26612 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.782735 26309 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:23:49.818101 26672 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:49.820622 26675 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.811692 26524 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:23:49.824852 26489 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:49.834112 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:49.853539 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:49.855461 26489 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.856271 26489 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:49.856812 26489 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:49.702422 26674 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:49.851346 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96736us
I20260709 12:23:49.851835 26524 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:49.861011 26524 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:49.862258 26524 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.864480 26524 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.836871 26353 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:49.844468 20850 leader_election.cc:336] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.535s (SENT)
I20260709 12:23:49.850847 26612 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:49.872815 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:49.876048 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:49.883720 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:49.884227 21431 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:23:49.862311 26663 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.877971 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:49.882750 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:49.894877 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:49.897954 21433 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:49.895044 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:49.899220 20802 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:49.900240 20808 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:49.903898 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:49.900257 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:49.910056 20807 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:23:49.907912 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:49.912617 20734 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:49.913079 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:49.917532 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.923853 21428 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:23:49.924553 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:49.924926 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 80483.6us
I20260709 12:23:49.925246 21427 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:49.921509 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:49.925846 20850 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.535s (SENT)
I20260709 12:23:49.926481 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:49.918572 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:49.921034 20809 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:23:49.927721 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:49.919317 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:49.921715 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:49.930637 21432 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:23:49.931770 20804 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:49.932544 21435 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:23:49.933682 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:49.903826 26657 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:49.935364 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:49.918133 26699 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.937966 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:49.938964 20805 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:23:49.917408 26495 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.917708 26674 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.939121 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:49.920903 26594 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:23:49.941035 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.944378 20809 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:49.921111 26353 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:49.933178 26672 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.946300 20804 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:49.935353 26511 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:49.936550 26401 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:49.952939 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:49.951370 20805 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:49.944326 26675 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:49.960783 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:49.939417 26691 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:49.915501 26489 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.946350 26590 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:49.998726 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:49.999992 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 74192.3us
W20260709 12:23:50.000762 20850 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.540s (SENT)
I20260709 12:23:50.001358 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:49.997571 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:49.998948 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:49.944870 26495 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.940872 26657 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:49.987128 26699 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:49.992535 26655 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.016089 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.017313 26495 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:50.017922 21399 leader_election.cc:336] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.628s (SENT)
W20260709 12:23:50.020401 26696 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:23:50.020797 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:49.859467 26682 consensus_queue.cc:1175] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 17 status { last_received { term: 16 index: 12 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:50.027143 26488 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:23:50.027896 26694 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:23:50.035542 26671 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:49.969113 26653 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 12:23:49.989957 26691 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:50.046319 26567 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:50.046609 26571 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:50.053704 26511 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.056905 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:50.057605 26415 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:50.058830 26525 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:23:50.069054 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56362.7us
I20260709 12:23:50.072872 26655 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:50.074488 26663 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:49.969384 26672 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.002370 26401 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
W20260709 12:23:50.143842 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 187701us
I20260709 12:23:50.061896 26699 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:50.097148 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79244.9us
I20260709 12:23:50.100747 21433 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:50.121134 26654 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:50.135537 21427 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:50.123494 20875 raft_consensus.cc:2376] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17. Re-sending same reply.
I20260709 12:23:50.127070 26511 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:50.122884 26571 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:50.145535 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 144816us
W20260709 12:23:50.119405 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45568.2us
W20260709 12:23:50.141793 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 181107us
W20260709 12:23:50.184641 20850 leader_election.cc:336] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.620s (SENT)
I20260709 12:23:50.185097 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:50.154104 26401 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:50.155242 21427 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:50.174665 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.143357 26653 consensus_queue.cc:1243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:23:50.159525 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:50.164691 26691 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:50.166170 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.168812 26672 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:50.176889 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.126663 26525 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:50.155776 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.187541 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.187846 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:50.205108 21427 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:50.205329 20879 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:23:50.149663 26699 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:50.207857 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:50.201146 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:50.208952 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.205245 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:50.209728 21428 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.207216 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.207857 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.195252 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2594) took 1232 ms. Trace:
I20260709 12:23:50.210953 20734 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:23:50.213609 20875 raft_consensus.cc:2410] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:23:50.211381 20881 rpcz_store.cc:276] 0709 12:23:48.963154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.963317 (+   163us) service_pool.cc:224] Handling call
0709 12:23:48.975900 (+ 12583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.104038 (+128138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.191939 (+1087901us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b540019cdd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:50.195159 (+  3220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123103616}
I20260709 12:23:50.216333 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:50.217276 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:50.186740 26401 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:50.217854 20876 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:50.209569 20735 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:50.216384 20732 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:23:50.222952 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.223062 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.222007 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:50.204707 26657 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:50.186172 26654 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.234965 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.235334 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.191826 26663 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:23:50.191681 26567 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:23:50.217592 26688 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:50.243260 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:50.244237 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:50.245286 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60672.6us
I20260709 12:23:50.236573 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:50.248346 26401 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.248828 20881 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:23:50.245018 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.195372 26565 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:23:50.249320 20734 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:50.249747 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:50.207638 26691 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 12:23:50.252919 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57109.5us
I20260709 12:23:50.213573 26695 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:23:50.232275 26469 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:50.284489 26469 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:50.271955 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:50.232583 26681 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:23:50.276358 20850 leader_election.cc:336] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.841s (SENT)
I20260709 12:23:50.277163 26401 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:50.277405 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.229513 26710 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:50.299225 20734 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:50.303632 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.294700 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:50.308928 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.310133 20880 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:50.310047 21427 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:50.315634 20876 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:50.316704 20881 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:50.312177 21428 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:50.321090 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:50.328625 21427 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:50.328903 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:50.331653 20727 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:50.331012 21430 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
W20260709 12:23:50.333132 21399 leader_election.cc:336] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.858s (SENT)
I20260709 12:23:50.333793 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:50.299431 26653 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.342696 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.307742 26710 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.351445 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:50.324440 26392 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:50.355857 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.343547 20734 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:50.354975 21428 pending_rounds.cc:85] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 12
I20260709 12:23:50.359846 21428 pending_rounds.cc:107] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:23:50.360301 21428 raft_consensus.cc:2889] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:50.361652 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:50.335450 26524 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:50.365635 21427 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:23:50.365540 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 97401.5us
W20260709 12:23:50.384836 20850 leader_election.cc:336] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.929s (SENT)
I20260709 12:23:50.335255 26695 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:50.388988 20810 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:23:50.356498 26710 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:50.357509 26353 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:50.338579 26488 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:23:50.394686 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:50.368173 26495 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:50.369022 26655 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:50.368594 26392 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:50.422964 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61379.8us
I20260709 12:23:50.383847 26670 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:50.357296 26489 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:23:50.342175 26653 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.428128 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.392781 26524 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:50.430313 20810 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:23:50.432328 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:50.425307 26657 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:50.427510 26670 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:50.416935 26695 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:50.446398 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.436661 26713 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:50.464406 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.486035 20810 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:50.437208 26657 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.486689 26713 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:50.504079 26713 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.448045 26697 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:50.507825 26670 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:50.477381 26714 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:23:50.451054 26392 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:50.636770 26723 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:23:50.638918 26538 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:50.454015 26511 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:50.638998 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:50.642203 20700 leader_election.cc:336] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 3.322s (SENT)
I20260709 12:23:50.648645 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:50.467867 26653 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:50.466421 26695 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:50.473443 26654 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:50.443979 26488 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.196s
I20260709 12:23:50.454236 26469 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:50.650177 26707 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.366s	user 0.001s	sys 0.003s
I20260709 12:23:50.697803 26511 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:50.523661 26657 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.652420 26513 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.705065 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1000) took 1661 ms. Trace:
I20260709 12:23:50.705821 21426 rpcz_store.cc:276] 0709 12:23:49.043350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.046956 (+  3606us) service_pool.cc:224] Handling call
0709 12:23:49.048160 (+  1204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.303502 (+255342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.704641 (+1401139us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400525154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:50.704959 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481119488}
I20260709 12:23:50.639905 26538 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:50.717861 20804 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:50.722786 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:50.468374 26524 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.653654 26670 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.750617 20731 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:50.753369 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.756862 20810 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:50.713155 26513 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:50.752785 26236 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:50.769639 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 122178us
I20260709 12:23:50.772863 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:50.777047 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:50.775072 26236 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.794380 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.779933 26725 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:50.811825 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:50.814257 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 274693us
I20260709 12:23:50.807773 26392 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:50.808548 26725 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:50.831465 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.837127 20801 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:50.824076 26469 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:50.853295 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.857385 20805 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:50.858451 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.870222 20726 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:50.876875 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.877015 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:50.877928 20733 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:50.878198 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:50.886328 20728 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:50.884256 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:50.900844 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:50.901799 20801 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:50.885056 26731 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:50.911808 26733 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:23:50.911479 26725 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.919656 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:50.922222 26732 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:23:50.908661 26392 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:50.899610 26511 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:50.885526 26740 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
W20260709 12:23:50.921478 26729 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:23:50.931962 20805 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:50.937209 20802 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:50.930824 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.943775 20809 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:23:50.921958 26731 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.949178 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.949234 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.952962 20802 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:50.923804 26710 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:50.928542 26513 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:50.928889 26576 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:50.939814 26644 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:50.936111 26511 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:50.960327 26699 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.741s	user 0.004s	sys 0.010s
W20260709 12:23:50.967494 20700 leader_election.cc:336] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.518s (SENT)
I20260709 12:23:50.963317 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.969278 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:50.969868 20802 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:50.951210 26731 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:50.970721 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:50.973299 20809 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:50.966670 26511 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.931423 26571 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:50.976719 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.979995 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:50.978425 20700 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.587s (SENT)
I20260709 12:23:50.978235 20802 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:50.982004 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:50.961460 26644 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.985102 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.994298 20805 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:50.982622 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:50.985045 26741 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 12:23:50.988387 26718 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.541s	user 0.006s	sys 0.003s
I20260709 12:23:50.967974 26713 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:50.992259 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.999570 20802 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:23:50.972095 26699 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.006996 20809 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:51.004065 26740 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:51.012943 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.965620 26525 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:51.020459 20700 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.632s (SENT)
I20260709 12:23:51.021184 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.059738 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2604) took 2075 ms. Trace:
I20260709 12:23:51.009007 26571 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:51.060715 20883 rpcz_store.cc:276] 0709 12:23:48.983972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.009535 (+ 25563us) service_pool.cc:224] Handling call
0709 12:23:49.031631 (+ 22096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.145610 (+113979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.059325 (+1913715us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003f98d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:51.059632 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260279168}
I20260709 12:23:51.066409 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.067059 26740 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:51.066545 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46152.8us
I20260709 12:23:51.075314 26525 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:51.080169 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.082077 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:50.998446 26392 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.091725 26538 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:51.020841 26710 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.094949 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1175) took 1193 ms. Trace:
I20260709 12:23:51.095536 21430 rpcz_store.cc:276] 0709 12:23:49.901248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.901714 (+   466us) service_pool.cc:224] Handling call
0709 12:23:51.094669 (+1192955us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b5400088c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:51.094833 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925925632}
I20260709 12:23:51.099466 26663 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:51.100047 26663 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:51.101125 26663 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.067358 20805 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:51.110913 26689 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.303s	user 0.004s	sys 0.006s
I20260709 12:23:51.107795 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.112993 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2608) took 1772 ms. Trace:
I20260709 12:23:51.110702 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.113351 20877 rpcz_store.cc:276] 0709 12:23:49.341034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.349654 (+  8620us) service_pool.cc:224] Handling call
0709 12:23:49.377255 (+ 27601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:49.384540 (+  7285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:49.384580 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:49.397430 (+ 12850us) log.cc:844] Serialized 26 byte log entry
0709 12:23:49.650011 (+252581us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:49.650073 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.650183 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:51.110252 (+1460069us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:51.112599 (+  2347us) raft_consensus.cc:1685] finished
0709 12:23:51.112630 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.112928 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34886,"raft.queue_time_us":94624,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":19980,"spinlock_wait_cycles":4931968,"thread_start_us":192425,"threads_started":1,"wal-append.queue_time_us":301919}
I20260709 12:23:51.113395 20807 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:23:51.113816 26236 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.128410 26699 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.102980 26663 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:51.129856 26236 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:51.131811 26745 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:23:51.133831 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.135028 20804 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:51.135521 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.136590 20804 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:23:51.139415 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.140792 26712 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:51.141774 26652 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:51.141409 26644 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:51.142791 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 757944us
I20260709 12:23:51.145005 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2564) took 1415 ms. Trace:
I20260709 12:23:51.162510 20882 rpcz_store.cc:276] 0709 12:23:49.730056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.743682 (+ 13626us) service_pool.cc:224] Handling call
0709 12:23:49.744220 (+   538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:49.808265 (+ 64045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.144664 (+1336399us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:51.144922 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87663488}
I20260709 12:23:51.163698 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.151698 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2596) took 2187 ms. Trace:
I20260709 12:23:51.164738 20884 rpcz_store.cc:276] 0709 12:23:48.963954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.964138 (+   184us) service_pool.cc:224] Handling call
0709 12:23:48.987077 (+ 22939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.638364 (+651287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.149280 (+1510916us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b5400366854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:51.151597 (+  2317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383836928}
I20260709 12:23:51.161274 20802 raft_consensus.cc:1217] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Deduplicated request from leader. Original: 16.13->[18.14-18.14]   Dedup: 18.14->[]
I20260709 12:23:51.165808 20802 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:23:51.167287 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.168372 20807 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:23:51.147737 26658 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.170007 20801 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:23:51.173511 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.173538 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.174422 20801 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:51.176414 20810 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
W20260709 12:23:51.170396 20850 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.065s (SENT)
I20260709 12:23:51.184563 20850 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.185731 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.186437 20805 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:51.186467 20801 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:23:51.191062 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1475) took 1248 ms. Trace:
I20260709 12:23:51.191582 21435 rpcz_store.cc:276] 0709 12:23:49.942195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.942333 (+   138us) service_pool.cc:224] Handling call
0709 12:23:49.943598 (+  1265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:50.031791 (+ 88193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:50.031839 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:50.032554 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 12:23:50.196574 (+164020us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:50.196613 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.190609 (+993996us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:51.190670 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:51.190748 (+    78us) raft_consensus.cc:1685] finished
0709 12:23:51.190764 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.190995 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36123,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":201712896,"thread_start_us":163571,"threads_started":1,"wal-append.queue_time_us":337053}
I20260709 12:23:51.148867 26657 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.199643 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.200822 20801 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:51.203960 20801 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 12:23:51.204908 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.205576 20807 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
W20260709 12:23:51.208649 20700 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.872s (SENT)
I20260709 12:23:51.208715 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:51.167142 26511 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.212006 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.213498 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.199103 26658 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:51.214128 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.218041 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:51.233304 20806 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:51.235167 26713 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.241120 21427 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:23:51.237093 26600 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.237237 26525 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:51.236382 26731 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.241906 26320 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:51.253664 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.255511 20807 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:23:51.264617 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102220us
W20260709 12:23:51.269850 20850 leader_election.cc:336] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.094s (SENT)
I20260709 12:23:51.270397 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:51.272223 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.246232 26600 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.275100 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.277302 20801 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:51.264449 26657 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:51.279138 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70466us
I20260709 12:23:51.286598 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.287343 20730 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:51.287356 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.288400 20728 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:51.300127 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.302063 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.303062 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:51.303259 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.303527 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.304087 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2772) took 1066 ms. Trace:
I20260709 12:23:51.304293 20804 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:23:51.304541 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:51.304471 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.304522 20731 rpcz_store.cc:276] 0709 12:23:50.238085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.311242 (+ 73157us) service_pool.cc:224] Handling call
0709 12:23:50.341412 (+ 30170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.751094 (+409682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.303852 (+552758us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54002977d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:51.304023 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940574464}
I20260709 12:23:51.305601 20733 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:51.305728 26713 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:51.306422 26713 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:51.306579 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.308156 26612 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:51.308681 26612 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.275878 26671 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:51.311616 26713 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.313832 20733 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:51.310968 26612 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.315613 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1008) took 1136 ms. Trace:
I20260709 12:23:51.315742 26511 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:51.316951 26752 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:23:51.318081 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:51.318360 26731 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.318991 26511 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:51.319789 26511 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:51.320171 26710 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.320672 26710 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.320879 26511 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:51.280411 26750 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:23:51.319684 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.271617 26525 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:51.324432 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.331085 20727 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:51.329779 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:51.364280 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98716.9us
I20260709 12:23:51.304131 20802 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:51.345377 20730 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:51.323552 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.324021 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.371054 20733 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
W20260709 12:23:51.372085 20850 leader_election.cc:336] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.098s (SENT)
I20260709 12:23:51.337131 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1483) took 1072 ms. Trace:
I20260709 12:23:51.330416 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.335937 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:51.348238 21399 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.833s (SENT)
I20260709 12:23:51.376566 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.333122 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.338752 20808 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:51.379132 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2467) took 1098 ms. Trace:
I20260709 12:23:51.379639 20808 rpcz_store.cc:276] 0709 12:23:50.281001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.281163 (+   162us) service_pool.cc:224] Handling call
0709 12:23:51.338243 (+1057080us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:23:51.338493 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:51.378794 (+ 40301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.379055 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14798080}
I20260709 12:23:51.334353 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.346009 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:51.383066 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:51.354126 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.316115 21428 rpcz_store.cc:276] 0709 12:23:50.178768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.322080 (+143312us) service_pool.cc:224] Handling call
0709 12:23:50.354669 (+ 32589us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:50.653945 (+299276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:50.653994 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:50.712888 (+ 58894us) log.cc:844] Serialized 52 byte log entry
0709 12:23:50.832511 (+119623us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:50.856082 (+ 23571us) log.cc:844] Serialized 14 byte log entry
0709 12:23:50.856382 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.856490 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:51.310004 (+453514us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b18000de8e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:51.315255 (+  5251us) raft_consensus.cc:1685] finished
0709 12:23:51.315360 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.315531 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5690,"prepare.queue_time_us":18569,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":41897,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":1271,"spinlock_wait_cycles":10683008,"thread_start_us":324136,"threads_started":2,"wal-append.queue_time_us":156998}
I20260709 12:23:51.391062 20732 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:51.374756 21427 rpcz_store.cc:276] 0709 12:23:50.265059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.420475 (+155416us) service_pool.cc:224] Handling call
0709 12:23:50.427730 (+  7255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:51.241592 (+813862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.336812 (+ 95220us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:51.337059 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1862268800}
W20260709 12:23:51.391816 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144207us
I20260709 12:23:51.394935 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.396059 20805 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:51.395709 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47530.7us
W20260709 12:23:51.397399 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:51.321334 26709 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.924s	user 0.007s	sys 0.004s
I20260709 12:23:51.399178 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:51.405797 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.406836 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.409241 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.410212 20810 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:23:51.363476 26392 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:51.415580 26392 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.418373 26392 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:51.390486 26652 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:51.419516 26320 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:51.420491 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2765) took 1221 ms. Trace:
I20260709 12:23:51.421223 26525 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.423048 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.340533 26681 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.403870 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.421757 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1015) took 1581 ms. Trace:
I20260709 12:23:51.430860 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.431061 20881 rpcz_store.cc:276] 0709 12:23:49.840071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.277649 (+437578us) service_pool.cc:224] Handling call
0709 12:23:50.316544 (+ 38895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.324796 (+  8252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.421564 (+1096768us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b5400171e50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:51.421692 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88615808}
I20260709 12:23:51.323112 26710 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.402300 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2469) took 1109 ms. Trace:
I20260709 12:23:51.411118 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.446589 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:51.394110 20700 leader_election.cc:336] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.677s (SENT)
W20260709 12:23:51.428743 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56888.4us
I20260709 12:23:51.423708 20726 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:51.423907 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.447557 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.432206 20882 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:23:51.422303 26725 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:51.374105 26647 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.400547 26612 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:51.428821 26688 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:51.420892 20729 rpcz_store.cc:276] 0709 12:23:50.198761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.199301 (+   540us) service_pool.cc:224] Handling call
0709 12:23:50.199608 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.199826 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:50.200065 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 12:23:51.419948 (+1219883us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b180038f7a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:23:51.420042 (+    94us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:51.420094 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:51.420149 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.420268 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.420394 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":363,"raft.queue_time_us":2115,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5443,"spinlock_wait_cycles":19097344,"thread_start_us":219974,"threads_started":2,"wal-append.queue_time_us":224946}
W20260709 12:23:51.438097 26751 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.426s	user 0.004s	sys 0.000s
I20260709 12:23:51.438686 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:51.480329 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86263.2us
I20260709 12:23:51.420898 26320 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:23:51.439479 26415 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:51.369865 26712 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 12:23:51.442193 26524 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:51.428947 26600 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:51.443750 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.444114 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:51.448791 26392 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:51.370353 26511 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:51.445585 20803 rpcz_store.cc:276] 0709 12:23:50.281698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.281984 (+   286us) service_pool.cc:224] Handling call
0709 12:23:51.334155 (+1052171us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:23:51.390915 (+ 56760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13293952}
I20260709 12:23:51.450317 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:51.470927 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.473559 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.401750 26571 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:51.471613 26612 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.476769 26469 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:23:51.479242 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:51.481560 20700 leader_election.cc:336] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.950s (SENT)
I20260709 12:23:51.482795 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.450727 26725 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:51.425836 26631 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:51.487082 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.487954 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.479558 20810 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:51.491650 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:51.507507 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57269.4us
I20260709 12:23:51.491932 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:51.493391 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.503933 20877 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:51.493537 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.510706 20808 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:51.504411 20726 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:51.498638 20882 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 18 because replica has last-logged OpId of term: 15 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
I20260709 12:23:51.503378 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.515995 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.516909 20802 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:51.528474 20882 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:51.486222 26688 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.529431 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.499568 26612 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:51.500648 26469 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:23:51.493204 26657 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:51.497164 26754 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.535529 26538 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:51.536655 26489 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:51.537672 26513 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:51.506136 26415 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.543414 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.585671 20733 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:51.543816 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.545096 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:51.546584 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 65065.9us
W20260709 12:23:51.547693 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.549412 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.550304 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.587522 20727 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:51.550705 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.588985 20728 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:23:51.551493 26697 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:51.555984 26688 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:51.558912 20879 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:51.599485 26513 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:51.560178 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.600802 26513 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.602674 21427 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:51.563820 20875 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:51.604586 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.604470 26725 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.606199 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.568388 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:51.607760 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:51.608253 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:51.609203 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.609483 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:51.570555 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.611270 26697 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:51.612046 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.613183 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.613734 20729 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:51.614524 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.576124 26320 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:23:51.615412 26526 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:51.514699 26511 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.616134 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.616603 26600 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:51.498865 26571 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:51.617446 26571 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:51.617647 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.617967 26511 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:51.580045 26754 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.620611 26754 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:51.592602 26652 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:51.621538 26652 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.499310 26671 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:51.565124 20882 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 12:23:51.623348 26671 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:51.623644 26652 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:51.534446 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:51.624418 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:51.624821 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90453.8us
I20260709 12:23:51.577091 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.449724 26681 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.484558 26647 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.533073 26565 raft_consensus.cc:2764] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:23:51.533603 26682 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:51.643661 26682 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.644192 26682 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:51.587558 20700 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.002s (SENT)
I20260709 12:23:51.571236 26712 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:51.593703 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:51.642827 26571 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.648473 21435 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:51.649098 20884 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:51.595969 21428 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:51.596688 26415 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:51.598253 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:51.580340 26489 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:51.606266 26725 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:51.615288 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.654247 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.655079 20733 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:51.578903 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:51.629256 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:51.629990 20727 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:51.625761 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.656271 20731 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:51.575829 20805 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:51.635612 21427 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:51.522681 26524 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.673920 26489 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.674824 26681 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:51.675297 26489 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:51.675570 26524 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:51.676050 20735 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:51.639740 20882 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:51.677353 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.678515 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.640462 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:51.679701 20732 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:51.680037 21430 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 10.
I20260709 12:23:51.680902 20732 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:51.681710 26236 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:51.682092 26236 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:51.580464 26657 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.647666 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:51.683840 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 96333.5us
I20260709 12:23:51.684620 26657 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:51.650847 26688 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:51.685428 26688 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.685822 26688 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:51.655488 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.687107 20729 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:51.616189 26526 raft_consensus.cc:2764] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:23:51.673022 26712 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:51.683959 20733 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:23:51.693414 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 986) took 2085 ms (client timeout 1969 ms). Trace:
I20260709 12:23:51.672883 20810 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:51.672914 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43739.1us
I20260709 12:23:51.696756 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:51.692967 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:51.701297 20882 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:51.693938 20806 rpcz_store.cc:269] 0709 12:23:49.607869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.611317 (+  3448us) service_pool.cc:224] Handling call
0709 12:23:51.693171 (+2081854us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540019aad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:51.693322 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3697136512}
I20260709 12:23:51.740809 20729 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:51.747592 21428 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
W20260709 12:23:51.749833 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 178011us
W20260709 12:23:51.753160 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 314551us
W20260709 12:23:51.755926 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 264046us
I20260709 12:23:51.757069 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:51.792907 20733 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:51.796608 20809 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:51.794836 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.699045 26524 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:51.804438 21428 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:23:51.805320 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 232992us
I20260709 12:23:51.806420 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:51.809196 21430 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:51.728367 26565 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.624302 26671 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.682938 26236 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.786194 26712 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:51.842840 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85886.1us
W20260709 12:23:51.846086 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 86374us
W20260709 12:23:51.843241 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89202.2us
W20260709 12:23:51.851094 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 164376us
I20260709 12:23:51.860288 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:51.856999 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 160358us
I20260709 12:23:51.862069 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:51.726359 26538 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.116s
I20260709 12:23:51.865674 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:51.869946 26600 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:51.871588 26576 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:51.870908 20880 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:23:51.878986 20731 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:51.880684 20730 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:51.830427 26415 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:51.889957 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.896478 20882 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:23:51.896982 20882 pending_rounds.cc:85] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 11
I20260709 12:23:51.897255 20882 pending_rounds.cc:107] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:23:51.897557 20882 raft_consensus.cc:2889] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:51.892136 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:51.841732 26657 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:51.840588 26469 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:51.870905 26600 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:23:51.911710 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:51.872032 26576 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:51.886650 26658 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:23:51.875950 26699 consensus_queue.cc:1175] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 15 status { last_received { term: 12 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.11-14.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:51.987076 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:51.872056 26565 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:51.876251 26712 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:51.941234 26526 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:52.024115 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.024755 20882 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
W20260709 12:23:52.016351 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 156590us
I20260709 12:23:51.904968 26670 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:23:52.027822 21427 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:51.932873 26657 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:52.030402 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140544us
I20260709 12:23:52.006369 26320 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:52.033607 20880 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
W20260709 12:23:52.032483 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 140474us
I20260709 12:23:52.017457 26571 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.043407 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.021889 26658 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:51.941495 26744 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:23:52.049757 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.038662 26670 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:52.023998 26612 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.040009 26655 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:52.076059 26767 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 12:23:52.079993 26775 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:23:52.090039 26415 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:23:52.057719 26511 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:52.104323 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.069020 26571 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:52.127622 20876 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:23:52.137807 20728 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:23:52.145864 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98967.8us
I20260709 12:23:52.155820 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:52.090493 26712 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.177016 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:52.045466 26576 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.123605 26782 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:52.105206 26565 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:52.197820 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.188083 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.201218 26612 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.201086 26793 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:52.203819 26794 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:52.160817 26652 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:52.207374 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.198076 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:52.211012 26576 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.123353 26657 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.211220 26779 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:52.212433 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:52.210536 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.179082 26571 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:52.214977 26759 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.542s	user 0.001s	sys 0.000s
I20260709 12:23:52.114490 26511 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.126659 26780 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:23:52.112015 26655 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.226666 26794 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:52.229310 26688 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.000s
W20260709 12:23:52.273425 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62977.3us
I20260709 12:23:52.073470 26771 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:23:52.228565 26731 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:52.195120 26783 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:23:52.242718 26699 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.271397 26794 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.280858 26699 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.230085 26769 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.283569 26769 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.286645 26769 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:52.245301 26778 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:23:52.246623 26787 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:23:52.252854 26697 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.641s	user 0.004s	sys 0.006s
I20260709 12:23:52.288161 26697 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:52.288920 26788 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:23:52.290597 26697 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.244443 26792 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:52.291056 26784 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:23:52.256955 26511 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:52.257534 26309 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:52.277456 26576 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:52.292506 26576 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.291374 20700 leader_election.cc:400] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14.
I20260709 12:23:52.293401 26712 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:52.294588 26576 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:52.295431 21427 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:52.295615 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:52.275857 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 239160us
I20260709 12:23:52.296552 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.295481 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:52.278188 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70925.5us
I20260709 12:23:52.241250 26565 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:52.257596 26644 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:52.299124 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:52.230016 26652 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:52.314754 26682 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
I20260709 12:23:52.351112 26699 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:52.351632 26647 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:52.353555 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:52.317832 26469 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:52.315991 26644 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:52.277679 26688 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:52.382946 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 322497us
I20260709 12:23:52.291474 26792 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.319545 26565 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.290663 26571 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.350172 26682 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.381644 26469 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:52.397465 20882 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:52.400080 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85591.6us
I20260709 12:23:52.403796 26682 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.404289 20877 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:52.404768 21399 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.722s (SENT) [suppressed 37 similar messages]
I20260709 12:23:52.411865 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:52.412837 20732 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:52.412230 26798 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:52.403857 26794 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.409090 26697 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:23:52.417471 20726 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:23:52.392570 26576 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:23:52.394495 26790 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:23:52.427973 20882 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:52.410518 26657 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:52.429337 26576 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:52.430147 26647 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:23:52.430858 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.393213 26644 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.438134 26790 consensus_queue.cc:1243] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:23:52.437584 26647 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:52.439522 26644 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.440249 26647 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.441394 26647 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:52.436697 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.442039 20881 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:23:52.443835 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:52.444499 20735 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:52.429395 26794 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:52.446854 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:52.431509 26655 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:52.451676 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.453820 21428 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:52.453274 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.431037 26469 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.458276 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.458760 21428 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:23:52.458987 20881 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 11.
I20260709 12:23:52.452153 26576 consensus_queue.cc:1243] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:52.460014 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.457537 26807 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.462819 26565 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.463303 26807 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.465564 26807 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:52.466763 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:52.466965 26804 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:23:52.467283 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.467410 20727 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:52.468158 26792 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:52.468343 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.469033 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:52.469440 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.470274 26792 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:52.470654 26792 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:52.471016 26792 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:52.472887 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 177325us
I20260709 12:23:52.473199 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:52.473660 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:52.474817 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 175744us
I20260709 12:23:52.475374 26320 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.475863 26320 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:52.477793 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.478698 26320 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:52.479496 26783 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:52.480268 26783 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:52.444350 26657 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.482853 26657 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:52.455899 26652 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:52.485261 20735 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:52.485410 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:52.490549 20879 raft_consensus.cc:2368] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14.
I20260709 12:23:52.491479 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.492239 20879 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:52.492611 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.493014 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:52.493388 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.494171 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.494527 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.474897 26392 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:52.495133 20810 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:52.454679 26671 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:52.486995 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.457846 26811 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.487360 26779 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:52.499106 20802 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:52.514030 20802 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:52.497611 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:52.502303 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.494059 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:52.517408 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.509994 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.510548 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.510524 26800 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:52.489333 20732 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:23:52.511370 26771 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.511453 26682 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.493395 26712 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:52.519044 20879 raft_consensus.cc:2368] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14.
I20260709 12:23:52.536062 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:52.521056 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.524267 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.512487 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:52.515143 20802 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.537284 20809 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:52.537801 20802 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:52.524859 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.535980 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.538470 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:52.512470 26565 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.538689 20807 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:52.539535 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.508661 26320 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:52.540246 20801 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:52.524809 26655 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:52.513092 26782 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:52.525441 26800 consensus_queue.cc:1243] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 17.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:52.566223 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.555879 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:52.564100 21435 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:52.565141 26658 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.587296 26809 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:52.587457 26658 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.589908 26658 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.564199 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.590941 20879 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:52.592093 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:52.592941 26809 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:52.593282 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.536731 26392 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:52.594110 26392 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:52.595052 26392 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.596942 26783 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:52.601886 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:52.575662 20807 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:52.578351 21426 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:23:52.579455 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70072.4us
I20260709 12:23:52.579905 21434 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:23:52.580459 26652 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.563619 26803 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:23:52.606485 26652 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.608215 26803 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:52.608726 26652 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:52.609081 26803 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:52.609460 26803 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:52.609826 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.610426 26712 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.611037 26790 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:52.521931 26309 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:23:52.581632 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.582278 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.546267 26807 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:52.525516 26792 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.621582 20807 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:23:52.617264 26807 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:52.622846 26807 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.583593 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.519418 26811 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.584620 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.624410 20808 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:52.624680 21435 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:52.586344 26576 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:52.573802 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.628473 20804 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:23:52.628754 26576 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:52.593501 26809 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.629143 26576 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:52.629329 26809 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.630051 26576 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.625041 26682 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:52.552229 26670 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:23:52.631522 26682 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:52.552874 26565 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.632445 26682 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.634235 26565 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:52.604064 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.634655 26811 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:52.604599 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48860us
I20260709 12:23:52.623821 26674 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:52.635494 26674 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:52.248777 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.636340 26674 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.637363 26674 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.621484 26792 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.610311 26803 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.639375 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:52.639981 26803 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:52.640376 26758 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.634s	user 0.003s	sys 0.000s
I20260709 12:23:52.641230 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.606405 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.642716 26670 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:52.643280 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.643440 26670 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:52.643708 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2612) took 1213 ms. Trace:
I20260709 12:23:52.644495 26670 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.645404 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.565786 26538 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:52.646059 20882 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:52.646306 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.641970 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 916) took 1782 ms. Trace:
I20260709 12:23:52.648000 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:52.647393 20734 rpcz_store.cc:276] 0709 12:23:50.859740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.859881 (+   141us) service_pool.cc:224] Handling call
0709 12:23:50.884667 (+ 24786us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:50.904610 (+ 19943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:50.904654 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:50.925904 (+ 21250us) log.cc:844] Serialized 52 byte log entry
0709 12:23:51.319606 (+393702us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b1800372700. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:51.527073 (+207467us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:23:51.582163 (+ 55090us) log.cc:844] Serialized 14 byte log entry
0709 12:23:51.583083 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.583219 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:52.639807 (+1056588us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:52.641742 (+  1935us) raft_consensus.cc:1685] finished
0709 12:23:52.641765 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:52.641913 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29329,"prepare.queue_time_us":55460,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":48327,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":5390,"spinlock_wait_cycles":27050112,"thread_start_us":207274,"threads_started":1,"wal-append.queue_time_us":931457}
I20260709 12:23:52.648976 20803 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:52.649457 26765 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:52.645404 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.649904 26765 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.650197 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.650796 20806 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.650885 26791 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.651549 26526 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:52.651727 26654 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:52.652347 26654 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.652732 26654 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.651732 26791 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:52.650316 26765 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:52.642772 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2629) took 1085 ms. Trace:
I20260709 12:23:52.662729 20876 rpcz_store.cc:276] 0709 12:23:51.556889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.557611 (+   722us) service_pool.cc:224] Handling call
0709 12:23:51.558028 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.151524 (+593496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.642527 (+491003us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540019cdd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:52.642711 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309124992}
I20260709 12:23:52.663290 26526 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:52.664094 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.664652 20876 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
W20260709 12:23:52.656311 26785 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.452s	user 0.000s	sys 0.001s
I20260709 12:23:52.665114 20876 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11.
I20260709 12:23:52.648849 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.665971 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2757) took 1281 ms. Trace:
I20260709 12:23:52.656056 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.666591 20801 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:52.658723 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.605530 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:52.644117 20884 rpcz_store.cc:276] 0709 12:23:51.430060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.430186 (+   126us) service_pool.cc:224] Handling call
0709 12:23:51.435710 (+  5524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:51.649490 (+213780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.643490 (+994000us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400366d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:52.643644 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490322816}
I20260709 12:23:52.659374 20879 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:52.659626 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2634) took 1099 ms. Trace:
I20260709 12:23:52.646976 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.659528 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:52.661031 26309 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:52.664201 26526 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.668776 20699 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11.
I20260709 12:23:52.688910 20699 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:52.669220 20801 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:23:52.634886 26576 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:52.674851 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:52.673792 20877 rpcz_store.cc:276] 0709 12:23:51.560523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.608905 (+ 48382us) service_pool.cc:224] Handling call
0709 12:23:52.659399 (+1050494us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:52.659527 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826275200}
I20260709 12:23:52.675882 20876 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:52.691502 20809 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:23:52.670013 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64633.3us
I20260709 12:23:52.692261 20804 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:23:52.692699 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:52.672163 26777 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.471s	user 0.000s	sys 0.002s
I20260709 12:23:52.667160 26236 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:52.668401 20810 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:23:52.678220 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.711341 20880 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:23:52.666378 20805 rpcz_store.cc:276] 0709 12:23:51.384792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.691132 (+306340us) service_pool.cc:224] Handling call
0709 12:23:51.691475 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:51.691826 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:51.691858 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:51.692329 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 12:23:51.866897 (+174568us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b18002665c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:51.958808 (+ 91911us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:51.958879 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.958996 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:52.655743 (+696747us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:52.665782 (+ 10039us) raft_consensus.cc:1685] finished
0709 12:23:52.665804 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:52.665917 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3843,"raft.queue_time_us":113,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":11186,"spinlock_wait_cycles":64920192,"thread_start_us":91597,"threads_started":1,"wal-append.queue_time_us":511868}
I20260709 12:23:52.712575 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.713578 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.715034 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.718257 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:52.719451 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2512) took 1155 ms. Trace:
I20260709 12:23:52.685808 20802 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:52.659050 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:52.669634 26657 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.677325 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.681092 20806 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:52.736222 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.719980 20728 rpcz_store.cc:276] 0709 12:23:51.564326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.617351 (+ 53025us) service_pool.cc:224] Handling call
0709 12:23:52.719235 (+1101884us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54001d6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:52.719380 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1155714816}
I20260709 12:23:52.686548 26652 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:52.742380 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.746104 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:52.747144 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69893.4us
I20260709 12:23:52.747949 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:52.698909 26320 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:52.672022 26725 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:23:52.753844 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.754945 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.755968 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.758051 20808 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:23:52.700496 26663 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:52.763008 20810 raft_consensus.cc:1217] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 12.6->[13.7-20.9]   Dedup: 13.7->[18.8-20.9]
I20260709 12:23:52.765409 20810 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 7
I20260709 12:23:52.765790 20810 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 17.8
I20260709 12:23:52.766113 20810 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:52.710769 26526 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.766983 20880 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:52.717425 26731 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:52.767397 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.769229 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1017) took 1179 ms. Trace:
I20260709 12:23:52.744252 26800 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:52.737717 26781 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:52.773103 21429 rpcz_store.cc:276] 0709 12:23:51.589830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.589949 (+   119us) service_pool.cc:224] Handling call
0709 12:23:51.596532 (+  6583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:51.687004 (+ 90472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:51.687033 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:51.687073 (+    40us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:51.687108 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.768976 (+1081868us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:52.769021 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:52.769170 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207618432}
I20260709 12:23:52.774482 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2630) took 1217 ms. Trace:
I20260709 12:23:52.699978 26783 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:23:52.747153 26652 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:52.765291 26663 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.768657 26565 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:52.771824 26792 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:52.761559 26712 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:52.777397 26697 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:52.690199 26309 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.780898 20883 rpcz_store.cc:276] 0709 12:23:51.557184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.569920 (+ 12736us) service_pool.cc:224] Handling call
0709 12:23:51.576518 (+  6598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:51.576744 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:23:51.584831 (+  8087us) log.cc:844] Serialized 14 byte log entry
0709 12:23:52.612502 (+1027671us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b18002df760. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:23:52.774071 (+161569us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:52.774135 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:23:52.774181 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.774266 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:52.774412 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15777,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":112,"spinlock_wait_cycles":11039488,"thread_start_us":161224,"threads_started":1}
I20260709 12:23:52.801649 20801 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:23:52.812028 20801 pending_rounds.cc:85] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 10
W20260709 12:23:52.808212 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 270827us
I20260709 12:23:52.812621 20801 pending_rounds.cc:107] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:23:52.747778 26657 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.761962 26469 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:52.813113 20801 raft_consensus.cc:2889] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:52.787899 26652 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.816741 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:52.816433 20618 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:52.795936 26731 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.796907 26663 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:52.773849 26571 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:23:52.795542 26655 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:52.811252 26697 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:52.828722 20809 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:52.809281 26565 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:52.810451 26712 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:52.817332 26469 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.816741 26792 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:52.841768 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.829560 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.849227 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:52.849341 20801 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:23:52.833350 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.848150 20809 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:52.855523 20732 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:52.835310 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 176297us
I20260709 12:23:52.858914 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:52.860857 26783 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:52.861379 26644 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:52.862035 26644 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:52.862524 26644 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:52.871011 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.874086 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:52.875144 20733 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:52.876029 20729 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:52.877395 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.878185 20729 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:52.880144 20731 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:52.881951 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.883136 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:52.886718 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.896396 21427 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:52.890609 20733 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:23:52.893535 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.827875 26800 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:52.900686 21430 raft_consensus.cc:1813] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Handling vote request from an unknown peer 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.899175 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:52.909129 20735 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:52.850014 26469 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:52.910849 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.855666 20809 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:52.911485 20880 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:23:52.911810 20809 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:52.903985 26815 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:23:52.903752 26655 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.905272 26792 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:52.828981 26681 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:52.928082 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.928782 21433 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:52.929265 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.907800 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.912695 21430 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11.
I20260709 12:23:52.913679 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:52.936713 20801 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:52.913749 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:52.914673 26813 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:23:52.914599 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.915366 26799 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:52.915221 26731 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:23:52.915692 26814 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:52.916137 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:52.916923 26657 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.917346 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:52.848243 26712 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.912226 26800 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:52.830729 26688 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:52.924774 20729 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:52.944065 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.836433 26697 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.944988 21432 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:52.940299 20809 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:23:52.964385 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:52.951538 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.954334 20731 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:52.960500 21426 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:52.948843 20880 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:52.973977 20809 pending_rounds.cc:85] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 10
I20260709 12:23:52.981825 20701 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 10 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11.
I20260709 12:23:52.983440 21433 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:52.981832 20809 pending_rounds.cc:107] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 14.11
W20260709 12:23:52.981025 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.984192 20809 raft_consensus.cc:2889] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:52.956427 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 243905us
W20260709 12:23:52.961233 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43963.8us
I20260709 12:23:52.985575 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.964887 20734 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:52.959775 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.986606 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:52.955817 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.943665 20735 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:52.973299 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:52.992429 20809 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:52.955287 26812 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:23:52.997397 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:52.993844 20809 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:23:52.994503 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:52.954881 26820 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:23:52.995908 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:53.002944 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.003306 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:53.003573 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:53.004712 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:53.013191 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:52.949952 26761 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.749s	user 0.001s	sys 0.003s
I20260709 12:23:52.964865 26800 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:53.002223 20801 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:53.014531 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:52.953465 26309 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:53.032444 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:53.075613 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.948668 26799 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:52.973196 26657 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:53.017477 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:53.086001 20806 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:23:53.015419 26769 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:53.020767 26780 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:53.089520 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72119.9us
I20260709 12:23:52.989454 26674 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:23:53.097155 26809 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.099169 26744 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:53.099802 26744 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:53.100592 26781 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:53.100761 26744 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.102203 26744 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:53.103739 20881 raft_consensus.cc:2410] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:53.090801 26769 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.112027 26769 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:53.112793 20876 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:53.114725 26799 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.115774 26799 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.011215 26771 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
W20260709 12:23:53.119210 26671 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.000s
I20260709 12:23:53.119522 26831 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:53.120322 26771 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.120893 26771 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.122574 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:53.121445 26831 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:53.124735 20807 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:23:53.125759 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 111303us
I20260709 12:23:53.128247 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:53.073663 26511 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:53.132840 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 120666us
I20260709 12:23:53.129761 26780 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:53.030444 26644 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.005535 26565 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.127521 26236 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:53.139256 26511 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.162650 20733 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:53.093176 26807 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.151396 26725 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.164793 26236 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.150519 26809 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:53.171806 26819 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.143805 26794 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
W20260709 12:23:53.228859 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100810us
I20260709 12:23:53.160039 26771 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:53.175982 26831 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:53.246790 26819 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.247364 26822 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:23:53.248968 26663 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:53.250070 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.250336 20805 raft_consensus.cc:2368] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18.
I20260709 12:23:53.252395 20808 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:53.257778 26819 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:53.257516 26780 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.171370 26816 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:53.260003 26816 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.262308 26816 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:53.262604 26697 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.263113 20882 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:53.263590 26644 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:53.186866 26631 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:53.150861 26652 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.264765 26631 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.265774 26681 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:53.266247 26681 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:53.267727 26631 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:53.264720 26697 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:53.232818 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.235505 26829 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.268749 26807 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:53.164337 26511 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:53.241622 26803 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.002s
W20260709 12:23:53.231367 26785 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:53.175273 26731 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:53.282111 26765 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.619s	user 0.006s	sys 0.004s
I20260709 12:23:53.283808 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:53.284500 20730 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:53.241535 20880 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:23:53.237555 26236 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:53.244625 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.293957 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:53.245328 26770 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:23:53.267812 26710 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:53.269018 26829 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.170547 26469 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.237349 26320 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 10. Current term is 11.
I20260709 12:23:53.242410 20884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37c593b286274096a8c431979866be8a"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36384
I20260709 12:23:53.331609 20884 raft_consensus.cc:606] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Received request to transfer leadership
I20260709 12:23:53.331689 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:53.332056 20884 raft_consensus.cc:614] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:53.292634 20727 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:53.296280 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.305307 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:53.311899 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:53.333016 20808 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:23:53.282723 26765 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.316653 20878 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:23:53.338289 20878 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 12
I20260709 12:23:53.338743 20878 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:23:53.339092 20878 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:53.326002 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93288.4us
I20260709 12:23:53.333413 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:53.316133 20729 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:23:53.334873 20803 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:23:53.340975 20803 pending_rounds.cc:85] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 8
I20260709 12:23:53.341126 20727 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:23:53.336026 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:53.341328 20803 pending_rounds.cc:107] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 15.9
W20260709 12:23:53.339756 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.193598 26725 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.316427 26793 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:53.342578 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36384 (request call id 0) took 1044 ms. Trace:
I20260709 12:23:53.342234 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:53.394343 20884 rpcz_store.cc:276] 0709 12:23:52.287971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.242261 (+954290us) service_pool.cc:224] Handling call
0709 12:23:53.332507 (+ 90246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:53.394749 20882 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:53.334650 26688 raft_consensus.cc:2764] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:23:53.324296 26807 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:53.372812 20803 raft_consensus.cc:2889] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:53.192508 26796 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.191s
I20260709 12:23:53.332808 26829 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.294591 26831 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:53.392606 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57889.9us
I20260709 12:23:53.343060 21399 leader_election.cc:400] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14.
I20260709 12:23:53.406872 21399 leader_election.cc:403] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:23:53.323939 26710 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:53.324936 26822 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.326004 26682 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.375079 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.331230 26699 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.369885 26809 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:53.375015 26526 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:53.394209 26793 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:53.414814 26663 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:53.335235 26731 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:53.416739 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73907.1us
I20260709 12:23:53.336386 26770 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.457286 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:53.417514 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 294968us
I20260709 12:23:53.412753 26652 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:53.442545 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:53.422032 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47801.5us
W20260709 12:23:53.470077 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.470034 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:53.471549 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.329454 26657 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:53.459740 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:53.460518 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:53.460558 26731 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:53.473532 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:53.473809 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:53.426592 26811 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:53.476832 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.463882 26793 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.466428 20882 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:53.468541 20884 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:53.472597 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:53.483070 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:53.485575 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:53.486860 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.488353 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:53.491418 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:53.492381 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:53.497041 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:53.497408 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.497972 20728 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:53.504798 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:53.499790 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:53.507047 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46608.5us
I20260709 12:23:53.507468 21428 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:23:53.423207 26838 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14.
I20260709 12:23:53.508657 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.510814 26840 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:53.511271 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:53.516515 21426 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
W20260709 12:23:53.498044 26712 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.002s
I20260709 12:23:53.511963 20732 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:53.513253 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:53.522832 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.505508 20803 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:53.519222 20882 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:23:53.511410 26782 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:53.520741 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:53.505860 26565 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.478291 26809 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:53.529341 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 121998us
I20260709 12:23:53.530288 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:53.532620 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:53.533591 20728 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.496660 26236 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:53.496403 26663 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:53.490305 26688 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:53.424489 26699 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.499012 26571 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:53.491742 26796 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:53.425890 26682 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.563536 20805 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:53.604081 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:53.546839 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1031) took 1290 ms. Trace:
I20260709 12:23:53.527918 26782 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:53.502704 26671 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:53.575241 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:53.506039 26786 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:53.615816 26836 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:23:53.540266 26565 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:53.616935 26565 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:53.618312 26846 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:23:53.583906 26647 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:23:53.527767 26712 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.622044 26710 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.420915 26822 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.511770 26526 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:53.512058 26731 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:53.624531 26731 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.515597 26843 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:53.584702 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71976us
I20260709 12:23:53.626461 26657 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:53.559352 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.566619 20726 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:53.539736 26309 raft_consensus.cc:2764] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:23:53.531965 26793 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:53.632606 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.604745 20878 rpcz_store.cc:276] 0709 12:23:52.256784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.666847 (+410063us) service_pool.cc:224] Handling call
0709 12:23:52.667113 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:53.374251 (+707138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.546466 (+172215us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:53.546753 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420610816}
I20260709 12:23:53.476670 26770 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.627010 21400 leader_election.cc:400] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18.
I20260709 12:23:53.633613 26782 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.634725 26782 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:53.588110 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79540.5us
I20260709 12:23:53.636121 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:53.626479 26843 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.636931 26652 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:53.637844 26652 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:53.638355 26836 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:53.638955 26836 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:53.640111 26836 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:53.643463 26837 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:23:53.645123 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2590) took 1144 ms. Trace:
I20260709 12:23:53.646612 26836 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:53.645707 20883 rpcz_store.cc:276] 0709 12:23:52.501098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.914173 (+413075us) service_pool.cc:224] Handling call
0709 12:23:52.914560 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.931884 (+ 17324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:52.931913 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:52.956679 (+ 24766us) log.cc:844] Serialized 26 byte log entry
0709 12:23:53.159820 (+203141us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:53.159862 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.159935 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:53.644790 (+484855us) raft_consensus.cc:1685] finished
0709 12:23:53.644820 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.645040 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37427,"raft.queue_time_us":31085,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":11462,"thread_start_us":89191,"threads_started":1,"wal-append.queue_time_us":338881}
I20260709 12:23:53.647401 26844 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 12:23:53.649260 26789 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:53.649737 26789 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:53.650555 26789 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.651772 26789 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:53.652420 26576 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.955s	user 0.007s	sys 0.004s
W20260709 12:23:53.653923 26841 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:23:53.562276 26236 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:53.657577 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:53.661168 20880 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:23:53.545624 26809 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:53.669050 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:53.670823 20732 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:23:53.671947 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.672135 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:53.580232 26840 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:23:53.689728 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.687314 26236 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:53.689810 26809 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:53.539682 26834 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.166s
I20260709 12:23:53.692283 26843 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:53.692523 26712 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:53.703392 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:53.704527 21435 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:53.672760 26688 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:53.701720 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 134374us
I20260709 12:23:53.649235 26822 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:53.693135 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:53.708848 20877 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:53.711169 20734 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:53.579150 26838 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 12:23:53.690128 26697 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:53.690361 26652 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.715901 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:53.722442 21427 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:53.611163 26392 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:23:53.666455 26674 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:23:53.694180 26644 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:53.737370 20883 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:53.732285 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:53.689493 26671 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
W20260709 12:23:53.734760 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 172967us
I20260709 12:23:53.742460 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:53.755553 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 227260us
W20260709 12:23:53.735944 26654 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.083s	user 0.007s	sys 0.007s
W20260709 12:23:53.740953 26839 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 12:23:53.735849 26688 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:53.612735 26831 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.735327 26658 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.721279 26663 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:23:53.735235 26731 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:53.721135 26526 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
W20260709 12:23:53.770409 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.749274 26320 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.735548 26822 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:23:53.730446 26652 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.740656 26811 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:23:53.746202 26794 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 12:23:53.818584 26845 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260709 12:23:53.819787 26712 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:23:53.823436 26842 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 12:23:53.819005 26765 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:53.800303 26856 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:53.812291 26320 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:53.767035 26654 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.740028 26644 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.805426 26658 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.767853 26725 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:53.772152 26783 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:23:53.864616 26809 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.810264 26817 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:53.865381 26682 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:23:53.865849 26769 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.011s
W20260709 12:23:53.866226 26800 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.809s	user 0.007s	sys 0.011s
I20260709 12:23:53.829818 26655 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.835340 26688 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.841603 26857 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:53.877082 21427 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 11.
W20260709 12:23:53.895437 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 168676us
I20260709 12:23:53.897068 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:53.974613 21432 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:23:53.898703 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:53.921602 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 189416us
W20260709 12:23:53.905625 26744 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.803s	user 0.017s	sys 0.000s
W20260709 12:23:53.956022 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 213631us
I20260709 12:23:53.926563 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:53.900086 26712 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:53.905331 26856 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.979508 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:53.982015 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:53.963452 20876 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:53.965581 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.925638 26796 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.979506 20883 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:53.901489 26697 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:53.990218 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 229059us
W20260709 12:23:53.991914 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:53.967664 26511 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.002s
I20260709 12:23:53.984431 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:53.993666 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:53.994552 20809 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:53.995541 26571 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:53.985132 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:53.997378 26844 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:53.998243 26844 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:53.998821 20731 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:23:53.999423 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:54.000994 26725 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:54.001578 26725 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.002032 26725 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.005128 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:54.006064 20804 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.007664 26526 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:54.008445 26526 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:54.009716 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.009629 26526 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:54.010622 20810 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:53.997714 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:54.008107 20806 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:23:54.015051 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:54.016007 20883 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:54.020412 20801 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:23:54.025321 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.029099 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:54.027467 20809 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:23:54.032687 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:54.042362 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:54.024500 26513 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:23:53.981405 26712 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:54.076665 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:54.077682 20727 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
W20260709 12:23:54.037904 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 332729us
I20260709 12:23:54.095474 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:54.039160 26842 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:53.969779 26769 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:53.990993 26856 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:54.040613 26765 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:53.988238 26682 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.145s
I20260709 12:23:54.178040 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.184123 26831 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:53.981209 26819 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.151s
I20260709 12:23:53.975728 26800 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.188707 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:53.978360 26655 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.128919 26840 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:54.190233 26831 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.184319 26869 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:23:54.027722 26697 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.200279 20883 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:54.209007 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2598) took 1571 ms. Trace:
I20260709 12:23:54.030576 26511 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.194298 26565 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.005s
I20260709 12:23:53.997453 26571 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.178s
I20260709 12:23:54.026989 26796 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.214890 20809 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:53.966549 26817 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:54.210669 26576 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.251763 20875 rpcz_store.cc:276] 0709 12:23:52.638086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.106690 (+468604us) service_pool.cc:224] Handling call
0709 12:23:53.106990 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.129439 (+ 22449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.129475 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:53.129955 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:23:53.221798 (+ 91843us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:53.321122 (+ 99324us) log.cc:844] Serialized 14 byte log entry
0709 12:23:53.328695 (+  7573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.328775 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.208666 (+879891us) raft_consensus.cc:1685] finished
0709 12:23:54.208700 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:54.208916 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7177,"prepare.queue_time_us":97488,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":8807,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":5629,"thread_start_us":45607,"threads_started":1,"wal-append.queue_time_us":259166}
I20260709 12:23:54.248026 26831 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:54.258936 26859 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.393s	user 0.000s	sys 0.001s
W20260709 12:23:54.261504 26851 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.453s	user 0.000s	sys 0.001s
W20260709 12:23:54.272439 20729 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.009s	sys 0.007s
W20260709 12:23:54.280864 20729 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.009s	sys 0.009s
W20260709 12:23:54.262181 26782 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
W20260709 12:23:54.262965 26836 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.612s	user 0.013s	sys 0.000s
W20260709 12:23:54.262900 26789 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.003s
I20260709 12:23:54.305831 21427 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:23:54.274827 26658 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:54.318447 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:54.319555 26866 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:54.320158 26853 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:54.320705 26864 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:23:54.324051 26854 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:23:54.325275 26877 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:54.328006 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:54.332283 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:54.336516 26697 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.336790 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2530) took 1410 ms. Trace:
I20260709 12:23:54.247800 26840 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.282749 26857 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:54.367511 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 153676us
I20260709 12:23:54.346787 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:54.288209 26652 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.287626 26809 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.270660 26565 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.362815 20733 rpcz_store.cc:276] 0709 12:23:52.926098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.055467 (+129369us) service_pool.cc:224] Handling call
0709 12:23:53.083916 (+ 28449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:53.163264 (+ 79348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.336559 (+1173295us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b54003ad9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:54.336728 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180468608}
I20260709 12:23:54.370500 21434 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:54.293974 26765 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:54.371731 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:54.355212 26320 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.358108 26866 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:54.408099 21426 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:54.362979 26872 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:54.361516 26854 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:54.398641 26697 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:54.484740 26697 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:54.488402 26817 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.502637 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:54.506054 20810 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:54.432588 26571 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:54.373062 26853 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.433519 26829 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.013s	user 0.010s	sys 0.002s
I20260709 12:23:54.508710 26829 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.506987 26571 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.511253 26829 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.414127 26872 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.511415 21429 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:54.511873 26872 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.512750 26571 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.354244 26725 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:54.442335 26888 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.513854 26888 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.441422 26681 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.175s	user 0.014s	sys 0.003s
I20260709 12:23:54.443799 26854 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.444439 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.486109 26697 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.444911 26682 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:54.511435 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:54.431671 20804 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:54.445391 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.445521 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:54.520100 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.445819 26882 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:54.525475 26882 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.526350 26236 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.818s	user 0.011s	sys 0.000s
I20260709 12:23:54.526890 26236 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.527539 26682 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.529250 26236 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.517179 26888 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.529999 26682 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:54.531417 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 86018.5us
I20260709 12:23:54.532859 21431 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:54.536916 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.538177 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:54.539132 21434 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:54.539072 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:54.540548 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:54.540807 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:54.541939 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:54.538188 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:54.544261 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:54.545126 21431 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:54.546007 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2603) took 1397 ms. Trace:
I20260709 12:23:54.546411 20876 rpcz_store.cc:276] 0709 12:23:53.148743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.190073 (+ 41330us) service_pool.cc:224] Handling call
0709 12:23:54.545828 (+1355755us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:54.545951 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1560497408}
I20260709 12:23:54.547789 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:54.548550 20876 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.517164 26681 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:54.553526 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.448992 26884 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:54.474515 26511 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.475476 26769 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:54.476758 26765 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:54.479722 26565 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.481192 26655 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.482863 26796 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.406723 26779 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.483815 26800 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.417182 26836 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:54.415052 26866 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.498905 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2796) took 1666 ms. Trace:
I20260709 12:23:54.510339 20631 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 5 to 8, term changed from 7 to 8, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196), VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 8 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:54.512843 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1188) took 1015 ms. Trace:
I20260709 12:23:54.441537 20806 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:54.442445 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:54.517836 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.446208 26831 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.555505 20883 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:54.557163 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.522310 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:54.622958 21435 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:54.446966 26840 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.524312 26853 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:54.524574 20801 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:23:54.613581 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:54.520972 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.568077 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2642) took 2069 ms. Trace:
I20260709 12:23:54.636137 20877 rpcz_store.cc:276] 0709 12:23:52.499022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.716860 (+217838us) service_pool.cc:224] Handling call
0709 12:23:52.763625 (+ 46765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:53.721995 (+958370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.567742 (+845747us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b5400305f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:54.567992 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173372672}
W20260709 12:23:54.572383 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.014s	sys 0.002s
W20260709 12:23:54.576327 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:54.638942 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.015s	sys 0.003s
W20260709 12:23:54.639962 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 597605us
I20260709 12:23:54.642361 20802 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:54.643481 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.578156 20726 rpcz_store.cc:276] 0709 12:23:52.832034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.832318 (+   284us) service_pool.cc:224] Handling call
0709 12:23:52.832594 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:53.630503 (+797909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.498591 (+868088us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54000c49d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:54.498821 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687600384}
W20260709 12:23:54.579792 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.580993 21429 rpcz_store.cc:276] 0709 12:23:53.497905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.499317 (+  1412us) service_pool.cc:224] Handling call
0709 12:23:53.504275 (+  4958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:54.511949 (+1007674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.512596 (+   647us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54007201d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:54.512777 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2315655040}
I20260709 12:23:54.556882 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:54.646731 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:54.647661 21428 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:54.662415 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.665588 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:54.666308 20729 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:23:54.667570 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.582731 26469 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.584558 26710 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.580837 26392 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:54.661307 26670 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:54.673508 26392 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:54.574796 26663 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:54.674515 26663 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:54.674613 26392 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:54.576035 26779 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.675881 26392 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.677663 26779 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.678277 26779 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.679647 20631 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 12 to 13, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 13 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:54.593145 26644 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:54.596686 26865 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.404s	user 0.000s	sys 0.002s
W20260709 12:23:54.597962 26875 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
I20260709 12:23:54.600807 26538 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.672870 26780 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:54.577534 26836 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.718902 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:54.684093 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1025) took 1029 ms. Trace:
W20260709 12:23:54.700181 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.720697 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:54.721383 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.721768 20801 raft_consensus.cc:2368] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:23:54.722491 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:54.702335 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:54.723677 20732 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:54.724859 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:54.726082 20877 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:54.719898 20803 rpcz_store.cc:276] 0709 12:23:53.654584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.655030 (+   446us) service_pool.cc:224] Handling call
0709 12:23:53.691784 (+ 36754us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:53.699876 (+  8092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.699908 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:53.710162 (+ 10254us) log.cc:844] Serialized 52 byte log entry
0709 12:23:53.926489 (+216327us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:53.926551 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.926663 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.595746 (+669083us) spinlock_profiling.cc:243] Waited 6.82 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:54.683714 (+ 87968us) raft_consensus.cc:1685] finished
0709 12:23:54.683757 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:54.684006 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2856,"raft.queue_time_us":2972,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":14280,"spinlock_wait_cycles":15682432,"thread_start_us":180259,"threads_started":1,"wal-append.queue_time_us":299548}
W20260709 12:23:54.603116 26793 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.970s	user 0.012s	sys 0.000s
W20260709 12:23:54.709384 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.037s	sys 0.082s
I20260709 12:23:54.709885 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:54.733239 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.038s	sys 0.083s
W20260709 12:23:54.713915 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 157129us
I20260709 12:23:54.735737 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:54.686229 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:54.569881 26854 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:54.707764 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:54.730028 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:54.739058 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:54.564728 26725 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.555924 26790 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:54.714017 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 77782.8us
I20260709 12:23:54.717521 20804 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:54.737179 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:54.786078 20804 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:54.730309 21429 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:23:54.678758 26779 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:54.589610 26882 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.831701 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:54.845448 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 110247us
I20260709 12:23:54.555145 26681 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:54.776166 20876 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:23:54.860457 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:54.593652 26699 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.777729 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.778028 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.588163 26876 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:54.590283 26799 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:23:54.868646 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.792176 21435 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:23:54.828195 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1024) took 1175 ms. Trace:
I20260709 12:23:54.872274 20805 rpcz_store.cc:276] 0709 12:23:53.652438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.652610 (+   172us) service_pool.cc:224] Handling call
0709 12:23:53.652993 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.653447 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.653475 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:53.653925 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 12:23:53.980469 (+326544us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:53.980531 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.980633 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.597143 (+616510us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:54.737389 (+140246us) raft_consensus.cc:1685] finished
0709 12:23:54.737419 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:54.828115 (+ 90696us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11626,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":21691,"spinlock_wait_cycles":30662528,"thread_start_us":324809,"threads_started":1,"wal-append.queue_time_us":497735}
W20260709 12:23:54.830199 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 122548us
I20260709 12:23:54.569252 26770 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.843384 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:54.741945 20734 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:23:54.846738 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:54.843653 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:54.876865 20734 pending_rounds.cc:85] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 13
I20260709 12:23:54.851253 20802 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:54.877274 20734 pending_rounds.cc:107] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 19.14
W20260709 12:23:54.853209 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 130838us
W20260709 12:23:54.777139 26873 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.416s	user 0.004s	sys 0.001s
I20260709 12:23:54.731673 26469 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:54.731025 26836 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:54.864596 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:54.718695 26769 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:54.867687 26670 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:23:54.718160 26817 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:54.868633 20732 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:54.843648 20804 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.723004 26655 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.145s
I20260709 12:23:54.883365 20807 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:23:54.889032 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:54.927278 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48908.8us
I20260709 12:23:54.877686 20734 raft_consensus.cc:2889] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:54.878649 20802 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:23:54.875648 21428 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:54.863914 26309 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.862917 26612 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:23:54.718144 26538 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.729401 26780 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:54.869414 26631 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:54.570341 26884 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.888151 26880 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:23:54.844352 26647 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.864795 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.912529 26670 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:54.986284 26670 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:54.940991 20802 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:54.987174 26670 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.975440 26792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.029s	sys 0.068s
W20260709 12:23:54.988147 26792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.029s	sys 0.068s
I20260709 12:23:54.912708 26817 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:54.908938 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62261.6us
I20260709 12:23:54.910782 26682 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:54.956773 26526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.046s	sys 0.045s
I20260709 12:23:54.916657 20804 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:54.935690 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.903841 20631 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 9 to 15, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 15 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:54.952714 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:54.958722 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.012660 20801 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
W20260709 12:23:54.959632 20847 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.037s	sys 0.047s
I20260709 12:23:54.955389 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:55.014150 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58884.1us
I20260709 12:23:54.851415 26809 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.988471 26670 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:54.896301 26712 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.000s
I20260709 12:23:55.037379 20883 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:23:54.982877 20729 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:54.911595 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47091.3us
I20260709 12:23:54.887929 26469 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.990280 20881 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:54.887250 26654 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:54.915405 26786 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:54.911957 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:55.049568 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 137730us
I20260709 12:23:54.900645 26731 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:55.051424 26883 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:55.051522 26731 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.051239 20631 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 7 to 14, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 14 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.053191 20631 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 12 to 17, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 17 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.054247 26731 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:54.893493 26779 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:54.975977 26576 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.000s
I20260709 12:23:55.056491 26631 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:55.056651 26576 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:55.057080 26631 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.057583 20809 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
W20260709 12:23:55.015594 20847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.037s	sys 0.047s
I20260709 12:23:55.054920 26889 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:23:55.061829 26631 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:55.062606 26631 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:55.063436 26682 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:55.063827 26631 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:55.064307 26631 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:55.022861 20804 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:23:54.867352 26834 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:54.882484 26896 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:54.845223 26790 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.044754 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1021) took 1574 ms. Trace:
I20260709 12:23:55.072435 20808 rpcz_store.cc:276] 0709 12:23:53.470483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.471362 (+   879us) service_pool.cc:224] Handling call
0709 12:23:53.471666 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.474464 (+  2798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.474515 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:53.477173 (+  2658us) log.cc:844] Serialized 26 byte log entry
0709 12:23:53.694626 (+217453us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180039afc0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:23:53.981837 (+287211us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:53.981878 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.981945 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.562727 (+580782us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:55.044527 (+481800us) raft_consensus.cc:1685] finished
0709 12:23:55.044549 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.044697 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4002,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":15265,"spinlock_wait_cycles":16471680,"thread_start_us":287018,"threads_started":1,"wal-append.queue_time_us":708029}
I20260709 12:23:54.889496 26799 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:54.882709 26571 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 12:23:54.912379 26769 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.055734 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.116468 20876 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:23:55.062990 26731 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:55.015368 26526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.046s	sys 0.046s
I20260709 12:23:54.953590 26655 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.852417 26887 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.306s
W20260709 12:23:55.023167 26688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.015s	sys 0.025s
I20260709 12:23:54.901413 26513 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:23:55.065084 20631 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 7 to 12, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 12 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:55.140116 26631 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.148396 26538 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:54.889123 26876 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.148808 26631 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:55.150763 26876 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:55.076452 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 323565us
I20260709 12:23:54.974232 26309 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.151331 26612 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.151934 26612 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.153064 26309 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:55.027257 26671 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:55.154021 26612 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:55.154079 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2607) took 1155 ms. Trace:
I20260709 12:23:55.154801 20700 leader_election.cc:400] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:23:55.154999 26771 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.155534 26831 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:55.155535 26771 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:55.157159 20630 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 272) took 34577 ms (client timeout 15000 ms). Trace:
I20260709 12:23:55.158187 26771 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:55.052069 26883 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.159505 26883 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:55.156215 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:55.028105 26803 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:55.160430 26803 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.161170 26817 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:55.162515 26803 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:55.050134 26469 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:55.034025 26392 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:55.164832 26392 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:55.165745 26392 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.070683 26896 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.167443 26896 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:55.157557 20630 rpcz_store.cc:269] 0709 12:23:20.421492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.535237 (+113745us) service_pool.cc:224] Handling call
0709 12:23:54.983843 (+34448606us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:43.615616 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:43.682429 (+ 66813us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:43.682469 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:23:43.682489 (+    20us) tablet.cc:662] Decoding operations
0709 12:23:43.964118 (+281629us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:43.987925 (+ 23807us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:43.992042 (+  4117us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:23:44.106800 (+114758us) tablet.cc:701] Row locks acquired
0709 12:23:44.106833 (+    33us) write_op.cc:260] PREPARE: finished
0709 12:23:44.107147 (+   314us) write_op.cc:270] Start()
0709 12:23:44.107361 (+   214us) write_op.cc:276] Timestamp: P: 1783599824107080 usec, L: 0
0709 12:23:44.107396 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:23:44.108470 (+  1074us) log.cc:844] Serialized 6816 byte log entry
0709 12:23:45.320822 (+1212352us) op_driver.cc:464] REPLICATION: finished
0709 12:23:46.133525 (+812703us) write_op.cc:301] APPLY: starting
0709 12:23:46.141882 (+  8357us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:46.819722 (+677840us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b180028aec0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:23:47.105003 (+285281us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:47.105049 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:47.686055 (+581006us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:47.686537 (+   482us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 12:23:47.686585 (+    48us) write_op.cc:312] APPLY: finished
0709 12:23:47.699846 (+ 13261us) log.cc:844] Serialized 156 byte log entry
0709 12:23:47.875343 (+175497us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:48.136348 (+261005us) write_op.cc:454] Released schema lock
0709 12:23:48.190508 (+ 54160us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:48.384594 (+194086us) spinlock_profiling.cc:243] Waited 70.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:48.414316 (+ 29722us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:48.414368 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":98472,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":646956,"apply.run_cpu_time_us":56558,"apply.run_wall_time_us":2276919,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75187,"num_ops":17,"prepare.queue_time_us":239863,"prepare.run_cpu_time_us":23850,"prepare.run_wall_time_us":1515220,"raft.queue_time_us":234438,"raft.run_cpu_time_us":26629,"raft.run_wall_time_us":960676,"replication_time_us":1213258,"spinlock_wait_cycles":209096832,"thread_start_us":2335582,"threads_started":5,"wal-append.queue_time_us":1083357,"wal-append.run_cpu_time_us":20213,"wal-append.run_wall_time_us":2927415}]]}
I20260709 12:23:55.005997 26647 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.043951 26681 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:23:55.159696 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.154603 20880 rpcz_store.cc:276] 0709 12:23:53.998638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.003389 (+  4751us) service_pool.cc:224] Handling call
0709 12:23:54.003820 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:54.209515 (+205695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:54.209562 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:54.231349 (+ 21787us) log.cc:844] Serialized 26 byte log entry
0709 12:23:54.633219 (+401870us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:54.633258 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.633324 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.153869 (+520545us) raft_consensus.cc:1685] finished
0709 12:23:55.153891 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.154023 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":325083,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":564807,"threads_started":2,"wal-append.queue_time_us":774390}
I20260709 12:23:55.174201 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:55.122695 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2648) took 1657 ms. Trace:
I20260709 12:23:55.175963 20878 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:23:55.075608 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:23:55.200469 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 124978us
I20260709 12:23:55.200292 20881 rpcz_store.cc:276] 0709 12:23:53.465194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.466177 (+   983us) service_pool.cc:224] Handling call
0709 12:23:53.466456 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.044952 (+1578496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.121106 (+ 76154us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400366d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:55.122623 (+  1517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3503826688}
I20260709 12:23:55.148206 20631 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 12, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 12 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:55.203011 26469 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.202864 20631 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.204571 20877 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:55.205883 26469 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:55.205099 20631 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 7 to 13, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 13 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.044713 26809 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.209124 26809 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:55.155154 20700 leader_election.cc:403] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:55.210610 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.211663 20881 raft_consensus.cc:2368] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:23:55.114382 26786 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:55.049911 26712 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:55.236007 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61960.5us
W20260709 12:23:55.238502 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83853.6us
I20260709 12:23:55.236539 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:55.212913 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:55.239951 20881 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:55.237592 20726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.021s
W20260709 12:23:55.241711 20726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.006s	sys 0.022s
W20260709 12:23:55.177325 26688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.015s	sys 0.025s
I20260709 12:23:55.125797 26881 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:23:55.245679 26856 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.055s	user 0.012s	sys 0.001s
I20260709 12:23:55.064745 26682 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.247393 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:55.081398 26834 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:55.189075 26880 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:23:55.158747 26771 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.139772 26731 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:55.261832 20631 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 5 to 15, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 15 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:23:55.269464 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 229719us
W20260709 12:23:55.236779 26697 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.000s	sys 0.039s
I20260709 12:23:55.234375 26831 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:55.264878 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:55.244271 26803 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:23:55.186141 26816 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:55.199779 26854 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:55.274179 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 258707us
I20260709 12:23:55.300504 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:55.302445 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:55.240118 26511 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:55.278198 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:55.266242 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 349963us
I20260709 12:23:55.242564 26853 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:55.241155 26852 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:55.246537 26819 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:23:55.282354 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.140743 26655 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.243537 26892 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:55.253086 26612 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:55.273995 26697 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.000s	sys 0.039s
I20260709 12:23:55.303160 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.303448 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.306161 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:55.306540 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:55.307732 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:55.311246 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.312093 20730 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:55.314585 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.316141 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:55.316968 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:55.317497 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.318564 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.320960 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:55.326452 20699 leader_election.cc:400] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:23:55.326887 20631 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 14, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 14 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:55.286019 26657 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:55.293910 26876 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:55.289237 26644 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:55.268267 26856 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.280049 26793 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:23:55.279137 26771 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.267663 26881 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.345840 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:55.530217 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 222536us
W20260709 12:23:55.531071 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 204739us
I20260709 12:23:55.530673 20809 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:23:55.392642 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:55.538247 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.343431 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:55.540796 20810 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:23:55.542647 20810 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:23:55.543320 20810 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 10
I20260709 12:23:55.543664 20810 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:23:55.543911 20810 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:55.544385 20802 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:23:55.545418 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:55.546101 20734 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:55.547380 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:55.547499 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:55.548004 20730 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:55.344126 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:55.301622 26854 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.532598 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.304699 26817 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:55.570025 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.572333 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:55.342250 26780 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:55.343704 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:55.446128 26807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.029s	sys 0.043s
I20260709 12:23:55.575594 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.375694 26819 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:55.576093 20631 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 9 to 13, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 13 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:55.539767 26872 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.027s	user 0.014s	sys 0.000s
I20260709 12:23:55.339120 26803 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:55.572355 20809 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:55.597517 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:55.450997 26674 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:55.605165 26899 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.358s	user 0.001s	sys 0.001s
I20260709 12:23:55.321086 26786 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.602022 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:55.644927 26710 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.936s	user 0.010s	sys 0.001s
I20260709 12:23:55.660558 26856 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:55.340500 26852 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.661474 20875 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:23:55.661828 20875 pending_rounds.cc:85] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 7
I20260709 12:23:55.662045 20875 pending_rounds.cc:107] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:23:55.662284 20875 raft_consensus.cc:2889] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:55.663305 20631 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:55.603639 21435 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:23:55.324862 26320 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.269s
I20260709 12:23:55.504884 26829 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.407759 26831 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:55.546201 26876 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.472368 26790 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:55.319541 26816 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:55.490039 26836 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.001s
I20260709 12:23:55.490458 26799 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:55.617244 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:55.553828 26647 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:55.631910 20730 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:55.669100 26897 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.634265 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:55.413492 26822 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:55.581751 26697 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.566834 26919 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:55.327978 26511 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:55.605777 26906 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.341s	user 0.001s	sys 0.001s
W20260709 12:23:55.604916 26807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.030s	sys 0.057s
W20260709 12:23:55.649770 26909 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 12:23:55.562819 26612 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.596593 26857 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:23:55.578085 26887 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:23:55.724243 26822 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.753882 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2657) took 1433 ms. Trace:
W20260709 12:23:55.653525 26902 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 12:23:55.654098 26819 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.754264 20882 rpcz_store.cc:276] 0709 12:23:54.320046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.322382 (+  2336us) service_pool.cc:224] Handling call
0709 12:23:54.322678 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:54.323026 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:54.323054 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:54.610193 (+287139us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:23:54.610795 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:23:55.007932 (+397137us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:55.007994 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.010003 (+  2009us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.753587 (+743584us) raft_consensus.cc:1685] finished
0709 12:23:55.753608 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.753813 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84353,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8264,"spinlock_wait_cycles":72295168,"thread_start_us":396661,"threads_started":1,"wal-append.queue_time_us":562455}
W20260709 12:23:55.756311 26670 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.713s	user 0.010s	sys 0.000s
I20260709 12:23:55.561846 26644 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:55.738097 26908 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 12:23:55.656312 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:55.757470 20728 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:55.657444 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:55.758054 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1035) took 1137 ms. Trace:
I20260709 12:23:55.758239 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2658) took 1437 ms. Trace:
I20260709 12:23:55.758684 20727 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:23:55.759119 20880 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:23:55.619432 26901 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 12:23:55.758457 20806 rpcz_store.cc:276] 0709 12:23:54.620669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.620960 (+   291us) service_pool.cc:224] Handling call
0709 12:23:54.632564 (+ 11604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:54.698289 (+ 65725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:54.698338 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:54.698959 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:23:55.109883 (+410924us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:55.109947 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.110048 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.757841 (+647793us) raft_consensus.cc:1685] finished
0709 12:23:55.757861 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.757992 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30436,"raft.queue_time_us":34726,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":112925,"thread_start_us":257538,"threads_started":1,"wal-append.queue_time_us":560106}
I20260709 12:23:55.719460 26897 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.552284 26657 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.668527 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:55.672444 20881 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:23:55.762884 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:55.749519 26911 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 12:23:55.676920 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 147955us
I20260709 12:23:55.678803 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.376271 26892 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.617516 26864 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:55.687359 20631 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:55.768074 20631 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 8 to 16, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 16 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.578316 26811 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:23:55.569154 26854 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:55.643770 26918 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:55.675139 26803 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:55.720182 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:55.690042 26876 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:55.684608 26829 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.682428 26852 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:55.740638 26771 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:55.741951 26890 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.873s	user 0.001s	sys 0.000s
I20260709 12:23:55.597572 20734 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
I20260709 12:23:55.674520 26674 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:55.742887 26875 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:23:55.745682 26903 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.400s	user 0.012s	sys 0.002s
I20260709 12:23:55.719219 26836 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:55.748172 26913 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:23:55.693539 26834 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:55.689893 26831 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.748986 26881 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:55.749133 26923 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:23:55.749639 26647 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.750190 26857 consensus_queue.cc:1243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:23:55.751582 26511 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:55.653187 26866 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.075s	user 0.012s	sys 0.001s
I20260709 12:23:55.756274 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.451268 26731 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.763516 20881 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:55.831411 26864 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.924837 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.340925 26853 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.764115 20735 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:55.758642 20884 rpcz_store.cc:276] 0709 12:23:54.320334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.323308 (+  2974us) service_pool.cc:224] Handling call
0709 12:23:54.323647 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:54.710375 (+386728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:54.710425 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:54.727925 (+ 17500us) log.cc:844] Serialized 52 byte log entry
0709 12:23:55.185923 (+457998us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:55.185982 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.186264 (+   282us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.758014 (+571750us) raft_consensus.cc:1685] finished
0709 12:23:55.758035 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.758179 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93659,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":65945,"thread_start_us":703136,"threads_started":2,"wal-append.queue_time_us":512207}
I20260709 12:23:55.653550 26883 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:55.647074 26872 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:55.814683 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.828318 26843 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:55.830332 20730 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:23:55.770157 20631 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.863155 20728 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:55.866719 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.764582 26655 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:55.754273 26822 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:55.772886 26631 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.004s
I20260709 12:23:55.888726 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:55.737402 26697 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:55.908424 21429 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:23:55.909602 20729 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:55.887959 26922 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.913698 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:55.919173 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 145725us
I20260709 12:23:55.923131 20882 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
W20260709 12:23:55.746306 26910 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 12:23:55.926363 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.900383 26887 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:23:55.977142 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.928275 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2817) took 1264 ms. Trace:
I20260709 12:23:56.006119 20881 raft_consensus.cc:2368] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:23:56.138926 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:55.996053 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.185316 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:56.114289 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1049) took 1579 ms (client timeout 1511 ms). Trace:
I20260709 12:23:56.006027 20735 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:23:55.983271 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:55.977360 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:56.228673 26935 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:23:56.038493 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2655) took 1719 ms. Trace:
I20260709 12:23:55.983742 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2608) took 1984 ms. Trace:
I20260709 12:23:56.186398 20726 rpcz_store.cc:276] 0709 12:23:54.663699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.665124 (+  1425us) service_pool.cc:224] Handling call
0709 12:23:54.681534 (+ 16410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:54.681973 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:54.682011 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:54.682754 (+   743us) log.cc:844] Serialized 52 byte log entry
0709 12:23:55.242198 (+559444us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:55.242255 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.242352 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.928084 (+685732us) raft_consensus.cc:1685] finished
0709 12:23:55.928105 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.928227 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48216,"raft.queue_time_us":3092,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":25639,"thread_start_us":558884,"threads_started":1,"wal-append.queue_time_us":769479}
I20260709 12:23:56.230011 20802 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:56.230295 20877 rpcz_store.cc:269] 0709 12:23:54.535121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.843330 (+308209us) service_pool.cc:224] Handling call
0709 12:23:56.113969 (+1270639us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540019d7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:56.114177 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752479616}
I20260709 12:23:56.104043 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:56.036092 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1053) took 1499 ms. Trace:
I20260709 12:23:56.233788 20880 rpcz_store.cc:276] 0709 12:23:54.536313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.236181 (+699868us) service_pool.cc:224] Handling call
0709 12:23:56.035876 (+799695us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400366854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:56.036027 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1194999040}
I20260709 12:23:56.234340 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.235757 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.929625 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:55.920895 26392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.013s	sys 0.016s
I20260709 12:23:56.240901 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:23:56.038897 26931 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 12:23:56.077720 26896 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.904s	user 0.006s	sys 0.006s
I20260709 12:23:56.118467 26769 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.267119 20875 rpcz_store.cc:276] 0709 12:23:54.319400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.321471 (+  2071us) service_pool.cc:224] Handling call
0709 12:23:54.321769 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.662701 (+1340932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.038225 (+375524us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540019bed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:56.038409 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3080063232}
I20260709 12:23:56.266418 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.275089 20734 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 12:23:56.277079 20735 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:23:56.145004 26657 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:56.019598 26905 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:23:56.016716 26923 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:23:56.285616 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.231961 20879 rpcz_store.cc:276] 0709 12:23:53.998905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.004920 (+  6015us) service_pool.cc:224] Handling call
0709 12:23:54.005354 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:54.212186 (+206832us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:23:54.212234 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:54.231970 (+ 19736us) log.cc:844] Serialized 26 byte log entry
0709 12:23:54.408604 (+176634us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:54.408920 (+   316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.409015 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.741395 (+1332380us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:55.949457 (+208062us) raft_consensus.cc:1685] finished
0709 12:23:55.949480 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.983672 (+ 34192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":323800,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":76402048,"thread_start_us":352632,"threads_started":2,"wal-append.queue_time_us":447851}
I20260709 12:23:56.285703 20631 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:56.286996 20726 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:23:56.029856 26864 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.145304 26836 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:56.151417 26674 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.029433 26829 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.296618 26793 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:56.291648 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.293138 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:56.264981 26392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.013s	sys 0.017s
I20260709 12:23:56.276227 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:56.056725 26936 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.933482 26809 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.299340 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.299677 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.144264 26883 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:56.120817 26526 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:55.921463 26647 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.144384 26712 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.035089 26644 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.109928 26897 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:56.023990 26612 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.304767 20877 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:23:56.152577 26853 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:56.306447 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 240047us
I20260709 12:23:56.365299 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.366179 26576 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:56.366606 26576 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:56.366698 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.367403 26655 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.367606 26819 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:56.368517 26309 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:56.368156 26655 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.368919 26309 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:56.370402 26819 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:56.370782 26769 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:56.372840 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.373055 26663 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.373782 26881 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:56.374104 26663 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:56.374380 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.375885 26697 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.376119 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.376730 26883 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:56.377697 26682 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:56.377641 26883 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:56.379292 26682 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:56.379827 26682 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:56.309692 20735 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:56.312469 20631 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:56.315390 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:56.388466 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.388656 20728 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:56.317859 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.265996 26943 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:56.395624 20879 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:56.276360 26892 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.235176 26843 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.323906 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:56.296298 26807 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:56.290163 21433 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:56.272567 26822 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:56.326746 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2820) took 1661 ms. Trace:
I20260709 12:23:56.330233 20726 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:56.410825 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2546) took 1070 ms. Trace:
I20260709 12:23:56.411379 20726 rpcz_store.cc:276] 0709 12:23:55.340956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.305709 (+964753us) service_pool.cc:224] Handling call
0709 12:23:56.327832 (+ 22123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.410548 (+ 82716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.410761 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.230979 26922 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.333618 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2660) took 2012 ms. Trace:
I20260709 12:23:56.414927 20875 rpcz_store.cc:276] 0709 12:23:54.320861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.332819 (+2011958us) service_pool.cc:224] Handling call
0709 12:23:56.333122 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.333287 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.333310 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.333365 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:56.333394 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.333449 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.333553 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.416247 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:55.932734 26511 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.382s
I20260709 12:23:56.334712 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:55.960610 26803 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.337232 20876 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:55.926781 26866 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.350014 26924 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
W20260709 12:23:56.350903 26925 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
W20260709 12:23:56.351812 26941 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:23:56.390254 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.336295 26674 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:56.390844 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2543) took 1061 ms. Trace:
I20260709 12:23:56.358922 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2662) took 2037 ms. Trace:
I20260709 12:23:56.446868 20882 rpcz_store.cc:276] 0709 12:23:54.321344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.357716 (+2036372us) service_pool.cc:224] Handling call
0709 12:23:56.358150 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.358427 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.358493 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.358555 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:56.358614 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.358691 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.358843 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.380774 26682 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.335883 26793 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:56.294486 26896 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.406839 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2614) took 1324 ms. Trace:
I20260709 12:23:56.459528 20879 rpcz_store.cc:276] 0709 12:23:55.081929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.317486 (+1235557us) service_pool.cc:224] Handling call
0709 12:23:56.406740 (+ 89254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.343722 26918 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.409271 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:56.410439 20733 rpcz_store.cc:276] 0709 12:23:54.664926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.696183 (+ 31257us) service_pool.cc:224] Handling call
0709 12:23:54.696624 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:54.700814 (+  4190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:54.700866 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:54.719428 (+ 18562us) log.cc:844] Serialized 26 byte log entry
0709 12:23:55.075872 (+356444us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:55.075916 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.075990 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:56.326526 (+1250536us) raft_consensus.cc:1685] finished
0709 12:23:56.326547 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.326683 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69082,"raft.queue_time_us":21249,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":52544,"thread_start_us":207976,"threads_started":1,"wal-append.queue_time_us":477332}
W20260709 12:23:56.330338 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54229.2us
I20260709 12:23:56.417778 20806 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
W20260709 12:23:56.431865 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 188929us
I20260709 12:23:56.423794 20631 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 12, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 12 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.382793 26655 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.435364 20728 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:56.465369 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:56.465855 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.430160 26892 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.440618 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:56.467747 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:56.337527 26936 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.341517 26809 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.446744 20735 rpcz_store.cc:276] 0709 12:23:55.329216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.293179 (+963963us) service_pool.cc:224] Handling call
0709 12:23:56.309446 (+ 16267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.390520 (+ 81074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.390777 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.469849 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:56.346985 26526 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.463470 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.489248 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.489686 26526 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.490844 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.436591 26576 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.491477 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:56.464370 26940 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:23:56.492185 21432 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:56.464136 26932 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:23:56.464700 26942 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:23:56.468663 21435 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:56.494194 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.472430 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:56.495086 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:56.495122 26792 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:56.495527 21428 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:56.495904 21435 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:56.473627 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:56.476675 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:56.496932 21434 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:23:56.491674 20734 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:23:56.498602 26771 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:56.499015 26771 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.495482 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:56.479175 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:36384 (request call id 4) took 2059 ms. Trace:
I20260709 12:23:56.499473 26771 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:56.499908 20875 rpcz_store.cc:276] 0709 12:23:54.416691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.430197 (+2013506us) service_pool.cc:224] Handling call
0709 12:23:56.476013 (+ 45816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.501240 20875 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:56.501518 26809 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.501792 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2609) took 1421 ms. Trace:
I20260709 12:23:56.502132 20875 rpcz_store.cc:276] 0709 12:23:55.080528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.500783 (+1420255us) service_pool.cc:224] Handling call
0709 12:23:56.501073 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.501581 (+   508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.501741 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.502847 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2543) took 1860 ms. Trace:
I20260709 12:23:56.503290 20805 rpcz_store.cc:276] 0709 12:23:54.642660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.874280 (+231620us) service_pool.cc:224] Handling call
0709 12:23:54.874666 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:54.925499 (+ 50833us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:54.925548 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:55.142223 (+216675us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:23:55.198839 (+ 56616us) log.cc:844] Serialized 26 byte log entry
0709 12:23:55.448246 (+249407us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:55.448301 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.448378 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:56.502531 (+1054153us) raft_consensus.cc:1685] finished
0709 12:23:56.502560 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.502784 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50271,"raft.queue_time_us":50402,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":20937,"spinlock_wait_cycles":65368320,"thread_start_us":171758,"threads_started":1,"wal-append.queue_time_us":657445}
W20260709 12:23:56.504628 20618 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:56.504853 20729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 938) took 1198 ms (client timeout 1511 ms). Trace:
W20260709 12:23:56.505260 20729 rpcz_store.cc:269] 0709 12:23:55.306031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.306312 (+   281us) service_pool.cc:224] Handling call
0709 12:23:56.504701 (+1198389us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54003b5454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:56.504798 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866754944}
I20260709 12:23:56.505203 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:56.505967 26392 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.506618 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.506536 26392 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.509662 26392 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.511910 20803 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:23:56.513542 26896 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.515950 26816 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:56.402417 26943 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.518625 26943 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.521466 26922 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.522217 26682 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:56.462847 20631 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 10 to 14, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 14 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:23:56.470374 26840 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.524287 26831 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.524077 26840 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.528936 20728 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:56.530402 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:56.479089 26916 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:56.480144 26631 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:56.481245 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:56.481578 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.446710 26674 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:56.482713 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:56.578346 21430 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:56.430027 26822 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.430408 26807 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.484190 26892 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.484367 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:56.487530 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:56.473322 20879 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: [15.12-19.13]
I20260709 12:23:56.609622 21426 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:56.610426 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1048) took 2075 ms. Trace:
I20260709 12:23:56.610877 20879 rpcz_store.cc:276] 0709 12:23:54.534654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.472503 (+1937849us) service_pool.cc:224] Handling call
0709 12:23:56.472966 (+   463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:56.610118 (+137152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.610369 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.612331 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.490790 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:56.478691 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:56.531152 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.534660 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:23:56.614464 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 79992.5us
I20260709 12:23:56.616860 20733 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:56.481830 26647 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.528998 26840 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.528324 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:56.544199 21432 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:56.575173 20726 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:23:56.576447 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:56.581079 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.629083 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 100834us
W20260709 12:23:56.581955 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51652.3us
I20260709 12:23:56.580631 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:56.630173 20729 raft_consensus.cc:2376] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16. Re-sending same reply.
I20260709 12:23:56.630306 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.617353 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:56.618124 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.537397 26844 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.532212 26916 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.632117 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:56.582873 26674 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:56.629855 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:56.636862 20879 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:56.641716 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2612) took 1560 ms. Trace:
I20260709 12:23:56.642127 20879 rpcz_store.cc:276] 0709 12:23:55.081402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.636346 (+1554944us) service_pool.cc:224] Handling call
0709 12:23:56.636671 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.641391 (+  4720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.641633 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.637738 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:56.649408 20728 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:56.614933 26831 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:56.659430 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:56.640589 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:56.670877 20631 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:56.673889 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.672690 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:56.539008 26771 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:56.476158 26918 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.680466 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.636107 26712 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:56.682794 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:56.656919 26840 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:56.635572 26943 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:56.639990 26881 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:56.642333 26916 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:56.687618 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.690667 20631 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 6 to 10, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 10 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.695174 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.671093 26853 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:56.697639 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:56.673537 26831 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:56.689428 26712 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.701774 20631 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 10 to 14, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 14 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.678514 26872 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:56.670965 26864 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:23:56.685642 26779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.626s	user 0.009s	sys 0.013s
I20260709 12:23:56.642737 26844 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:56.706526 20806 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:56.736613 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1050) took 2201 ms (client timeout 1669 ms). Trace:
W20260709 12:23:56.858130 20883 rpcz_store.cc:269] 0709 12:23:54.535468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.888783 (+353315us) service_pool.cc:224] Handling call
0709 12:23:56.736387 (+1847604us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540005b754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:56.736534 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255177472}
I20260709 12:23:56.689203 26612 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:56.859505 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.861125 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:56.860276 26779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.801s	user 0.013s	sys 0.016s
I20260709 12:23:56.840229 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1047) took 2421 ms. Trace:
I20260709 12:23:56.944906 20876 rpcz_store.cc:276] 0709 12:23:54.419029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.456640 (+2037611us) service_pool.cc:224] Handling call
0709 12:23:56.457037 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.457301 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:23:56.679172 (+221871us) log.cc:844] Serialized 14 byte log entry
0709 12:23:56.839801 (+160629us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.839867 (+    66us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:23:56.839929 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.840020 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.840161 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":423924,"raft.queue_time_us":92,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":7084,"thread_start_us":372235,"threads_started":2}
I20260709 12:23:56.940999 26612 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.700632 26881 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:56.989138 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.060813 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:56.992565 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 143261us
I20260709 12:23:56.842393 26883 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:56.867297 26831 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:56.796231 26655 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.052788 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:56.861446 26712 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:56.700961 26792 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:56.985308 26843 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:56.699335 26697 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.290s
W20260709 12:23:57.062191 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.866971 26864 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:57.063863 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:56.707954 26888 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:57.070220 20876 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:57.077119 20806 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:23:56.988754 26612 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:56.986011 26866 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:56.689893 26793 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.109609 26822 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.111220 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:57.112248 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.113564 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.114223 20801 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:57.062919 26936 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.102707 20884 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:57.103399 21430 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
W20260709 12:23:57.085328 26944 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.478s	user 0.000s	sys 0.001s
W20260709 12:23:57.105543 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.116742 21430 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:23:57.077924 26881 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.115150 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.118852 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:57.096935 26946 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.489s	user 0.000s	sys 0.001s
I20260709 12:23:57.123699 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.139279 20884 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:57.140488 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.141589 20884 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:57.142637 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:57.134016 20806 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:57.136225 21429 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1045) took 1571 ms (client timeout 1810 ms). Trace:
W20260709 12:23:57.144322 21429 rpcz_store.cc:269] 0709 12:23:55.565329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.576004 (+ 10675us) service_pool.cc:224] Handling call
0709 12:23:57.135934 (+1559930us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:57.136136 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485613824}
I20260709 12:23:57.147372 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:57.102017 26864 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:57.148078 20801 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.104456 26888 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:57.107501 26697 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:57.122697 26892 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.086246 26655 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:57.153095 26949 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.473s	user 0.001s	sys 0.001s
I20260709 12:23:57.098588 26792 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.148857 26771 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.188499 20806 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:57.196987 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.161253 26888 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:57.210716 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2547) took 2546 ms. Trace:
I20260709 12:23:57.213500 20803 rpcz_store.cc:276] 0709 12:23:54.650035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.880412 (+230377us) service_pool.cc:224] Handling call
0709 12:23:54.887733 (+  7321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:56.512383 (+1624650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.196005 (+683622us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003a1454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:57.196247 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3734471552}
I20260709 12:23:57.222154 20801 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:57.219276 26513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.014s	sys 0.003s
I20260709 12:23:57.223905 20806 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:23:57.231581 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.193488 26918 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.235252 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.232872 20801 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:57.237931 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:57.228376 26513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.015s	sys 0.003s
W20260709 12:23:57.234161 26682 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.711s	user 0.006s	sys 0.005s
I20260709 12:23:57.227505 26936 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:57.253575 26963 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:23:57.254019 26936 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:57.255704 26964 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:23:57.256738 26936 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:57.249050 26663 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.873s	user 0.014s	sys 0.000s
I20260709 12:23:57.258375 26811 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:57.258956 26811 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.259301 26792 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.260416 20810 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:57.261777 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2541) took 1933 ms. Trace:
I20260709 12:23:57.262239 20730 rpcz_store.cc:276] 0709 12:23:55.328486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.273085 (+944599us) service_pool.cc:224] Handling call
0709 12:23:56.275555 (+  2470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:56.297313 (+ 21758us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:56.533301 (+235988us) log.cc:844] Serialized 14 byte log entry
0709 12:23:56.702926 (+169625us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:56.703763 (+   837us) log.cc:844] Serialized 52 byte log entry
0709 12:23:56.703959 (+   196us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:56.735575 (+ 31616us) log.cc:844] Serialized 14 byte log entry
0709 12:23:56.860184 (+124609us) log.cc:844] Serialized 14 byte log entry
0709 12:23:56.860498 (+   314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.860593 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.261440 (+400847us) raft_consensus.cc:1685] finished
0709 12:23:57.261473 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.261679 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":210595,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":130,"raft.queue_time_us":12380,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":55144,"spinlock_wait_cycles":183208832,"thread_start_us":283752,"threads_started":2,"wal-append.queue_time_us":489864}
I20260709 12:23:57.280902 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2661) took 2959 ms. Trace:
I20260709 12:23:57.281425 20877 rpcz_store.cc:276] 0709 12:23:54.321110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.349236 (+2028126us) service_pool.cc:224] Handling call
0709 12:23:56.404144 (+ 54908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:56.404640 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.404682 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:56.405330 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:23:56.710172 (+304842us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:56.710235 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.710337 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.280624 (+570287us) raft_consensus.cc:1685] finished
0709 12:23:57.280655 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.280833 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":206,"raft.queue_time_us":1555,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":23737,"spinlock_wait_cycles":2304,"thread_start_us":254511,"threads_started":1,"wal-append.queue_time_us":719491}
I20260709 12:23:57.283679 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.285104 20877 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:23:57.248759 26844 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:57.286500 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.287428 20877 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
W20260709 12:23:57.288118 26854 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.422s	user 0.007s	sys 0.007s
I20260709 12:23:57.288601 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.288739 26854 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.290407 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.292703 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2615) took 2210 ms. Trace:
I20260709 12:23:57.293273 20877 rpcz_store.cc:276] 0709 12:23:55.082205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.291810 (+2209605us) service_pool.cc:224] Handling call
0709 12:23:57.292098 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.292299 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.292333 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.292381 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:57.292422 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.292482 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.292650 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.295701 20877 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:57.302784 20730 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:57.304930 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2663) took 2182 ms. Trace:
I20260709 12:23:57.305514 20877 rpcz_store.cc:276] 0709 12:23:55.122059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.295054 (+2172995us) service_pool.cc:224] Handling call
0709 12:23:57.295453 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.304559 (+  9106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.304851 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.316054 26655 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.294881 26513 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:57.316936 26513 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:57.317910 26513 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:57.318713 26916 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.003s
W20260709 12:23:57.326252 26674 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.640s	user 0.006s	sys 0.003s
I20260709 12:23:57.327327 26854 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:57.335350 26674 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.337925 26674 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.338485 26916 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.342310 20733 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:23:57.293375 26876 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.415s	user 0.011s	sys 0.002s
I20260709 12:23:57.348022 20730 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:57.353468 20877 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:57.319065 26513 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:57.362105 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2665) took 2176 ms. Trace:
I20260709 12:23:57.362880 20877 rpcz_store.cc:276] 0709 12:23:55.185812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.306813 (+2121001us) service_pool.cc:224] Handling call
0709 12:23:57.353189 (+ 46376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.355219 (+  2030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.362008 (+  6789us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.364042 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:57.370898 20730 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:57.346649 26916 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.322922 26888 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.373566 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.379657 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:57.380502 20803 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.380386 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:57.381223 20877 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:57.349253 26876 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.393046 20728 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:57.394953 26876 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.390296 26888 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.388862 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:57.401518 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:57.405624 20803 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:23:57.406705 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.411322 26671 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:57.413642 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:57.414577 20877 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:57.378576 26612 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.415937 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:57.419243 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2669) took 2160 ms. Trace:
I20260709 12:23:57.419770 20877 rpcz_store.cc:276] 0709 12:23:55.260322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.417436 (+2157114us) service_pool.cc:224] Handling call
0709 12:23:57.417874 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.418095 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.418123 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.418166 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:57.418702 (+   536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.418908 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.419032 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.422068 20877 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:57.422736 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2673) took 2097 ms. Trace:
I20260709 12:23:57.413082 26671 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.427207 26881 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.428314 20879 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:57.430500 26671 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.430555 26876 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:57.431447 20804 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:57.431351 26876 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.431846 20804 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:57.432822 20804 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.437135 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2832) took 1006 ms. Trace:
I20260709 12:23:57.437367 26309 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.438285 26712 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.437640 20806 rpcz_store.cc:276] 0709 12:23:56.430287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.236222 (+805935us) service_pool.cc:224] Handling call
0709 12:23:57.246828 (+ 10606us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:57.252528 (+  5700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.252563 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:57.260333 (+  7770us) log.cc:844] Serialized 52 byte log entry
0709 12:23:57.376957 (+116624us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:57.386361 (+  9404us) log.cc:844] Serialized 14 byte log entry
0709 12:23:57.389731 (+  3370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.389813 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.436874 (+ 47061us) raft_consensus.cc:1685] finished
0709 12:23:57.436896 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.437065 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7943,"prepare.queue_time_us":9193,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":8160,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":187,"thread_start_us":116181,"threads_started":1,"wal-append.queue_time_us":148814}
I20260709 12:23:57.439850 20884 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:57.441443 20804 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:57.429239 26831 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.446700 26881 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.446900 26309 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:57.447566 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:57.448351 20735 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.450465 20884 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:57.451157 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2616) took 2111 ms. Trace:
I20260709 12:23:57.451503 20884 rpcz_store.cc:276] 0709 12:23:55.340012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.449696 (+2109684us) service_pool.cc:224] Handling call
0709 12:23:57.450293 (+   597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.450905 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.451099 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.453234 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2617) took 2112 ms. Trace:
I20260709 12:23:57.456995 26895 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:57.457705 26895 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.460297 26895 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.462219 20876 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:57.453563 20884 rpcz_store.cc:276] 0709 12:23:55.340335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.452260 (+2111925us) service_pool.cc:224] Handling call
0709 12:23:57.452557 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.452856 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.452884 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.452933 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:57.452965 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.453036 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.453180 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.463107 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2620) took 2122 ms. Trace:
I20260709 12:23:57.466449 20876 rpcz_store.cc:276] 0709 12:23:55.341151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.461581 (+2120430us) service_pool.cc:224] Handling call
0709 12:23:57.461970 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.462786 (+   816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.463040 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.423116 20877 rpcz_store.cc:276] 0709 12:23:55.325255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.421562 (+2096307us) service_pool.cc:224] Handling call
0709 12:23:57.421890 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.422511 (+   621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.422683 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.442552 26864 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:57.470424 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 834646us
I20260709 12:23:57.476864 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:57.479496 26831 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.480368 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:57.482038 20877 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:57.482762 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2625) took 1296 ms. Trace:
I20260709 12:23:57.483105 20877 rpcz_store.cc:276] 0709 12:23:56.186021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.468074 (+1282053us) service_pool.cc:224] Handling call
0709 12:23:57.481832 (+ 13758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.482500 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.482706 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.484753 20877 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:57.485358 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1055) took 1091 ms. Trace:
I20260709 12:23:57.485708 20877 rpcz_store.cc:276] 0709 12:23:56.394228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.484003 (+1089775us) service_pool.cc:224] Handling call
0709 12:23:57.484391 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.485093 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.485302 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.487413 20804 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:57.492259 20876 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
W20260709 12:23:57.491606 26933 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:23:57.493589 26881 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:57.494429 20804 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:57.494779 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2624) took 1309 ms. Trace:
I20260709 12:23:57.495442 20735 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:57.495175 20876 rpcz_store.cc:276] 0709 12:23:56.185151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.467605 (+1282454us) service_pool.cc:224] Handling call
0709 12:23:57.492072 (+ 24467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.494500 (+  2428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.494702 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.493785 26884 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.504443 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.505513 20804 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:23:57.509367 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:57.509852 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.510174 20807 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 12.
I20260709 12:23:57.501717 26881 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:57.512421 26612 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.516803 26864 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:57.521505 26712 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:57.523630 26876 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.524603 26881 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:57.553984 26975 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:23:57.541273 26838 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:57.558121 26838 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:57.563553 26960 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.432s	user 0.003s	sys 0.000s
I20260709 12:23:57.608928 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2678) took 1110 ms. Trace:
I20260709 12:23:57.609915 20877 rpcz_store.cc:276] 0709 12:23:56.498349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.486587 (+988238us) service_pool.cc:224] Handling call
0709 12:23:57.486979 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.488573 (+  1594us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:23:57.488839 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 12:23:57.607920 (+119081us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.608055 (+   135us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:57.608164 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.608341 (+   177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.608600 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11439,"prepare.queue_time_us":2958,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":118265,"threads_started":1,"wal-append.queue_time_us":46384}
I20260709 12:23:57.611838 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:57.614144 20877 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:57.616356 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:57.617087 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:57.617633 20806 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:57.617951 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36394 (request call id 0) took 2291 ms. Trace:
I20260709 12:23:57.618480 20877 rpcz_store.cc:276] 0709 12:23:55.326344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.616019 (+2289675us) service_pool.cc:224] Handling call
0709 12:23:57.617857 (+  1838us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.624039 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:57.639156 20804 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:57.625691 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2613) took 2543 ms. Trace:
I20260709 12:23:57.638906 26811 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.618983 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:23:57.632436 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 2.257s (SENT) [suppressed 12 similar messages]
I20260709 12:23:57.667912 20803 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.659863 20804 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:57.632292 26791 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.660326 20879 rpcz_store.cc:276] 0709 12:23:55.081671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.643408 (+1561737us) service_pool.cc:224] Handling call
0709 12:23:56.643792 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.428721 (+784929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.524010 (+ 95289us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:57.625480 (+101470us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1803692288}
I20260709 12:23:57.689690 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:57.696880 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2847) took 1190 ms. Trace:
I20260709 12:23:57.697403 20730 rpcz_store.cc:276] 0709 12:23:56.506051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.475934 (+969883us) service_pool.cc:224] Handling call
0709 12:23:57.610602 (+134668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:57.611497 (+   895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.611534 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:57.612632 (+  1098us) log.cc:844] Serialized 26 byte log entry
0709 12:23:57.647954 (+ 35322us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:57.648024 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.648140 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.696437 (+ 48297us) raft_consensus.cc:1685] finished
0709 12:23:57.696457 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.696627 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21192,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":354,"thread_start_us":32292,"threads_started":1,"wal-append.queue_time_us":64548}
I20260709 12:23:57.698158 26791 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.700088 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:57.713691 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:57.717960 20731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 942) took 2407 ms (client timeout 1811 ms). Trace:
I20260709 12:23:57.717797 20804 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:57.718468 20731 rpcz_store.cc:269] 0709 12:23:55.310620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.310761 (+   141us) service_pool.cc:224] Handling call
0709 12:23:57.717479 (+2406718us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b5400226ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:57.717754 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3758422272}
I20260709 12:23:57.719648 20804 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:57.721374 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2622) took 2063 ms. Trace:
I20260709 12:23:57.721668 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2842) took 1216 ms. Trace:
I20260709 12:23:57.724123 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2659) took 3403 ms. Trace:
W20260709 12:23:57.725348 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2667) took 2487 ms (client timeout 2187 ms). Trace:
I20260709 12:23:57.751689 26838 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.753655 26838 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:57.756515 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1039) took 2440 ms (client timeout 1869 ms). Trace:
W20260709 12:23:57.757121 21433 rpcz_store.cc:269] 0709 12:23:55.315780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.315919 (+   139us) service_pool.cc:224] Handling call
0709 12:23:57.755867 (+2439948us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54007602d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:57.756189 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681987968}
I20260709 12:23:57.779348 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2821) took 1421 ms. Trace:
I20260709 12:23:57.739105 20878 rpcz_store.cc:276] 0709 12:23:54.320606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.335143 (+ 14537us) service_pool.cc:224] Handling call
0709 12:23:54.335538 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.199957 (+864419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.723540 (+2523583us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b540022f6d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:57.723950 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931682560}
W20260709 12:23:57.796907 26965 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.531s	user 0.002s	sys 0.000s
W20260709 12:23:57.690560 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56203.8us
I20260709 12:23:57.806772 26791 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.807862 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:57.808574 26576 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.809207 20731 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:57.810765 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:57.816449 20806 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:23:57.738792 20733 rpcz_store.cc:276] 0709 12:23:56.504701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.110929 (+606228us) service_pool.cc:224] Handling call
0709 12:23:57.721215 (+610286us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54000c3fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:57.721466 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529141632}
W20260709 12:23:57.739449 20881 rpcz_store.cc:269] 0709 12:23:55.238023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.924567 (+686544us) service_pool.cc:224] Handling call
0709 12:23:57.724912 (+1800345us) spinlock_profiling.cc:243] Waited 79.9 ms on lock 0x7b54000965d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:57.725167 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183682304}
I20260709 12:23:57.780058 20802 rpcz_store.cc:276] 0709 12:23:56.302317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.302572 (+   255us) service_pool.cc:224] Handling call
0709 12:23:56.302939 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.258038 (+955099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.258073 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.258112 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:57.258155 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.722807 (+464652us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:57.722878 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.723089 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2223,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"spinlock_wait_cycles":2195621632}
I20260709 12:23:57.825556 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:23:57.826007 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.828933 26924 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:57.833066 26924 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:57.829099 26955 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.729s	user 0.002s	sys 0.000s
I20260709 12:23:57.838701 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:57.840492 26852 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.841441 26852 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.842131 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:57.847757 21434 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:23:57.827174 20726 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:57.852061 20803 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:57.852533 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:57.861173 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:57.844385 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2819) took 1554 ms. Trace:
I20260709 12:23:57.855690 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:57.823572 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2848) took 1154 ms. Trace:
I20260709 12:23:57.849809 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:57.848860 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2824) took 1465 ms. Trace:
I20260709 12:23:57.843152 20729 raft_consensus.cc:2376] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18. Re-sending same reply.
I20260709 12:23:57.860610 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:57.861624 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2611) took 2780 ms. Trace:
I20260709 12:23:57.870494 21426 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.865931 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:57.869032 20875 rpcz_store.cc:276] 0709 12:23:55.081130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.502846 (+1421716us) service_pool.cc:224] Handling call
0709 12:23:56.737084 (+234238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.737292 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.737324 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.737374 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:56.737416 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.737470 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.861166 (+1123696us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540005b750. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:57.861398 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538007424}
I20260709 12:23:57.738380 20884 rpcz_store.cc:276] 0709 12:23:55.658057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.491345 (+1833288us) service_pool.cc:224] Handling call
0709 12:23:57.491958 (+   613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.492166 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:57.504889 (+ 12723us) log.cc:844] Serialized 14 byte log entry
0709 12:23:57.662232 (+157343us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.671634 (+  9402us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:57.688989 (+ 17355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.693263 (+  4274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.721175 (+ 27912us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16511,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1479,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":10073,"thread_start_us":138968,"threads_started":1,"wal-append.queue_time_us":44445}
I20260709 12:23:57.878041 21434 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:57.862088 20809 rpcz_store.cc:276] 0709 12:23:56.289483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.291281 (+  1798us) service_pool.cc:224] Handling call
0709 12:23:56.291797 (+   516us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:56.607954 (+316157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.607989 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:56.620703 (+ 12714us) log.cc:844] Serialized 52 byte log entry
0709 12:23:56.755995 (+135292us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:57.230404 (+474409us) log.cc:844] Serialized 14 byte log entry
0709 12:23:57.230671 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.843564 (+612893us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:57.843658 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.843750 (+    92us) raft_consensus.cc:1685] finished
0709 12:23:57.843881 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.844179 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":967,"prepare.queue_time_us":95338,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":43009,"raft.queue_time_us":1987,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":14442,"spinlock_wait_cycles":667251584,"thread_start_us":454210,"threads_started":2,"wal-append.queue_time_us":435085}
I20260709 12:23:57.881772 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:57.859676 26924 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:57.884234 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.885326 21433 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:57.864462 26791 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.866411 20727 rpcz_store.cc:276] 0709 12:23:56.383234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.384121 (+   887us) service_pool.cc:224] Handling call
0709 12:23:56.397644 (+ 13523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:56.398041 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.398067 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:56.408193 (+ 10126us) log.cc:844] Serialized 26 byte log entry
0709 12:23:56.657418 (+249225us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:56.657474 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.657582 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.795320 (+1137738us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:57.848215 (+ 52895us) raft_consensus.cc:1685] finished
0709 12:23:57.848248 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.848617 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23360896,"thread_start_us":248801,"threads_started":1,"wal-append.queue_time_us":450363}
I20260709 12:23:57.864763 20735 rpcz_store.cc:276] 0709 12:23:56.506312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.600480 (+1094168us) service_pool.cc:224] Handling call
0709 12:23:57.601483 (+  1003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.602115 (+   632us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:23:57.602778 (+   663us) log.cc:844] Serialized 14 byte log entry
0709 12:23:57.658851 (+ 56073us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.658926 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:57.658986 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.659074 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.660736 (+  1662us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45402,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":378,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":9661,"thread_start_us":54443,"threads_started":1,"wal-append.queue_time_us":195256}
I20260709 12:23:57.890959 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2853) took 1345 ms. Trace:
I20260709 12:23:57.891449 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:57.896019 20729 rpcz_store.cc:276] 0709 12:23:56.542460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.876708 (+1334248us) service_pool.cc:224] Handling call
0709 12:23:57.882320 (+  5612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.885700 (+  3380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.885735 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.887771 (+  2036us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:57.887824 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.887894 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.888011 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:57.900260 20726 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.901582 21434 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 12: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:57.902550 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 939) took 2580 ms. Trace:
W20260709 12:23:57.905860 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.905479 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:57.909963 26921 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.885337 26881 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:57.904379 26807 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.916386 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2682) took 1076 ms. Trace:
I20260709 12:23:57.904497 20732 rpcz_store.cc:276] 0709 12:23:55.307216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.307334 (+   118us) service_pool.cc:224] Handling call
0709 12:23:55.307688 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:57.614572 (+2306884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.614626 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:57.615291 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 12:23:57.659387 (+ 44096us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:57.659431 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.878174 (+218743us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b54002990d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:23:57.878239 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.878290 (+    51us) raft_consensus.cc:1685] finished
0709 12:23:57.878303 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.887674 (+  9371us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18729,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":154,"spinlock_wait_cycles":5303317376,"thread_start_us":43598,"threads_started":1,"wal-append.queue_time_us":205407}
I20260709 12:23:57.914683 26921 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.918251 20880 rpcz_store.cc:276] 0709 12:23:56.837992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.521856 (+683864us) service_pool.cc:224] Handling call
0709 12:23:57.522196 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:57.522552 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.522578 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:57.523028 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 12:23:57.811827 (+288799us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:57.824214 (+ 12387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.865006 (+ 40792us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.908000 (+ 42994us) raft_consensus.cc:1685] finished
0709 12:23:57.908026 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.914225 (+  6199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1883,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":170074,"threads_started":1,"wal-append.queue_time_us":301585}
I20260709 12:23:57.913012 26852 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.910562 26791 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.933995 26934 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:23:57.929638 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:57.935482 20700 connection.cc:256] RPC call timeout handler was delayed by -1.21848778399999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 12:23:57.941392 20700 connection.cc:256] RPC call timeout handler was delayed by -1.2090166340003634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 12:23:57.941887 20700 connection.cc:256] RPC call timeout handler was delayed by -1.1178653010001653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 12:23:57.944180 20700 connection.cc:256] RPC call timeout handler was delayed by -1.0582877840001856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
I20260709 12:23:57.941339 20730 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:57.945104 26817 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.945900 26817 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.957373 20727 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:57.958954 26807 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:57.958042 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:57.968364 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:57.979169 26807 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:57.980129 26807 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.979446 26889 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:57.985407 26889 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:23:57.986608 26996 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:23:57.988387 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2841) took 1483 ms. Trace:
I20260709 12:23:57.988929 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:57.989516 26921 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:57.970355 26881 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.987167 26889 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:57.990007 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:57.992985 26852 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:57.993758 26852 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:57.988854 20728 rpcz_store.cc:276] 0709 12:23:56.504439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.458185 (+953746us) service_pool.cc:224] Handling call
0709 12:23:57.458627 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:57.459064 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.459105 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:57.604019 (+144914us) log.cc:844] Serialized 26 byte log entry
0709 12:23:57.676444 (+ 72425us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:57.684067 (+  7623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.689646 (+  5579us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.987938 (+298292us) raft_consensus.cc:1685] finished
0709 12:23:57.987962 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.988202 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1752,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":60416,"threads_started":1,"wal-append.queue_time_us":274455}
W20260709 12:23:57.996245 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 77900.8us
I20260709 12:23:57.999989 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:58.006131 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:58.007436 21431 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:23:58.007663 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50757us
W20260709 12:23:58.015448 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.023289 26876 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:58.010497 26807 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.009343 26881 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:58.035385 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.035583 26881 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:58.044644 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.048768 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:58.049186 20726 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:23:58.051966 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.053690 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:58.054818 26831 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:58.055330 26831 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.055691 26831 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:58.060277 26852 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.039284 26791 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:58.071691 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60175.1us
I20260709 12:23:58.064010 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:58.080013 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.074072 20729 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:58.090857 26852 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:58.091585 26852 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.092099 26852 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:58.095373 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 138216us
I20260709 12:23:58.097829 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:58.100623 26881 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.101329 26881 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.102227 26817 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:58.103706 26883 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:58.104148 26883 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:58.067065 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:58.105043 21434 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:58.104533 26883 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:58.112313 26838 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:58.123631 26809 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:23:58.124249 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.133822 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:58.136781 20804 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:23:58.137207 20804 pending_rounds.cc:85] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 11
I20260709 12:23:58.137622 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:58.136384 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 136629us
I20260709 12:23:58.144109 26809 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.144693 26924 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:58.144425 26933 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:23:58.146354 26933 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:58.146809 26809 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.149467 26647 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:23:58.151710 26924 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:58.152047 26647 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:58.152760 26647 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:58.154243 26647 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.155947 26647 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:58.156589 26924 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:58.147008 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 120289us
I20260709 12:23:58.159685 27004 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:58.176153 21434 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:58.176465 27004 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:58.176898 27004 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:58.137640 20804 pending_rounds.cc:107] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:23:58.179972 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2849) took 1655 ms. Trace:
W20260709 12:23:58.181636 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49695.9us
I20260709 12:23:58.183915 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:58.182731 20804 raft_consensus.cc:2889] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:58.133062 26881 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:23:58.146878 26933 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:58.201699 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101674us
I20260709 12:23:58.183121 20731 rpcz_store.cc:276] 0709 12:23:56.506562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.879876 (+1373314us) service_pool.cc:224] Handling call
0709 12:23:57.904468 (+ 24592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.918800 (+ 14332us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:23:57.937838 (+ 19038us) log.cc:844] Serialized 14 byte log entry
0709 12:23:58.143795 (+205957us) log.cc:844] Serialized 14 byte log entry
0709 12:23:58.161439 (+ 17644us) log.cc:844] Serialized 14 byte log entry
0709 12:23:58.161744 (+   305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:58.161801 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:58.161869 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.162110 (+   241us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:58.162241 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11628,"prepare.queue_time_us":8320,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":8093,"raft.queue_time_us":25899,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":15505,"spinlock_wait_cycles":30272640,"thread_start_us":164023,"threads_started":2,"wal-append.queue_time_us":143169}
I20260709 12:23:58.201408 26943 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:58.210302 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:58.212553 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:58.215255 26511 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:58.187278 26886 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
W20260709 12:23:58.231897 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134381us
I20260709 12:23:58.208348 26817 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:23:58.233661 21427 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:58.234539 26817 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:58.230204 27009 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:58.235067 26817 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:58.235306 26912 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:58.237789 26817 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.239326 26817 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:58.248425 21426 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:58.252796 21433 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:58.226943 26943 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:23:58.257936 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55760.2us
I20260709 12:23:58.258925 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:58.263834 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:23:58.265908 20681 auto_rebalancer.cc:893] Removing replace marker failed: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:23:58.266003 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:58.266762 20681 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:23:58.267077 21432 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 13: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:23:58.267217 20681 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:23:58.267756 20880 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:58.268137 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:58.268409 26852 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.268976 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:23:58.269209 26889 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:58.269888 26889 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:58.247690 26886 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:58.271550 26886 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:58.271169 26889 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.271642 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.272578 26886 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.273689 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.274324 26886 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:58.266659 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:58.275471 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:58.275940 26943 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.276847 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:58.277609 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.265973 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.278951 27012 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:58.279539 27012 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.280030 27012 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:58.280720 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.281383 20881 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:23:58.278654 27011 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:58.282723 27011 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:58.255874 26934 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:58.283890 27011 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:58.286662 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.286510 26933 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.286561 21431 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:58.287636 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.287701 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.290102 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:58.290601 20880 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: [15.12-19.13]
I20260709 12:23:58.291306 20878 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 13: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:58.292030 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.293952 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.289685 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:58.296592 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.297452 26933 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:58.297374 26876 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:58.279706 20884 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:58.298499 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:58.301805 20775 connection.cc:256] RPC call timeout handler was delayed by -1.6894760199998018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 12:23:58.302742 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:58.313542 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 78815.5us
I20260709 12:23:58.333868 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:58.335116 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76338us
I20260709 12:23:58.336174 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:23:58.314718 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75706.9us
I20260709 12:23:58.329555 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:58.347031 21432 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:58.349288 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.352682 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:58.353451 27005 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:58.348583 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:58.324637 26852 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.374312 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.374392 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:58.384298 20775 connection.cc:256] RPC call timeout handler was delayed by -1.3807760199997574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
W20260709 12:23:58.385008 20775 connection.cc:256] RPC call timeout handler was delayed by -1.1683760199998687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260709 12:23:58.388716 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:58.391427 20878 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:58.396369 27009 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:58.397043 27009 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:58.397552 27009 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:58.399484 26791 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:58.400286 26791 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.367291 27005 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:58.406682 26934 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:58.418733 27005 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:58.418787 26934 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:58.419454 26934 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:58.423578 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:58.426605 27017 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.099s	user 0.003s	sys 0.000s
I20260709 12:23:58.407722 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:58.427364 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63296.2us
I20260709 12:23:58.428903 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:58.358501 26889 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:23:58.430610 20775 leader_election.cc:400] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15.
I20260709 12:23:58.431602 26809 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:58.432212 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57484.3us
I20260709 12:23:58.432977 26921 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:58.411702 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54446us
I20260709 12:23:58.432273 26809 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:58.403156 26791 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:58.431319 20618 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:23:58.435964 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.438366 26876 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:23:58.441545 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.435173 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:58.440900 20618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63800.2us
I20260709 12:23:58.443563 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:58.448544 27020 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:58.451047 27020 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:23:58.474524 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:58.497280 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:58.514506 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80174.9us
I20260709 12:23:58.466567 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:58.475845 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:23:58.475967 26876 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.477061 26889 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:58.439422 26883 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:58.479267 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48920.6us
I20260709 12:23:58.485260 26809 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.376523 26831 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.490576 26576 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:23:58.507140 27004 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.377374 26807 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.131s
I20260709 12:23:58.463508 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.497277 26838 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:58.510386 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.512499 27012 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.513396 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.539705 20876 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:58.464568 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.522696 20880 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:58.540122 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.525703 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.491468 26791 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:58.542182 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.509855 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:58.548509 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.549515 20803 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:23:58.528828 26817 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.553032 26817 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.556149 26511 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:58.538394 26889 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:58.557052 26886 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:23:58.542557 26791 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:58.558187 26889 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.559679 26791 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.561618 26791 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:58.565055 26883 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:58.566267 26883 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:58.570398 26817 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.571545 26817 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.572402 26817 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.576876 27012 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:58.577097 26817 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:58.583715 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:58.586489 21430 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:58.587487 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:58.596161 20806 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:23:58.589285 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:58.601143 26511 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:58.604395 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63376us
I20260709 12:23:58.604435 26511 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:58.605975 26511 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:58.615389 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 279380us
I20260709 12:23:58.613421 26647 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:23:58.626468 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 273884us
I20260709 12:23:58.604646 27005 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.632859 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:58.636292 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:23:58.663820 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 229716us
I20260709 12:23:58.668069 27023 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:58.668298 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45042.4us
I20260709 12:23:58.668854 27023 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:58.669580 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:58.672163 27023 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.672978 27023 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:58.673146 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.673986 27023 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:58.674520 27023 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:58.675966 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:58.676673 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:23:58.627993 26807 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:58.683123 27005 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:23:58.687536 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58673.5us
I20260709 12:23:58.691954 26831 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.692355 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:58.702402 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:58.708139 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:58.675181 27023 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:58.713357 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:58.714291 20729 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:23:58.710975 20731 raft_consensus.cc:2376] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16. Re-sending same reply.
I20260709 12:23:58.637187 26934 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.712005 27005 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:23:58.725731 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.726212 26831 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:58.726753 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.707361 26647 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:58.741803 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:58.746794 20806 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:23:58.741572 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.748878 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:58.765511 20802 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:58.774513 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:58.779382 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:58.780048 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.784618 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:58.821183 20806 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
W20260709 12:23:58.786566 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 276837us
I20260709 12:23:58.787220 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.830848 20808 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:58.833907 26511 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:58.836225 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.775629 27004 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:58.856441 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:58.856624 20802 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
W20260709 12:23:58.858167 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 185556us
W20260709 12:23:58.862416 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 113684us
I20260709 12:23:58.784991 26831 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:23:58.862298 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 182242us
I20260709 12:23:58.867218 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.873761 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:23:58.876663 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:58.905838 20802 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:58.776337 27009 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.909052 20881 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:58.911495 20805 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:23:58.913654 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:58.866262 26511 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:58.921238 20808 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:23:58.922001 26511 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.923326 21432 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:58.916051 26809 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:58.927965 26809 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.862020 26647 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:58.947011 20808 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:58.951606 20881 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:23:58.949015 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:58.960328 27013 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:58.960528 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90826.6us
I20260709 12:23:58.963398 20875 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:58.965549 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.966812 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:58.968001 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.968189 20875 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:58.969229 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:58.949226 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 158494us
W20260709 12:23:58.971521 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.980257 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:58.983848 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76940.4us
I20260709 12:23:58.981160 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:58.969238 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:58.985172 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:58.983719 20805 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:23:58.987404 27015 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:58.971385 20876 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:58.994179 20802 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:58.994413 27014 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:58.995432 27014 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.998260 26731 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:23:58.980376 26647 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:59.000880 27025 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:23:59.001132 26647 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.002688 26809 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:58.985256 26817 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:59.003893 26817 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:58.961021 27013 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:58.946995 26807 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:59.022073 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:23:59.002797 27032 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:23:59.041057 20805 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:23:59.041703 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:59.041883 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48665.2us
I20260709 12:23:58.985571 20729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46798
I20260709 12:23:58.987957 27015 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:59.046626 20729 raft_consensus.cc:606] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:59.049834 20729 raft_consensus.cc:614] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:59.053161 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72107.7us
I20260709 12:23:59.021474 26934 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.004933 26817 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.058568 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.051577 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.059532 20802 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:23:59.060005 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.043285 26881 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:59.062065 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.066757 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.045047 26809 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:59.055454 27013 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.062454 27004 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:59.050390 26807 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:59.104488 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.105422 26807 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.078349 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.106724 20802 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:23:59.078791 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.108183 20808 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:59.079536 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:59.082185 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:59.082366 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:59.109041 20803 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:59.084812 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:59.085712 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.075115 26934 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.074532 26809 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:59.088205 26817 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.091684 27038 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.072252 26881 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.094529 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:59.070621 26731 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:59.100106 27009 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.076393 26943 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:59.101315 27015 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.139802 27015 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.102455 27023 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:59.103044 27004 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.076756 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.121238 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:59.131754 27020 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.134634 20801 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:59.134975 20880 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:23:59.122110 26852 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.122188 27019 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:59.136596 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 80945.7us
I20260709 12:23:59.138037 27014 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.159538 20806 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:59.164294 20805 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
W20260709 12:23:59.170686 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.176079 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.181044 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1215) took 1180 ms. Trace:
I20260709 12:23:59.181504 21430 rpcz_store.cc:276] 0709 12:23:57.988404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.989679 (+  1275us) service_pool.cc:224] Handling call
0709 12:23:59.168165 (+1178486us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:23:59.168291 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1271874944}
I20260709 12:23:59.147969 26731 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:59.159757 27004 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.194839 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.198817 21431 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:59.221328 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.163229 27009 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:23:59.195282 26731 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.158713 27035 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:23:59.230552 21427 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:23:59.240456 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 154875us
W20260709 12:23:59.254340 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67909.1us
W20260709 12:23:59.260789 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68861.6us
I20260709 12:23:59.202628 27020 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:23:59.185604 27038 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:59.256423 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51900.1us
W20260709 12:23:59.239483 26791 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.677s	user 0.012s	sys 0.006s
I20260709 12:23:59.266326 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.274034 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:23:59.272655 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108033us
I20260709 12:23:59.282414 20803 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:23:59.283921 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 322147us
I20260709 12:23:59.289019 20806 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:23:59.280507 26838 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:59.309847 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43614.4us
W20260709 12:23:59.331681 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 69111.6us
I20260709 12:23:59.332708 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.336107 26809 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.304093 26790 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:59.337670 26790 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:59.338682 26790 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.340183 26790 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:59.293268 26886 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:23:59.344377 26886 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:59.344802 26886 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:59.345830 26886 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.347280 27047 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:23:59.347492 26731 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:59.349498 27050 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:23:59.349877 26731 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.350579 27049 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 12:23:59.380990 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:59.387034 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:59.369781 26886 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:59.396356 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.392513 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.400662 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.351572 26791 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:23:59.407212 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:59.406234 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 227994us
W20260709 12:23:59.409195 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 135331us
I20260709 12:23:59.409409 26809 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:59.410537 26647 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:59.410890 26647 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:23:59.419121 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 86504.6us
W20260709 12:23:59.432654 27058 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:23:59.411783 26647 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.433343 27029 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:59.434059 27029 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:59.434187 26647 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:59.435611 26921 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:59.436050 26921 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:23:59.436622 27046 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:59.437049 27046 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:59.435042 27029 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:59.408576 27023 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.423491 26943 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:59.441447 26876 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:59.450129 27046 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.437083 26921 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.451927 27046 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.452293 27038 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.454109 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:59.454561 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 126522us
I20260709 12:23:59.456024 20880 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:59.456491 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:23:59.461068 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 80219.5us
I20260709 12:23:59.462059 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:23:59.462771 27036 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.469s	user 0.002s	sys 0.002s
I20260709 12:23:59.463042 26838 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.466552 26881 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.468094 27005 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:59.468673 27005 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:59.469118 27005 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:59.472101 27034 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:59.472558 27034 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:59.473526 27034 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.476815 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.478063 26876 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:59.479449 27041 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:23:59.480108 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.480762 21435 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:59.481011 20726 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:23:59.482219 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:59.483667 26790 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.458906 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.484522 26790 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.424937 26883 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:23:59.486580 21432 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:59.487205 26876 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:59.487522 26790 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.487656 26876 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.487771 26831 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.488145 27074 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.488471 26876 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:59.434983 27004 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:59.488611 26831 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.490880 27004 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.501905 20801 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:23:59.507428 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:23:59.512907 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70626.6us
I20260709 12:23:59.515480 20884 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.506551 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.517287 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.475065 27034 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:59.520195 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.521068 20726 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:59.522137 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:59.522846 27030 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.487s	user 0.000s	sys 0.002s
I20260709 12:23:59.522924 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:59.523665 20880 raft_consensus.cc:606] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:59.527810 27005 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:59.529327 27037 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.453s	user 0.000s	sys 0.001s
I20260709 12:23:59.530525 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.532828 20731 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:59.531127 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.534443 20728 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:59.531519 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:59.490965 26831 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.538887 20727 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:59.506791 26912 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:59.531697 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.550525 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.550925 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.551590 20801 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:59.552197 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.532033 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.552843 20735 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:23:59.553464 20733 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:59.553751 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.543066 20726 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:59.555440 26912 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.556272 26912 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.557264 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.558400 20726 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:59.559712 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:59.566084 27038 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:59.554919 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.543000 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.546917 20734 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:59.543529 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.544505 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.544737 27020 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.545452 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.546859 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:59.541893 20729 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:59.554682 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.568521 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:59.577366 20730 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:59.575661 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.579916 20726 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:59.580307 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:23:59.580889 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 186074us
I20260709 12:23:59.572556 20733 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:23:59.588274 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.589248 20801 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:23:59.592240 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:59.592432 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:59.594079 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.595625 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:59.597008 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.600069 20879 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:23:59.600390 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:59.598023 20728 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:23:59.601707 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:59.578708 27033 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.506s	user 0.001s	sys 0.004s
I20260709 12:23:59.608657 21430 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:59.530138 27005 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.607986 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.592968 21432 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:23:59.609236 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.609467 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.614739 20734 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:59.615370 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:23:59.601037 20877 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
W20260709 12:23:59.617866 20849 connection.cc:256] RPC call timeout handler was delayed by -1.4042226510000546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
I20260709 12:23:59.619151 20728 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:59.621918 20730 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:23:59.622551 27072 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:59.624534 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:59.625221 20877 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.625556 20877 raft_consensus.cc:614] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:59.626036 20731 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:59.627153 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:59.627266 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:23:59.628077 27053 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:59.600812 27004 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:59.628741 27053 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.628877 27004 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.631340 20732 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.631548 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1059) took 1110 ms. Trace:
I20260709 12:23:59.632004 20804 rpcz_store.cc:276] 0709 12:23:58.521360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.526612 (+  5252us) service_pool.cc:224] Handling call
0709 12:23:58.527025 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:58.614436 (+ 87411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:58.614484 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:58.615069 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:23:58.837616 (+222547us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:58.837692 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.837795 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.565990 (+728195us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:23:59.631259 (+ 65269us) raft_consensus.cc:1685] finished
0709 12:23:59.631281 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.631485 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12191,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5215,"spinlock_wait_cycles":13983488,"thread_start_us":208868,"threads_started":1,"wal-append.queue_time_us":391830}
I20260709 12:23:59.634459 26876 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:59.635793 20729 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:23:59.636782 26876 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.592155 26831 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 12:23:59.639369 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 145055us
W20260709 12:23:59.644070 20849 connection.cc:256] RPC call timeout handler was delayed by -1.190022651000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 12:23:59.644454 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1621226510001179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260709 12:23:59.644794 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1549226510001063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
I20260709 12:23:59.642452 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:23:59.645186 20849 connection.cc:256] RPC call timeout handler was delayed by -1.0671864880000612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
I20260709 12:23:59.646487 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.647840 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.650727 20884 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:23:59.651428 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.650822 20726 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:59.655323 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:59.657428 20735 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:59.660007 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:59.661083 20728 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
W20260709 12:23:59.663050 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64078.6us
I20260709 12:23:59.663785 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.666306 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.667855 20726 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:23:59.650638 26881 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.668774 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:23:59.669296 26881 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.661913 27062 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:59.670925 27062 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:23:59.671384 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:59.672246 20883 raft_consensus.cc:2368] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:23:59.671825 27062 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:23:59.672549 27073 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:23:59.673018 27062 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.673115 27076 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:59.673666 27076 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:23:59.674067 27076 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:59.674540 27020 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:59.631089 27053 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.678802 20883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46401ecb9b443118647ee361081a01e"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:23:59.679551 20883 raft_consensus.cc:606] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:23:59.679953 20883 raft_consensus.cc:614] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:59.684158 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:59.696451 27038 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:59.697120 27038 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.697496 27038 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.701242 26912 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.703567 27029 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:23:59.703943 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.704370 27029 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:59.704948 27070 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:59.705379 26791 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.705948 26791 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:59.706961 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.706849 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:59.707554 27015 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.708076 27015 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.709215 27064 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:59.709692 27064 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.709810 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.710122 27064 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:59.712080 26790 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.712340 26647 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:23:59.712888 26576 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:59.712657 26790 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.713019 26647 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.716784 27015 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.725878 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.727747 20732 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:59.684737 26817 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.755721 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.685377 27072 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:59.766985 20882 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:59.772269 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.707983 26881 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:23:59.768699 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83117.6us
W20260709 12:23:59.778524 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 112425us
I20260709 12:23:59.773283 21427 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:23:59.778985 20727 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:23:59.782723 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.785300 20879 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:59.779026 21432 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:59.787395 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.746570 27004 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.789494 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:59.792891 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67113.1us
I20260709 12:23:59.744680 26790 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.789494 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:59.771101 27015 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.767635 27024 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:59.793924 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:59.802433 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.795154 20883 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:23:59.780550 26647 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.809661 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:23:59.810913 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149653us
I20260709 12:23:59.813661 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.812947 20875 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:23:59.747551 26876 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:23:59.818470 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:59.816560 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.796810 27014 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:59.807763 27020 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:23:59.832742 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52826.8us
I20260709 12:23:59.831343 26886 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:59.834197 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.835588 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:59.839855 27020 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:59.841643 27020 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:59.841737 26831 raft_consensus.cc:2764] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:23:59.839968 26647 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.816586 26912 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.839669 26790 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:59.843544 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.843675 26790 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.844185 26790 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:59.845500 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.846495 27054 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:59.809443 26934 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:23:59.847258 27054 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:59.847293 26791 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.833140 27014 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:23:59.848327 27014 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.819994 27024 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:59.809268 27015 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:23:59.840507 26886 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.851397 26886 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.852717 27015 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:23:59.852227 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.853170 26790 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.855201 26934 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:59.857614 26790 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.850183 27024 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.863705 27054 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:23:59.864956 26934 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:59.865319 26934 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:23:59.866333 26934 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.866636 27072 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.846994 27076 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.868547 20878 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:59.868705 26934 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:59.877885 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:59.878203 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:23:59.879014 20883 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:23:59.872884 27046 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:59.876637 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60185.7us
I20260709 12:23:59.872716 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.881237 27044 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:23:59.881913 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85419us
I20260709 12:23:59.882318 26886 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:23:59.882660 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:23:59.884061 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.884420 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.887914 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.897385 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.883127 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.901480 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:23:59.897264 27015 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:59.898033 27070 raft_consensus.cc:2764] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:23:59.898221 21431 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:23:59.891846 26943 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:23:59.884152 27048 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:59.898633 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.870393 27076 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:23:59.910420 20731 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:59.900717 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:59.925246 21435 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.911870 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:59.912103 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.926421 21426 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.926520 21434 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:59.913904 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.938100 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.938349 21426 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:59.939318 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:23:59.942291 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:23:59.945904 21435 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:23:59.948719 21429 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:23:59.948653 27046 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.942101 27012 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:59.911688 26912 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:59.942324 27044 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:59.952524 27044 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.952869 27044 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.956799 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:23:59.957311 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58750.7us
I20260709 12:23:59.958575 20883 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:23:59.918807 26790 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.924127 27048 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:23:59.932410 27005 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:59.972827 20728 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:23:59.885929 27014 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:23:59.962217 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60810.8us
I20260709 12:23:59.968039 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:23:59.942615 27034 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:59.947937 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:23:59.971890 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:59.995301 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:23:59.995904 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:23:59.973335 20728 pending_rounds.cc:85] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 10
I20260709 12:23:59.996184 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:23:59.996627 20728 pending_rounds.cc:107] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:23:59.997008 20728 raft_consensus.cc:2889] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:59.961372 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:23:59.956001 26576 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:00.001067 27034 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:00.001684 26886 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.002115 27034 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.003331 27076 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:00.003674 27014 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.003873 27076 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:23:59.997759 21429 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:23:59.984249 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:00.006222 20804 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:24:00.006040 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58242.4us
I20260709 12:24:00.008214 20802 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:23:59.972883 27048 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:59.999644 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:00.013741 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:00.017319 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:00.016106 20806 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:00.022471 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.024345 20810 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:24:00.024901 20810 pending_rounds.cc:85] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 13
I20260709 12:24:00.023665 20883 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.025254 20810 pending_rounds.cc:107] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:24:00.027967 20810 raft_consensus.cc:2889] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:00.036580 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:00.004884 27076 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.015838 26886 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:00.044687 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.010982 27070 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:00.042685 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.041576 20728 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:00.046522 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:00.027489 26511 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:00.016204 26817 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:00.021396 26912 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:00.036247 26790 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:00.069278 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:23:59.980505 27038 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.039031 27047 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:00.029767 27029 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:00.078045 27029 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.078747 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:00.076077 26809 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:24:00.080121 26809 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:00.080349 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:24:00.080607 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:00.080850 26807 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:00.081226 26807 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:00.081329 27029 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.058377 27070 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.082394 27070 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:00.042817 27020 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:00.084566 27094 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:24:00.086020 27020 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.038626 27064 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.087265 27059 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:00.087900 27059 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:00.088325 27064 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.063336 27046 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:00.091502 27046 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:00.092696 27046 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.059702 26886 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.096258 26886 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.096982 27038 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.100503 20699 leader_election.cc:400] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:00.100976 20699 leader_election.cc:403] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:00.086881 27044 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.061863 27014 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.099282 27047 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:00.109899 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:00.116986 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.120508 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.129622 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:00.133358 20883 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:00.045141 27011 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 12:24:00.087101 26817 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:00.144713 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:00.115324 27086 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:00.112892 26790 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:00.155824 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 114026us
I20260709 12:24:00.111446 26791 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:00.111464 27014 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:00.158051 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:00.159078 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134509us
I20260709 12:24:00.118505 27038 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:00.192484 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62996.2us
I20260709 12:24:00.193866 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:00.221534 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:00.158771 26912 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:00.222489 21429 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:00.138106 27075 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:00.160189 27014 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.230309 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:00.185062 26647 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:00.195168 26511 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:00.089006 27059 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.238029 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.238909 20804 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:00.241123 20810 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:00.244460 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:00.244774 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.230710 26933 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:00.195475 26943 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:24:00.228884 26912 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:00.195928 27024 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:24:00.217445 26852 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:24:00.232393 27075 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:00.195134 27038 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.245914 26647 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:00.288945 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93495.1us
I20260709 12:24:00.283056 27048 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.292755 26647 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.256657 27104 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 12:24:00.290894 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50344us
I20260709 12:24:00.256610 26912 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:00.286911 26886 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:00.306149 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:00.289158 26790 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:00.288096 27075 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:00.309774 20804 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
W20260709 12:24:00.311389 27099 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:24:00.313511 27109 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
W20260709 12:24:00.314407 27101 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:24:00.315083 27090 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:24:00.301152 27020 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:00.288393 27012 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 12:24:00.331413 27100 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:24:00.335350 27029 raft_consensus.cc:2764] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:24:00.247985 26817 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:00.373140 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70845.6us
I20260709 12:24:00.309258 26790 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:00.378557 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76651.1us
I20260709 12:24:00.379479 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:00.384027 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:00.389268 26886 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:00.375643 27044 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.255666 27047 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.382310 26790 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.394316 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:00.410235 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:00.397696 27095 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
W20260709 12:24:00.398131 27096 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:24:00.399343 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.400035 27109 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.416663 27038 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.400090 26807 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.399904 26576 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:00.401865 26921 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:00.399417 27098 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.300s	user 0.006s	sys 0.000s
I20260709 12:24:00.403234 27048 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:00.405987 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.407510 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:00.411039 20802 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:24:00.412705 27064 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:24:00.415125 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.415740 27114 raft_consensus.cc:2764] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:24:00.420676 20804 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
W20260709 12:24:00.451771 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72623.9us
I20260709 12:24:00.385875 26511 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:00.378079 27020 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.431654 27046 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.434000 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.454931 20803 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.447639 20810 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:00.449481 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.456045 20877 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:00.452780 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:00.453542 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.457217 20802 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:00.459043 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.458076 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:00.417156 27109 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.473613 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:00.448917 27044 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:24:00.498945 20801 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:00.473580 26934 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.001s
I20260709 12:24:00.466318 27048 raft_consensus.cc:2764] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:24:00.468725 27062 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:00.470183 26807 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:00.507988 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:00.501220 27015 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.000s
I20260709 12:24:00.512526 20803 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:00.516587 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.518299 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:00.518934 20803 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:00.469043 27046 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.523293 27118 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:00.523962 27046 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.517836 27015 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:00.527485 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.527623 20807 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:00.528784 27109 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.530236 27020 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.532414 26943 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.532907 26943 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.535359 26731 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:00.519994 27111 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:00.497444 26912 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.509375 27012 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:24:00.543159 20618 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:00.543532 20802 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:00.511633 27062 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.546370 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:00.549572 21427 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.534984 26943 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.552870 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.553579 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.554931 27120 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.555615 27120 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.558020 27120 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.555070 20809 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:00.558622 26511 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.567461 20802 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:00.575615 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:00.523962 27118 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:00.582144 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.582846 20876 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:00.552879 27116 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:24:00.587612 20880 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:00.553143 27062 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:00.593937 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.581589 27111 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.594672 20880 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:00.622825 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:00.586635 27015 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:00.631017 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.629714 20880 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:00.563090 27070 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.591996 27012 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:00.637362 20878 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:24:00.637320 20880 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:00.625730 27113 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:00.636054 26852 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:00.648178 20880 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.594115 27014 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.650843 20880 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:00.656759 27014 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.658115 26790 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.662528 27075 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.663548 20880 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:00.665674 20878 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:00.664579 27075 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.668504 27111 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.650590 26852 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.674010 26852 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:00.651268 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:00.678056 21428 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.681408 20810 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:00.630187 26943 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:24:00.635962 27015 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.650797 27113 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.685343 27113 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.651269 27118 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.685482 27015 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.686651 26912 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.649283 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:00.689488 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:00.692054 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:00.694597 21428 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:24:00.693751 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53298.3us
I20260709 12:24:00.698222 20806 raft_consensus.cc:2410] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 12:24:00.692423 27122 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.074s	user 0.002s	sys 0.006s
I20260709 12:24:00.691179 26943 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.707849 26790 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.708151 21427 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:24:00.709414 20805 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:24:00.709813 20805 pending_rounds.cc:85] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 9
I20260709 12:24:00.699234 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:00.710541 26943 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.711009 26943 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.711175 26889 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:00.711974 26889 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:00.693500 26912 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.712899 26889 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.713608 26912 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.716301 26912 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.718173 26886 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:00.722507 27029 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:24:00.725044 27107 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 12:24:00.726444 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:00.710296 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.728469 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:00.728641 21434 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:00.710029 20805 pending_rounds.cc:107] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:00.731652 20805 raft_consensus.cc:2889] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:00.733856 20876 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:00.728689 20804 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:00.750083 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:00.751536 20802 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:00.773772 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.786567 20731 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:00.788846 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:00.794513 20802 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:24:00.795535 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.794480 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:00.795718 20727 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:00.794416 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:00.802484 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:00.804845 20730 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:00.805576 20734 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:00.808346 20808 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:00.808872 20802 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:24:00.809695 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:00.810736 20802 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:00.812532 20807 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:00.782636 27012 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:00.791741 26817 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.826788 20808 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:00.868994 20808 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:00.871440 20808 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:00.874712 27105 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:00.875300 27105 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:00.876360 27105 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:00.878618 27127 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:24:00.884382 20808 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:00.812871 27121 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:24:00.877763 27105 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:00.903055 20808 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:24:00.905112 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 107807us
I20260709 12:24:00.825232 27076 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:24:00.922492 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:00.889323 27047 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:00.906353 27111 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.923985 27047 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.924279 27111 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.926316 27111 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.927049 27076 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:00.927631 27076 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:00.928762 27076 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.930451 27076 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:00.845455 26817 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.922644 21430 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:00.936729 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.926399 27047 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.945463 21427 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:24:00.947448 26886 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:00.947836 20878 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:00.948004 20733 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:00.950042 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:00.934814 27070 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.951193 21430 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:00.931088 26852 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:00.953866 26852 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.955598 26817 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:00.957975 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.934652 27062 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:00.963835 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.965230 27062 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:00.958674 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:00.967404 20809 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:00.971743 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:00.972765 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58531.3us
I20260709 12:24:00.972980 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:00.973757 20732 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:00.973869 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:00.977989 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.978559 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.987118 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.987219 20879 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:00.990516 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:00.994854 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:00.994262 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:00.999209 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.993932 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:00.998708 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.010212 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:00.988302 26817 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.990968 27103 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.013409 20727 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:01.031102 20808 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:00.987880 27047 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:24:00.992071 27070 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:24:00.987478 26886 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 12:24:01.048367 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:01.020998 26817 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.035043 27054 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:01.067623 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.069692 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:01.059302 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.060617 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:01.064433 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.065887 26889 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:01.072386 20700 leader_election.cc:400] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:01.072525 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.073921 20700 leader_election.cc:403] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:01.073872 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.074455 20729 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:24:01.077407 20879 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:01.088205 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:01.088341 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.089943 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:01.092394 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.093303 20730 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:24:01.091959 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:01.091123 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:01.097481 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:01.100299 20732 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:01.112532 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.113677 21434 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:01.113525 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:01.114569 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:01.115391 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.116284 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:01.072626 27013 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.076036 26886 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.119704 21428 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:24:01.072731 27015 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:01.118947 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:01.073813 26889 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:01.122493 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.123103 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.124987 21428 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:01.088160 27111 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:01.128338 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:01.129738 20729 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:01.117983 26647 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.121907 27053 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.120775 27077 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:01.086539 27054 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.122167 26852 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:01.123057 27109 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:01.120972 26943 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.128620 27005 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:01.126488 27013 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.160178 20879 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:24:01.128296 27015 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.144081 27062 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:24:01.166158 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 160718us
W20260709 12:24:01.172073 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53180us
I20260709 12:24:01.172950 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:01.151533 27005 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:01.149134 27077 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:01.136982 26647 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.177680 20879 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:01.128919 26886 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.156337 26852 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:01.170614 27075 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:24:01.183645 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69150.3us
I20260709 12:24:01.182061 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:01.199970 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:01.201011 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:01.170970 27062 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:01.198047 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72857.4us
I20260709 12:24:01.149290 26921 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:24:01.204200 26921 consensus_queue.cc:1243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:24:01.202292 27062 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:01.208578 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 86774.2us
I20260709 12:24:01.194592 27109 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.199349 27075 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.177310 27070 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:24:01.225014 27108 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.225198 27015 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.227239 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:01.227599 27142 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:24:01.227735 27015 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.188635 27121 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:24:01.220986 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:01.241598 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.217413 27014 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.205390 27113 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:24:01.223883 27075 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:01.271121 26852 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.275504 27077 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.276386 27054 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.279619 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:01.282444 20803 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:01.267033 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66141.1us
I20260709 12:24:01.228484 27015 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.283951 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63120.9us
I20260709 12:24:01.284119 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.212553 27160 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:01.266724 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.261982 27108 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:01.269270 27153 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
W20260709 12:24:01.269624 27148 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:01.284950 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:01.305243 27072 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:01.310896 27054 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:01.212697 26881 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 12:24:01.264562 27145 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:24:01.248852 26817 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.221899 27109 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:01.333338 27146 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 12:24:01.342061 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 135772us
W20260709 12:24:01.336136 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 129959us
I20260709 12:24:01.312270 26943 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.372543 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:01.248310 27151 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 12:24:01.379251 27156 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:24:01.380302 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98113.8us
I20260709 12:24:01.367585 27054 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.390741 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:01.392462 20801 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:24:01.335631 27158 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:01.316735 27015 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:01.377018 27144 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
W20260709 12:24:01.392340 27153 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:24:01.367925 27004 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.371665 27070 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:01.367333 27014 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.403258 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118076us
I20260709 12:24:01.419258 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.420047 20805 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:01.389616 27155 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:01.402482 27077 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:24:01.429569 27015 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.431044 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.431885 21434 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
W20260709 12:24:01.432031 27149 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:24:01.434580 27108 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.413882 26943 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:24:01.441414 26943 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:01.441871 26943 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:01.444706 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:01.462771 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:01.365124 27072 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.468453 20805 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:01.470373 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.390162 26881 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:24:01.464082 20875 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:01.480002 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:01.483796 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 198915us
I20260709 12:24:01.485140 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:01.486076 21434 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:01.486933 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:01.445886 26647 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.500180 20729 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:24:01.441403 27109 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.473268 26934 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:01.505184 20805 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:01.487511 27076 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:01.443176 27014 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.420981 27004 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.508180 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:01.511121 20875 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:24:01.504678 27180 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:01.502454 27118 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:01.519968 20728 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:01.520248 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.518013 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102001us
I20260709 12:24:01.518965 20735 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:01.518815 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.442751 26943 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.506546 26934 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:01.521919 20729 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:01.523092 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:01.508996 26647 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:01.525394 20805 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:01.508805 27076 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:01.528468 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:01.530869 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:01.531152 20775 leader_election.cc:400] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:24:01.535692 20728 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:01.532202 27175 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.538386 20775 leader_election.cc:403] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:01.541898 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:01.543083 21434 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:01.546339 26647 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:01.550189 27076 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.551265 27161 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:01.551435 27076 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:01.551780 27161 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.552178 27161 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:01.517752 27112 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:24:01.531944 26934 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:01.560151 27004 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.561542 20805 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:01.562443 27075 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.507612 27176 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:24:01.566888 20879 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:24:01.567238 27011 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:01.568105 27012 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:01.570164 27161 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.570629 20803 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:24:01.576275 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45352.4us
I20260709 12:24:01.524302 27118 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.579644 26943 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:01.538319 27064 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 12:24:01.581526 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 208868us
I20260709 12:24:01.608747 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:01.584424 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:01.590510 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59690.6us
I20260709 12:24:01.610577 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:01.585875 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:01.598246 26934 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.594992 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:01.613214 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:01.613456 20878 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:01.590207 27054 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.532112 26817 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:24:01.611457 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:01.584753 20883 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:01.614491 20875 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:01.614261 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:01.599443 27004 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:01.593665 27011 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:01.622387 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:01.595676 27076 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.629120 27121 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:01.629531 26511 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:01.632001 20735 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:24:01.644786 20735 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 12:24:01.627836 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.600759 27012 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:01.638079 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:01.628839 27072 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.604524 27109 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.654508 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:01.656219 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:01.648099 20729 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:01.659091 27014 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.659641 26647 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:24:01.659994 27111 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:01.660403 27179 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:01.660681 27111 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.623703 27113 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:24:01.663872 27108 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:01.651825 20735 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:01.664022 27171 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:01.664361 27170 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:01.620465 27181 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:01.663327 27038 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:01.650961 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.661118 26647 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:01.537997 27175 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.650892 27180 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:24:01.707988 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51872.6us
I20260709 12:24:01.663990 27183 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.710757 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.661140 27156 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:01.623852 26881 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:24:01.663080 27111 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.688570 27181 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.623533 27185 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 12:24:01.688090 27014 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.676625 27077 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:01.717900 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:01.689699 26817 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:01.702507 27108 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:01.748698 27164 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:24:01.752213 27075 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.753078 26934 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.676174 27109 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.757860 27014 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.760291 27014 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.710916 27170 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.760955 27077 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.715782 27188 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:24:01.763367 27188 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:01.763738 27188 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:01.764597 27188 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.718521 27183 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.767712 27108 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.769044 27171 raft_consensus.cc:993] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:24:01.769410 27188 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:01.769721 27171 raft_consensus.cc:1081] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
W20260709 12:24:01.760310 27062 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.001s
I20260709 12:24:01.725230 27012 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.771821 27118 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:01.772138 27012 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:01.772855 27182 consensus_queue.cc:1175] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 20 status { last_received { term: 19 index: 12 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:01.770941 27062 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.774725 27161 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.775439 20730 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:01.718946 27180 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:01.776863 27180 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.778390 20730 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 12:24:01.778826 27005 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.003s
I20260709 12:24:01.711634 27103 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:24:01.779392 27005 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.780126 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.780231 27103 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:01.780539 20735 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:01.780725 27103 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:01.781993 27103 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.783494 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2597) took 1368 ms. Trace:
I20260709 12:24:01.711803 27121 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:24:01.784245 27151 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:01.784662 27151 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:01.785108 20802 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:01.785481 27151 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.788695 27005 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.790503 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2865) took 1010 ms. Trace:
I20260709 12:24:01.710579 27038 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.790680 20878 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:01.780728 20807 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:01.783110 26817 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.790935 20802 rpcz_store.cc:276] 0709 12:24:00.776861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.827817 (+ 50956us) service_pool.cc:224] Handling call
0709 12:24:01.784870 (+957053us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.785932 (+  1062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.787012 (+  1080us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b5400127350. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:01.787163 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2200125312}
I20260709 12:24:01.783674 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.783877 20804 rpcz_store.cc:276] 0709 12:24:00.415473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.458753 (+ 43280us) service_pool.cc:224] Handling call
0709 12:24:01.783309 (+1324556us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5400127354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:01.783417 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595352320}
W20260709 12:24:01.734333 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 113528us
I20260709 12:24:01.795437 20803 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:01.789188 26889 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.666299 27076 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.811969 20809 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:01.724885 27181 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:01.748809 27186 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:24:01.826987 20804 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:01.829344 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.830652 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:01.780787 27062 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.787058 26886 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:01.840301 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.839150 20803 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:24:01.841334 20803 pending_rounds.cc:85] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 10
I20260709 12:24:01.841760 20803 pending_rounds.cc:107] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:24:01.842056 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:01.842206 20803 raft_consensus.cc:2889] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:01.850293 20884 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:01.858296 20804 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:01.824488 26881 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.857247 26886 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.876902 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 156900us
W20260709 12:24:01.878521 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47976.5us
I20260709 12:24:01.823470 27112 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:24:01.878782 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:01.837910 27109 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:01.879930 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:01.878508 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:01.826517 27118 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:24:01.716235 27011 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:01.925372 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45523.8us
I20260709 12:24:01.928875 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:01.862898 27184 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.930068 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:01.876075 26889 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:01.937736 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.863867 27196 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:01.869244 26807 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:01.965858 27121 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:01.973461 27048 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.924185 27173 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:01.982693 27163 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:01.983280 27197 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:01.983655 27159 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:01.983980 27121 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.865358 26886 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:01.924767 27112 consensus_queue.cc:1243] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:01.877807 26881 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:01.964668 27155 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:01.999672 20803 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:24:02.001626 27203 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:02.009115 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.012612 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:01.931723 27185 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:24:01.978255 26807 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:01.975971 27113 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 12:24:02.028119 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.989297 27077 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.006335 27197 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.050849 20730 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:02.054001 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 91938.5us
I20260709 12:24:02.054658 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:01.994248 27171 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 20.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:24:01.995759 27048 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.059844 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:02.031641 26831 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:02.031358 27155 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:02.043893 27203 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:01.981361 27196 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.051887 27197 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.054291 27064 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:02.059458 27170 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.145218 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:02.149282 27205 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:24:02.183715 27108 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.184319 27108 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.186697 27108 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.157836 27113 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:02.142434 26807 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.186969 27013 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.133504 27175 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:02.153085 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93328.2us
I20260709 12:24:02.160844 26831 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:02.200364 26831 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.204581 27202 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:02.205024 27070 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:02.206137 27202 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:02.207738 27164 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:24:02.208113 27215 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:24:02.208252 27164 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:02.154062 27173 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:02.144400 27076 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.143533 27038 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.148138 27171 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.166121 20803 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:02.168002 27121 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:02.169775 27131 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.377s	user 0.005s	sys 0.000s
I20260709 12:24:02.147710 27183 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:02.153891 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.214058 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.188472 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:02.215781 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:02.229242 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.234823 21434 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:24:02.190419 27113 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:02.236387 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48057.9us
I20260709 12:24:02.240597 27113 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:02.242040 26886 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.243932 26881 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.244555 26881 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:02.214787 27202 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:02.191788 27210 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.245048 26881 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.245572 27210 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.248427 27210 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:02.249961 27202 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:02.251400 27202 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.253103 27202 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.256953 27183 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:02.257460 27183 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.257788 27183 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:02.260385 27203 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.264143 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.265677 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.271718 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.213828 27173 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:02.266919 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.283926 20733 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:02.289208 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:02.292158 27206 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:24:02.292769 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:02.286238 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.293738 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:02.294682 20807 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.210829 27064 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:02.307585 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.283167 27105 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:02.308648 27105 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:02.309396 27105 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.209609 27155 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.311736 27214 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:02.312276 27214 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.314613 27214 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:02.305749 26881 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.233385 27171 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.322491 27171 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:02.209594 27164 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.324026 27164 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.337157 27064 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.338724 27196 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.340389 27048 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:02.343590 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109245us
I20260709 12:24:02.336423 27173 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.345012 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1095) took 1651 ms. Trace:
I20260709 12:24:02.348937 20880 rpcz_store.cc:276] 0709 12:24:00.693204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.693353 (+   149us) service_pool.cc:224] Handling call
0709 12:24:00.694309 (+   956us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:00.697954 (+  3645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.697985 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:00.709934 (+ 11949us) log.cc:844] Serialized 52 byte log entry
0709 12:24:02.060859 (+1350925us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b18004c04e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:24:02.220938 (+160079us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:02.221002 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.221107 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.343278 (+122171us) raft_consensus.cc:1685] finished
0709 12:24:02.343300 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.344942 (+  1642us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8958,"spinlock_wait_cycles":48334080,"thread_start_us":159685,"threads_started":1,"wal-append.queue_time_us":1630920}
I20260709 12:24:02.347872 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.351799 20732 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
W20260709 12:24:02.359455 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.358803 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:02.375293 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:02.350659 27181 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.005s
I20260709 12:24:02.380499 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.376556 27173 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:02.382231 27196 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:02.393275 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.388892 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.376065 27211 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:02.400995 27212 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.401580 27212 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.401914 27197 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.381657 27181 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.409348 27181 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:02.412060 27216 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:24:02.413969 27113 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:02.419725 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.420730 20806 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:24:02.418514 27183 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.424072 20727 raft_consensus.cc:2368] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19.
I20260709 12:24:02.426433 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.393409 27048 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.436262 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.429155 27197 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.438874 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:02.440382 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.441630 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.442288 21434 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:02.443105 20733 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:02.445221 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.445636 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.446245 21434 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
W20260709 12:24:02.448652 27217 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:24:02.434821 27048 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.452502 27048 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:02.454351 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:02.457988 27188 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.003s
W20260709 12:24:02.460644 27174 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.081s	user 0.002s	sys 0.005s
I20260709 12:24:02.468987 27113 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.470319 27212 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.470846 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:02.471801 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:02.472043 27203 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:02.472565 27203 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.477062 20807 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:02.479305 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.480331 21427 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:02.472990 27203 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:02.484476 20727 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:02.472594 27108 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:02.487680 27108 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.488090 27108 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.487911 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:02.489809 27038 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:02.490265 27038 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.490685 27038 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:02.496132 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.497041 27103 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.499607 20807 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [14.9-16.10]
I20260709 12:24:02.504691 20807 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 12:24:02.507752 27182 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:02.511286 27182 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:02.513840 27209 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.204s	user 0.001s	sys 0.003s
I20260709 12:24:02.518157 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.519865 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1087) took 1238 ms. Trace:
I20260709 12:24:02.520253 20801 rpcz_store.cc:276] 0709 12:24:01.281098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.283866 (+  2768us) service_pool.cc:224] Handling call
0709 12:24:02.519710 (+1235844us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54003cad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:02.519809 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247280512}
I20260709 12:24:02.533030 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:02.541680 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 663190us
I20260709 12:24:02.542557 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:02.543588 27109 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:02.544271 27109 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:02.549759 27038 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.554018 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:02.554781 27158 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.122s	user 0.000s	sys 0.005s
W20260709 12:24:02.555150 27158 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.000s	sys 0.005s
I20260709 12:24:02.555414 27158 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:02.556203 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.556290 27158 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.557660 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:02.558285 21427 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:02.553335 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:02.578171 20880 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:24:02.514660 27182 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.585045 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.585815 20878 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:02.587481 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.526818 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:02.588152 20875 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:02.589315 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.590283 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.502889 27105 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.588021 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.593466 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.593695 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.588024 21434 pending_rounds.cc:85] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 9
I20260709 12:24:02.596609 21434 pending_rounds.cc:107] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:24:02.596694 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.596931 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:02.592025 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.597591 20879 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:02.596953 21434 raft_consensus.cc:2889] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:02.597864 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.590704 20881 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:24:02.600262 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.602933 21432 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:02.591778 27210 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:02.604569 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.605026 27038 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.605304 20878 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.605536 27201 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:02.605551 27181 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:02.606541 27181 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.564345 26912 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:02.584615 27013 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:02.564567 27164 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:02.623416 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.618881 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:02.610548 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1103) took 1008 ms. Trace:
I20260709 12:24:02.604628 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.610992 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.607519 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.628134 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.591540 27169 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:02.625454 27229 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.070s	user 0.000s	sys 0.002s
I20260709 12:24:02.650566 20876 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:02.662946 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:02.593384 26886 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.650696 20884 rpcz_store.cc:276] 0709 12:24:01.602450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.602938 (+   488us) service_pool.cc:224] Handling call
0709 12:24:02.610226 (+1007288us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540022f6d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:02.610433 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553257856}
I20260709 12:24:02.671340 20882 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:24:02.657617 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56433.8us
I20260709 12:24:02.651583 20879 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:02.647081 20727 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:02.664893 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:02.653270 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 126553us
I20260709 12:24:02.658366 21435 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:02.607105 27038 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:02.674832 27038 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.653420 21429 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:02.652165 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.653079 21426 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:02.598026 27155 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:02.677927 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.678815 27155 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:02.679006 27038 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.656085 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:02.679584 20727 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:02.605139 27210 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:02.699427 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.699983 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:02.701586 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.701974 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.702455 20883 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:02.657289 27164 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.706848 27164 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.618463 27198 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.090s
I20260709 12:24:02.623947 26647 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.681514 27230 raft_consensus.cc:2764] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:24:02.618155 26921 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 12:24:02.708518 26647 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.598217 27197 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:02.710495 27197 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.711689 27024 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:02.712299 27024 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.610136 27203 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.714218 26647 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.674125 27169 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.659693 27013 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.716399 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1100) took 1124 ms. Trace:
I20260709 12:24:02.716820 20805 rpcz_store.cc:276] 0709 12:24:01.591606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.591793 (+   187us) service_pool.cc:224] Handling call
0709 12:24:01.606857 (+ 15064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.607353 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.607395 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.608033 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.471720 (+863687us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b18003fdd80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:24:02.555468 (+ 83748us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:02.555531 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.555633 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.624790 (+ 69157us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:02.716185 (+ 91395us) raft_consensus.cc:1685] finished
0709 12:24:02.716207 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.716338 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3271,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8440,"spinlock_wait_cycles":79260288,"thread_start_us":83521,"threads_started":1,"wal-append.queue_time_us":925035}
I20260709 12:24:02.718859 20734 raft_consensus.cc:2368] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20.
I20260709 12:24:02.719276 27210 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.657362 26912 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.700058 27109 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:02.655066 27180 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:02.722026 27109 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.722481 27109 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.699359 26511 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.723466 26511 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.695425 27156 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:02.724412 27156 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:02.696050 26886 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.725286 27156 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.652968 27185 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:02.726545 27185 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.728690 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2894) took 1109 ms. Trace:
I20260709 12:24:02.729118 20735 rpcz_store.cc:276] 0709 12:24:01.619305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.702968 (+ 83663us) service_pool.cc:224] Handling call
0709 12:24:01.703356 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.780980 (+ 77624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.728374 (+947394us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b54001d6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:02.728625 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177062144}
I20260709 12:24:01.516645 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:02.587131 27112 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:24:02.740998 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:02.676311 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:02.749298 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93305.8us
I20260709 12:24:02.749503 20878 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
W20260709 12:24:02.750161 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73957.7us
I20260709 12:24:02.773048 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:02.762411 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.742147 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:02.764488 20729 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 12:24:02.776444 27163 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:02.776998 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.778074 27163 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:02.778317 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:02.770105 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.780010 27163 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:02.779181 20729 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:24:02.782272 27163 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:02.780717 27170 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:02.780108 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.784843 26790 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:02.785251 27170 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:02.785634 26790 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.786290 27170 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.787668 27170 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:02.784075 27075 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.788375 27075 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.788798 26790 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.790722 27075 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.799579 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2613) took 1141 ms. Trace:
W20260709 12:24:02.799463 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57375.8us
I20260709 12:24:02.810132 20849 leader_election.cc:400] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19.
I20260709 12:24:02.809980 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.810681 20849 leader_election.cc:403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:02.811156 20881 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:02.820967 20879 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:02.822512 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.824946 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.826087 20883 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:24:02.826673 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:02.728770 27139 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.840581 20877 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:02.842536 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:02.844041 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.844746 20801 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:24:02.847693 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.848183 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.849002 20881 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 21
W20260709 12:24:02.849481 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.765157 27038 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:02.747490 27029 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.865826 20801 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:02.837352 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.801313 20809 rpcz_store.cc:276] 0709 12:24:01.667420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.826328 (+158908us) service_pool.cc:224] Handling call
0709 12:24:01.857511 (+ 31183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.857972 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.858014 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.869509 (+ 11495us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.032767 (+163258us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:02.053073 (+ 20306us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.799200 (+746127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.799284 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.799336 (+    52us) raft_consensus.cc:1685] finished
0709 12:24:02.799351 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.799489 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":499273,"prepare.queue_time_us":20334,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":9063,"raft.queue_time_us":555521,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":16319,"thread_start_us":409264,"threads_started":2,"wal-append.queue_time_us":291595}
I20260709 12:24:02.837716 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:02.814215 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.869645 20878 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:02.870149 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:02.768675 27053 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:02.870997 20726 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:02.867468 20805 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:02.877036 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.606294 27201 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.883898 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:02.885069 20803 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:02.871924 20809 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.874037 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.836879 27202 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19.
I20260709 12:24:02.894690 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.886500 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.860489 27197 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.884667 20805 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:02.829640 27020 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:24:02.889971 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:02.892848 20849 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:24:02.893190 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.859395 27212 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:02.740614 27155 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.918298 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:02.919834 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.920624 20878 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:02.857290 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:02.898826 20883 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:02.792634 27180 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:24:02.859124 27159 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:02.907651 20884 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:02.908577 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.909991 20849 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:24:02.910874 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.824560 27113 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:02.914783 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.929162 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:02.885756 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.929893 20877 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:02.884512 27114 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:24:02.933099 27182 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:24:02.933566 27169 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.933807 26817 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.912539 27203 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.903396 26790 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:02.934962 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2873) took 1323 ms. Trace:
I20260709 12:24:02.916306 27220 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:24:02.935422 20806 rpcz_store.cc:276] 0709 12:24:01.611451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.611957 (+   506us) service_pool.cc:224] Handling call
0709 12:24:02.934766 (+1322809us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b540027c254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:02.934901 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778719488}
I20260709 12:24:02.936237 27201 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.936807 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.937619 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:24:02.938725 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.912909 27212 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:02.934679 26790 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.939332 26511 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:02.939848 27212 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:02.940994 27200 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.060s	user 0.003s	sys 0.000s
I20260709 12:24:02.941057 26790 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.941325 26886 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.894790 27053 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.941321 27212 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.944499 27053 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:02.944725 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.944993 20728 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:24:02.923098 27181 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.922709 27157 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.927469 27185 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:02.896615 27103 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:24:02.902343 27054 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:02.948588 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.984560 20880 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:24:02.878335 27029 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.855063 27139 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:02.949735 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.993638 20878 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:02.951644 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.968338 27121 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.953896 20805 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:02.954308 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:02.922369 27233 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:02.957545 20849 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:24:02.968006 27181 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.949399 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2891) took 1332 ms. Trace:
I20260709 12:24:02.881328 27108 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.951836 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1566) took 1999 ms. Trace:
I20260709 12:24:03.001353 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:02.947180 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.957813 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:02.968163 20881 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:03.000624 20877 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:03.002594 20849 leader_election.cc:403] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:02.968359 27072 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:02.956802 27113 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.968672 27201 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:03.022089 27201 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:02.963847 27157 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.023929 27188 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.024531 27188 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.024984 27201 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.025336 27157 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.928125 27202 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:24:03.026602 27072 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:03.026249 27113 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:02.986869 26790 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.028359 26790 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:03.028781 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71438.9us
I20260709 12:24:03.029037 27072 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.030068 27072 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:03.030387 26790 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:02.991703 27185 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:02.982839 26886 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.031021 27188 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.030238 27202 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:24:03.031673 26886 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.033871 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:03.034044 26886 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:02.956045 27109 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.034662 27202 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:03.035076 27202 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.035463 27202 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.037066 27183 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.038726 27105 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.039310 26912 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.040654 27013 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.015692 27121 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.043498 27121 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.045020 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.046325 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.047637 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.047825 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2899) took 1405 ms. Trace:
I20260709 12:24:03.048174 20730 rpcz_store.cc:276] 0709 12:24:01.642827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.784056 (+141229us) service_pool.cc:224] Handling call
0709 12:24:01.786835 (+  2779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.053154 (+266319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.047591 (+994437us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54000e6fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:03.047763 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602873088}
I20260709 12:24:03.049324 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.025974 27181 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.049949 20730 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:03.014842 20731 rpcz_store.cc:276] 0709 12:24:01.617094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.617241 (+   147us) service_pool.cc:224] Handling call
0709 12:24:01.617935 (+   694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.642644 (+ 24709us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.642686 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.643164 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.713247 (+ 70083us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:01.713288 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.713365 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.940357 (+1226992us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:02.949078 (+  8721us) raft_consensus.cc:1685] finished
0709 12:24:02.949109 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.949335 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14333,"raft.queue_time_us":25006,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":34137,"spinlock_wait_cycles":53917440,"thread_start_us":50169,"threads_started":1,"wal-append.queue_time_us":163674}
I20260709 12:24:03.050518 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.050897 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.051214 20883 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:03.052037 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.022367 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.059429 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.056107 20810 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:03.020058 20728 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:03.071522 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.075398 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.075268 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.075994 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.079833 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:24:03.083174 21432 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:03.086163 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:03.086936 21429 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:03.018384 21430 rpcz_store.cc:276] 0709 12:24:00.952483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.987843 (+ 35360us) service_pool.cc:224] Handling call
0709 12:24:01.003310 (+ 15467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.019111 (+ 15801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.019160 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.019891 (+   731us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.547003 (+1527112us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18003f1a20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:24:02.853356 (+306353us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:02.853402 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.853474 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.951503 (+ 98029us) raft_consensus.cc:1685] finished
0709 12:24:02.951523 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.951747 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24065,"raft.queue_time_us":21995,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":11591,"spinlock_wait_cycles":2978560,"thread_start_us":306140,"threads_started":1,"wal-append.queue_time_us":1921824}
I20260709 12:24:03.089053 20732 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:02.995915 27236 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.091271 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:03.092417 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.092610 20802 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:03.091290 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:03.094661 21427 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:03.092087 21426 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:24:03.017830 27075 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.098865 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.099124 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.048189 27105 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:03.031162 27185 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:03.104758 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.102166 20730 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:24:03.096565 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:02.992223 27054 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.107008 20728 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:24:03.086490 27188 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:03.106022 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:03.073925 27013 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.083284 27109 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.109803 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.110237 20728 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:03.114600 20881 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:03.103673 27114 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:03.147480 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.153318 21427 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:03.144685 20735 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:03.145795 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.133718 21426 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:03.113724 27233 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:03.169011 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.169710 20810 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:03.180194 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2888) took 1564 ms. Trace:
I20260709 12:24:03.188990 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.101866 27113 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:03.199466 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.184597 20726 rpcz_store.cc:276] 0709 12:24:01.616124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.616263 (+   139us) service_pool.cc:224] Handling call
0709 12:24:01.616551 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.891274 (+1274723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.179870 (+288596us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:03.180113 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880098048}
I20260709 12:24:03.102829 27236 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.148480 27234 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.209721 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.115375 27105 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.103842 27075 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.227769 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.171839 27233 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:03.234326 20879 raft_consensus.cc:2368] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20.
I20260709 12:24:03.167284 27188 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:03.149813 27013 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.239094 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2614) took 1258 ms. Trace:
I20260709 12:24:03.243891 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:03.241619 20807 rpcz_store.cc:276] 0709 12:24:01.981047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.505625 (+524578us) service_pool.cc:224] Handling call
0709 12:24:02.505935 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.636500 (+130565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.636553 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.637234 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.883619 (+246385us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:03.237962 (+354343us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.238704 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.238779 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.238824 (+    45us) raft_consensus.cc:1685] finished
0709 12:24:03.238835 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.239033 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130395,"raft.queue_time_us":133169,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":6502,"thread_start_us":511141,"threads_started":2,"wal-append.queue_time_us":358899}
I20260709 12:24:03.244654 20881 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:03.254029 20808 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:24:03.256682 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.265262 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.267865 20807 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:03.204710 27202 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.236286 27233 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.238530 26912 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:03.240906 27188 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.275980 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.278450 20807 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:03.280295 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.291419 20881 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
W20260709 12:24:03.295325 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 190642us
I20260709 12:24:03.296887 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.301070 20807 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:03.302409 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.239928 27234 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.191365 27185 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.282539 27188 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:03.305763 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.306322 20807 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:03.310310 20809 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:03.292870 27029 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.317773 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.318678 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.319443 20881 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:03.320892 20881 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:03.277590 27139 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:03.336987 27054 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.317981 20809 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 12:24:03.340328 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.299052 27164 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:03.327461 26647 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.342726 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.279671 26912 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.346431 26912 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.333194 26807 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:03.349208 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:03.350093 21433 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:03.351364 27202 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:03.311643 20803 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:24:03.352319 20803 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:03.352713 20803 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:03.336251 27013 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.303572 27108 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.355435 27185 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:03.356076 27105 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.357125 27113 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.364274 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.364699 27234 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.366029 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:03.314831 27181 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.368901 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:03.369310 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.369822 21428 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:03.342763 27164 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:03.372454 27024 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.372380 27164 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.373435 27164 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:03.378976 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2905) took 1174 ms. Trace:
I20260709 12:24:03.379442 20732 rpcz_store.cc:276] 0709 12:24:02.204108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.472687 (+268579us) service_pool.cc:224] Handling call
0709 12:24:02.473009 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.089518 (+616509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.378721 (+289203us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54000e92d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:03.378904 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416190208}
I20260709 12:24:03.380164 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.381500 27156 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.382267 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.382109 27156 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.383968 20802 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 12:24:03.389317 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.391670 27212 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:03.393540 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.392333 27212 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.394311 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.394560 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2909) took 1079 ms. Trace:
I20260709 12:24:03.396422 27212 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.395035 20729 rpcz_store.cc:276] 0709 12:24:02.315621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.782168 (+466547us) service_pool.cc:224] Handling call
0709 12:24:02.782481 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.782704 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:02.872344 (+ 89640us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.879649 (+  7305us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.394066 (+514417us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18000dd4a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:03.394212 (+   146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.394260 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:03.394323 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.394388 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.394501 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11147,"prepare.queue_time_us":247475,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":528201,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":6886,"spinlock_wait_cycles":27202304,"thread_start_us":76971,"threads_started":1}
I20260709 12:24:03.399364 26647 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:03.400207 27170 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:24:03.400393 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 543172us
I20260709 12:24:03.399999 26647 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.401772 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.403064 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.403265 26647 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.403499 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.403755 20734 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:03.404060 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.404927 20729 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:03.405426 27044 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.406247 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.409611 27156 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.409711 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.409657 27151 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:03.411819 26817 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:03.413342 26817 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:03.413954 26817 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:03.404121 20726 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:03.415222 21429 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:03.402930 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.418416 27024 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:03.419081 27024 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:03.420396 27054 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:03.421533 27188 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.422253 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.406800 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.415000 27075 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.382210 20803 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.422253 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:03.425030 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.425678 20807 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:03.456015 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.458066 20883 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:03.432138 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.434437 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.424187 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.463553 20878 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:03.434762 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:03.436450 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.436807 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.437124 20805 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:03.437124 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.440663 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:03.470489 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49778.5us
I20260709 12:24:03.446890 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:03.471124 21429 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:03.449203 20803 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.472929 21399 leader_election.cc:400] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20.
W20260709 12:24:03.452960 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.474977 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.476692 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:03.479375 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.447386 20806 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:03.437844 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:03.484848 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47105.5us
I20260709 12:24:03.486009 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.431308 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:03.493615 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62432.3us
I20260709 12:24:03.476707 21433 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:03.435168 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.473346 21399 leader_election.cc:403] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:03.500646 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65953.8us
W20260709 12:24:03.501192 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66088.9us
I20260709 12:24:03.506498 20732 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:03.508980 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1092) took 1027 ms. Trace:
I20260709 12:24:03.509645 21433 rpcz_store.cc:276] 0709 12:24:02.467986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.476864 (+  8878us) service_pool.cc:224] Handling call
0709 12:24:03.348979 (+872115us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:24:03.494941 (+145962us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4963584}
I20260709 12:24:03.498162 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:03.508464 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.521852 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:03.426720 27236 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.429330 20628 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 9 to 17, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 17 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:03.535394 27108 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:03.535621 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.536078 27108 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:03.536898 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.537153 27108 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:03.538169 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 115993us
I20260709 12:24:03.539700 26934 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.540477 26934 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:03.540915 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:03.543874 27194 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:03.544449 27194 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:03.545418 27194 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.434974 27169 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:03.434907 27158 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:03.556121 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83371.7us
I20260709 12:24:03.556960 27195 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20.
I20260709 12:24:03.424094 27188 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.560617 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.561308 21428 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 11.
I20260709 12:24:03.562273 26790 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.514257 27234 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:03.563179 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.563822 21433 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:03.563668 27234 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.565163 21426 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:24:03.565793 27234 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.441759 27233 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.567572 20879 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 11
I20260709 12:24:03.567883 20879 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:24:03.568149 20879 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:03.568683 27233 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.570334 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.570936 21426 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:03.572858 20882 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:03.574244 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.520346 27183 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:03.575022 20877 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:03.575141 27183 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:03.432071 26647 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.575848 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.576038 27183 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.576202 26647 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.577139 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.578454 26647 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:03.525883 27203 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:03.579810 27203 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:03.580156 27203 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:03.583699 27165 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:03.427012 27179 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.158s
I20260709 12:24:03.585251 27165 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:03.585564 27179 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:03.585747 27165 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:03.586001 27179 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.586412 27179 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:03.587298 27020 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:03.588068 27159 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:03.588497 27159 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.508375 27011 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.588856 27159 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:03.589907 27183 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.508895 27075 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:03.509027 27156 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:03.591817 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 112929us
I20260709 12:24:03.575857 27182 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.593325 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:03.593686 27182 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.595464 27175 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:03.525557 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 1047135f71834dbab6b869eb3c081b81: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:24:03.596622 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59807.4us
W20260709 12:24:03.601456 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60616.1us
I20260709 12:24:03.602382 21400 leader_election.cc:400] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20.
I20260709 12:24:03.595916 27175 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:03.614864 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:03.604321 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.615626 20726 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:03.607822 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.609967 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.611646 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.616539 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.617218 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:03.611680 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 113603us
I20260709 12:24:03.618605 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.620582 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.579044 26647 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:03.614200 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.507134 26807 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:03.614336 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.603777 26912 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.621842 26886 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:03.562785 26790 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.623571 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.649813 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.624919 27156 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:03.627415 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.635708 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.613198 21429 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
W20260709 12:24:03.639520 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 117751us
I20260709 12:24:03.642267 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.643786 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.675552 21432 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:24:03.671391 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.668478 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:03.672027 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:03.653261 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:03.677503 20806 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:03.655833 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:03.646339 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:03.619946 27075 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.621608 27183 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.666630 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:03.679046 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:03.677629 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.688155 27230 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:03.689018 20878 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:03.681350 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:03.691134 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.640322 26807 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:03.682904 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47294.7us
I20260709 12:24:03.635964 26647 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:03.646279 26886 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.735548 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:03.736382 20808 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.634694 27233 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:24:03.709380 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.709466 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.648193 27197 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:03.654505 27236 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:03.715902 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:03.719035 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63321.6us
W20260709 12:24:03.726723 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.656842 27180 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:03.665809 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.692875 27155 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:03.655071 27024 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:24:03.760672 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:03.658383 27156 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.742600 20807 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:03.676524 27157 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:03.734531 20628 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 8 to 12, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 12 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:03.744562 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:03.745374 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:03.762816 20805 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.684001 27169 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.644841 27158 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:24:03.747836 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.781129 20878 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:03.679600 27014 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.788908 20805 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:03.684073 27164 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.754125 20810 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 12:24:03.718744 26807 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:03.586769 27165 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:03.765344 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.763815 27156 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:03.771505 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:03.653371 26912 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.731043 26647 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:03.764972 27157 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.778167 27253 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.740631 26886 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:03.783856 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:03.779430 20806 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:03.785884 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.742982 27182 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:03.749933 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:03.847467 20884 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:03.745894 27183 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.787840 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:03.754951 27236 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.758620 27180 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.777546 20731 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:03.775563 27197 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:03.832130 20808 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
W20260709 12:24:03.835719 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49938.4us
I20260709 12:24:03.841511 21426 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:03.866683 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:03.877840 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.880342 20803 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:03.831763 27014 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:03.873360 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:03.834199 27157 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:03.790338 27169 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.891482 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.883399 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.790795 27184 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
W20260709 12:24:03.870797 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 6dc725b7c88e47f693e3a16879798246: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:24:03.886730 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.746767 27175 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:03.897692 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:03.848270 27201 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:03.875911 27180 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:03.880811 27048 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:03.873100 27236 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:03.883664 27257 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:03.901087 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:03.796731 27164 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:03.907974 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 199937us
I20260709 12:24:03.872641 27183 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.867834 27253 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.926458 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:03.914062 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 176761us
I20260709 12:24:03.917346 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:03.932083 20878 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:03.906698 27012 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:03.932196 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:03.890767 27182 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:03.921821 27222 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:03.938413 27109 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:03.948853 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 233057us
I20260709 12:24:03.950220 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:03.933123 27048 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:03.930998 27072 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:03.906901 27151 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:24:03.943470 27158 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:03.938068 27012 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:03.944136 27181 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:03.985313 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.947975 27155 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:04.002784 27155 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:04.004559 27155 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.004643 27164 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.011442 27151 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:04.011997 27151 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:04.012928 27151 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.014248 27151 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.015378 27048 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:04.016419 27048 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.017904 27047 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.018443 27047 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:03.944347 26790 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:03.955152 27159 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.019810 27201 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.020356 27201 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.020617 27047 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.021332 27159 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.022442 27201 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.022645 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:04.025100 27276 raft_consensus.cc:2764] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:24:04.027350 27012 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:04.038367 20880 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 10. (term mismatch)
I20260709 12:24:03.998728 26912 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.047758 27164 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.051156 20876 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:24:04.057315 27253 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.059425 27169 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.061511 27183 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.064898 27181 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:04.067619 27259 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.199s	user 0.000s	sys 0.003s
W20260709 12:24:04.069844 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97687us
I20260709 12:24:04.072947 20880 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 9
W20260709 12:24:04.076386 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 331873us
I20260709 12:24:04.087083 20880 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 21.10
I20260709 12:24:04.092025 20880 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:04.087306 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.088097 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:04.095124 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.100065 20880 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
W20260709 12:24:04.102847 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 205220us
I20260709 12:24:04.106051 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:04.114962 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:03.944926 27197 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.081408 27029 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.074528 27158 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:04.153559 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65533.4us
W20260709 12:24:04.115945 27239 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:24:04.086731 27181 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:04.156399 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70617.6us
I20260709 12:24:04.156781 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:04.165545 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.149425 26790 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.160193 27276 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:04.151959 27272 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:04.168133 27272 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.164259 27253 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:04.171783 27253 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:04.172291 27253 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:04.172365 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.173465 20732 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:04.166742 26790 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.157331 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.150575 27169 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:04.159981 27038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.036s	sys 0.016s
W20260709 12:24:04.178818 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 250285us
I20260709 12:24:04.088222 27164 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.084995 27203 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.185865 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:04.174693 27236 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.171087 27272 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.177191 26912 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:04.175453 27182 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.187377 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.197530 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.199237 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:04.184044 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:04.160116 27270 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:04.160092 27029 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.209246 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.164835 26807 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:04.242106 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51149.4us
I20260709 12:24:04.189462 27169 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:04.259603 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:04.189359 27278 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:04.270262 27230 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.001s
I20260709 12:24:04.281134 27230 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:04.183382 27038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.036s	sys 0.016s
I20260709 12:24:04.227031 27272 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:04.250402 26807 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.200186 27072 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:04.251602 27274 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:04.251497 20808 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:04.252364 27266 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
I20260709 12:24:04.166445 27158 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.255688 27194 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:04.257584 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58431.3us
I20260709 12:24:04.342053 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.270681 27234 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:04.272034 26790 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.350960 26790 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:04.351387 26790 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.351763 26790 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:04.274452 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.274994 27164 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.277810 27180 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.279253 27155 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:04.280649 27179 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.694s	user 0.009s	sys 0.004s
I20260709 12:24:04.281512 27157 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.282249 26886 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.283849 27012 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.284860 27159 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:04.285538 27278 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.241551 26912 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:04.243346 27273 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:04.288472 27236 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.187531 27188 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:24:04.245093 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:04.247845 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50392.9us
I20260709 12:24:04.261098 27014 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.349939 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.291134 27201 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.228250 27182 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.264577 26647 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.393195 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:04.394860 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45018.2us
I20260709 12:24:04.395630 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.398294 27210 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:04.398821 27210 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:04.399253 27210 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:04.400009 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:04.400811 26807 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:04.401211 26807 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.401692 26807 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:04.401867 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.404162 27188 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:04.404453 20804 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:04.404609 27188 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:04.404801 26647 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.405423 26647 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:04.405588 27188 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.389045 27278 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.408722 27012 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.408587 27236 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:04.409485 27278 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.411410 27201 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.412554 27014 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.413688 27201 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.406132 20801 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:04.414670 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 169668us
I20260709 12:24:04.415485 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:04.418174 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.414254 27164 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:04.388914 27273 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.429848 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.430925 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.431265 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.431839 20729 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:04.426069 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.432935 20804 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:04.434934 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:04.437608 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:04.427300 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.406895 27188 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.428915 20727 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:04.429061 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.440551 20733 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:04.367235 27179 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.454703 20729 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:04.456503 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.457528 20804 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:04.458956 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.460546 20804 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:04.461730 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.462419 20734 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:04.461906 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:04.421041 27011 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:04.444649 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:04.475499 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.430269 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.389285 27159 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.485810 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:04.428558 27047 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:04.443540 27195 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:04.396723 27194 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 12:24:04.492996 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.494029 20879 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
W20260709 12:24:04.488389 27285 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
I20260709 12:24:04.464190 27185 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:04.442767 27273 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:04.529968 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.463337 27280 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.483366 27164 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:04.537956 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55213.1us
I20260709 12:24:04.489634 27072 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:04.536743 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.537653 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.543062 20806 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
W20260709 12:24:04.543552 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 115053us
I20260709 12:24:04.389406 27169 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.540372 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.497762 27048 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:04.544705 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.528851 27014 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.530678 27185 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.530023 27011 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:04.561157 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.561877 20806 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:04.565485 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.566968 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:24:04.558583 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74437.3us
I20260709 12:24:04.568799 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:04.552464 27156 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.705s	user 0.002s	sys 0.010s
I20260709 12:24:04.569264 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:04.569599 20879 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:04.570293 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:04.541584 27072 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.576161 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.578256 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.552821 27048 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.579427 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.542092 27278 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.531639 27195 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.584592 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.559697 27185 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:04.580955 27072 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:04.593227 27155 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.593595 27300 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.568202 27012 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:04.433176 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:04.594172 27293 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:24:04.579768 27156 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.590514 27048 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.623107 27195 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.626396 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.661896 20883 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:24:04.662119 20804 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:04.662401 20883 pending_rounds.cc:85] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 9
I20260709 12:24:04.662768 20883 pending_rounds.cc:107] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:04.663069 20883 raft_consensus.cc:2889] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:04.541817 27280 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.665048 27280 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.666057 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.628047 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.667438 20803 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:04.671036 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.631649 27182 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.673194 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.631752 26511 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:04.674412 20877 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:04.633105 27179 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:04.675453 27262 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.724s	user 0.011s	sys 0.000s
I20260709 12:24:04.634506 21432 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:04.633751 27300 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.641788 27155 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:04.677515 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1117) took 1720 ms. Trace:
I20260709 12:24:04.677400 27155 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.678757 27300 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.677915 20802 rpcz_store.cc:276] 0709 12:24:02.956928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.384852 (+427924us) service_pool.cc:224] Handling call
0709 12:24:03.385169 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:03.429899 (+ 44730us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:03.430202 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.907326 (+477124us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.914493 (+  7167us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:03.926986 (+ 12493us) log.cc:844] Serialized 26 byte log entry
0709 12:24:03.934578 (+  7592us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:03.934640 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.935030 (+   390us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.674795 (+739765us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:04.677306 (+  2511us) raft_consensus.cc:1685] finished
0709 12:24:04.677328 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.677454 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75336,"prepare.queue_time_us":1778,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":96978,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":5938,"spinlock_wait_cycles":47400448,"thread_start_us":455589,"threads_started":2,"wal-append.queue_time_us":473710}
I20260709 12:24:04.572817 27201 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:04.595006 27301 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:04.680261 27155 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.680341 27301 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:04.680785 27255 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.928s	user 0.010s	sys 0.013s
I20260709 12:24:04.680979 27301 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:04.642290 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:04.682265 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:04.585830 27253 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.683357 27230 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:04.581548 27180 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.686017 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:04.587687 27193 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:04.609319 27260 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.800s	user 0.006s	sys 0.017s
I20260709 12:24:04.612826 27029 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.598673 27278 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:04.614327 27286 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 12:24:04.654605 27258 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.759s	user 0.004s	sys 0.000s
I20260709 12:24:04.614773 27222 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:04.600880 26807 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.615587 27121 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:04.616143 27210 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.617586 27047 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:04.531409 26912 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.630239 27159 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.683146 27038 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:04.603760 27263 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.676s	user 0.001s	sys 0.013s
I20260709 12:24:04.542369 27164 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.656670 20805 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:04.657153 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2683) took 1861 ms. Trace:
I20260709 12:24:04.633325 20801 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:24:04.740456 20876 rpcz_store.cc:276] 0709 12:24:02.795416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.977652 (+182236us) service_pool.cc:224] Handling call
0709 12:24:04.656863 (+1679211us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540026d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:04.657073 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2371661568}
W20260709 12:24:04.660807 20631 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 294) took 23821 ms (client timeout 15000 ms). Trace:
I20260709 12:24:04.674476 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:04.690184 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.708212 27165 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:24:04.691326 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.692844 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.746600 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:04.747380 20729 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:24:04.693320 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:04.695051 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:04.696096 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 102992us
I20260709 12:24:04.748281 20733 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:04.697124 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:04.694489 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:04.698385 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 158126us
I20260709 12:24:04.748848 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:04.749204 20727 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:04.701841 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:04.657925 27156 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.752036 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:04.755167 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60201.9us
I20260709 12:24:04.755967 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.756786 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:04.757746 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:04.758576 21426 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:24:04.598881 27197 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:04.630239 27151 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.000s
W20260709 12:24:04.655664 27266 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:24:04.772603 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:04.701498 27295 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:04.773092 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:04.773452 27295 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:04.774401 27295 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.774559 27048 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.714255 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.714105 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:04.674405 26511 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:04.778934 27278 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.778643 26511 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.779835 26511 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:04.780135 27048 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.780397 27164 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:04.632047 27157 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.783305 27197 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.784278 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ccae903e8484f8d8c3af847e2c4c3d4 with delete_type TABLET_DATA_TOMBSTONED (TS 49323f66286f4798b268f216bb62b3be not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:39562
W20260709 12:24:04.716212 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47123.7us
I20260709 12:24:04.786212 27210 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.713418 27222 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:04.720547 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:04.705029 27201 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:24:04.788183 27201 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:04.788537 27201 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:04.789335 27201 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.790494 27201 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:04.701174 27253 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:04.748134 27290 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 12:24:04.641937 27053 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:24:04.708436 27013 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:04.793164 27013 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.793555 27013 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:04.638556 26790 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.795506 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2757) took 1094 ms. Trace:
I20260709 12:24:04.769760 27281 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:24:04.796996 26790 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.673249 27181 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.768078 27180 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:04.655792 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.744227 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.599732 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.800143 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:04.800800 20879 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:04.706455 27155 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:24:04.649889 20628 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 14 to 19, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 19 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:04.747403 20807 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:04.746696 20732 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:04.753034 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.759842 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:04.762753 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.769193 20883 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
W20260709 12:24:04.820432 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57780.5us
I20260709 12:24:04.820878 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.821141 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.761201 27179 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:04.634756 27261 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.731s	user 0.000s	sys 0.014s
I20260709 12:24:04.776093 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1116) took 1806 ms. Trace:
I20260709 12:24:04.828204 20881 rpcz_store.cc:276] 0709 12:24:02.969650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.332884 (+363234us) service_pool.cc:224] Handling call
0709 12:24:03.333288 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:03.333686 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.333719 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:03.334323 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:24:03.653486 (+319163us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:03.653523 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.653610 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.653850 (+1000240us) spinlock_profiling.cc:243] Waited 66.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:04.775669 (+121819us) raft_consensus.cc:1685] finished
0709 12:24:04.775689 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.776032 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2352,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":153372544,"thread_start_us":318731,"threads_started":1,"wal-append.queue_time_us":467345}
I20260709 12:24:04.777410 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.659247 27203 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.832887 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.745047 27038 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:04.813205 20882 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:04.750573 26817 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:24:04.835107 20628 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:04.795919 20878 rpcz_store.cc:276] 0709 12:24:03.700986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.937252 (+236266us) service_pool.cc:224] Handling call
0709 12:24:03.956426 (+ 19174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:03.959354 (+  2928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.959405 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:03.960077 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 12:24:04.357433 (+397356us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:04.357494 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.357593 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.795274 (+437681us) raft_consensus.cc:1685] finished
0709 12:24:04.795297 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.795444 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2370,"raft.queue_time_us":3444,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9223,"thread_start_us":266587,"threads_started":1,"wal-append.queue_time_us":388387}
I20260709 12:24:04.809172 27029 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:04.821480 20883 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:04.813460 27165 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:04.823078 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:04.823467 27256 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.082s	user 0.010s	sys 0.007s
I20260709 12:24:04.847307 21432 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:04.823145 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.820298 21428 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:04.770242 27121 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:04.810014 20880 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:24:04.858022 20880 pending_rounds.cc:85] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 8
I20260709 12:24:04.858407 20880 pending_rounds.cc:107] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:24:04.858687 20880 raft_consensus.cc:2889] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:04.811175 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.772614 27182 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.736605 27047 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:04.835872 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.814821 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43620.4us
I20260709 12:24:04.871238 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:04.825579 27151 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:04.786269 27197 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:04.778582 27185 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.838287 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.835953 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.854890 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:04.802134 27024 consensus_queue.cc:1175] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 20 status { last_received { term: 19 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:04.855829 20628 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 13 to 19, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 19 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:04.856122 27038 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.857100 20733 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:04.927929 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.857352 20807 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:24:04.794548 27273 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:04.857594 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1108) took 1924 ms. Trace:
I20260709 12:24:04.858922 27165 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:04.931181 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:04.821380 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:04.931363 20809 rpcz_store.cc:276] 0709 12:24:02.933427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.339896 (+406469us) service_pool.cc:224] Handling call
0709 12:24:03.416365 (+ 76469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.416603 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:03.433988 (+ 17385us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.792452 (+358464us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.793416 (+   964us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:03.793492 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.793575 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.675883 (+882308us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:24:04.857409 (+181526us) spinlock_profiling.cc:243] Waited 76.2 ms on lock 0x7b54003d6dd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:04.857531 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33722,"prepare.queue_time_us":17927,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":103377,"raft.run_cpu_time_us":19417,"raft.run_wall_time_us":784513,"spinlock_wait_cycles":194416384,"thread_start_us":351814,"threads_started":1,"wal-append.queue_time_us":522830}
W20260709 12:24:04.932687 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 111369us
I20260709 12:24:04.933426 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.829825 27181 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:04.872963 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:04.879339 20883 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:04.876592 27300 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.849112 27203 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:04.838366 27029 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.903146 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:04.925464 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:04.857811 27121 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.928509 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.927271 20628 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 15 to 18, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 18 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:04.949657 20881 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:04.935694 20807 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:24:04.937767 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64894.5us
I20260709 12:24:04.836593 27188 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:04.938529 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.938498 20727 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:04.925549 27180 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:04.884539 27280 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.848992 27159 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:24:04.836953 27072 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.947877 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.971648 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37c593b286274096a8c431979866be8a"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:04.972267 20878 raft_consensus.cc:606] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:04.870200 27164 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:04.957074 20807 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:04.741895 20631 rpcz_store.cc:269] 0709 12:23:40.838695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.838925 (+   230us) service_pool.cc:224] Handling call
0709 12:24:04.660112 (+23821187us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:57.305995 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:57.404074 (+ 98079us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:57.410641 (+  6567us) write_op.cc:435] Acquired schema lock
0709 12:23:57.411618 (+   977us) tablet.cc:662] Decoding operations
0709 12:23:57.423277 (+ 11659us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:57.423383 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:57.423416 (+    33us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:23:57.425338 (+  1922us) tablet.cc:701] Row locks acquired
0709 12:23:57.425388 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:23:57.425553 (+   165us) write_op.cc:270] Start()
0709 12:23:57.425706 (+   153us) write_op.cc:276] Timestamp: P: 1783599837425488 usec, L: 0
0709 12:23:57.425729 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:23:57.432904 (+  7175us) log.cc:844] Serialized 9081 byte log entry
0709 12:23:57.571635 (+138731us) op_driver.cc:464] REPLICATION: finished
0709 12:23:57.682327 (+110692us) write_op.cc:301] APPLY: starting
0709 12:23:57.694110 (+ 11783us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:57.849356 (+155246us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:57.849610 (+   254us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:58.685223 (+835613us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:58.686220 (+   997us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 12:23:58.686296 (+    76us) write_op.cc:312] APPLY: finished
0709 12:23:58.687063 (+   767us) log.cc:844] Serialized 204 byte log entry
0709 12:23:58.987055 (+299992us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:58.987572 (+   517us) write_op.cc:454] Released schema lock
0709 12:23:58.987655 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:59.073381 (+ 85726us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:59.075269 (+  1888us) write_op.cc:454] Released schema lock
0709 12:23:59.525883 (+450614us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b180030d1e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
Metrics: {"mutex_wait_us":780209,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":1176,"apply.run_cpu_time_us":69079,"apply.run_wall_time_us":1471702,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100267,"num_ops":23,"prepare.queue_time_us":25659,"prepare.run_cpu_time_us":33342,"prepare.run_wall_time_us":207200,"raft.queue_time_us":36033,"raft.run_cpu_time_us":1538,"raft.run_wall_time_us":1538,"replication_time_us":145705,"spinlock_wait_cycles":8154880,"thread_start_us":370869,"threads_started":5,"wal-append.queue_time_us":477335,"wal-append.run_cpu_time_us":17021,"wal-append.run_wall_time_us":2043413}]]}
I20260709 12:24:04.933931 27281 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:04.957957 20806 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:04.962498 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:04.960567 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:04.965785 20875 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:04.937306 27165 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:04.966603 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 256954us
I20260709 12:24:04.884209 27182 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.959400 20628 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 9 to 19. New cstate: current_term: 19 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:04.945356 27197 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:04.944092 27151 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:04.948107 27029 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:04.927874 27156 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:24:04.956981 27121 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:05.005996 20628 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:04.972792 20878 raft_consensus.cc:614] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:04.954807 20808 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:04.957491 27188 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:04.987550 20733 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:04.988723 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:04.990427 21427 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:24:04.999289 20881 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:05.016222 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:04.946100 27300 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.008499 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.008880 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:05.008467 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 255526us
I20260709 12:24:05.003664 27169 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.967960 27180 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:04.977622 27164 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.011833 20628 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 15 to 18, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 18 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:04.982384 26807 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:05.019217 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:05.059881 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:05.060425 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 126648us
I20260709 12:24:05.024212 20700 leader_election.cc:400] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:05.024900 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:05.024721 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:05.062950 20801 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:05.018922 27188 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:04.994601 27299 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:04.993510 27151 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.991106 27197 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.100783 27151 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:05.100893 27197 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:04.972720 27280 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.102895 27180 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.103039 27300 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.104171 27280 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.105808 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.106429 21426 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:05.108088 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.062572 20700 leader_election.cc:403] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:05.064980 21429 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:05.061368 20628 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 12 to 17, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 17 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.066552 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2768) took 1141 ms. Trace:
I20260709 12:24:05.109751 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.110210 20881 rpcz_store.cc:276] 0709 12:24:03.925129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.026505 (+1101376us) service_pool.cc:224] Handling call
0709 12:24:05.065999 (+ 39494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.066175 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.066199 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.066239 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:05.066281 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.066353 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.066485 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.072953 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.074779 27047 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.108752 21431 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:05.086432 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2766) took 1378 ms. Trace:
I20260709 12:24:05.113404 20875 rpcz_store.cc:276] 0709 12:24:03.708485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.256053 (+547568us) service_pool.cc:224] Handling call
0709 12:24:04.256438 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.982278 (+725840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.086010 (+103732us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b5400200b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:05.086340 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630561152}
I20260709 12:24:05.099725 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:05.100095 27304 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:05.108201 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:05.124833 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 100759us
W20260709 12:24:05.111945 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50302us
I20260709 12:24:05.039706 27281 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:05.055027 27164 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:05.132108 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:05.134407 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1129) took 1064 ms. Trace:
I20260709 12:24:05.139330 20875 rpcz_store.cc:276] 0709 12:24:04.070505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.131099 (+1060594us) service_pool.cc:224] Handling call
0709 12:24:05.133908 (+  2809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.134088 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.134112 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.134162 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:05.134199 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.134250 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.134350 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.141322 20875 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:05.142189 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1130) took 1071 ms. Trace:
I20260709 12:24:05.050908 27257 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:05.137183 20628 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 9 to 13, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 13 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:05.142719 20875 rpcz_store.cc:276] 0709 12:24:04.070870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.140655 (+1069785us) service_pool.cc:224] Handling call
0709 12:24:05.141048 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.141841 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.142115 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.151976 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:05.153710 20881 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:05.154366 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1131) took 1083 ms. Trace:
I20260709 12:24:05.157202 21429 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:05.154700 20881 rpcz_store.cc:276] 0709 12:24:04.071165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.153185 (+1082020us) service_pool.cc:224] Handling call
0709 12:24:05.153505 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.154116 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.154307 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.160333 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:05.159632 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.168654 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:05.168862 20628 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:05.169368 20807 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:05.190022 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:05.192068 20801 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:05.138289 27175 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:05.131285 27159 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:05.123543 27280 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:05.047457 27169 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:05.175102 27296 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:24:05.203533 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:05.180619 27284 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:24:05.164381 27257 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:05.198866 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:05.207624 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.215693 20807 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:05.122913 27314 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:24:05.169867 27281 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:05.235843 20628 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:05.215596 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:05.237309 20810 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:05.163313 27194 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:24:05.226702 20801 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:05.200362 27203 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:05.218122 27301 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.000s
I20260709 12:24:05.245198 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 10 to 18. New cstate: current_term: 18 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.233506 21428 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:05.235713 27253 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:05.268091 20628 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 10 to 13, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 13 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.248383 27312 tablet_replica.cc:333] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:24:05.272171 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.239562 27175 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:05.211876 27280 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.280437 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:05.255592 27273 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:05.324422 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:05.278364 27305 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:24:05.254575 27203 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.254786 27301 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.354306 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:05.432591 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88246.9us
I20260709 12:24:05.433138 21430 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:05.422917 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.436811 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.347534 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:05.437386 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.417898 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:05.354589 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.444518 20628 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 11 to 16, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 16 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.450659 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:05.274739 27072 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.274986 27297 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:05.360648 20802 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:05.413720 27273 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:05.338357 27283 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.972s	user 0.000s	sys 0.004s
I20260709 12:24:05.342767 27169 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.486543 20628 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 11 to 15, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 15 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:05.461508 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:05.459777 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 112309us
I20260709 12:24:05.324615 27012 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.330998 27195 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:05.424413 27155 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:05.334270 27109 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:05.430274 27075 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:05.283252 27253 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:05.537839 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1125) took 1110 ms. Trace:
I20260709 12:24:05.500885 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:05.537964 27253 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.538255 20805 rpcz_store.cc:276] 0709 12:24:04.427153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.429507 (+  2354us) service_pool.cc:224] Handling call
0709 12:24:04.442587 (+ 13080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.740024 (+297437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.537583 (+797559us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400127d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:05.537779 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491863296}
I20260709 12:24:05.539887 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:05.540277 20805 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:05.541983 27180 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:05.345705 27280 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.356364 27047 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:05.543452 27180 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:05.413373 27300 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.138s
I20260709 12:24:05.523491 20732 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:05.543643 27047 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.523147 20628 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 2 to 15, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 15 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:05.544703 27300 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:05.545111 27300 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:05.545959 27047 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.545436 20628 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 11 to 15, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 15 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.546033 27300 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.324494 27222 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.547209 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:05.548509 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:05.549032 27222 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:05.549608 27053 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:05.549940 27299 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:05.550820 27299 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.553170 27299 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.553681 27299 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:05.554015 27299 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:05.554368 27299 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:05.489609 27156 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:05.447708 27184 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:24:05.290830 27175 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:05.532930 27317 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
W20260709 12:24:05.534569 27322 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 12:24:05.535007 27292 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.000s	user 0.000s	sys 0.001s
W20260709 12:24:05.535282 27311 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 12:24:05.536114 27319 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:05.448123 27044 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:24:05.447283 27267 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:24:05.542052 20805 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:05.502219 26790 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:05.340858 27182 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.561142 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.562568 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.549942 20628 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 11 to 13, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 13 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:05.669700 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:05.277734 27281 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.560508 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:05.510206 27169 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.453389 27048 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.509989 26912 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:05.574501 27011 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:05.574790 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.576330 26807 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:05.580456 27151 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:05.583940 26511 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.804s	user 0.002s	sys 0.010s
I20260709 12:24:05.615267 27193 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:24:05.585192 27320 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:24:05.585712 27201 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.004s
I20260709 12:24:05.586571 20810 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:05.567229 27075 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:05.590979 27029 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
I20260709 12:24:05.592635 27301 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.779024 27307 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:05.596210 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:05.826936 20881 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:05.500181 27273 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.522853 27109 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:05.515004 27012 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.599133 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2764) took 1896 ms. Trace:
I20260709 12:24:05.835528 20880 rpcz_store.cc:276] 0709 12:24:03.703049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.163717 (+460668us) service_pool.cc:224] Handling call
0709 12:24:04.164031 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.859791 (+695760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.598841 (+739050us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:05.599045 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484521728}
I20260709 12:24:05.627264 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:05.629555 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:05.637581 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:05.647076 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.839272 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 209820us
I20260709 12:24:05.648623 21432 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:05.840883 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:05.648875 20726 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:05.599141 27323 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:24:05.612336 20876 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:24:05.851294 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2771) took 1652 ms. Trace:
I20260709 12:24:05.851747 20876 rpcz_store.cc:276] 0709 12:24:04.199210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.611713 (+1412503us) service_pool.cc:224] Handling call
0709 12:24:05.612076 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:05.850914 (+238838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.851221 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.822431 27108 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:05.494634 27297 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.671739 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.612237 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:05.567459 27195 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:05.628773 27072 consensus_queue.cc:1175] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 14 status { last_received { term: 14 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 14 index: 10 } }
I20260709 12:24:05.869067 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:05.772011 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 211589us
I20260709 12:24:05.815498 27048 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:05.772079 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.675308 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:05.693259 20732 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:05.606181 27280 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.137s
I20260709 12:24:05.784567 26912 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:05.624215 26886 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.786453 20734 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:05.787070 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 114024us
W20260709 12:24:05.618566 27321 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:24:05.602808 27121 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.001s
I20260709 12:24:05.635685 27047 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:05.550042 27053 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:05.783051 27322 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:24:05.617619 27185 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.889775 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.890715 27297 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.891040 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.890911 27053 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.892197 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.893430 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.809784 21426 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:05.894369 27151 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.624235 27203 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.625748 27175 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:05.743180 26790 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:05.792712 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 345545us
I20260709 12:24:05.886039 27038 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.897809 27038 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.898581 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:05.892225 27012 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.901337 27325 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:05.902494 27325 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:05.779981 27011 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.819021 20878 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:05.903419 27011 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:05.794987 27013 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.001s	user 0.015s	sys 0.000s
I20260709 12:24:05.866873 27108 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.905323 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1129) took 1145 ms. Trace:
W20260709 12:24:05.819077 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 279250us
I20260709 12:24:05.905818 20801 rpcz_store.cc:276] 0709 12:24:04.759600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.243252 (+483652us) service_pool.cc:224] Handling call
0709 12:24:05.266567 (+ 23315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:05.266894 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.266932 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:05.267520 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:24:05.635414 (+367894us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:05.644557 (+  9143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.644645 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:05.905125 (+260480us) raft_consensus.cc:1685] finished
0709 12:24:05.905147 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.905270 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":254995,"threads_started":1,"wal-append.queue_time_us":545000}
I20260709 12:24:05.906665 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:05.907881 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:05.908068 20801 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:05.909020 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:05.910048 27159 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:05.910454 27159 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:05.791818 27314 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:05.528012 27155 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.911373 27314 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:05.897073 26790 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.912361 27314 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.913000 27155 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:05.609562 27302 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.205s
I20260709 12:24:05.609091 27112 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.206s
I20260709 12:24:05.914191 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1133) took 1153 ms. Trace:
I20260709 12:24:05.693172 27299 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.914636 20802 rpcz_store.cc:276] 0709 12:24:04.760552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.494267 (+733715us) service_pool.cc:224] Handling call
0709 12:24:05.494598 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.859371 (+364773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.859419 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.859475 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:05.859531 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.913980 (+ 54449us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540006d1d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:05.914026 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.914141 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838265216}
W20260709 12:24:05.915375 27329 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:24:05.915431 27108 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:05.916255 27325 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:05.916574 27299 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.917287 20876 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:05.916976 26912 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.917531 27175 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:05.917951 27175 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:05.918407 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2710) took 1326 ms. Trace:
I20260709 12:24:05.802261 27193 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:05.918725 27175 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.918756 20876 rpcz_store.cc:276] 0709 12:24:04.592283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.852691 (+1260408us) service_pool.cc:224] Handling call
0709 12:24:05.870546 (+ 17855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.917656 (+ 47110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.918184 (+   528us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b54001ffc54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:05.918350 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107063680}
I20260709 12:24:05.920249 27273 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:05.920506 27203 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:05.819800 27324 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:05.920945 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2711) took 1326 ms. Trace:
I20260709 12:24:05.614850 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:05.921941 20804 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:05.922257 27257 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
I20260709 12:24:05.921510 20876 rpcz_store.cc:276] 0709 12:24:04.594779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.920142 (+1325363us) service_pool.cc:224] Handling call
0709 12:24:05.920471 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.920647 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.920673 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.920709 (+    36us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:05.920742 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.920798 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.920892 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.922710 27257 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.923760 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:47514 (request call id 1) took 1370 ms. Trace:
I20260709 12:24:05.923915 20876 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:24:05.924168 20804 rpcz_store.cc:276] 0709 12:24:04.553312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.922696 (+1369384us) service_pool.cc:224] Handling call
0709 12:24:05.923652 (+   956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.862748 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.924877 27257 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:05.863716 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.864539 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.920945 27203 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:05.932037 27203 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.866761 20807 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:05.861948 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:05.870251 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:05.600310 27230 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:05.870651 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:05.942799 20726 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:24:05.855289 27291 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:05.850231 27272 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:05.869951 27188 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.000s
W20260709 12:24:05.939325 27197 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.838s	user 0.012s	sys 0.000s
I20260709 12:24:05.871021 27195 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:05.872126 26852 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:24:05.887006 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:05.965957 20734 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:05.887998 27020 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:05.674937 27300 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.903661 27038 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:05.828127 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2650) took 1157 ms. Trace:
I20260709 12:24:05.833518 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:05.972656 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:05.942210 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.815460 27169 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:05.943091 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72929.3us
I20260709 12:24:05.945056 21431 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:24:05.934162 27075 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:05.946785 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84969.6us
I20260709 12:24:05.947089 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1132) took 1871 ms. Trace:
W20260709 12:24:05.903888 27328 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:24:05.904860 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2769) took 1979 ms. Trace:
I20260709 12:24:05.991705 20878 rpcz_store.cc:276] 0709 12:24:03.925931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.053380 (+1127449us) service_pool.cc:224] Handling call
0709 12:24:05.087763 (+ 34383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.903480 (+815717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.904643 (+  1163us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54000965d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:05.904807 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681026432}
I20260709 12:24:05.843005 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:05.993736 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 150789us
I20260709 12:24:05.994400 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.994730 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:05.922807 20808 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:24:05.937189 27164 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.767s	user 0.006s	sys 0.004s
I20260709 12:24:05.933189 27203 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:05.940809 27165 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.894s	user 0.010s	sys 0.001s
I20260709 12:24:05.829100 27029 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:05.955852 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46898.2us
I20260709 12:24:06.001523 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:05.921406 27324 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.914636 27112 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:24:05.976310 27163 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.022s	sys 0.024s
I20260709 12:24:05.920250 27175 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:05.942538 27230 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:05.886221 27307 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:24:05.971177 20809 rpcz_store.cc:276] 0709 12:24:04.670943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.941076 (+270133us) service_pool.cc:224] Handling call
0709 12:24:04.941405 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.946241 (+  4836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.946291 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:04.946882 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 12:24:05.206605 (+259723us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:05.206662 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.206752 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:05.827830 (+621078us) raft_consensus.cc:1685] finished
0709 12:24:05.827860 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.828046 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1731,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2454,"thread_start_us":229431,"threads_started":1,"wal-append.queue_time_us":362736}
I20260709 12:24:05.924746 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1134) took 1280 ms. Trace:
I20260709 12:24:05.973667 20881 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:06.018464 20876 rpcz_store.cc:276] 0709 12:24:04.644435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.923320 (+1278885us) service_pool.cc:224] Handling call
0709 12:24:05.923601 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.924529 (+   928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.924694 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:06.005841 27163 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.022s	sys 0.024s
I20260709 12:24:05.885952 27121 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.982658 20884 rpcz_store.cc:276] 0709 12:24:04.075398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.274696 (+1199298us) service_pool.cc:224] Handling call
0709 12:24:05.275018 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.878832 (+603814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.878863 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.878907 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:05.878949 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.946846 (+ 67897us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b540022fbd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:05.946895 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.947032 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388142464}
I20260709 12:24:05.959389 27188 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.989676 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.935839 27047 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.003978 27038 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.965623 27273 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:05.959173 26807 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:05.946116 27013 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.002266 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:05.959503 27337 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:05.894376 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:06.021606 27272 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:06.019611 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:05.948552 27210 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:05.959705 27197 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.066558 20804 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:06.067809 20882 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
W20260709 12:24:05.979357 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 327847us
I20260709 12:24:06.035607 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:06.037904 20883 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:06.041173 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:05.911226 27159 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:05.997083 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:06.044922 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:06.069974 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:06.003364 27075 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:06.004276 27324 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:06.012899 27112 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:06.015550 27291 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:06.081413 20803 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:05.970767 27048 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:06.023854 27307 consensus_queue.cc:1243] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:24:06.079437 27272 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:06.092651 27047 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:06.093050 20727 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:06.071521 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:06.071821 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.094401 20878 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:06.094957 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:06.096125 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.071521 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.096966 20734 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:06.097306 20807 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:06.095019 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1135) took 1450 ms. Trace:
I20260709 12:24:06.098295 20878 rpcz_store.cc:276] 0709 12:24:04.644856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.080910 (+1436054us) service_pool.cc:224] Handling call
0709 12:24:06.094069 (+ 13159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.094792 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.094965 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.100314 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:36384 (request call id 7) took 1404 ms. Trace:
I20260709 12:24:06.100595 27011 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.072027 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:06.072690 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:06.105046 20881 raft_consensus.cc:606] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:06.105674 20881 raft_consensus.cc:614] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:06.073045 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:06.065132 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:06.001412 27164 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.005373 27165 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:06.111074 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43584us
I20260709 12:24:06.075408 27273 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:06.088927 27334 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:06.090862 27197 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:06.100644 20878 rpcz_store.cc:276] 0709 12:24:04.695760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.099044 (+1403284us) service_pool.cc:224] Handling call
0709 12:24:06.100217 (+  1173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.072937 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:06.109843 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2717) took 1276 ms. Trace:
I20260709 12:24:06.044241 27163 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:24:06.056151 27038 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.118705 20808 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:06.047430 27156 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:24:06.139957 20730 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.126452 20884 rpcz_store.cc:276] 0709 12:24:04.833135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.108976 (+1275841us) service_pool.cc:224] Handling call
0709 12:24:06.109280 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.109469 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.109505 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.109548 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:06.109586 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.109642 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.109779 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.132516 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2707) took 1576 ms. Trace:
I20260709 12:24:06.133800 20879 raft_consensus.cc:2368] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21.
I20260709 12:24:06.103592 27280 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:06.146931 20802 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:06.147480 20884 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:06.104480 27299 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:06.151749 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2775) took 1024 ms. Trace:
I20260709 12:24:06.152168 20884 rpcz_store.cc:276] 0709 12:24:05.127576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.141407 (+1013831us) service_pool.cc:224] Handling call
0709 12:24:06.141760 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.151464 (+  9704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.151685 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.145784 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:06.118445 20807 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:24:06.154381 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 186623us
I20260709 12:24:06.184214 20876 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
W20260709 12:24:06.148447 27340 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:24:06.131438 27273 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:06.194983 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 192826us
I20260709 12:24:06.196611 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:06.092671 27272 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.188648 20878 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:06.190210 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1133) took 2111 ms. Trace:
I20260709 12:24:06.190479 20876 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:06.198815 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:06.157204 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:06.158020 27257 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:06.159826 27316 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:06.238368 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81289.3us
I20260709 12:24:06.242918 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:06.238636 20878 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:06.246495 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:06.249147 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:06.241328 27291 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:06.240199 27047 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.239645 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:06.253618 27307 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:06.255226 27188 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:06.256647 27201 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:06.254813 27195 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:06.273993 27201 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.274482 27201 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:06.266044 27029 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:06.190133 27112 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.279987 27112 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:06.280656 27230 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.281978 27230 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:06.238106 27335 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:24:06.283298 27307 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.241192 27109 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.284502 27165 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:06.285228 20801 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:06.285535 27013 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:06.285631 27307 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:06.269385 27075 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:06.288504 27273 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.252283 27316 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.290433 27314 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:06.291831 27164 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:06.252763 27048 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:06.262678 27169 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:06.254170 27121 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:06.251166 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:06.204510 27295 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:06.157621 27299 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.266252 20878 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:06.301915 20733 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:24:06.304399 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53293.5us
I20260709 12:24:06.307407 20730 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:06.308091 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:06.312464 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:06.313179 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:06.314548 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:06.316120 21431 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:06.292070 27316 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:06.319216 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:06.321092 27188 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:06.320573 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:06.274492 27302 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:06.316845 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:06.324708 20801 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:06.325801 27338 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:24:06.311398 27169 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:06.316466 27029 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:06.336826 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:06.323135 27164 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:06.335038 27354 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:24:06.343500 27257 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:06.338838 27188 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:06.344116 27182 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:06.379088 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:06.347741 27345 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:24:06.347826 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1264) took 1240 ms. Trace:
W20260709 12:24:06.348647 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 3894130cfc2549b5a8c99d056675ef94: Illegal state: Replica 657bc62d20374e539b23641715f22745 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:06.321770 27210 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.028s
W20260709 12:24:06.349916 27346 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:24:06.311594 27048 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:24:06.350276 27169 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:06.350541 27038 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:06.352044 27193 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.353183 27011 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:06.356741 27299 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:06.358433 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:06.425808 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67452.2us
I20260709 12:24:06.361976 27344 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.202410 20877 rpcz_store.cc:276] 0709 12:24:04.078751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.597156 (+1518405us) service_pool.cc:224] Handling call
0709 12:24:05.597532 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.597772 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:05.870916 (+273144us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.189858 (+318942us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.189915 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:06.189974 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.190038 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.190155 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1236,"prepare.queue_time_us":452186,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":734,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":8453,"thread_start_us":580084,"threads_started":2}
I20260709 12:24:06.372488 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:06.372653 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:06.374430 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:06.375437 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:06.431514 20727 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:06.323967 27302 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:06.378070 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:06.378439 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:06.432646 20729 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:06.432796 20810 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:06.379241 27156 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:06.380134 27312 raft_consensus.cc:2243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:06.363408 27336 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:24:06.441663 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:06.382274 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.444314 20809 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:06.378131 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:06.404265 21426 rpcz_store.cc:276] 0709 12:24:05.107253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.108046 (+   793us) service_pool.cc:224] Handling call
0709 12:24:05.108314 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.894523 (+786209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.347450 (+452927us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:06.347726 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594643712}
I20260709 12:24:06.414397 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.410645 27072 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:06.380270 27354 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:24:06.435079 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1139) took 1635 ms (client timeout 1787 ms). Trace:
I20260709 12:24:06.436540 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.421365 27257 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:06.450778 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.460503 20804 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:06.453835 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:06.380271 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1137) took 1684 ms. Trace:
I20260709 12:24:06.406170 27324 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.458572 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:06.454816 20879 rpcz_store.cc:269] 0709 12:24:04.800009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.810856 (+ 10847us) service_pool.cc:224] Handling call
0709 12:24:06.434795 (+1623939us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400059e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:06.434988 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2930102016}
I20260709 12:24:06.435940 27038 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:06.463886 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:06.468418 20805 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:06.463289 20881 rpcz_store.cc:276] 0709 12:24:04.696066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.106461 (+1410395us) service_pool.cc:224] Handling call
0709 12:24:06.106821 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.322399 (+215578us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:06.322665 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.379005 (+ 56340us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b18002cd280. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a
0709 12:24:06.379226 (+   221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.379286 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:06.379341 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.380022 (+   681us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:06.380067 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.380209 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":106,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3241,"raft.queue_time_us":1712,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":503363712,"thread_start_us":10985,"threads_started":1,"wal-append.queue_time_us":15614}
I20260709 12:24:06.472489 20729 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:06.443174 27299 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.481240 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2664) took 1065 ms. Trace:
I20260709 12:24:06.481626 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:06.444131 26852 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:06.481719 20806 rpcz_store.cc:276] 0709 12:24:05.415498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.838941 (+423443us) service_pool.cc:224] Handling call
0709 12:24:05.844240 (+  5299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:05.844590 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.844617 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:05.845050 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.077469 (+232419us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:06.077523 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.077607 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.481020 (+403413us) raft_consensus.cc:1685] finished
0709 12:24:06.481043 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.481177 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":9495,"spinlock_wait_cycles":21376,"thread_start_us":211039,"threads_started":1,"wal-append.queue_time_us":338591}
I20260709 12:24:06.455930 27047 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:06.490372 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:06.495700 20881 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:06.496865 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:06.454654 27354 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:06.494029 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2709) took 1910 ms. Trace:
I20260709 12:24:06.464918 27210 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.502430 20880 rpcz_store.cc:276] 0709 12:24:04.584023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.836726 (+1252703us) service_pool.cc:224] Handling call
0709 12:24:05.837055 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:05.837383 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.837410 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:05.837906 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.085968 (+248062us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:06.086052 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.086154 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.493829 (+407675us) raft_consensus.cc:1685] finished
0709 12:24:06.493850 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.493972 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9708,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7684,"thread_start_us":244560,"threads_started":1,"wal-append.queue_time_us":325914}
I20260709 12:24:06.503984 20878 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:06.442883 27339 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:24:06.439743 27344 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.443014 27184 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:24:06.482663 27038 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.442602 27335 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:24:06.486034 26790 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:06.461017 27072 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:06.540946 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.518536 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:06.678774 20804 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
W20260709 12:24:06.820755 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.016s (SENT) [suppressed 10 similar messages]
I20260709 12:24:06.677771 20809 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:24:06.582569 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.717546 27075 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.489765 26852 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:06.498059 27047 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.746631 27354 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.497865 20881 raft_consensus.cc:606] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:06.575704 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:06.514900 27300 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
W20260709 12:24:06.851018 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 275444us
I20260709 12:24:06.855970 20877 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:06.498468 27272 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:06.709820 26511 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:06.814755 27230 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.000s
I20260709 12:24:06.559496 27195 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:06.518864 27210 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.556391 27302 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.915870 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:06.856242 27121 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:06.678414 27011 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.823472 27312 raft_consensus.cc:2272] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:06.880505 20809 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
W20260709 12:24:06.881477 20618 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:06.861027 27354 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:06.989415 20618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108025us
I20260709 12:24:06.895241 20881 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:06.990829 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1141) took 1010 ms. Trace:
I20260709 12:24:06.991370 20881 rpcz_store.cc:276] 0709 12:24:05.980198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.894341 (+914143us) service_pool.cc:224] Handling call
0709 12:24:06.894923 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.990467 (+ 95544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.990755 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.863178 27300 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:06.502483 27169 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.892242 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:07.018752 21431 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:06.897531 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:06.899346 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:06.558380 27048 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.829188 27075 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.824440 27336 consensus_queue.cc:1243] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:24:06.909278 20735 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:24:06.816078 27112 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:07.024427 20735 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:24:07.024803 20735 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:07.025107 20735 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:06.824399 27072 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:06.912357 20882 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2772) took 2224 ms (client timeout 2016 ms). Trace:
I20260709 12:24:06.493322 27299 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:06.922446 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:06.950613 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.026129 20882 rpcz_store.cc:269] 0709 12:24:04.688096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.835685 (+147589us) service_pool.cc:224] Handling call
0709 12:24:06.912034 (+2076349us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400268254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:06.912250 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2692326272}
I20260709 12:24:06.855105 27047 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:06.857004 27316 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:06.993440 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1142) took 1013 ms. Trace:
I20260709 12:24:06.141252 20875 rpcz_store.cc:276] 0709 12:24:04.556581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.625398 (+1068817us) service_pool.cc:224] Handling call
0709 12:24:05.625741 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.625974 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:05.942842 (+316868us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.132130 (+189288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.132192 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:06.132243 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.132313 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.132450 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6445,"prepare.queue_time_us":435762,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1208,"thread_start_us":505400,"threads_started":2}
I20260709 12:24:06.862710 27351 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:07.048594 20881 rpcz_store.cc:276] 0709 12:24:05.980539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.992458 (+1011919us) service_pool.cc:224] Handling call
0709 12:24:06.992812 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.993018 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.993059 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.993120 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:06.993167 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.993237 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.993372 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.946830 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:06.682837 27257 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.828296 26790 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:06.858995 27182 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.823719 27038 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:06.986070 27367 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
W20260709 12:24:07.026309 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 204877us
I20260709 12:24:06.905135 27272 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:07.032763 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.076510 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2787) took 1092 ms. Trace:
I20260709 12:24:07.105553 20882 rpcz_store.cc:276] 0709 12:24:05.984246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.029792 (+1045546us) service_pool.cc:224] Handling call
0709 12:24:07.075788 (+ 45996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.076047 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.076086 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:07.076145 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:07.076201 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.076277 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.076434 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.909909 26511 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.086521 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2788) took 1100 ms. Trace:
I20260709 12:24:07.107888 20875 rpcz_store.cc:276] 0709 12:24:05.986029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.050349 (+1064320us) service_pool.cc:224] Handling call
0709 12:24:07.079348 (+ 28999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.082040 (+  2692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.082278 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:07.085847 (+  3569us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:07.085972 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.086116 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.086388 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:07.102722 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.112756 21427 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:24:06.993577 27011 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:07.086081 27363 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.299s	user 0.000s	sys 0.007s
I20260709 12:24:07.079993 27324 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:07.109977 27369 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.552s	user 0.001s	sys 0.000s
I20260709 12:24:07.028354 27300 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.036512 27112 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.032095 27048 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:06.949028 27195 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.031783 26852 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.104039 27316 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.085019 27351 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:07.277045 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 247617us
W20260709 12:24:07.254058 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 130439us
I20260709 12:24:07.328593 21431 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
W20260709 12:24:07.339437 21434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.000s	sys 0.008s
W20260709 12:24:07.340284 21434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.000s	sys 0.009s
I20260709 12:24:07.165433 20875 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:07.121232 27343 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:07.321877 27201 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.043s	user 0.013s	sys 0.000s
I20260709 12:24:07.340739 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:07.319655 27393 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.176249 27374 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.265942 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2779) took 1679 ms. Trace:
I20260709 12:24:07.389637 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2790) took 1028 ms. Trace:
I20260709 12:24:07.215049 27184 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.393519 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 52905.3us
W20260709 12:24:07.371647 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 284233us
W20260709 12:24:07.244500 26912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.002s	sys 0.006s
I20260709 12:24:07.400799 20876 rpcz_store.cc:276] 0709 12:24:05.586197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.241264 (+655067us) service_pool.cc:224] Handling call
0709 12:24:06.254601 (+ 13337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.256158 (+  1557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.256206 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.256682 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.347051 (+ 90369us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b18001216c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:24:06.441748 (+ 94697us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:06.441820 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.441936 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:07.253828 (+811892us) raft_consensus.cc:1685] finished
0709 12:24:07.253849 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.265851 (+ 12002us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4638,"spinlock_wait_cycles":7932160,"thread_start_us":94379,"threads_started":1,"wal-append.queue_time_us":233588}
I20260709 12:24:07.285876 27378 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.407794 20805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.012s	sys 0.008s
I20260709 12:24:07.288432 27335 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:07.414883 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:07.267243 26790 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.284082 27344 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:07.117889 27182 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.305346 27380 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.156s
W20260709 12:24:07.652397 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 305123us
I20260709 12:24:07.527691 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2934) took 1456 ms. Trace:
W20260709 12:24:07.584664 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.571775 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2702) took 3160 ms (client timeout 3088 ms). Trace:
W20260709 12:24:07.665689 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 250921us
W20260709 12:24:07.683880 20883 rpcz_store.cc:269] 0709 12:24:04.411132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.962148 (+551016us) service_pool.cc:224] Handling call
0709 12:24:07.571494 (+2609346us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400267854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:07.571672 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2427286912}
I20260709 12:24:07.624951 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:07.685619 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:07.572221 20884 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: [22.10-22.10]
I20260709 12:24:07.400892 20882 rpcz_store.cc:276] 0709 12:24:06.361183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.106964 (+745781us) service_pool.cc:224] Handling call
0709 12:24:07.114188 (+  7224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.114466 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:07.132148 (+ 17682us) log.cc:844] Serialized 14 byte log entry
0709 12:24:07.388598 (+256450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:07.389227 (+   629us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:07.389300 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.389383 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.389548 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17490,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"thread_start_us":216540,"threads_started":1}
W20260709 12:24:07.663013 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.624768 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.666803 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.004s	sys 0.021s
W20260709 12:24:07.691656 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.004s	sys 0.021s
I20260709 12:24:07.691859 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:07.461584 21398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.033s	sys 0.055s
W20260709 12:24:07.693333 21398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.044s	sys 0.066s
W20260709 12:24:07.664335 20805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.016s	sys 0.013s
I20260709 12:24:07.686668 20883 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:24:07.398320 27272 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:07.620044 20730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 2947) took 1741 ms (client timeout 1798 ms). Trace:
W20260709 12:24:07.766080 20730 rpcz_store.cc:269] 0709 12:24:05.867520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.868788 (+  1268us) service_pool.cc:224] Handling call
0709 12:24:07.608276 (+1739488us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54003b4a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:07.608414 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621172480}
W20260709 12:24:07.562048 27348 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.291s	user 0.000s	sys 0.002s
I20260709 12:24:07.581728 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:07.700086 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:07.768332 20876 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
W20260709 12:24:07.768728 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.398579 27299 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:07.674535 27121 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.013s	sys 0.011s
W20260709 12:24:07.800429 27121 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.015s	sys 0.012s
I20260709 12:24:07.800877 27121 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:07.802120 27121 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:07.390990 27201 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:07.437175 27184 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:07.486835 27075 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.689869 20808 rpcz_store.cc:276] 0709 12:24:06.071234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.095901 (+ 24667us) service_pool.cc:224] Handling call
0709 12:24:07.527405 (+1431504us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b54003c9e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:07.527603 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37241728}
I20260709 12:24:07.486665 27210 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.412910 27393 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:07.676090 27378 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:07.426828 27371 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 12:24:07.622810 27011 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:07.737461 27164 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.361s	user 0.014s	sys 0.000s
W20260709 12:24:07.425429 26912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.002s	sys 0.007s
W20260709 12:24:07.623659 27314 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.328s	user 0.002s	sys 0.011s
I20260709 12:24:07.681834 27188 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:07.772596 27316 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.608857 27324 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:07.767303 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 99655.3us
I20260709 12:24:07.421089 27374 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.565896 27400 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:07.573869 27195 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.637880 27385 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:07.667985 27388 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:07.807168 27169 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:07.810642 27398 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 12:24:07.678299 27335 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:07.812775 27354 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.797s	user 0.002s	sys 0.010s
I20260709 12:24:07.803562 27406 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:24:07.814035 20881 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:07.815121 27201 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:07.702099 27015 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:07.937544 27314 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:07.818969 27401 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:24:07.816592 27394 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:07.827553 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:07.961275 27015 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.965526 27314 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.389931 27343 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:07.808384 27038 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.003s
I20260709 12:24:07.935949 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:07.937017 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:07.972211 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.852051 27312 ts_tablet_manager.cc:1916] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:24:07.874390 27047 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.793s	user 0.011s	sys 0.000s
I20260709 12:24:07.827049 27300 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.975946 27047 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.861431 27307 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.977411 27307 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:07.961120 27403 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.909425 27212 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.917610 27048 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:07.878970 27112 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:07.877171 27182 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:07.873658 27375 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:07.935010 27164 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:07.998103 27393 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:07.878629 27380 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:07.830330 27412 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:07.961232 27374 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:07.803912 27407 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:24:07.943174 27188 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:08.042395 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.957927 27385 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.104780 27184 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.926656 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:07.835917 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:07.813280 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:07.813802 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.112216 20804 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:08.112588 27188 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.115433 27300 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:08.116128 20803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2653) took 3421 ms (client timeout 2016 ms). Trace:
I20260709 12:24:08.116401 27300 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:08.116822 20803 rpcz_store.cc:269] 0709 12:24:04.694989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.706125 (+ 11136us) service_pool.cc:224] Handling call
0709 12:24:08.115553 (+3409428us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003b6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:08.115843 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2997094784}
I20260709 12:24:07.816015 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:07.981683 27403 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:08.120848 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 304742us
W20260709 12:24:08.043942 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 409034us
W20260709 12:24:08.046257 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 464499us
I20260709 12:24:07.822846 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2724) took 1329 ms. Trace:
I20260709 12:24:07.834372 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:07.863499 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2947) took 1306 ms. Trace:
I20260709 12:24:08.132773 20805 rpcz_store.cc:276] 0709 12:24:06.556920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.557061 (+   141us) service_pool.cc:224] Handling call
0709 12:24:06.557519 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.557989 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:06.573701 (+ 15712us) log.cc:844] Serialized 14 byte log entry
0709 12:24:07.703082 (+1129381us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:07.703715 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:24:07.703909 (+   194us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:07.703957 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.704046 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:07.863256 (+159210us) raft_consensus.cc:1685] finished
0709 12:24:07.863275 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.863438 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156070,"prepare.queue_time_us":24868,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":8129,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":150249,"thread_start_us":1121194,"threads_started":1,"wal-append.queue_time_us":1278760}
I20260709 12:24:07.870172 20879 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:07.872115 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2783) took 1902 ms. Trace:
I20260709 12:24:08.055076 20882 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:24:08.142922 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.079790 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:08.082281 27013 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.142227 20880 rpcz_store.cc:276] 0709 12:24:05.969324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.508016 (+538692us) service_pool.cc:224] Handling call
0709 12:24:06.508452 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.508750 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:06.555074 (+ 46324us) log.cc:844] Serialized 14 byte log entry
0709 12:24:07.871510 (+1316436us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b1800349f00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:24:07.871644 (+   134us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:07.871711 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:07.871796 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.871886 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.872040 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49344,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":10465,"raft.queue_time_us":185590,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":19549,"spinlock_wait_cycles":46456064,"thread_start_us":611004,"threads_started":2,"wal-append.queue_time_us":1182840}
I20260709 12:24:08.147819 27013 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.149343 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.149634 27300 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.149996 26852 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:08.082302 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.150743 20880 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:07.968084 27015 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.167857 27015 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.168943 27015 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.170931 27230 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:08.171571 27230 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:08.172082 27222 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:08.173349 27164 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:08.086079 27047 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:07.995170 27053 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:08.174340 27164 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.176838 27339 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:08.177346 27339 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:07.982820 27038 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.178356 27164 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:07.957087 27195 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.989301 27212 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.192221 27195 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.986955 27307 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.197024 27212 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.001451 27314 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:07.997589 27335 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.201116 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.202648 27354 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.993989 27343 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:08.208663 27368 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.453s	user 0.007s	sys 0.010s
I20260709 12:24:08.001277 27351 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.212129 27272 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.213454 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.213256 27272 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:07.995357 27378 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.214646 21431 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:07.961454 27405 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.216774 27272 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.216935 27405 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.004571 27415 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:08.219317 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58425.5us
I20260709 12:24:08.219928 27415 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.221035 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1124) took 1909 ms. Trace:
I20260709 12:24:07.862779 27408 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 12:24:08.003491 27423 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:07.979923 27394 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.999411 27375 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.099347 27295 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:08.099975 27163 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.002981 27412 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.001603 27380 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:07.961995 27388 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.101127 27406 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.101890 27332 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.955169 27400 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:08.043478 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.018s	sys 0.050s
I20260709 12:24:08.106938 20728 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:08.107990 27273 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:08.108395 27385 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.122412 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:08.124090 27413 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:08.124956 27403 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.125792 27280 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:08.134500 27422 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:24:08.135592 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.136137 27374 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:08.136786 27393 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.136301 20875 rpcz_store.cc:276] 0709 12:24:06.493175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.109081 (+615906us) service_pool.cc:224] Handling call
0709 12:24:07.109466 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.341556 (+232090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.822582 (+481026us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:07.822778 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127718144}
I20260709 12:24:08.137059 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2781) took 2504 ms. Trace:
I20260709 12:24:07.833954 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.138942 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:08.139322 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2776) took 3009 ms. Trace:
I20260709 12:24:08.136163 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:08.143162 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2946) took 1590 ms. Trace:
I20260709 12:24:08.075384 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2794) took 1217 ms. Trace:
W20260709 12:24:08.248574 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.248416 27393 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.154421 27013 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:08.168959 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89707.5us
I20260709 12:24:08.178716 27339 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.184108 27038 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:07.877066 27337 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:08.221547 21433 rpcz_store.cc:276] 0709 12:24:06.311963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.312197 (+   234us) service_pool.cc:224] Handling call
0709 12:24:06.321237 (+  9040us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.321735 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.321766 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.330622 (+  8856us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.426535 (+ 95913us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:06.426575 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.426642 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:08.200107 (+1773465us) spinlock_profiling.cc:243] Waited 871 us on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:08.220404 (+ 20297us) raft_consensus.cc:1685] finished
0709 12:24:08.220428 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.220824 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2146,"raft.queue_time_us":638,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":6031,"spinlock_wait_cycles":2002816,"thread_start_us":62934,"threads_started":1,"wal-append.queue_time_us":147167}
W20260709 12:24:08.264078 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.020s	sys 0.051s
W20260709 12:24:08.264829 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 869682us
I20260709 12:24:08.264096 27337 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:08.236299 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.258507 27431 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:08.264191 27273 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.267434 27413 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.270301 27394 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.258139 27339 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:08.266716 27337 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.250198 20884 rpcz_store.cc:276] 0709 12:24:05.129542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.153058 (+1023516us) service_pool.cc:224] Handling call
0709 12:24:07.571969 (+1418911us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:07.688584 (+116615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.138855 (+450271us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400267850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:08.139110 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3262294272}
W20260709 12:24:08.212145 27399 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.637s	user 0.000s	sys 0.003s
I20260709 12:24:08.275492 27273 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.266917 27163 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:08.276360 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140538us
I20260709 12:24:08.278275 20879 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:08.249900 20878 rpcz_store.cc:276] 0709 12:24:05.632367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.305827 (+673460us) service_pool.cc:224] Handling call
0709 12:24:06.319942 (+ 14115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.509268 (+189326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.136495 (+1627227us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:08.136858 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422803200}
I20260709 12:24:08.279561 27378 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:08.279433 27406 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:08.281419 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 159290us
I20260709 12:24:08.282452 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.279476 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.250273 20876 rpcz_store.cc:276] 0709 12:24:06.857847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.769157 (+911310us) service_pool.cc:224] Handling call
0709 12:24:07.821593 (+ 52436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:07.822050 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.822093 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:07.822731 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.071932 (+249201us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:08.072607 (+   675us) log.cc:844] Serialized 14 byte log entry
0709 12:24:08.074439 (+  1832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.074609 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:08.074721 (+   112us) raft_consensus.cc:1685] finished
0709 12:24:08.074739 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.075092 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25785,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":30482,"raft.run_cpu_time_us":622,"raft.run_wall_time_us":78245,"thread_start_us":248456,"threads_started":1,"wal-append.queue_time_us":238259}
I20260709 12:24:08.225541 27415 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.282899 27380 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.248736 27423 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:08.293624 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.298506 27400 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.294871 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.301993 27415 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.303135 27415 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.304694 27423 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.290031 27280 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:08.249557 20806 rpcz_store.cc:276] 0709 12:24:06.552319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.552498 (+   179us) service_pool.cc:224] Handling call
0709 12:24:06.682380 (+129882us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:07.022140 (+339760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.022189 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:07.032807 (+ 10618us) log.cc:844] Serialized 52 byte log entry
0709 12:24:07.319509 (+286702us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:07.342990 (+ 23481us) log.cc:844] Serialized 14 byte log entry
0709 12:24:08.126582 (+783592us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b180010f480. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:08.142328 (+ 15746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.142467 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:08.142562 (+    95us) raft_consensus.cc:1685] finished
0709 12:24:08.142694 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.142932 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":165,"prepare.queue_time_us":22774,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":10120,"raft.queue_time_us":365125,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":14032,"spinlock_wait_cycles":32566016,"thread_start_us":543381,"threads_started":3,"wal-append.queue_time_us":437726}
I20260709 12:24:08.302101 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.283366 27332 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.309938 27375 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.310345 27163 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.310784 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.313504 27412 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.313912 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.314751 20808 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:08.315780 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.317576 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.323493 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.295565 27394 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:08.323916 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2793) took 1799 ms. Trace:
I20260709 12:24:08.327950 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1122) took 2125 ms. Trace:
I20260709 12:24:08.328416 21434 rpcz_store.cc:276] 0709 12:24:06.201951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.202149 (+   198us) service_pool.cc:224] Handling call
0709 12:24:06.202578 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.240836 (+ 38258us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:06.438599 (+197763us) log.cc:844] Serialized 14 byte log entry
0709 12:24:07.384646 (+946047us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:07.406502 (+ 21856us) log.cc:844] Serialized 26 byte log entry
0709 12:24:07.406722 (+   220us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:07.406773 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.406873 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:08.211040 (+804167us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:08.325378 (+114338us) raft_consensus.cc:1685] finished
0709 12:24:08.325475 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.327714 (+  2239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83177,"prepare.queue_time_us":70778,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":343632,"raft.run_cpu_time_us":439,"raft.run_wall_time_us":20002,"spinlock_wait_cycles":77617408,"thread_start_us":1031908,"threads_started":2,"wal-append.queue_time_us":946290}
I20260709 12:24:08.330641 27109 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:08.332826 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.331821 20880 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:08.321985 27354 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.351475 20875 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:08.355407 27388 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.356904 27393 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:08.358534 27332 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.359035 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.359824 21434 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:08.361327 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:08.364805 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 52414.7us
I20260709 12:24:08.365085 20729 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:08.369143 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.370172 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:08.376350 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.377519 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.377782 20730 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:08.378942 27413 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.379124 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.376210 27302 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:08.345659 27280 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.118s
I20260709 12:24:08.506470 20730 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:08.518798 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.408809 20878 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:24:08.326277 20883 rpcz_store.cc:276] 0709 12:24:06.501132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.768433 (+1267301us) service_pool.cc:224] Handling call
0709 12:24:07.768805 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:07.793893 (+ 25088us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:08.080654 (+286761us) log.cc:844] Serialized 14 byte log entry
0709 12:24:08.131341 (+ 50687us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.275029 (+143688us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.275273 (+   244us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:08.275309 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.275423 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:08.284270 (+  8847us) raft_consensus.cc:1685] finished
0709 12:24:08.284299 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.299915 (+ 15616us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30373,"prepare.queue_time_us":211167,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":37,"raft.queue_time_us":35367,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":3538,"thread_start_us":321126,"threads_started":2,"wal-append.queue_time_us":48192}
I20260709 12:24:08.401860 20884 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:08.518800 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 192825us
I20260709 12:24:08.506368 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.488902 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.462924 21428 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:24:08.459648 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 139376us
I20260709 12:24:08.410477 20876 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:24:08.409294 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.518389 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.539230 20806 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:08.379974 27015 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.454907 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.547587 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:08.560519 27394 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:08.534956 27195 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.564194 20731 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:08.565032 20734 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:08.565667 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:08.566071 20726 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:24:08.567699 20883 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:24:08.567893 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.569222 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.568147 27203 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.569941 20805 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:08.570242 27203 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.554894 27393 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.579221 27344 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:08.582240 20731 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:08.583796 20775 connection.cc:256] RPC call timeout handler was delayed by -1.0376264330002414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 12:24:08.584772 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:08.584961 27385 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:08.585024 27273 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:08.585975 27013 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:08.585749 27385 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.585852 27273 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.589085 26511 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:08.590642 27436 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:24:08.590909 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2792) took 2229 ms. Trace:
I20260709 12:24:08.591396 20877 rpcz_store.cc:276] 0709 12:24:06.361679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.493827 (+132148us) service_pool.cc:224] Handling call
0709 12:24:08.590439 (+2096612us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b54003f98d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:08.590685 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80573824}
I20260709 12:24:08.593218 20803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad57e68a6a114847ac252579307658f7"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:08.594152 20803 raft_consensus.cc:606] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:08.594643 26807 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:08.594602 27393 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.595589 26807 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.609272 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2960) took 2536 ms. Trace:
I20260709 12:24:08.610073 20808 raft_consensus.cc:1217] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 13.9->[14.10-14.10]   Dedup: 14.10->[]
I20260709 12:24:08.610662 20808 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:08.609747 20735 rpcz_store.cc:276] 0709 12:24:06.072654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.073293 (+   639us) service_pool.cc:224] Handling call
0709 12:24:06.073568 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.025676 (+952108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.608770 (+1583094us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54003ae8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:08.609076 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920983808}
W20260709 12:24:08.626731 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.633368 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.596308 27175 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:08.580780 27394 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:08.634454 20805 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:08.639290 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.644196 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36394 (request call id 3) took 2200 ms. Trace:
I20260709 12:24:08.644687 20878 rpcz_store.cc:276] 0709 12:24:06.441125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.548999 (+2107874us) service_pool.cc:224] Handling call
0709 12:24:08.641120 (+ 92121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:08.648447 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.649434 20727 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:08.650597 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.650868 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.651398 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:08.651527 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.651800 20809 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:08.651561 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:08.653177 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.578509 27015 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:08.657634 20804 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:24:08.657717 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.658604 20729 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:08.659963 20735 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:24:08.661099 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 53078.5us
I20260709 12:24:08.635732 27175 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:08.659932 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2722) took 2650 ms. Trace:
I20260709 12:24:08.661260 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.664546 20881 rpcz_store.cc:276] 0709 12:24:05.988258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.078776 (+1090518us) service_pool.cc:224] Handling call
0709 12:24:07.079104 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.960651 (+881547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.576850 (+616199us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b540019bed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:08.636834 (+ 59984us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689646464}
I20260709 12:24:08.671875 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:08.663323 27280 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:08.673629 20803 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:08.657871 27385 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.677706 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.682766 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:08.672650 27015 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.693408 27385 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:08.693641 27379 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.704324 27379 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.706456 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.693712 27272 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.707612 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.708385 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.708598 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:08.710362 20805 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:24:08.711468 20802 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:08.712466 27438 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:24:08.686888 27354 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.703415 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.712967 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.713467 20801 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:08.707362 27272 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.714740 26852 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:08.707769 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.715339 26852 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:08.715876 20730 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:24:08.716243 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:08.715847 26852 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:08.716826 27272 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:08.713690 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.717698 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:08.718539 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.717042 20805 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:08.703051 27433 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:08.721216 20808 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: [14.9-17.10]
W20260709 12:24:08.768982 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 116455us
I20260709 12:24:08.770455 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.776738 27312 ts_tablet_manager.cc:1929] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.7
I20260709 12:24:08.777539 27312 log.cc:1199] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 49323f66286f4798b268f216bb62b3be: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/wals/7ccae903e8484f8d8c3af847e2c4c3d4
W20260709 12:24:08.778270 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54843.5us
I20260709 12:24:08.767673 27400 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:08.797724 27015 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.797958 27312 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39562 (request call id 3) took 4118 ms. Trace:
I20260709 12:24:08.794469 27393 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:08.779121 27379 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.752261 27044 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.747681 27385 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.807682 27044 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.810129 27385 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.813488 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.813720 27433 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.794821 27280 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.815968 27280 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.813230 27044 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.809008 20802 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:08.818222 27433 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.821700 26807 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.823032 27415 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.823887 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:08.824235 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.824756 20735 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:08.825492 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:08.826056 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.826712 21426 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:08.827224 20801 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:08.827409 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.827565 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.827682 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.828235 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.828555 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.828698 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.828917 20801 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:08.829411 20803 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:24:08.829860 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:08.830075 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.830651 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.830644 27273 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.830014 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.833024 27405 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:08.833491 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1279) took 1933 ms (client timeout 2239 ms). Trace:
W20260709 12:24:08.834009 21428 rpcz_store.cc:269] 0709 12:24:06.900472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.900652 (+   180us) service_pool.cc:224] Handling call
0709 12:24:08.833283 (+1932631us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54004da8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:08.833410 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3173963648}
I20260709 12:24:08.835918 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.798620 27312 rpcz_store.cc:276] 0709 12:24:04.680018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.716673 (+ 36655us) service_pool.cc:224] Handling call
0709 12:24:05.121658 (+404985us) ts_tablet_manager.cc:1151] Deleting tablet 7ccae903e8484f8d8c3af847e2c4c3d4
0709 12:24:05.171734 (+ 50076us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:24:07.997932 (+2826198us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:08.682865 (+684933us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:08.797818 (+114953us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1141611,"tablet-delete.queue_time_us":268926,"thread_start_us":159454,"threads_started":1}
I20260709 12:24:08.839012 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.826303 20809 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:08.839179 27230 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.842916 27408 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:08.843031 27230 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:08.843515 27408 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:08.810575 27175 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.844628 27408 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.803013 27400 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:08.845870 27408 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:08.846791 27400 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.852197 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.852674 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2796) took 1158 ms. Trace:
I20260709 12:24:08.852981 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.853173 20882 rpcz_store.cc:276] 0709 12:24:07.694314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.965378 (+271064us) service_pool.cc:224] Handling call
0709 12:24:07.965762 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.143304 (+177542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.852369 (+709065us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b54000965d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:08.852607 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203076864}
I20260709 12:24:08.854914 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.855360 27389 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:08.855965 20877 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:08.857860 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.860190 27406 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:08.863787 27403 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:08.864298 27403 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:08.864480 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1617) took 1539 ms. Trace:
I20260709 12:24:08.865342 27403 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.866473 27403 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:08.866958 27029 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.845188 27175 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:08.864511 20726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1041) took 2552 ms (client timeout 2103 ms). Trace:
W20260709 12:24:08.868146 20726 rpcz_store.cc:269] 0709 12:24:06.312080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.312190 (+   110us) service_pool.cc:224] Handling call
0709 12:24:08.864236 (+2552046us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b54003ad9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:08.864410 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5180430080}
I20260709 12:24:08.868443 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.864894 21435 rpcz_store.cc:276] 0709 12:24:07.324606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.324782 (+   176us) service_pool.cc:224] Handling call
0709 12:24:07.325129 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.413952 (+ 88823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.413990 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:07.414031 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:07.414066 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.864010 (+1449944us) spinlock_profiling.cc:243] Waited 88.5 ms on lock 0x7b540042ed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:08.864069 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.864285 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203530112}
I20260709 12:24:08.870579 21429 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:24:08.855026 27163 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:24:08.871470 27163 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:08.871886 27163 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:08.873368 27163 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.877344 27163 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:08.878782 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 84896.4us
W20260709 12:24:08.878952 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74906.4us
I20260709 12:24:08.879835 27281 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:08.879876 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:08.883749 27024 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:08.884179 27024 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.882973 27203 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.884495 27024 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:08.888857 27281 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:08.889322 27281 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:08.890507 27316 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:08.890296 27281 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.891091 27316 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.891708 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 7ccae903e8484f8d8c3af847e2c4c3d4 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:24:08.893428 27281 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:08.893653 27316 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:08.895718 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.896278 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2797) took 1201 ms. Trace:
I20260709 12:24:08.896642 20884 rpcz_store.cc:276] 0709 12:24:07.694631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.298654 (+604023us) service_pool.cc:224] Handling call
0709 12:24:08.370050 (+ 71396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.534223 (+164173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.896087 (+361864us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b54003f6e50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:08.896214 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158419840}
I20260709 12:24:08.898922 27337 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:08.902478 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet c0b73d72c3ae4b17abbf64b777ddb8c2: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:08.902618 27188 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:08.903257 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:08.903620 27188 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:08.906635 27188 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:08.907707 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.912035 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:08.912784 27193 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:08.919938 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.920517 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:08.921247 21426 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:08.922366 20884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46401ecb9b443118647ee361081a01e"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:08.923058 20884 raft_consensus.cc:606] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:08.923158 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.911351 27159 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:08.924269 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.925019 20881 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:08.867600 27029 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:08.928783 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.929428 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.930119 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:08.930546 27029 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:08.923452 20884 raft_consensus.cc:614] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:08.930969 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.930567 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.931835 20882 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:08.931970 20810 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:08.932890 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:08.933738 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 43775us
W20260709 12:24:08.932930 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46060.6us
I20260709 12:24:08.939872 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.940203 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.940881 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:08.941562 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.942351 20884 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:08.943454 27158 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:08.943953 27158 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:08.944329 27158 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:08.948351 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:08.954625 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.957726 20881 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:08.962924 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.963773 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.964488 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.965229 20879 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:24:08.964032 20878 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:08.960038 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:08.966528 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:08.968567 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.968729 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:08.969777 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.970413 20882 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:08.964537 20876 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:08.958377 27044 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:08.970055 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.979627 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.981426 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.977502 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:08.986827 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.987685 20728 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:24:08.983783 20734 raft_consensus.cc:2376] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17. Re-sending same reply.
I20260709 12:24:08.991322 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.984887 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:08.959096 27273 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:24:08.997097 20882 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:08.997932 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:08.998651 20804 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:24:08.995476 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:08.999151 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:09.002393 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:09.003113 20809 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:09.003625 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:08.979043 27044 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:09.005306 27044 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:09.006691 27075 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:09.007108 27075 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:09.007510 27075 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:09.007310 26807 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:09.008946 26807 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:09.009728 27385 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.009841 26807 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:09.010322 26807 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:09.010249 27385 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.012884 27385 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:09.014042 20883 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:24:09.015064 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.015986 20732 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:09.016260 27373 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:09.016753 27373 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.017112 27373 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:09.017159 27075 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:09.017963 27360 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.421s	user 0.005s	sys 0.041s
W20260709 12:24:09.018335 27360 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.471s	user 0.005s	sys 0.042s
I20260709 12:24:09.019809 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:08.981777 27280 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:09.028307 20876 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:08.972729 26852 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:09.033638 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 67304.9us
I20260709 12:24:09.019294 27390 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.037514 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.040585 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.040895 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.041944 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.042768 21430 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:09.046200 20730 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:09.047591 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.047842 20727 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:09.048804 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:09.049471 20726 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:09.050674 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.051517 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.052374 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:09.055603 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.056789 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.057117 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.057536 20801 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:09.057756 20803 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:09.058455 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.058568 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.047075 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.065922 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:09.059065 20801 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:09.060042 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.021348 27188 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:09.072990 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78593.5us
W20260709 12:24:09.077164 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.059095 20803 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 12.
I20260709 12:24:09.067853 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.068866 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.102695 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.105815 20879 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:09.106307 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.107033 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:24:09.033902 27376 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:09.113648 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.115211 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.115677 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.118171 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.115967 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.120664 20733 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:09.106469 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.117957 20801 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:24:09.124250 21428 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:24:09.122640 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.129326 20729 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:09.123080 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:09.134826 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.126381 20727 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:09.136144 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.143060 20882 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:24:09.143654 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.142932 20804 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 12:24:09.071609 27158 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:09.150877 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.144052 20807 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:09.150941 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.156030 20803 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:09.149381 20876 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:24:09.122114 27075 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.117621 27376 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:09.156665 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:09.028460 26807 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:09.165239 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.167021 20810 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:09.070411 27390 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:09.168550 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.169039 27291 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:09.169551 27054 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.170240 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.255029 20727 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:24:09.170466 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.171447 20731 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:09.171799 20809 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:09.162473 27024 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:09.172720 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59174.9us
I20260709 12:24:09.173868 27158 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.173916 20805 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:09.174969 27193 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:09.177881 27316 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:09.183862 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:09.184954 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2725) took 1032 ms. Trace:
I20260709 12:24:09.146871 27378 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:09.189734 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.271348 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.193094 27273 raft_consensus.cc:993] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:24:09.249706 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:09.251973 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 80665.9us
I20260709 12:24:09.276180 20876 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:09.279984 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.243794 20726 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:09.260301 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.167649 27376 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:09.285501 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.282205 20810 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:24:09.229943 20879 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:09.264058 20875 rpcz_store.cc:276] 0709 12:24:08.152765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.381478 (+228713us) service_pool.cc:224] Handling call
0709 12:24:08.606168 (+224690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.644233 (+ 38065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.644268 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.670598 (+ 26330us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.789726 (+119128us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:08.789766 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.789833 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.013761 (+223928us) raft_consensus.cc:1685] finished
0709 12:24:09.013783 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.184668 (+170885us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b54003f98d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:09.184887 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5107,"raft.queue_time_us":10435,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3415,"spinlock_wait_cycles":202657024,"thread_start_us":77751,"threads_started":1,"wal-append.queue_time_us":290165}
I20260709 12:24:09.294323 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:09.263692 27316 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.270118 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.272722 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.274057 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:09.274502 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90695.4us
I20260709 12:24:09.301054 20732 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:09.275656 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:09.283911 27385 raft_consensus.cc:1081] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:24:09.296096 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:09.274556 27105 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:09.300870 20878 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:09.304090 20734 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:09.312577 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.313592 20726 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
W20260709 12:24:09.315128 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 359136us
I20260709 12:24:09.283613 27193 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.293377 27188 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:09.330369 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54827.1us
W20260709 12:24:09.268857 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.260576 27291 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:09.300913 27316 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.335067 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:09.332191 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.332089 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.351389 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.351934 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.351575 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.317615 26807 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.306059 27105 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:09.353729 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.367676 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.295884 27373 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:09.368458 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 92455.7us
I20260709 12:24:09.368732 20878 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:24:09.371656 20878 pending_rounds.cc:85] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 12
I20260709 12:24:09.360736 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:09.269733 27054 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.336087 27188 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:09.334159 27378 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:09.396190 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:09.374504 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:09.374464 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.372014 20878 pending_rounds.cc:107] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:24:09.401182 20878 raft_consensus.cc:2889] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:09.349337 26852 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.405062 20875 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:09.403405 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:09.368134 27302 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.834s	user 0.010s	sys 0.006s
I20260709 12:24:09.350708 27273 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:09.410224 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.370098 27075 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.370078 27105 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:09.403321 26817 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:09.403573 21432 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:09.415145 20730 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:09.419167 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.419167 21427 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:09.428648 27385 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:09.429160 27385 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:09.429607 27385 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.435555 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.436587 27054 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:09.437539 27024 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.439877 27373 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:09.441319 27072 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.441879 27072 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:09.443487 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49176.7us
I20260709 12:24:09.445366 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.451772 21428 raft_consensus.cc:2410] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 12:24:09.454391 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59345.3us
W20260709 12:24:09.457381 20806 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.016s	sys 0.026s
W20260709 12:24:09.461093 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93101.1us
I20260709 12:24:09.422958 27394 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:09.465330 20806 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.017s	sys 0.026s
I20260709 12:24:09.476400 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:09.427263 26817 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:09.477924 26817 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.479064 26817 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:09.481693 27044 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:09.494666 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.496181 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:09.502640 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:09.502807 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47105.4us
I20260709 12:24:09.503336 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:09.504664 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:09.499593 27400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.015s	sys 0.019s
I20260709 12:24:09.524200 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:09.475059 27394 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:09.527221 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1614) took 3464 ms (client timeout 2249 ms). Trace:
W20260709 12:24:09.537302 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.532380 21432 rpcz_store.cc:269] 0709 12:24:06.035005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.035241 (+   236us) service_pool.cc:224] Handling call
0709 12:24:09.467521 (+3432280us) spinlock_profiling.cc:243] Waited 3.33 s on lock 0x7b54007602d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:09.499677 (+ 32156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7668168064}
W20260709 12:24:09.529682 27400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.015s	sys 0.020s
I20260709 12:24:09.538977 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.531329 27394 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:09.538904 27281 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.004s
W20260709 12:24:09.539517 20808 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.014s
I20260709 12:24:09.533900 27195 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:09.561244 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.567601 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:09.562424 20808 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.008s	sys 0.014s
I20260709 12:24:09.564911 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.538717 27382 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:24:09.553181 27230 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:09.566121 27337 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:09.566900 27390 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.559772 27193 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:09.590969 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.592672 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.595129 27400 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:09.596318 27447 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:24:09.597030 27291 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.597656 27291 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:09.566077 27029 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:24:09.581764 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57699.4us
I20260709 12:24:09.545802 27203 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 12:24:09.606575 21426 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.007s	sys 0.015s
I20260709 12:24:09.609560 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:09.610126 21426 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.007s	sys 0.015s
I20260709 12:24:09.604754 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:09.610365 21433 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:09.621973 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.586175 27337 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:09.636247 27337 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:09.624754 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.640223 27280 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:09.640713 27280 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.641129 27280 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:09.627053 27072 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:09.648032 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.643601 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:09.648102 20876 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:09.573566 27433 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:24:09.644526 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.630170 27316 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.602437 27044 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:09.630606 27376 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:09.649751 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89275.7us
I20260709 12:24:09.664800 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:09.665969 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60568.1us
I20260709 12:24:09.668550 21429 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:09.670166 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.670624 20801 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:09.671696 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.672824 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.674464 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:09.676568 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:09.606299 27400 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 12:24:09.678226 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:09.678612 20735 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:24:09.672151 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:09.697261 20727 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:09.699822 20807 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:09.701519 21433 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:09.704373 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.705179 20726 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:09.706493 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.708787 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:09.709460 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:09.687683 27072 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:09.714509 20729 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:09.702216 27203 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:09.711354 27415 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:09.748656 27462 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:24:09.750502 27163 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.873s	user 0.010s	sys 0.004s
I20260709 12:24:09.751591 27029 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:09.752202 27029 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:09.721616 27230 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:09.758746 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:09.754968 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45638.3us
I20260709 12:24:09.739424 27302 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:09.762441 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85950.7us
I20260709 12:24:09.765486 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:09.677898 27105 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.776006 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:09.778978 27316 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.780184 27376 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:09.781068 26852 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:09.781539 26852 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:09.781934 26852 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:09.782590 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:09.783484 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:09.747346 27195 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:09.785247 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.753280 27029 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.788059 27203 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.799491 20879 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.17-19.17]
I20260709 12:24:09.800910 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:09.804697 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46030.8us
I20260709 12:24:09.801155 27281 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:09.790710 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:09.802701 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:09.761632 27230 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:09.805220 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.764000 27302 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.810425 20876 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:09.813959 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.814034 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:09.818338 20877 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.018s	sys 0.034s
W20260709 12:24:09.819924 20877 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.018s	sys 0.034s
W20260709 12:24:09.823557 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.811595 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:09.809494 27382 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:09.833835 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:09.834723 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:09.836946 20735 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:09.839138 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:09.839803 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52841.5us
I20260709 12:24:09.840847 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:09.841704 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:09.809723 27193 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:09.812265 27230 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:09.847324 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:09.848426 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.848492 27382 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:09.849182 27382 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.849556 27382 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:09.849710 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:09.850451 20807 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:09.851125 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:09.851567 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.846765 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:09.832012 27302 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:09.853068 20734 raft_consensus.cc:2368] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21.
I20260709 12:24:09.846766 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:09.854794 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:09.858376 20807 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.853960 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:09.834580 27291 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:09.869371 27163 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:09.869536 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:09.869859 27163 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:09.870170 20803 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:09.870292 27163 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:09.870913 27433 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:09.876204 27105 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:09.837357 27273 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:09.854187 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:09.842733 27054 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:09.880239 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.888135 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:09.889006 20806 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:09.812304 27281 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.810189 27195 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.888159 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:09.896962 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.892948 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:09.900205 26807 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.003s
I20260709 12:24:09.887745 27054 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:09.901226 27281 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:09.901479 27054 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:09.902092 27267 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:24:09.865950 27072 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:09.812323 27203 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.911228 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.907063 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:09.881574 27291 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:09.900960 27159 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:09.884649 27376 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:09.893422 27075 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:09.923688 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:09.906587 27072 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.927932 21400 leader_election.cc:400] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21.
I20260709 12:24:09.866210 27403 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:24:09.925305 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:09.930895 21430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad57e68a6a114847ac252579307658f7"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44460
I20260709 12:24:09.935626 21400 leader_election.cc:403] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:24:09.935771 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95025us
I20260709 12:24:09.937130 21428 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 12.
I20260709 12:24:09.915885 27291 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.918395 27376 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.942612 21432 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:24:09.943143 20802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.003s	sys 0.029s
W20260709 12:24:09.968395 20802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.003s	sys 0.030s
I20260709 12:24:09.925956 27158 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:09.937227 27072 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.888203 26852 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.967123 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:09.943450 20880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.010s	sys 0.022s
I20260709 12:24:09.916982 27385 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:09.971342 20880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.010s	sys 0.022s
I20260709 12:24:09.942761 26817 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.971388 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1146) took 1076 ms. Trace:
I20260709 12:24:09.972273 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2733) took 1070 ms. Trace:
I20260709 12:24:09.973038 20880 rpcz_store.cc:276] 0709 12:24:08.902327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.916171 (+ 13844us) service_pool.cc:224] Handling call
0709 12:24:08.916534 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.916817 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:09.301866 (+385049us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.971944 (+670078us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.972003 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:09.972051 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.972110 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.972212 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":626503,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"thread_start_us":963012,"threads_started":2}
I20260709 12:24:09.924355 27337 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:09.943383 27376 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.994566 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2696) took 1065 ms. Trace:
I20260709 12:24:10.000090 20804 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:09.972461 21435 rpcz_store.cc:276] 0709 12:24:08.894517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.927383 (+ 32866us) service_pool.cc:224] Handling call
0709 12:24:08.927677 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.927968 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:09.397409 (+469441us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.971007 (+573598us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.971068 (+    61us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:09.971119 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.971207 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.971336 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11903,"prepare.queue_time_us":663352,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":964415,"threads_started":2}
I20260709 12:24:10.001543 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:09.995213 20884 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.010s	sys 0.056s
I20260709 12:24:10.004735 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.005560 20803 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 12: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:10.008320 27105 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.008884 27105 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.012583 26852 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.015673 21430 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:10.016165 21430 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:10.022115 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:10.005496 27337 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.014514 27230 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:10.003052 20884 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.010s	sys 0.056s
I20260709 12:24:09.996877 20802 rpcz_store.cc:276] 0709 12:24:08.929368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.929483 (+   115us) service_pool.cc:224] Handling call
0709 12:24:08.929772 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.930102 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:09.394772 (+464670us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.994218 (+599446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.994273 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:09.994324 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.994385 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.994505 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":705145,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"thread_start_us":1017252,"threads_started":2}
I20260709 12:24:10.011062 27105 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:10.078884 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 112858us
W20260709 12:24:10.080768 20881 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.024s	sys 0.012s
W20260709 12:24:10.081261 20881 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.025s	sys 0.012s
I20260709 12:24:10.035518 27075 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:10.052011 27430 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:10.126006 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1155) took 1172 ms. Trace:
W20260709 12:24:10.130601 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.009s	sys 0.017s
W20260709 12:24:10.131147 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.009s	sys 0.017s
I20260709 12:24:10.078024 27337 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:10.122110 20700 leader_election.cc:400] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:10.133764 20700 leader_election.cc:403] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:24:10.135376 21426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.010s	sys 0.014s
W20260709 12:24:10.136049 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 568529us
W20260709 12:24:10.136219 21426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.010s	sys 0.014s
I20260709 12:24:10.023099 20806 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:10.112414 20728 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.114956 21429 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:10.023742 27386 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.159s
I20260709 12:24:10.137917 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1149) took 1160 ms. Trace:
W20260709 12:24:10.138329 21431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.023s
I20260709 12:24:10.088923 27344 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:10.164968 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48044.8us
I20260709 12:24:10.169005 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.163097 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:10.127095 20884 rpcz_store.cc:276] 0709 12:24:08.953582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.957000 (+  3418us) service_pool.cc:224] Handling call
0709 12:24:08.978838 (+ 21838us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:09.012728 (+ 33890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.012772 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:09.013488 (+   716us) log.cc:844] Serialized 52 byte log entry
0709 12:24:09.398157 (+384669us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:09.593724 (+195567us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b18001d2c00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:10.053310 (+459586us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.071496 (+ 18186us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.074003 (+  2507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.074104 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.116861 (+ 42757us) raft_consensus.cc:1685] finished
0709 12:24:10.116935 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.125870 (+  8935us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187242,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":4075,"raft.queue_time_us":7419,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":31159,"spinlock_wait_cycles":14688000,"thread_start_us":1018477,"threads_started":2,"wal-append.queue_time_us":571217}
W20260709 12:24:10.175616 20808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.004s	sys 0.013s
W20260709 12:24:10.176128 20808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.004s	sys 0.013s
W20260709 12:24:10.163254 21431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.000s	sys 0.024s
W20260709 12:24:10.158768 27457 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:24:10.179942 20850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.006s	sys 0.008s
I20260709 12:24:10.139838 27337 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.159502 27281 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21.
I20260709 12:24:10.182072 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.140764 27316 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:10.184842 20850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.006s	sys 0.008s
I20260709 12:24:10.185304 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2699) took 1225 ms. Trace:
I20260709 12:24:10.196599 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.203753 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.203752 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:10.205175 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 298154us
I20260709 12:24:10.190235 20808 rpcz_store.cc:276] 0709 12:24:08.968031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.974605 (+  6574us) service_pool.cc:224] Handling call
0709 12:24:08.994255 (+ 19650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.024138 (+ 29883us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:09.584532 (+560394us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.184462 (+599930us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.184610 (+   148us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:10.184738 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.184924 (+   186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.185191 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":611392,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"thread_start_us":1063754,"threads_started":2}
I20260709 12:24:10.210263 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:10.186218 27158 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:10.186432 27337 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.223011 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:10.224656 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:10.224558 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.240566 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.240885 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.241464 20882 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:10.243520 20876 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:10.204650 27446 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.020s
I20260709 12:24:10.206136 27029 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.250845 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.251689 20880 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.255788 20884 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:10.257091 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.205721 27430 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:10.257627 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 419406us
I20260709 12:24:10.261749 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:10.271081 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:10.253204 27446 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.007s	sys 0.021s
I20260709 12:24:10.228863 27105 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:10.228623 27159 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.233140 27024 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:10.262557 27463 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:10.286249 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61798.3us
I20260709 12:24:10.296175 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:10.304394 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 361493us
W20260709 12:24:10.309224 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:10.277693 27169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.013s	sys 0.024s
I20260709 12:24:10.240572 27403 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:10.276197 27475 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:10.330690 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:10.278726 27385 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:10.331744 27188 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.919s	user 0.011s	sys 0.002s
W20260709 12:24:10.344467 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 134299us
I20260709 12:24:10.375905 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.306704 27159 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.378947 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 82908.8us
I20260709 12:24:10.380059 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.349438 20801 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:10.360437 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.363787 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:10.343129 27169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.014s	sys 0.025s
I20260709 12:24:10.391496 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.392452 27478 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:10.385426 27273 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.395629 27273 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:10.397439 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:10.398653 27478 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:10.399041 27478 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:10.409039 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 240114us
I20260709 12:24:10.351732 27195 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.346793 27403 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:10.350632 27400 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:10.400162 26807 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:10.416415 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:10.412705 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 142681us
I20260709 12:24:10.382524 27475 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:10.387028 27463 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:10.415445 27478 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.298094 27163 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:10.431789 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 217787us
I20260709 12:24:10.452993 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:10.408295 27385 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:10.431067 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:10.409930 27193 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.448477 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 178637us
I20260709 12:24:10.386439 27188 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.389911 27159 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:10.453512 27467 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.321s	user 0.001s	sys 0.001s
W20260709 12:24:10.474396 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 110706us
I20260709 12:24:10.423384 27403 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:10.475057 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 114724us
I20260709 12:24:10.431461 27475 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.429844 27493 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:10.482358 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.483747 21430 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:10.489194 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72847.6us
I20260709 12:24:10.489717 21428 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 12. Current term is 14. Ops: []
W20260709 12:24:10.489843 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58847.2us
W20260709 12:24:10.487210 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 279592us
I20260709 12:24:10.465720 27373 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:10.419715 27029 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:24:10.461787 27044 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.456426 27385 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.491087 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:10.466696 27169 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:10.504133 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:10.508081 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132288us
I20260709 12:24:10.511029 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:10.477569 27390 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:10.492448 27382 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.007s
I20260709 12:24:10.483232 27230 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.479758 27487 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:10.495805 27473 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.355s	user 0.000s	sys 0.002s
I20260709 12:24:10.481588 27463 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:10.520774 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67877.1us
I20260709 12:24:10.504891 27105 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:10.505995 27373 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:10.543717 27469 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.302s	user 0.001s	sys 0.001s
I20260709 12:24:10.544164 27373 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:10.476210 27193 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:10.523810 27474 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:24:10.546572 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2749) took 1505 ms. Trace:
I20260709 12:24:10.546933 27193 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.517983 27493 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.547487 27492 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
W20260709 12:24:10.534228 27476 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:24:10.548794 27492 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:10.549212 27492 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:10.533808 27487 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.550047 27487 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:10.550266 27492 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.533718 20700 leader_election.cc:400] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
W20260709 12:24:10.539752 27468 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
W20260709 12:24:10.532295 27483 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:24:10.536471 27393 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:10.575314 27393 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.575851 27393 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:10.543494 27486 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:24:10.547755 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:10.551860 27482 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:24:10.552284 27163 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.531495 27382 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.579936 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.580829 27029 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:10.581419 27382 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.581554 27029 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:10.582159 27472 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 12:24:10.582388 27382 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:10.582875 27382 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.536837 27491 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:24:10.583261 27480 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:24:10.554023 27503 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:10.162282 21426 rpcz_store.cc:276] 0709 12:24:08.977338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.981931 (+  4593us) service_pool.cc:224] Handling call
0709 12:24:08.995772 (+ 13841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.996047 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:09.620463 (+624416us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.137355 (+516892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.137431 (+    76us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:10.137508 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.137613 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.137844 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3088,"prepare.queue_time_us":595590,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":135,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":22902,"thread_start_us":1103356,"threads_started":2}
I20260709 12:24:10.546976 20881 rpcz_store.cc:276] 0709 12:24:09.040676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.111026 (+ 70350us) service_pool.cc:224] Handling call
0709 12:24:09.111540 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.543238 (+431698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.543283 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.567038 (+ 23755us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.081721 (+514683us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:10.081783 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.082896 (+  1113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.546343 (+463447us) raft_consensus.cc:1685] finished
0709 12:24:10.546363 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.546509 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43348,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":11561,"thread_start_us":878500,"threads_started":2,"wal-append.queue_time_us":627404}
W20260709 12:24:10.584365 27479 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 12:24:10.536167 27477 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.159s	user 0.001s	sys 0.001s
I20260709 12:24:10.585276 27203 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:10.585528 27488 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:24:10.586248 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1290) took 1019 ms. Trace:
I20260709 12:24:10.586753 26807 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:10.587455 26807 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:10.587600 27382 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:10.538034 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:10.590581 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52719.7us
W20260709 12:24:10.569476 20734 consensus_meta.cc:287] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.000s
I20260709 12:24:10.521005 27385 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:10.591603 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.592841 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:10.593936 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:10.581305 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.604766 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.606401 20880 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:10.595420 20734 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 12.
I20260709 12:24:10.603556 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:10.565277 20700 leader_election.cc:403] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:24:10.543828 27466 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:24:10.613693 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:10.613904 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.615167 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
W20260709 12:24:10.621007 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 130002us
I20260709 12:24:10.539049 27195 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:10.586666 21431 rpcz_store.cc:276] 0709 12:24:09.566493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.568163 (+  1670us) service_pool.cc:224] Handling call
0709 12:24:09.583108 (+ 14945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.587832 (+  4724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.587881 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.605730 (+ 17849us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.177271 (+571541us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:10.177484 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.177580 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.586016 (+408436us) raft_consensus.cc:1685] finished
0709 12:24:10.586045 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.586191 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":699,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":7427,"thread_start_us":549653,"threads_started":1,"wal-append.queue_time_us":631811}
W20260709 12:24:10.601046 27280 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.959s	user 0.011s	sys 0.001s
I20260709 12:24:10.629678 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:10.595170 27054 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.693s	user 0.006s	sys 0.002s
W20260709 12:24:10.598711 27376 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.004s
I20260709 12:24:10.631543 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:10.632375 21429 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:10.600620 27433 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.598196 27159 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.616061 27024 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.617233 27169 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.615764 27491 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:10.624643 27496 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:10.617712 27507 raft_consensus.cc:2764] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:24:10.631424 27505 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:10.644040 27501 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.632239 27029 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:10.634194 27400 raft_consensus.cc:2764] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:24:10.560922 27463 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.647370 27484 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:24:10.653864 20882 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:10.648332 27499 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:10.636229 27376 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.648758 27169 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:10.649286 27291 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.001s
W20260709 12:24:10.649891 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 122762us
I20260709 12:24:10.647455 27497 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:10.631217 27075 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:10.635763 27195 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:10.635049 27492 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:10.647606 27024 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:10.692085 26817 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.695s	user 0.007s	sys 0.004s
I20260709 12:24:10.692574 26817 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.692803 27024 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:10.650779 27302 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.000s
I20260709 12:24:10.693069 27478 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.693509 27302 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.695443 21427 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:10.662926 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 151996us
I20260709 12:24:10.696872 27302 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:10.663408 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 158301us
I20260709 12:24:10.699518 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.699677 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.700196 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:10.668692 27433 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:10.701085 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:10.701452 27433 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.703648 27433 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.704166 27433 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:10.704639 27433 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:10.705485 27433 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.706413 27390 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.707561 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:10.634609 27462 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:24:10.708329 27463 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.634788 27280 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.711061 27499 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.711256 27280 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:10.711644 27499 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.634819 27054 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.659860 27380 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:10.647373 27344 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:10.713059 27344 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:10.712985 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:10.714005 27054 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.714177 21428 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:10.714387 27499 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.707880 20808 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:10.712393 27506 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:10.715919 21429 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:24:10.715658 27506 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.716296 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.716301 21429 pending_rounds.cc:85] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:24:10.717549 21429 pending_rounds.cc:107] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:24:10.717873 21429 raft_consensus.cc:2889] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:10.651180 27496 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:10.687594 27509 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:24:10.688649 27415 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.930s	user 0.001s	sys 0.010s
I20260709 12:24:10.688521 27385 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.690412 27382 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.708199 20805 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:10.708329 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:10.684005 27491 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.678226 27075 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:10.682507 27501 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:10.731920 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:10.682755 27394 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.116s	user 0.013s	sys 0.001s
I20260709 12:24:10.720326 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:10.740832 20806 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:10.723481 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:10.747128 21431 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:10.725684 27507 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:10.750808 21431 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:10.727573 27496 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:10.724434 26817 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:10.678614 27291 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.762426 27502 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:10.752398 27478 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.763363 27193 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:10.739957 27415 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:10.763711 21431 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:10.808643 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51721.4us
I20260709 12:24:10.809116 21431 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:10.765606 27385 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.765959 27376 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:10.766222 27195 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.766959 27169 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:10.728085 27393 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.768332 27493 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.770081 27382 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.679098 27505 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:24:10.727121 27273 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.733796 27509 consensus_queue.cc:1243] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:10.748183 27075 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:10.776968 27403 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.827032 27496 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.828415 27403 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.829249 27496 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:10.755489 27394 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.763000 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:10.836615 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73681.9us
W20260709 12:24:10.836962 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 134463us
I20260709 12:24:10.838325 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:10.753619 27501 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.794566 27188 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:10.807089 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 133921us
W20260709 12:24:10.811096 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.822808 20801 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:10.784581 26807 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.816938 27484 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:10.825003 27054 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:10.823113 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:10.825922 27193 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:10.826203 27393 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:10.859673 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 370276us
W20260709 12:24:10.779066 27072 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.775s	user 0.011s	sys 0.000s
I20260709 12:24:10.844367 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:10.817224 27497 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:10.851778 20728 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:10.888836 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:10.855878 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:10.857440 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:10.859419 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.890345 20731 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:10.860628 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:10.862859 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.863322 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:10.826401 27291 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.892155 20879 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:24:10.864297 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.862897 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:10.879050 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.892704 20807 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:10.893571 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:10.894371 20729 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:10.896173 20805 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:24:10.883800 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.884328 20878 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:24:10.885267 20881 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:10.885712 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.846726 27403 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:10.863806 27337 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.001s
W20260709 12:24:10.901479 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.902882 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.907878 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.909278 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.911105 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:10.904930 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:10.817359 26817 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:10.914283 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53707us
I20260709 12:24:10.914908 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:10.916625 20728 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.917075 20881 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:10.917793 20701 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:10.864094 27463 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.918147 20701 leader_election.cc:403] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:10.824714 27344 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.863786 27316 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:10.861515 27462 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:10.920358 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:10.923615 27393 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:10.917804 27390 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.925048 27169 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.906570 27502 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.928217 27512 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:10.928789 27491 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:10.930584 26807 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:10.888689 27280 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:24:10.936453 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:10.938829 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:10.940310 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:10.941202 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.942219 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.923720 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:10.917279 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59897.8us
I20260709 12:24:10.961148 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:10.898618 27072 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:10.974629 27316 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.965818 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.965521 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:10.966897 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:10.972389 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:10.976632 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:10.990649 20728 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:10.993567 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.907011 27403 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.994869 20807 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:10.926218 27193 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.003448 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:10.959856 27477 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:10.918696 27337 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.062999 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:10.998785 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 84209.8us
I20260709 12:24:11.056109 20879 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
W20260709 12:24:11.065871 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.852356 27475 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.012665 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.016057 21431 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:24:11.015087 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:11.024749 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.025013 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:10.962219 27376 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:11.027664 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:10.965473 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:11.026254 20732 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.863556 27484 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.071604 20809 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:11.053364 20810 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
W20260709 12:24:11.037518 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 415592us
I20260709 12:24:11.052961 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.006084 27380 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:11.002737 27497 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:10.915812 27506 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.061259 20878 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:11.098737 27497 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.101436 27497 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.101553 27302 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:11.102036 27302 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:11.103271 27302 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.061506 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:11.108273 27344 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:11.113150 20732 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:11.114650 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.056362 20805 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:11.055794 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62321.8us
I20260709 12:24:11.072715 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.117673 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:11.012212 27500 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:11.073668 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:11.073763 20735 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:11.118571 20732 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:24:10.973337 27463 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.077800 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.080442 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:11.086205 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58660.4us
I20260709 12:24:11.005529 27393 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.004894 27390 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:10.997882 27075 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.092144 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.142110 20729 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:11.092258 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.094473 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:11.052412 27188 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:24:11.086889 27507 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.154125 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.115532 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.116241 27281 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:11.077085 20803 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:11.073380 21431 pending_rounds.cc:85] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 13
I20260709 12:24:11.026235 27491 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.054486 27316 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:11.119762 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94834us
I20260709 12:24:11.012254 27462 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.026482 27512 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.056775 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:11.177134 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:11.178045 20884 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:11.060388 27403 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:11.132167 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:11.132463 20699 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:11.133359 20806 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:11.087065 27373 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 12:24:11.138152 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:11.146418 27024 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:11.012545 27169 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.069486 27475 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.150298 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.068195 27394 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:11.103602 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:11.196270 21430 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.149776 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:11.198436 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 281891us
I20260709 12:24:11.158632 21431 pending_rounds.cc:107] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:24:11.199373 21431 raft_consensus.cc:2889] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:11.199592 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:11.131357 27230 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:11.135905 27344 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:11.125912 20728 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:11.170714 20734 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: [18.13-19.14]
I20260709 12:24:11.218693 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:11.168318 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.126858 27376 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:11.128019 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:11.221774 21429 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:11.181769 27491 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:11.186982 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:11.187999 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55689.4us
I20260709 12:24:11.263943 20801 raft_consensus.cc:2368] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22.
I20260709 12:24:11.264834 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:11.130915 27484 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.156220 27511 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:11.198313 20732 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:11.199061 21435 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:11.176923 27487 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.626s	user 0.012s	sys 0.000s
I20260709 12:24:11.265729 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:11.157752 27302 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:11.221479 26852 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:24:11.026895 27502 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.157255 27054 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:11.223371 20882 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 21
W20260709 12:24:11.225801 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 145496us
I20260709 12:24:11.226364 27528 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:11.227574 27394 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:11.228969 27393 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.228705 27376 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.224105 27430 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:11.225630 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49289.3us
I20260709 12:24:11.227174 27495 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:24:11.279739 27487 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.229389 27513 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:11.231688 27514 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:24:11.232369 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:44460 (request call id 1641) took 1273 ms. Trace:
I20260709 12:24:11.202664 27523 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.371630 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.239418 20734 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:11.245712 27501 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.245980 21429 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:11.369710 21430 rpcz_store.cc:276] 0709 12:24:09.930395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.930693 (+   298us) service_pool.cc:224] Handling call
0709 12:24:11.165473 (+1234780us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b5400690fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec96a3d 00007f9e23c66cd0 00007f9e1d1eb254 00007f9e1d1eb1e2 00007f9e1d1eb111 00007f9e1d1eb08c 00007f9e1d1ea342 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:11.198702 (+ 33229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178602368}
I20260709 12:24:11.250469 27503 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:11.253885 27158 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.193794 27281 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:11.256801 27512 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:11.257150 27273 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.193638 27515 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:11.274320 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.280336 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:11.145767 27500 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.423599 27075 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.425300 27500 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:11.281515 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57236.5us
I20260709 12:24:11.425869 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.153452 27386 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:24:11.427827 27386 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:11.428408 27386 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:11.291431 20805 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:11.430053 20810 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: [14.7-15.8]
I20260709 12:24:11.293573 20729 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:11.433706 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.434168 27487 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.434944 20727 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:11.435815 27462 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.432346 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.436810 27503 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:11.437309 27503 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:11.437544 27492 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 12:24:11.340931 27054 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:11.441048 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 160793us
I20260709 12:24:11.368050 27535 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:11.325305 20884 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:11.221961 27072 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:11.334916 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:11.335605 20878 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:11.169992 27507 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.198316 27373 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.346685 20732 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:24:11.453125 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 118315us
I20260709 12:24:11.348387 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:11.454236 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:11.454738 20875 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:11.285388 27344 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:11.292976 27044 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:11.295630 27382 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:11.438515 27503 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.366973 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:11.371209 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 152607us
I20260709 12:24:11.377699 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.462118 20734 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:11.328418 27528 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:11.380630 20882 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:11.373327 27513 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.378409 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.463888 21427 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:24:11.376472 26817 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.410275 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:11.411518 20807 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:11.415534 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:11.417603 27390 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.412843 27158 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.372723 27514 consensus_queue.cc:1243] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:24:11.426379 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:11.145275 27380 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 12:24:11.210246 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 81355.4us
I20260709 12:24:11.474797 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:11.476321 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:11.444903 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.359880 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:11.412985 27496 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.459784 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:11.387012 27506 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 12:24:11.450150 27525 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.451125 27493 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:11.479393 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:11.457782 27044 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.485982 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.470309 27487 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:11.465483 27273 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:11.469069 27344 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:11.487428 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:11.378340 27523 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.501516 21431 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:11.467270 27528 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:11.501274 20734 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:11.505817 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.484905 27501 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:11.485702 27492 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:11.514130 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 87834.3us
I20260709 12:24:11.488837 27044 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.488387 27493 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.525223 27493 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:11.522964 27281 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.527284 27492 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.527832 27492 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:11.528119 27344 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.524312 27158 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.531414 27463 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:11.482589 27496 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.454013 27535 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.494531 27433 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:11.497308 27430 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:11.511708 27481 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.073s	user 0.006s	sys 0.003s
I20260709 12:24:11.498327 27390 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:11.506007 27503 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.547895 27507 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.557926 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.554366 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:11.550518 21435 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
W20260709 12:24:11.604825 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51305.1us
I20260709 12:24:11.512028 27539 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:11.605109 21431 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:11.583060 27390 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:11.609246 27502 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.610095 20727 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:24:11.611884 27386 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.613422 27169 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.586055 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:11.640196 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54183.4us
I20260709 12:24:11.620824 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.622246 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.641901 20878 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:11.641664 20877 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:11.625965 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.488688 27525 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.626959 20875 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:11.623805 20735 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:11.549597 27430 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:11.641144 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:11.645777 20730 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:11.643543 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:11.567567 27054 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.658600 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.659310 20727 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.651485 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:11.663856 27230 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:11.664511 27230 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.665126 27230 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:11.665277 27525 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:11.672122 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:11.673115 27537 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:24:11.682566 27523 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.685811 27513 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.692025 27373 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:11.696957 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 38d64dd1339c4abfb4ef05669402e801: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:11.664201 27203 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:11.703516 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:11.710691 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69629.5us
I20260709 12:24:11.712540 20884 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:11.650386 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:11.729517 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 79274us
I20260709 12:24:11.730834 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:11.553936 27158 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:24:11.714339 27506 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:11.656080 27487 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:11.743536 20727 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:11.745046 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.657279 27430 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:11.751483 27506 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:11.714504 27507 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:11.755869 27507 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:11.756794 27507 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.757946 27507 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:11.758324 27513 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:11.759353 27513 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:11.740449 27380 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:11.761271 27513 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:11.761696 27513 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:11.764873 27487 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:11.765985 27487 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.768639 27497 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:11.769208 27497 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:11.762657 27513 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.773015 27513 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:11.773097 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.774029 20731 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:24:11.779800 27403 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.000s
I20260709 12:24:11.780340 27403 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.782750 27403 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.783944 27535 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:11.788972 27316 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.587s	user 0.005s	sys 0.005s
W20260709 12:24:11.789278 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.789486 27316 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.791909 27496 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:11.793030 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.793260 27380 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:11.794010 27380 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.799342 27380 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.800911 27528 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.775143 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.770237 27497 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.805473 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.809181 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:11.826944 20878 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:11.828509 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.651682 27539 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:11.712239 27203 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.852859 27203 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:11.853382 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2753) took 1011 ms. Trace:
I20260709 12:24:11.853786 20882 rpcz_store.cc:276] 0709 12:24:10.841610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.841855 (+   245us) service_pool.cc:224] Handling call
0709 12:24:11.853212 (+1011357us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:11.853323 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765172480}
I20260709 12:24:11.856594 21433 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:24:11.831660 20884 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:11.832078 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.858855 21429 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:11.859720 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:11.860850 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:11.833570 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:11.835441 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.836297 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.869504 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:11.879463 27169 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:11.880407 27072 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:11.838086 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107394us
I20260709 12:24:11.881544 20700 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:11.883369 20700 leader_election.cc:400] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22.
I20260709 12:24:11.883647 20700 leader_election.cc:403] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:11.839716 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:11.884249 20877 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.884507 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:11.886271 27072 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:11.889878 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:11.891403 27497 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.891997 27489 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:11.892580 27489 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:11.893445 27489 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.894119 21431 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:11.895068 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:11.897119 27548 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:11.897840 27548 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22.
I20260709 12:24:11.900976 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:11.901810 21433 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:11.905582 20806 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:11.907343 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:11.908985 27337 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.909693 27548 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:11.910300 27548 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.912395 27535 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:11.914024 27548 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:11.914108 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.844597 27539 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.845459 27528 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:11.846807 27491 consensus_meta.cc:287] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.001s
I20260709 12:24:11.836308 27475 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:24:11.847470 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:11.849728 27302 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.003s
I20260709 12:24:11.850777 27316 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:11.851970 27493 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:11.830873 20875 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:11.872071 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:11.873332 21428 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:11.875569 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:11.876935 27509 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:24:11.867039 27506 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:11.915450 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:11.917651 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:11.842065 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:11.921305 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:11.860481 27403 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:11.947168 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.962069 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:11.963407 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.964551 27403 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:11.965725 27403 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:11.955448 21430 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:11.967005 27403 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:11.971873 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:11.972854 20734 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:11.974146 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:11.975701 27475 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:11.976320 27475 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.976788 27475 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.982949 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:11.984778 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:11.986225 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:11.992923 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 117449us
W20260709 12:24:11.995766 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 153777us
I20260709 12:24:11.996860 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:11.957664 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85699.9us
I20260709 12:24:11.961144 21429 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:24:12.007302 20877 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
W20260709 12:24:12.010076 20726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.003s	sys 0.015s
W20260709 12:24:12.013653 20726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.004s	sys 0.016s
I20260709 12:24:12.014731 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:12.016211 20878 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:12.019555 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.023120 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:12.025085 20734 raft_consensus.cc:2368] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:24:12.026191 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:12.027051 20734 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:12.027304 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:12.028302 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:12.030385 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.031384 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:12.033907 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:12.035408 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 2999) took 1168 ms. Trace:
I20260709 12:24:12.037256 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:12.043244 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.045459 20735 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:24:12.020819 27029 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:24:11.934443 27044 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.047592 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:12.057900 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:12.011310 27072 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:12.017400 27280 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:12.017227 27513 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:12.018617 27492 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.042307 27516 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:12.070051 27492 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:12.053356 20726 rpcz_store.cc:276] 0709 12:24:10.850038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.850165 (+   127us) service_pool.cc:224] Handling call
0709 12:24:10.887653 (+ 37488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.960244 (+ 72591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.960300 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.011581 (+ 51281us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.328981 (+317400us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:11.468476 (+139495us) log.cc:844] Serialized 14 byte log entry
0709 12:24:12.014944 (+546468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.015026 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.015088 (+    62us) raft_consensus.cc:1685] finished
0709 12:24:12.015102 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.017881 (+  2779us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39623,"prepare.queue_time_us":146205,"prepare.run_cpu_time_us":329,"prepare.run_wall_time_us":5930,"raft.queue_time_us":44995,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":4443,"thread_start_us":908790,"threads_started":3,"wal-append.queue_time_us":357768}
I20260709 12:24:12.071552 21431 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [15.12-18.13]
I20260709 12:24:12.073326 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.075412 27158 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:12.075562 27403 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:12.076897 26852 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:12.056299 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:12.024771 27024 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:12.078331 27024 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:12.080101 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.079685 27024 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.011705 27302 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:12.058977 20734 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:12.081370 27302 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:12.082343 20734 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:24:11.949079 27491 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.083976 27500 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:12.084512 27230 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.085916 27491 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.057703 27495 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:12.060895 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.087458 20877 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:12.087549 20727 raft_consensus.cc:2368] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21.
I20260709 12:24:12.041836 27509 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:12.088603 27509 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:12.089823 27509 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.024276 27373 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:12.058543 27273 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:24:12.091527 27373 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.092653 20733 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 13: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:12.091980 27273 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:12.093854 27273 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:12.094210 27373 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:12.094455 27499 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:12.093880 27528 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:12.094805 27273 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.095625 27528 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:12.051224 27548 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:24:12.096249 27273 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:12.097019 27548 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:12.097431 27548 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:12.094271 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46787.2us
I20260709 12:24:12.098670 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:12.099509 27029 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:12.099975 27029 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:12.082391 27302 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.100477 27496 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:12.100917 27496 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.101173 20728 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17.
I20260709 12:24:12.101305 27496 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:12.101567 27280 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:12.102861 27203 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.100849 27029 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.104990 27029 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:12.105507 27203 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:12.106108 27382 consensus_queue.cc:1175] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 23 status { last_received { term: 21 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:12.096632 27528 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:12.110882 27541 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.675s	user 0.004s	sys 0.005s
I20260709 12:24:12.111742 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:12.112753 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.121629 20877 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:12.077164 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:12.116669 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:12.115331 20804 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:12.123421 20726 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:24:12.117659 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.125397 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:12.128047 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96768.4us
I20260709 12:24:12.129276 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:12.055141 20803 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:12.098533 27548 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:12.053740 20876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.017s	sys 0.012s
W20260709 12:24:12.143946 20876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.018s	sys 0.012s
I20260709 12:24:12.144850 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.145200 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.145733 20803 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.147619 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.148819 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.149144 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:12.154883 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.155546 20804 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:12.102607 27280 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.100903 27044 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:12.132611 27230 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.133323 27373 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.142086 27495 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:12.142462 27499 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:12.179070 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:12.055483 27430 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.197636 27499 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:12.193357 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.152782 27491 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:12.147387 27337 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:12.144624 27403 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:12.209839 27344 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:12.211833 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:12.278663 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66903.9us
I20260709 12:24:12.279616 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.280565 20877 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.280669 20804 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:12.280824 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:12.216651 20878 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:12.220118 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:12.283865 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.602s (SENT) [suppressed 16 similar messages]
I20260709 12:24:12.215157 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:12.223356 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:12.227754 20809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2719) took 1356 ms (client timeout 1602 ms). Trace:
I20260709 12:24:12.230011 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.229899 27499 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.231587 27493 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.204577 27554 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:24:12.236234 27535 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:12.195712 27548 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:12.236807 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2677) took 1358 ms. Trace:
I20260709 12:24:12.238072 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:12.239018 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.332630 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:12.197257 27373 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.341048 27373 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.244656 21435 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:12.226975 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1075) took 1370 ms. Trace:
I20260709 12:24:12.341732 20730 rpcz_store.cc:276] 0709 12:24:10.857019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.169764 (+312745us) service_pool.cc:224] Handling call
0709 12:24:11.645471 (+475707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.652529 (+  7058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.226741 (+574212us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54003acfd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:12.226901 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014415104}
I20260709 12:24:12.260783 20801 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:12.244606 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2976) took 1390 ms. Trace:
I20260709 12:24:12.343079 20802 rpcz_store.cc:276] 0709 12:24:10.854071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.854195 (+   124us) service_pool.cc:224] Handling call
0709 12:24:10.895996 (+ 41801us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.058513 (+162517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.058544 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.240739 (+182195us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b180009a600. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a
0709 12:24:11.241230 (+   491us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:24:11.241834 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.353315 (+111481us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:11.353361 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.353443 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.110559 (+757116us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b2400168758. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf109d 00007f9e1e82310e 00007f9e1e83a88c 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221
0709 12:24:12.244313 (+133754us) raft_consensus.cc:1685] finished
0709 12:24:12.244345 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.244529 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23219,"raft.queue_time_us":160502,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":8966,"spinlock_wait_cycles":27435008,"thread_start_us":265252,"threads_started":2,"wal-append.queue_time_us":193341}
W20260709 12:24:12.264611 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.163055 27503 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:12.288044 20880 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:12.225908 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2715) took 1355 ms. Trace:
I20260709 12:24:12.291543 20803 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:12.296541 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:12.349135 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:12.349637 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53236.4us
I20260709 12:24:12.348435 20810 rpcz_store.cc:276] 0709 12:24:10.870274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.198147 (+327873us) service_pool.cc:224] Handling call
0709 12:24:11.429636 (+231489us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:11.430627 (+   991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.225617 (+794990us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540006ccd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:12.225825 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364676480}
I20260709 12:24:12.297683 27075 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.298604 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.357823 20878 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:12.299350 20731 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
W20260709 12:24:12.286935 20809 rpcz_store.cc:269] 0709 12:24:10.870964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.910761 (+ 39797us) service_pool.cc:224] Handling call
0709 12:24:12.227512 (+1316751us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54002bff54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:12.227642 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366795008}
I20260709 12:24:12.161808 27158 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:24:12.319909 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:12.210354 27390 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.206657 27495 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:12.164304 27496 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:12.334112 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2713) took 1464 ms (client timeout 1938 ms). Trace:
I20260709 12:24:12.329231 20733 rpcz_store.cc:276] 0709 12:24:10.878491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.143505 (+265014us) service_pool.cc:224] Handling call
0709 12:24:12.236535 (+1093030us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54003ad9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:12.236698 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2006161792}
I20260709 12:24:12.262702 27497 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:12.299775 27487 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.007s	sys 0.044s
I20260709 12:24:12.282517 27169 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:12.336416 20881 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:12.336684 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:12.388388 20883 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:12.263289 27337 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:12.325241 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.298169 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.360826 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:12.361897 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.391626 20805 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 13: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:12.333918 27514 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:12.367455 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.282945 27403 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.376035 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:12.370543 20807 rpcz_store.cc:269] 0709 12:24:10.869800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.061216 (+191416us) service_pool.cc:224] Handling call
0709 12:24:12.333873 (+1272657us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54003ca354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:12.334040 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650723712}
I20260709 12:24:12.380285 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.401458 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:12.347600 27029 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:12.349222 27475 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.363474 27503 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:12.365278 27535 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:12.298852 27344 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.402132 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:12.471095 20807 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:12.396046 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:12.379520 27487 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.008s	sys 0.045s
I20260709 12:24:12.405296 20802 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.402344 27491 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:12.393148 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 68013.4us
I20260709 12:24:12.462275 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:12.366779 27075 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.475301 20805 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:12.476200 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.402251 20803 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:12.390926 27337 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.471318 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:12.388206 27463 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
W20260709 12:24:12.473968 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 78026.1us
I20260709 12:24:12.478024 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.473724 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:12.488606 20810 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:12.381450 27497 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.399777 20880 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:12.383977 20877 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:12.467396 27516 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:12.509349 20807 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:24:12.462425 27514 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:12.464812 27373 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:12.408236 27029 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.510913 20807 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:12.510407 20805 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:24:12.490856 27502 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.013s	sys 0.015s
I20260709 12:24:12.518375 20804 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:12.553864 20804 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:12.600557 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:12.601401 21432 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:24:12.526466 20810 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:24:12.506500 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 478736us
W20260709 12:24:12.543502 27502 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.013s	sys 0.015s
I20260709 12:24:12.489868 27193 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:12.565184 27195 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.570313 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1081) took 1011 ms. Trace:
I20260709 12:24:12.571646 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:12.571703 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2979) took 1672 ms. Trace:
I20260709 12:24:12.608206 20808 rpcz_store.cc:276] 0709 12:24:10.899694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.226638 (+326944us) service_pool.cc:224] Handling call
0709 12:24:11.227069 (+   431us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:11.550808 (+323739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.550850 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:11.551696 (+   846us) log.cc:844] Serialized 78 byte log entry
0709 12:24:11.860819 (+309123us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:11.878300 (+ 17481us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.878749 (+   449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.571118 (+692369us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:12.571206 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.571280 (+    74us) raft_consensus.cc:1685] finished
0709 12:24:12.571403 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.571627 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8069,"prepare.queue_time_us":21829,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":8617,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":210,"spinlock_wait_cycles":713020672,"thread_start_us":308779,"threads_started":1,"wal-append.queue_time_us":508956}
I20260709 12:24:12.573792 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:12.492097 27158 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:12.513075 27044 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:12.589378 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:12.611809 21429 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:12.525125 20805 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:12.473167 27535 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.519447 27487 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:12.615469 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:12.616762 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.606235 20727 rpcz_store.cc:276] 0709 12:24:11.558810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.744707 (+185897us) service_pool.cc:224] Handling call
0709 12:24:12.570064 (+825357us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54001d7754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:12.570233 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785538304}
I20260709 12:24:12.519253 27491 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
W20260709 12:24:12.627880 27558 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:24:12.629434 27514 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.627812 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.632273 27539 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:12.634683 27559 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:24:12.587571 27169 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:24:12.635776 27555 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 12:24:12.527782 27373 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.646025 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2668) took 1771 ms. Trace:
W20260709 12:24:12.597146 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 252845us
I20260709 12:24:12.635712 27169 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:12.654012 20878 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:12.657145 21432 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:24:12.626933 27557 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
W20260709 12:24:12.669266 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 385383us
I20260709 12:24:12.670734 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:12.671473 21431 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:12.646620 20729 rpcz_store.cc:276] 0709 12:24:10.875120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.437373 (+562253us) service_pool.cc:224] Handling call
0709 12:24:11.437891 (+   518us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:11.439724 (+  1833us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:11.947271 (+507547us) log.cc:844] Serialized 14 byte log entry
0709 12:24:12.369928 (+422657us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:12.392435 (+ 22507us) log.cc:844] Serialized 52 byte log entry
0709 12:24:12.392627 (+   192us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:12.392661 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.392736 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.645699 (+252963us) raft_consensus.cc:1685] finished
0709 12:24:12.645730 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.645946 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78588,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1976,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":242,"thread_start_us":872836,"threads_started":2,"wal-append.queue_time_us":611186}
I20260709 12:24:12.493769 27495 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:12.682440 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.688086 21429 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:12.696627 20878 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:12.701475 20877 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:12.669790 27430 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.718957 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:12.726971 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.724407 27561 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:12.598699 27503 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:12.658339 27195 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.659572 27487 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.713363 27496 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.709182 27491 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:12.749617 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:12.899288 21432 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:12.769606 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.721606 27502 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.145s
I20260709 12:24:12.876914 20808 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:12.715515 27515 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:12.879844 20805 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
W20260709 12:24:12.881114 27573 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:24:12.733466 27564 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.156s
I20260709 12:24:12.881441 27478 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:12.882897 27195 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:12.729014 26852 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.144s
I20260709 12:24:12.733242 27561 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:12.910810 27496 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:12.896441 20802 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:12.873454 21435 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:12.886370 27337 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:12.905025 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.009s	sys 0.013s
W20260709 12:24:12.949057 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.009s	sys 0.013s
W20260709 12:24:12.949488 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 629591us
I20260709 12:24:12.899633 27484 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:12.899919 27475 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:12.908188 27344 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:12.942694 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.945492 21432 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:12.953218 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:12.965584 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:12.911406 27491 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.936007 27515 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:12.801746 27169 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:12.961671 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 53730.8us
W20260709 12:24:12.970235 20883 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.000s
W20260709 12:24:12.971077 27273 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.007s
I20260709 12:24:12.945116 27029 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.919836 27578 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:24:12.953019 27561 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:12.954114 27484 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:13.005640 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:12.974525 27075 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.976064 20883 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:12.973014 27195 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:12.983808 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.020231 21432 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:12.986503 27587 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:12.987227 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.024431 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.024309 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:12.991549 27497 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:12.994146 27337 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.032115 27584 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.995405 27029 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.032837 27029 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:13.032770 27584 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.033396 27029 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:12.997421 27571 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:13.035400 27584 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:12.998080 27567 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:24:12.948707 27478 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:12.966599 27496 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.024663 20806 raft_consensus.cc:2376] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24. Re-sending same reply.
I20260709 12:24:13.044003 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2725) took 1134 ms. Trace:
I20260709 12:24:13.044369 20806 rpcz_store.cc:276] 0709 12:24:11.909816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.911975 (+  2159us) service_pool.cc:224] Handling call
0709 12:24:12.987050 (+1075075us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:24:13.043930 (+ 56880us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16607488}
I20260709 12:24:12.992609 27487 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.019645 27195 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:13.045212 27496 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:12.995100 27583 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:13.030321 27587 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.040536 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.046931 20802 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:13.007408 27273 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.052841 20803 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:13.056247 20808 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:24:13.060088 20805 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:13.046061 27195 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:13.062127 27548 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.001s
I20260709 12:24:13.064167 27587 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.066903 27273 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.019096 27578 consensus_queue.cc:1243] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:13.045132 27478 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.077129 27072 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:13.086382 27535 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.086437 26852 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:24:13.087791 21430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1162) took 1697 ms (client timeout 1708 ms). Trace:
I20260709 12:24:13.088119 27373 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:13.088311 21430 rpcz_store.cc:269] 0709 12:24:11.390033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.425435 (+ 35402us) service_pool.cc:224] Handling call
0709 12:24:13.087582 (+1662147us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400729a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:13.087720 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183034240}
I20260709 12:24:13.090417 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:13.092286 27373 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:13.092386 20728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2680) took 1838 ms (client timeout 1553 ms). Trace:
I20260709 12:24:13.092924 27373 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:13.093037 20728 rpcz_store.cc:269] 0709 12:24:11.253675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.274018 (+ 20343us) service_pool.cc:224] Handling call
0709 12:24:13.092108 (+1818090us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54002e4fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:13.092292 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1581188352}
I20260709 12:24:13.093631 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:13.055809 27382 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:13.094031 27373 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.095029 27484 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.096324 27484 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:13.099175 20806 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:24:13.099645 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:13.100662 27548 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:13.101516 27491 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.101274 27548 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.102173 27491 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:13.102731 27491 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:13.104171 27548 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.105396 27382 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:13.105516 27489 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:13.106573 27382 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.108076 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:13.110140 27591 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:24:13.110544 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:13.111632 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:13.111933 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 146440us
I20260709 12:24:13.113261 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.113358 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:13.056205 27415 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:13.116685 21430 raft_consensus.cc:2368] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:13.116631 27415 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:13.111466 27587 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:13.117316 20882 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:13.117585 27587 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:13.117831 27415 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.118435 27587 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.114044 27548 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:13.116663 21434 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:13.120170 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.121107 20877 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:24:13.122819 21400 leader_election.cc:400] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21.
I20260709 12:24:13.124300 21400 leader_election.cc:403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:13.124536 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:13.124893 20734 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:24:13.125089 20877 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:13.126370 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:13.127876 20727 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:13.127990 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.128865 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.129900 20726 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:13.113982 27565 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:24:13.130642 27072 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:13.131067 27072 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:13.130049 27569 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.400s	user 0.001s	sys 0.001s
I20260709 12:24:13.135434 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.132999 20882 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:24:13.134888 20728 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:13.138051 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.138118 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:13.139551 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.140475 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:13.147291 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.148696 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.149040 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.149796 20807 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:13.157132 20877 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:13.119124 27415 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:13.160180 20728 pending_rounds.cc:85] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 9
I20260709 12:24:13.182451 20728 pending_rounds.cc:107] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:24:13.182842 20728 raft_consensus.cc:2889] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.185626 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.173842 27548 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:13.113433 27273 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.191908 20730 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:13.134994 27499 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.193321 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.132797 27535 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21.
I20260709 12:24:13.195119 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.145656 27478 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.197085 20733 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:13.132030 27072 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.199049 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b31661191c14b44ac964555853900f1"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:13.220399 20877 raft_consensus.cc:606] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:13.220896 20877 raft_consensus.cc:614] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:13.213498 27561 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.218508 20807 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:13.218281 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.222813 20807 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:13.224325 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.225757 20807 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:13.230057 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:13.236153 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:13.237602 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 95378.5us
I20260709 12:24:13.180902 27316 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:13.238586 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:13.242216 20810 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:13.149892 27583 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:24:13.246991 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.249451 20734 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:13.157280 27393 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:24:13.247543 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:13.249464 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.253362 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.241319 27273 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:13.255273 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:13.169696 27577 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:24:13.255344 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:13.246106 27570 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.365s	user 0.001s	sys 0.002s
I20260709 12:24:13.241914 27578 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.241945 27316 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.266495 27316 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:13.258591 27273 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.263140 20735 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:13.262661 27548 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.251709 27373 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:13.256865 27588 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:13.272886 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.254158 27072 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:13.252262 27592 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:13.259248 27514 raft_consensus.cc:2764] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:24:13.220506 27499 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.315006 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.240403 27478 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.291365 27578 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.290082 27588 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.315752 21427 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:13.240343 27491 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.303761 27587 raft_consensus.cc:2764] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:24:13.311347 27463 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:24:13.330308 27577 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:13.310936 27565 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:24:13.339920 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.314574 27576 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:13.360808 20883 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:13.321762 27514 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:13.328262 27588 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:13.356230 27577 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.390161 27386 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:13.390942 27578 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:13.391922 20731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2689) took 1627 ms (client timeout 1749 ms). Trace:
I20260709 12:24:13.410162 27577 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:13.410472 20731 rpcz_store.cc:269] 0709 12:24:11.764304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.774842 (+ 10538us) service_pool.cc:224] Handling call
0709 12:24:13.391707 (+1616865us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54000c44d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:13.391831 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1138889984}
I20260709 12:24:13.363373 27576 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:13.336710 27302 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:24:13.325279 27564 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:24:13.399724 27517 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:13.370640 27514 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:13.358372 27599 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:13.368331 27597 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:24:13.325991 27029 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.368602 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:13.438678 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2760) took 2296 ms. Trace:
W20260709 12:24:13.438809 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70349.1us
I20260709 12:24:13.310925 27592 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.450187 27592 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:13.455466 27561 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.454798 27514 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.457573 27478 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.457772 27514 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.458966 27273 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.459661 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:13.461328 27602 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:24:13.463310 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.463560 20701 leader_election.cc:400] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:13.464062 20701 leader_election.cc:403] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:13.464243 21434 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:13.420149 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.370244 27579 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:24:13.424063 20802 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:24:13.439321 20876 rpcz_store.cc:276] 0709 12:24:11.134567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.134745 (+   178us) service_pool.cc:224] Handling call
0709 12:24:11.135102 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.265247 (+130145us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:11.486917 (+221670us) log.cc:844] Serialized 14 byte log entry
0709 12:24:12.150250 (+663333us) log.cc:844] Serialized 14 byte log entry
0709 12:24:12.150459 (+   209us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:12.150507 (+    48us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:12.150558 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.048022 (+897464us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b180039eb60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:24:13.079105 (+ 31083us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b180039eb60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:13.430366 (+351261us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540005ad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:13.430439 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.430609 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5434,"prepare.queue_time_us":252767,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":3068,"raft.queue_time_us":5613,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":11744,"spinlock_wait_cycles":395043456,"thread_start_us":850530,"threads_started":2,"wal-append.queue_time_us":909641}
I20260709 12:24:13.423738 27517 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:13.388976 27515 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.445888 27599 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.469843 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.469998 20877 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:24:13.472896 21434 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:24:13.473249 20802 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 14
I20260709 12:24:13.513965 20802 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:24:13.514313 20802 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.428386 27499 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.506810 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.445266 27029 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.516880 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:13.517313 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:13.517907 20731 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.519129 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.522297 20728 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:13.522523 20735 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:13.524134 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:13.525942 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:13.470765 27478 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.526993 21430 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:13.534737 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:13.534950 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.535627 20733 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:13.537932 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:13.538676 20726 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:24:13.540225 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.541157 20735 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:13.545969 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.412007 27576 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.548029 27576 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:13.506631 27291 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:13.550199 27291 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:13.551360 27291 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.552558 27499 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.553339 27499 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:13.556259 27572 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.827s	user 0.002s	sys 0.000s
I20260709 12:24:13.558151 27478 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.560608 27478 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:13.560810 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.038s	sys 0.070s
W20260709 12:24:13.561230 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.039s	sys 0.071s
I20260709 12:24:13.561094 27302 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:24:13.561550 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 840608us
I20260709 12:24:13.564983 27583 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:13.565459 27583 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:13.566335 27583 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.568308 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2846) took 1354 ms. Trace:
I20260709 12:24:13.568704 20875 rpcz_store.cc:276] 0709 12:24:12.213469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.213733 (+   264us) service_pool.cc:224] Handling call
0709 12:24:12.214109 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.492574 (+278465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.492605 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:12.492655 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:12.492695 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.568049 (+1075354us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:13.568101 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.568246 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639943424}
I20260709 12:24:13.571910 20882 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 12:24:13.574401 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:13.576455 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.569751 27515 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.520156 27380 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:13.578228 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 333015us
I20260709 12:24:13.588819 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:13.579746 20728 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:13.579754 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.591948 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.592278 20727 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:24:13.589360 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:13.590309 20849 leader_election.cc:400] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:24:13.590338 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.580802 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.593000 20849 leader_election.cc:403] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:13.593809 20882 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:24:13.523783 27587 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:24:13.596588 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.601013 21430 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:13.595615 20733 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:13.602910 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:13.603535 20875 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:13.604202 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:13.597329 27602 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:24:13.614944 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.616815 20875 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:24:13.587862 26817 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:13.578081 27430 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.589042 27565 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.562983 27316 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.599707 27072 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:24:13.599860 27592 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:13.560771 27024 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:24:13.626214 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:13.630285 21432 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:13.631706 27273 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:13.632145 27273 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.632483 27273 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:13.632624 27316 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:13.633538 27553 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.163s	user 0.006s	sys 0.000s
I20260709 12:24:13.617277 27499 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.600301 27601 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:24:13.619654 27478 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:24:13.637247 26852 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.644361 26852 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:13.648818 27568 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.776s	user 0.000s	sys 0.004s
I20260709 12:24:13.648994 21426 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 12:24:13.649962 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.609354 27579 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:24:13.626323 27430 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.639180 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2669) took 2763 ms. Trace:
I20260709 12:24:13.653518 27554 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:13.654479 27554 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:13.655026 27554 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:13.655611 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:13.656311 20880 raft_consensus.cc:606] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:13.656488 27554 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:13.657395 27562 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.191s	user 0.001s	sys 0.002s
I20260709 12:24:13.657840 27554 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:13.627432 27565 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.658440 20876 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:24:13.659566 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.603245 27517 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.660241 27565 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:13.654138 20732 rpcz_store.cc:276] 0709 12:24:10.875539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.453532 (+577993us) service_pool.cc:224] Handling call
0709 12:24:11.454026 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.640893 (+186867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.640924 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.651660 (+ 10736us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.923163 (+271503us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:11.923248 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.923340 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.632942 (+1709602us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:13.635695 (+  2753us) raft_consensus.cc:1685] finished
0709 12:24:13.635727 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.639101 (+  3374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11852,"raft.queue_time_us":21919,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":4658,"spinlock_wait_cycles":59099904,"thread_start_us":441604,"threads_started":2,"wal-append.queue_time_us":710234}
I20260709 12:24:13.669369 27403 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.670432 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:13.671591 27599 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.672502 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:13.673789 27390 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:13.673930 27502 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:13.675420 27230 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:13.675848 27230 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:13.680832 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.699518 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1197) took 1578 ms. Trace:
I20260709 12:24:13.725327 20803 rpcz_store.cc:276] 0709 12:24:12.120914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.491660 (+370746us) service_pool.cc:224] Handling call
0709 12:24:12.491968 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.053483 (+561515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.699202 (+645719us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b540006d1d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:13.699439 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289251712}
W20260709 12:24:13.726038 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet c6fe576036c04c149240cd6107285eca: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:24:13.700794 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1203) took 1578 ms (client timeout 1864 ms). Trace:
I20260709 12:24:13.706941 20731 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:13.713925 27230 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.715574 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:13.726820 20804 rpcz_store.cc:269] 0709 12:24:12.122643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.297906 (+175263us) service_pool.cc:224] Handling call
0709 12:24:13.700536 (+1402630us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b540027a954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:13.700692 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128918912}
I20260709 12:24:13.716735 27195 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.718168 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:13.722703 27273 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.727895 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:13.729775 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:13.731036 20880 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:13.737277 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:13.702895 27380 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16.
I20260709 12:24:13.750432 20804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:13.753245 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:13.754618 20804 raft_consensus.cc:606] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:13.765002 27580 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.804s	user 0.002s	sys 0.000s
I20260709 12:24:13.764799 27588 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.768131 27592 raft_consensus.cc:993] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:24:13.768778 27592 raft_consensus.cc:1081] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
I20260709 12:24:13.769003 27373 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.769778 27373 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.751873 27514 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:13.773974 27514 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:13.774381 27514 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.774544 27273 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:13.775838 27588 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.776549 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:13.776973 27592 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:24:13.779820 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 200420us
I20260709 12:24:13.789088 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1307) took 1775 ms. Trace:
I20260709 12:24:13.654362 27430 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:13.793265 21429 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:24:13.793610 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.798091 20804 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:24:13.792503 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:13.749254 27509 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:13.789651 21428 rpcz_store.cc:276] 0709 12:24:12.013405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.013769 (+   364us) service_pool.cc:224] Handling call
0709 12:24:12.037117 (+ 23348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.481984 (+444867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.482029 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.482638 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.674157 (+191519us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:12.674198 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.674271 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.764276 (+1090005us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:13.782329 (+ 18053us) raft_consensus.cc:1685] finished
0709 12:24:13.782361 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.789027 (+  6666us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33155,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":11701504,"thread_start_us":592312,"threads_started":2,"wal-append.queue_time_us":437254}
W20260709 12:24:13.807772 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.751365 27496 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:13.810168 21435 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:13.751899 27566 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:13.814404 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:13.759963 27302 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:13.730677 27463 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.063s
W20260709 12:24:13.821102 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 6a458432a57843e4af30e5d9af5dfcfe: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:13.821938 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.822945 20731 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:13.819351 21426 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:24:13.824081 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.824899 20733 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:24:13.825264 21435 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [15.12-18.13]
I20260709 12:24:13.826514 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:13.827369 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:13.829408 20733 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:24:13.790918 27273 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.834810 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.835343 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.788475 27024 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:13.839301 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.840114 20802 raft_consensus.cc:2368] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16.
I20260709 12:24:13.840047 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:13.806188 27507 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.838634 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:13.784328 27478 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:24:13.850029 20733 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:24:13.852412 20733 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:13.852914 20733 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:13.846747 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:13.866425 20804 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:13.820293 27564 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.806010 27029 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:24:13.793375 27565 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:24:13.950183 20734 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:13.848425 27613 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:13.819947 27566 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:13.988831 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 141083us
I20260709 12:24:14.013747 20805 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:24:14.014446 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.845438 27273 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:13.949568 27024 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.024771 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:13.982901 27463 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:13.848100 27507 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:13.981424 27599 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:14.017306 27554 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.987682 27613 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:14.042915 20806 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:13.956952 27564 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:14.070099 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.029675 27390 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:14.100226 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69065.6us
I20260709 12:24:14.033344 27029 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:14.022749 27169 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:24:14.029111 27478 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:24:14.034400 27024 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.044742 27302 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:14.105387 27316 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:14.042609 27554 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.040694 27599 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.109052 27390 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.142006 27302 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.146420 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2788) took 1022 ms. Trace:
I20260709 12:24:14.147006 20883 rpcz_store.cc:276] 0709 12:24:13.123874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.124238 (+   364us) service_pool.cc:224] Handling call
0709 12:24:14.146169 (+1021931us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b540022ecd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:14.146335 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486219648}
W20260709 12:24:14.148788 27619 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:24:14.150446 27620 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:24:14.156385 27613 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.158295 27613 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:14.163945 27624 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:24:14.166754 27566 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.139128 27316 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:14.169502 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.170387 27316 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.105187 27484 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:24:14.172441 27484 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.172603 27554 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.121129 27585 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:24:14.173095 27484 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.175405 27024 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.176476 27564 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:14.178915 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:14.179647 20878 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:24:14.183933 21432 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:14.184973 27586 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:14.185384 27586 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:14.186506 27611 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:14.186228 27586 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.186882 27611 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:14.188550 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:14.189617 27554 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:14.190019 27554 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.190379 27554 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:14.192437 27507 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.194152 27024 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.137310 20733 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.194787 27024 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.195591 20726 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:14.196367 27273 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.196853 27024 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.198155 20733 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.199968 21429 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:24:14.201257 20875 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:24:14.202122 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.157209 27576 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:14.207365 27502 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.207736 20881 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:24:14.207883 27502 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.210544 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:14.211256 20880 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:14.215452 20884 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:14.218235 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:14.218988 20884 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:14.219383 20884 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:14.220480 20884 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.221655 20884 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:14.224105 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.224779 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:14.225210 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2998) took 1069 ms. Trace:
W20260709 12:24:14.229511 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 437100us
W20260709 12:24:14.233074 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 413897us
I20260709 12:24:14.234438 27509 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:14.236357 27509 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.236863 27509 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.240360 27373 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:14.241536 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.241871 27475 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.242404 27475 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.225764 20810 rpcz_store.cc:276] 0709 12:24:13.155786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.243039 (+ 87253us) service_pool.cc:224] Handling call
0709 12:24:13.243466 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.243723 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:13.359508 (+115785us) log.cc:844] Serialized 14 byte log entry
0709 12:24:13.798154 (+438646us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.159838 (+361684us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b180048c9a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1
0709 12:24:14.160565 (+   727us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b180048c9a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:14.224542 (+ 63977us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b180048c9a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:14.224733 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.224813 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:14.224885 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.224973 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.225129 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27032,"prepare.queue_time_us":198589,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":123,"raft.queue_time_us":16931,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":8350,"spinlock_wait_cycles":17937792,"thread_start_us":516234,"threads_started":2,"wal-append.queue_time_us":486837}
I20260709 12:24:14.243772 27291 raft_consensus.cc:993] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:24:14.212080 27576 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:14.245476 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2999) took 1077 ms. Trace:
W20260709 12:24:14.245942 27625 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.117s	user 0.003s	sys 0.000s
I20260709 12:24:14.245857 20808 rpcz_store.cc:276] 0709 12:24:13.168048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.394405 (+226357us) service_pool.cc:224] Handling call
0709 12:24:13.394863 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.395139 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:13.618919 (+223780us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.245115 (+626196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.245166 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:14.245221 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.245298 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.245407 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":352211,"prepare.queue_time_us":46737,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"thread_start_us":480503,"threads_started":2,"wal-append.queue_time_us":416229}
I20260709 12:24:14.247592 27385 raft_consensus.cc:1081] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:24:14.248396 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 2997) took 1094 ms. Trace:
I20260709 12:24:14.248799 20807 rpcz_store.cc:276] 0709 12:24:13.153637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.227034 (+ 73397us) service_pool.cc:224] Handling call
0709 12:24:13.249499 (+ 22465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.315529 (+ 66030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.315562 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:13.315604 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:13.315644 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.248138 (+932494us) spinlock_profiling.cc:243] Waited 65.8 ms on lock 0x7b54003a0f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:14.248201 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.248339 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151323392}
I20260709 12:24:14.245105 27576 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.250306 27230 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:14.250847 27576 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:14.252187 27484 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.253268 27514 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.240515 27496 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.255399 27496 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.256347 21431 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 16. (term mismatch)
I20260709 12:24:14.256709 21431 pending_rounds.cc:85] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:24:14.256937 21431 pending_rounds.cc:107] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 17.16
I20260709 12:24:14.257161 21431 raft_consensus.cc:2889] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:14.239172 27509 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.258271 27203 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.260097 27517 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:14.260720 27044 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:14.262202 27618 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.262135 27044 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:24:14.258752 27203 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:14.264218 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.264935 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.265543 27506 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:14.265579 27495 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:14.267292 27495 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:14.268122 27382 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.267874 27495 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.268663 27382 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.269816 21429 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44460
I20260709 12:24:14.270418 21429 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.270738 21429 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.267817 27273 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.262801 27618 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.272002 27273 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.274343 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:14.274950 21428 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:14.267130 27506 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:14.278398 27599 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.283994 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:14.286165 20883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:14.287107 20883 raft_consensus.cc:606] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:14.287649 20883 raft_consensus.cc:614] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:14.223680 20878 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:14.292028 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:14.291684 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60500.1us
I20260709 12:24:14.295104 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:14.298413 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.299118 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:14.299893 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:14.302043 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.303011 20882 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:14.304845 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.305639 20877 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:14.307621 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:14.258948 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1663) took 1275 ms. Trace:
I20260709 12:24:14.315733 21431 rpcz_store.cc:276] 0709 12:24:12.983282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.984483 (+  1201us) service_pool.cc:224] Handling call
0709 12:24:12.984892 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.257608 (+1272716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.258723 (+  1115us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540042ed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:14.258894 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2923111296}
I20260709 12:24:14.314878 20884 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:14.325603 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:14.327852 27273 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.328608 27484 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:14.329113 27484 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:14.329473 27496 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.330052 27612 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:14.329553 27484 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.330880 27380 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:24:14.333158 27380 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:14.336628 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:14.336436 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:14.336016 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:14.345944 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:14.356783 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.366374 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:14.366531 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.368945 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:14.379901 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:14.381533 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:14.382318 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:14.383428 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:14.384243 20728 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:14.385247 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:14.387419 20735 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:24:14.342252 21429 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.395026 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:14.397749 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.401284 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64732.7us
I20260709 12:24:14.334846 27487 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:14.293421 27583 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:14.327982 27495 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:14.429577 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46737.3us
I20260709 12:24:14.322839 27528 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:14.431066 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:14.434370 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 94151.4us
I20260709 12:24:14.356478 27072 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.360176 27029 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.438782 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:14.435472 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:14.439064 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:14.450484 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:14.480845 20804 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:14.481402 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:14.443389 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 117921us
I20260709 12:24:14.460021 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.476280 21429 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.361218 27506 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.482371 20881 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:14.401391 27280 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:14.360968 27523 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:24:14.491418 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60477.7us
I20260709 12:24:14.495960 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:14.363266 27514 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.493324 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:14.403613 27382 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.404101 27380 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:14.500491 20801 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:14.437091 27592 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:14.450904 27507 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.500509 20884 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:14.440869 27584 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.394951 27491 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 12:24:14.431690 27487 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:14.452489 27475 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.443849 27203 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.519073 27487 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:14.500819 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:14.520102 26852 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.443682 27495 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:14.490820 27029 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.477309 27585 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.517148 27612 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:14.521610 27373 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:14.521401 27585 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.521420 27612 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.522310 27373 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:14.523334 27373 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.499732 27599 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:14.525403 27599 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.525779 27599 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:14.526297 27507 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.518147 27584 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.387172 27576 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 12:24:14.528504 27507 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.528964 27584 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.520859 27385 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:14.531801 27611 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.512043 27380 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.487783 27554 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.486106 27072 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.536087 27592 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.516505 27523 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:14.515389 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 81929.2us
I20260709 12:24:14.392513 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:14.567070 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:14.570497 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:14.509330 27382 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:14.571091 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:14.585201 20807 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:24:14.575852 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:14.569372 27385 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.496702 27506 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.576304 20804 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:14.589893 27612 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.597175 27072 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.584474 27382 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.584789 27592 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:14.598749 21435 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:14.574023 27380 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:14.587258 27611 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:14.601354 27385 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:14.615042 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 492552us
I20260709 12:24:14.622068 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:14.611423 27382 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:14.678067 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:14.667591 20732 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:14.682760 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:14.686355 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:14.666167 27072 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:14.666383 27514 raft_consensus.cc:2764] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:24:14.688488 27029 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:14.706427 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:14.672772 27280 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:24:14.723528 27585 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:14.725571 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.729784 20877 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:24:14.733887 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.696870 27072 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.690563 27523 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:14.805840 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.766203 27502 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:14.818066 20700 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.022s
W20260709 12:24:14.820358 20700 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.000s	sys 0.022s
W20260709 12:24:14.820758 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.820680 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 515054us
W20260709 12:24:14.822741 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.823899 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 421079us
I20260709 12:24:14.824856 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:14.784721 27554 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:14.790047 27506 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:14.793104 27390 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:14.866632 21432 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.016s	sys 0.015s
W20260709 12:24:14.878839 21432 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.016s	sys 0.016s
W20260709 12:24:14.877534 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.816612 27054 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.884296 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.884491 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.885658 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.892941 20882 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:24:14.885974 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.898741 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:14.898766 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:14.900638 20731 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.792541 27599 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.841470 27588 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:14.847627 27514 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:14.868137 27586 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:14.920001 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:14.934207 20728 raft_consensus.cc:2368] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:24:14.937583 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:14.938796 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:14.908300 27054 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.924430 27514 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:14.966563 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 531164us
W20260709 12:24:14.967382 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 528351us
I20260709 12:24:14.978827 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:14.927258 27588 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:14.914469 27487 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:14.947810 27024 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:14.979449 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:15.048575 21435 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
W20260709 12:24:14.978979 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.035072 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.011s	sys 0.030s
W20260709 12:24:15.051842 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.011s	sys 0.030s
I20260709 12:24:15.028731 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:15.028733 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:15.052242 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 569448us
I20260709 12:24:14.979444 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:15.067056 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:15.067775 20806 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:15.015041 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:15.067991 20875 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:15.001999 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.067937 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:15.072371 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:15.074138 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59207.7us
W20260709 12:24:15.040619 27643 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
W20260709 12:24:15.031973 27344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.023s	sys 0.022s
I20260709 12:24:15.033054 27660 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.093s
I20260709 12:24:15.013162 27044 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:15.014014 27230 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.763s	user 0.014s	sys 0.003s
W20260709 12:24:15.015816 27528 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.001s
I20260709 12:24:15.111775 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.130034 20801 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:15.124320 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:15.049099 27024 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:15.138638 20875 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:15.083868 27535 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:15.084340 27564 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:15.086139 27640 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 12:24:15.013190 27503 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:15.013752 27514 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:15.124167 20726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.014s	sys 0.009s
I20260709 12:24:15.091115 27380 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.111315 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:15.134330 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:15.140209 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:15.120615 27635 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.616s	user 0.006s	sys 0.000s
W20260709 12:24:15.112921 27344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.023s	sys 0.022s
W20260709 12:24:15.107697 27651 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 12:24:15.128158 27169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.014s	sys 0.057s
W20260709 12:24:15.127302 27484 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.795s	user 0.016s	sys 0.001s
I20260709 12:24:15.146848 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.147576 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.257966 20876 raft_consensus.cc:2368] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:15.100332 27599 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:15.112388 27618 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.102398 27487 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:15.047034 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.006s	sys 0.014s
I20260709 12:24:15.076570 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:15.171031 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1195) took 1700 ms. Trace:
I20260709 12:24:15.122292 27072 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.221180 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:15.272850 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.277623 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 201146us
W20260709 12:24:15.200172 20878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.015s	sys 0.016s
W20260709 12:24:15.221623 20700 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.602s (SENT)
I20260709 12:24:15.123399 27507 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:15.111351 27495 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.278878 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:15.078514 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:15.236261 20726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.015s	sys 0.009s
I20260709 12:24:15.111250 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:15.236312 20805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.035s	sys 0.039s
W20260709 12:24:15.291939 20805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.035s	sys 0.039s
I20260709 12:24:15.119968 27044 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.239610 20807 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:15.293191 20807 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:15.239913 21434 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.238273 20809 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
W20260709 12:24:15.250083 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.260238 27385 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
I20260709 12:24:15.239960 27535 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:15.257190 20882 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
W20260709 12:24:15.245577 27382 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.563s	user 0.022s	sys 0.000s
I20260709 12:24:15.235690 27576 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:15.294296 20807 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.240244 27503 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:15.279646 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58060.8us
I20260709 12:24:15.245584 27655 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:15.280376 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.007s	sys 0.014s
W20260709 12:24:15.332649 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 836698us
W20260709 12:24:15.254721 27169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.014s	sys 0.057s
I20260709 12:24:15.277412 20879 rpcz_store.cc:276] 0709 12:24:13.470607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.470919 (+   312us) service_pool.cc:224] Handling call
0709 12:24:14.147638 (+676719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.148062 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.148095 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.148610 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.510643 (+362033us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:14.510709 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.510799 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.907488 (+396689us) raft_consensus.cc:1685] finished
0709 12:24:14.907513 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.147283 (+239770us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b540022ecd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:15.170924 (+ 23641us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2350,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":1555349760,"thread_start_us":361591,"threads_started":1,"wal-append.queue_time_us":632356}
I20260709 12:24:15.334753 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:15.296264 27660 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:15.278589 27024 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.299681 20876 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:24:15.296288 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 185112us
I20260709 12:24:15.302580 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:15.258543 20729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.018s	sys 0.009s
W20260709 12:24:15.347563 20729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.018s	sys 0.009s
I20260709 12:24:15.296507 27273 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.349478 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3032) took 1238 ms. Trace:
I20260709 12:24:15.350760 20879 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:15.261232 27484 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:15.338023 27611 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.005s
I20260709 12:24:15.282233 27507 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.285300 27528 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:24:15.330742 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet c1303aa66bdb4474b6bcb604461e74a5: Illegal state: Replica 657bc62d20374e539b23641715f22745 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:24:15.363548 20809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 2983) took 3210 ms (client timeout 2046 ms). Trace:
W20260709 12:24:15.363988 20809 rpcz_store.cc:269] 0709 12:24:12.153472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.380000 (+226528us) service_pool.cc:224] Handling call
0709 12:24:15.363349 (+2983349us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b54002b57d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:15.363471 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6252685056}
I20260709 12:24:15.278576 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:15.354689 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.355572 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:15.280627 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.020s	sys 0.015s
W20260709 12:24:15.356817 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.279268 20878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.016s	sys 0.018s
W20260709 12:24:15.389103 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.021s	sys 0.015s
I20260709 12:24:15.360800 27502 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:15.361708 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:15.304541 27385 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.342235 27655 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.351972 27663 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:15.397866 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 458876us
I20260709 12:24:15.296790 27495 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.396445 27528 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.390779 27588 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:15.404428 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.349972 20729 rpcz_store.cc:276] 0709 12:24:14.110963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.111180 (+   217us) service_pool.cc:224] Handling call
0709 12:24:14.125654 (+ 14474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.125890 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:14.614389 (+488499us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.348979 (+734590us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.349148 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.349192 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.349255 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.349318 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.349426 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3675,"prepare.queue_time_us":597441,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":29,"thread_start_us":1207729,"threads_started":2}
I20260709 12:24:15.339401 27660 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:15.406890 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45286.7us
I20260709 12:24:15.355207 27373 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:15.413326 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:15.367331 27652 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:15.331226 27382 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.356289 27273 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.330453 27535 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.392834 27507 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:15.401295 27054 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:15.472043 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58799.9us
I20260709 12:24:15.483906 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.457307 27528 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:15.339813 27576 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.406224 27510 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.452503 27203 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:15.480783 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.421396 27663 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.482847 20849 leader_election.cc:400] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 15. Current term is 16.
I20260709 12:24:15.484174 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:15.484802 20881 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:15.475414 20803 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:15.471987 27660 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:15.488070 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:15.556494 20882 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 20
W20260709 12:24:15.482400 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 261277us
I20260709 12:24:15.462669 27618 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.475759 27430 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.476569 27072 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:15.548390 20879 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:15.553192 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70480.5us
W20260709 12:24:15.554538 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70464.3us
I20260709 12:24:15.558707 21399 leader_election.cc:400] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:15.558633 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:15.559147 21399 leader_election.cc:403] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:15.558940 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:15.560020 20879 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.564635 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:15.493076 21432 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.002s	sys 0.042s
W20260709 12:24:15.567727 21432 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.003s	sys 0.042s
W20260709 12:24:15.600337 27657 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 12:24:15.601348 27658 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:24:15.601238 27655 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:15.602596 27659 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.253s	user 0.000s	sys 0.000s
I20260709 12:24:15.605311 27672 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:15.606279 27672 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:24:15.618129 27514 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.625310 20880 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:15.625811 27385 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:15.631421 27554 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:15.633725 27554 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:15.633119 27654 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:15.638993 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1097) took 1270 ms. Trace:
I20260709 12:24:15.639600 20726 rpcz_store.cc:276] 0709 12:24:14.368225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.371041 (+  2816us) service_pool.cc:224] Handling call
0709 12:24:14.371555 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.397465 (+ 25910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.397521 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.398083 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.291373 (+893290us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:15.291431 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.291529 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.638758 (+347229us) raft_consensus.cc:1685] finished
0709 12:24:15.638779 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.638928 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25295,"raft.queue_time_us":59463,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":40511,"thread_start_us":802056,"threads_started":1,"wal-append.queue_time_us":1018132}
I20260709 12:24:15.632606 27652 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:15.476806 27373 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.648515 27373 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:15.649494 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1675) took 1369 ms. Trace:
I20260709 12:24:15.650089 21428 rpcz_store.cc:276] 0709 12:24:14.280446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.285043 (+  4597us) service_pool.cc:224] Handling call
0709 12:24:14.322866 (+ 37823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.323352 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.323395 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.362567 (+ 39172us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.949190 (+586623us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:14.949239 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.949315 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.649186 (+699871us) raft_consensus.cc:1685] finished
0709 12:24:15.649218 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.649414 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9719,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":495992,"threads_started":1,"wal-append.queue_time_us":847365}
I20260709 12:24:15.568302 27618 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.655807 27618 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.564461 27203 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:15.658983 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2880) took 1170 ms. Trace:
I20260709 12:24:15.547098 27663 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:15.660064 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.659456 20883 rpcz_store.cc:276] 0709 12:24:14.488969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.489395 (+   426us) service_pool.cc:224] Handling call
0709 12:24:14.512542 (+ 23147us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:14.545800 (+ 33258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.545847 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:14.572697 (+ 26850us) log.cc:844] Serialized 52 byte log entry
0709 12:24:15.043612 (+470915us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.044813 (+  1201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.465727 (+420914us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:15.465799 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.658636 (+192837us) raft_consensus.cc:1685] finished
0709 12:24:15.658660 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.658916 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":715,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":2753,"spinlock_wait_cycles":75134848,"thread_start_us":436862,"threads_started":1,"wal-append.queue_time_us":968351}
I20260709 12:24:15.635814 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:15.600528 27662 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.251s	user 0.000s	sys 0.000s
I20260709 12:24:15.677088 20883 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:15.555130 27599 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:15.567301 27380 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.693336 20883 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:15.694784 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:15.661178 20881 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:24:15.636462 27044 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:15.634495 27495 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.582719 27586 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:15.701524 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.562718 27072 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:15.568591 27054 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:15.705138 27344 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:15.641079 27382 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.547706 27510 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.751803 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:15.752388 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.686817 27612 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:15.654989 27554 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:15.581772 27646 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:24:15.697669 20882 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:15.677409 27484 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:15.699654 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1676) took 1418 ms. Trace:
I20260709 12:24:15.679997 21434 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:15.702291 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1224) took 1250 ms. Trace:
I20260709 12:24:15.797891 20802 rpcz_store.cc:276] 0709 12:24:14.452565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.454895 (+  2330us) service_pool.cc:224] Handling call
0709 12:24:14.455304 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.908754 (+453450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.908784 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.909270 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.357557 (+448287us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.357616 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.357709 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.702052 (+344343us) raft_consensus.cc:1685] finished
0709 12:24:15.702073 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.702231 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43361,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":825532,"threads_started":2,"wal-append.queue_time_us":618669}
I20260709 12:24:15.799888 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:15.636559 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 558102us
I20260709 12:24:15.802304 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:15.802523 20849 leader_election.cc:400] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:24:15.802920 20849 leader_election.cc:403] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:24:15.704178 27599 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.705286 27576 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:15.698908 27203 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.708173 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1231) took 1139 ms. Trace:
I20260709 12:24:15.708360 20808 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
W20260709 12:24:15.686007 27592 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.066s	user 0.010s	sys 0.005s
I20260709 12:24:15.685539 20879 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
W20260709 12:24:15.689709 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.761675 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.762487 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.570269 27430 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:15.725605 27650 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:24:15.726967 27496 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.025s	sys 0.122s
I20260709 12:24:15.777958 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:15.779206 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.727713 27495 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.728600 27072 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:15.776952 27291 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:24:15.754146 27054 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:15.767256 27506 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:15.828794 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.829733 20881 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 16: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:15.830005 27430 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:15.824573 27496 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.025s	sys 0.122s
I20260709 12:24:15.828812 27506 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.833993 27506 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.821995 27592 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.838557 20875 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:15.839767 27592 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:15.806185 27382 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.770769 27523 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:15.770285 27586 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:15.800793 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.846331 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:15.803768 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:15.809118 20803 rpcz_store.cc:276] 0709 12:24:14.569150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.570850 (+  1700us) service_pool.cc:224] Handling call
0709 12:24:15.707935 (+1137085us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54002b5cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:15.708104 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2038231296}
I20260709 12:24:15.767611 27655 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:24:15.814033 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:15.814729 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:15.852614 20879 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:15.817750 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 657bc62d20374e539b23641715f22745 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:15.856164 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:15.810017 27554 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:15.812553 20810 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:15.820091 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.820708 27576 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.861541 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:15.820052 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.822381 20806 raft_consensus.cc:2376] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19. Re-sending same reply.
I20260709 12:24:15.771484 27652 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.776521 27612 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:15.765173 27344 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:15.805842 27484 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:15.811914 27583 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:24:15.863044 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:15.863740 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:15.842522 27054 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:15.825695 27385 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:15.894328 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:15.895267 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:15.897521 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.797173 21432 rpcz_store.cc:276] 0709 12:24:14.280757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.283145 (+  2388us) service_pool.cc:224] Handling call
0709 12:24:14.283548 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:14.283944 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.283976 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:14.284568 (+   592us) log.cc:844] Serialized 52 byte log entry
0709 12:24:14.524925 (+240357us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4800280d58. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:24:14.879397 (+354472us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:15.568408 (+689011us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.677990 (+109582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.678074 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.678120 (+    46us) raft_consensus.cc:1685] finished
0709 12:24:15.699373 (+ 21253us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.699590 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104915,"prepare.queue_time_us":887450,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2489,"raft.queue_time_us":9760,"raft.run_cpu_time_us":2451,"raft.run_wall_time_us":212936,"spinlock_wait_cycles":5707648,"thread_start_us":1249988,"threads_started":2,"wal-append.queue_time_us":605204}
I20260709 12:24:15.884970 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:15.900229 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:15.820465 27681 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:24:15.859475 27592 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:15.866910 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 306360us
I20260709 12:24:15.901207 20806 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 12:24:15.864248 27430 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:15.865216 27668 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:15.908427 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:15.870644 27344 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:15.842118 27495 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.918248 20729 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:24:15.918759 20729 pending_rounds.cc:85] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 7
I20260709 12:24:15.918993 20729 pending_rounds.cc:107] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:24:15.919301 20729 raft_consensus.cc:2889] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.919443 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:15.922698 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:15.897992 27373 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:15.928992 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.861060 27496 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:15.860647 27382 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.904081 27169 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.860905 27586 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.935817 20806 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:15.936118 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.936811 20802 raft_consensus.cc:2376] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17. Re-sending same reply.
W20260709 12:24:15.864921 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 306343us
W20260709 12:24:15.913573 27676 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:24:15.912722 27430 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.939649 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.956897 20809 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:15.914572 27679 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:15.941401 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:15.946947 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 143251us
I20260709 12:24:15.941869 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:15.957938 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:15.958434 20803 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:15.871591 27652 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.941166 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:15.875092 27230 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:24:15.875928 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:15.865700 27554 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.897248 27618 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:24:15.912990 27668 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.868346 27576 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.893965 27484 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:15.905398 27682 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:15.938418 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:15.933233 27670 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.247s	user 0.001s	sys 0.001s
I20260709 12:24:15.965838 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:15.986591 20801 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:24:15.986791 20806 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:15.962000 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:15.932281 27496 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.967063 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:15.896823 27683 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:24:15.945732 27517 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:15.912112 27592 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:15.988266 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 112412us
I20260709 12:24:15.988989 20803 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:16.021843 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.022579 20803 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:16.023494 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.024570 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:16.025117 20803 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:15.955430 27510 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:15.964537 27373 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:15.968962 27430 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:15.963295 27689 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.959605 27169 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:15.996202 27496 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:15.967622 27281 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:16.019279 20883 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:15.974586 27668 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:15.997645 20809 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:15.979815 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:16.043730 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 105418us
I20260709 12:24:16.048969 20803 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:16.060189 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.063443 27316 raft_consensus.cc:2764] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:24:16.064208 27316 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.064699 27316 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:16.066046 27667 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.574s	user 0.004s	sys 0.000s
I20260709 12:24:16.066504 20806 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:16.034751 27230 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:16.041026 27685 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:16.071264 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1222) took 1625 ms. Trace:
W20260709 12:24:16.070787 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49959.6us
I20260709 12:24:16.073495 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.074857 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1198) took 1193 ms. Trace:
I20260709 12:24:16.073401 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.085625 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:16.086556 20803 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:16.072244 27029 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:16.087074 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:16.086702 20806 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:16.087935 27029 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.089568 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:16.090173 27029 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.090541 21432 raft_consensus.cc:2368] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:24:16.090782 27029 raft_consensus.cc:2764] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:24:16.084558 20805 rpcz_store.cc:276] 0709 12:24:14.446097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.446234 (+   137us) service_pool.cc:224] Handling call
0709 12:24:14.446628 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.606275 (+159647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.606307 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.606861 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.292511 (+685650us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.292558 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.062287 (+769729us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54003b7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:16.062364 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.065415 (+  3051us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:16.071024 (+  5609us) raft_consensus.cc:1685] finished
0709 12:24:16.071039 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.071148 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":818,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3963,"spinlock_wait_cycles":490193024,"thread_start_us":620720,"threads_started":1,"wal-append.queue_time_us":787529}
I20260709 12:24:16.093310 20805 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:16.092289 20806 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:16.094019 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:16.095510 27029 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:16.097262 20803 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 12:24:16.096257 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:16.097875 27029 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.098671 21428 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:24:16.097143 20805 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:16.085739 20884 rpcz_store.cc:276] 0709 12:24:14.881216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.881353 (+   137us) service_pool.cc:224] Handling call
0709 12:24:14.881854 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.882308 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.882342 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.882827 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.362438 (+479611us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.362494 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.362586 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.074561 (+711975us) raft_consensus.cc:1685] finished
0709 12:24:16.074593 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.074778 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1105,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":479152,"threads_started":1,"wal-append.queue_time_us":603916}
I20260709 12:24:15.982089 27612 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.108151 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.060535 27281 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.111660 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:16.106735 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.056586 27599 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:16.118227 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:16.067653 27695 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.071265 27316 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.124351 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.127636 20805 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:16.116309 20803 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:24:16.118539 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:16.103751 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50614.9us
I20260709 12:24:16.046060 27689 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.124107 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:16.132773 20884 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:16.133322 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.105672 27685 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:16.115270 27696 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:16.138402 27495 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:16.139355 27576 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.140138 27169 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.158443 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:16.140349 27660 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 12:24:16.141978 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:16.100360 27230 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.150714 27281 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:16.155807 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:16.129526 27683 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.167485 27682 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:16.159004 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.157467 27700 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:16.129731 27507 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.640s	user 0.010s	sys 0.000s
I20260709 12:24:16.168430 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:16.191134 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:16.162839 27495 raft_consensus.cc:2764] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:24:16.160377 27696 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:16.116624 27373 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.212180 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:16.189056 27672 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.582s	user 0.004s	sys 0.008s
I20260709 12:24:16.187667 27385 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:16.212985 21426 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:16.216413 27652 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:16.187740 27029 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.168015 27316 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:16.167330 27382 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:16.268044 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:16.130834 27695 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.166971 27660 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.271864 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:16.276378 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.279286 20805 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:16.280294 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:16.280869 20805 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:16.276978 27385 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.283726 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:16.284056 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:16.284991 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:16.190583 27507 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.264673 27484 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:16.286630 21435 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:24:16.287274 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:16.290465 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.293944 21426 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:16.214352 27713 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:16.295032 27691 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
W20260709 12:24:16.286545 27697 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:24:16.209884 27704 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 12:24:16.188664 27683 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.275539 27672 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
W20260709 12:24:16.286788 27687 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 12:24:16.204310 27700 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.271697 27681 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.278899 27592 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:16.288293 27554 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:16.397791 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 272268us
I20260709 12:24:16.405038 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:16.405412 20805 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:16.317818 27660 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:16.299273 27692 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.418020 20810 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19.
I20260709 12:24:16.288596 27316 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.433482 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:16.434732 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:16.456467 20805 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:16.458052 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:16.399240 27291 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:16.289284 27382 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:16.288383 27024 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:16.267287 27685 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.508373 27695 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.493727 27382 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.511926 27382 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:16.314824 27713 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.517632 27700 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.526660 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2883) took 1062 ms. Trace:
I20260709 12:24:16.527663 27230 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:16.530104 27507 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:16.533499 27072 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.005s
I20260709 12:24:16.426995 27696 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.535671 27713 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.432220 27681 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.539861 27672 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:16.540702 27385 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:16.548728 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:16.527328 20877 rpcz_store.cc:276] 0709 12:24:15.464160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.464557 (+   397us) service_pool.cc:224] Handling call
0709 12:24:15.464976 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.479932 (+ 14956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.479970 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.485763 (+  5793us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.860557 (+374794us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:15.860598 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.860660 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.526386 (+665726us) raft_consensus.cc:1685] finished
0709 12:24:16.526413 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.526572 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14456,"raft.queue_time_us":15332,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6109,"thread_start_us":286405,"threads_started":1,"wal-append.queue_time_us":379139}
I20260709 12:24:16.563695 27463 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.597412 27695 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:16.597306 27463 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.597970 27695 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:16.550784 27672 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.599462 27672 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:16.564273 27528 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.068s	user 0.009s	sys 0.004s
I20260709 12:24:16.725246 27528 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.727713 27584 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:16.728188 27584 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:16.728205 20806 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:16.729157 27584 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:16.564494 27708 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.410s	user 0.002s	sys 0.000s
I20260709 12:24:16.725726 27463 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:16.631723 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.000s	sys 0.036s
I20260709 12:24:16.458768 27692 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:16.729899 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:16.987280 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.000s	sys 0.048s
W20260709 12:24:16.988306 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 1.02628e+06us
I20260709 12:24:16.985152 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:16.994401 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.000398 20773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.013s	sys 0.016s
W20260709 12:24:17.001426 20773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.013s	sys 0.017s
W20260709 12:24:16.729166 27430 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.683s	user 0.006s	sys 0.003s
W20260709 12:24:16.972689 27496 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.913s	user 0.011s	sys 0.000s
W20260709 12:24:16.832647 27663 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.147s	user 0.012s	sys 0.000s
W20260709 12:24:16.820182 27712 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.346s	user 0.000s	sys 0.000s
W20260709 12:24:16.631656 27698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.006s	sys 0.014s
W20260709 12:24:16.814929 27054 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.913s	user 0.010s	sys 0.001s
W20260709 12:24:16.736217 27344 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.000s
I20260709 12:24:16.991215 27683 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:16.834239 27707 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:16.815955 27528 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:16.985019 27584 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.891562 27592 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:16.568974 27072 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.010645 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:17.026021 27344 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.030025 27054 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.030638 27692 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:17.019851 27698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.007s	sys 0.018s
I20260709 12:24:17.031437 27230 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:16.552671 27024 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.026324 27700 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:24:17.034083 27586 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:16.992849 27689 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.008265 27503 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.015931 27496 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.016510 27430 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:17.074025 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.165277 20879 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:17.303481 20807 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:17.119251 21427 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:17.079113 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2821) took 1252 ms. Trace:
W20260709 12:24:17.221390 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.015s	sys 0.020s
W20260709 12:24:17.307572 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.017s	sys 0.022s
W20260709 12:24:17.308118 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 842722us
I20260709 12:24:17.307407 20884 rpcz_store.cc:276] 0709 12:24:15.826336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.077963 (+1251627us) service_pool.cc:224] Handling call
0709 12:24:17.078399 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.078657 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.078695 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.078754 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:17.078810 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.078882 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.079036 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.314031 27385 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:17.314746 27385 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.316900 27514 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.317199 27385 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.317565 27514 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:17.318040 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2823) took 1460 ms. Trace:
I20260709 12:24:17.318217 27514 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.320432 27072 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.318511 20883 rpcz_store.cc:276] 0709 12:24:15.857682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.862752 (+  5070us) service_pool.cc:224] Handling call
0709 12:24:17.317816 (+1455064us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:17.317940 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339820416}
W20260709 12:24:17.321698 27674 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.538s	user 0.001s	sys 0.017s
I20260709 12:24:17.324229 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.324900 20805 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 13.
I20260709 12:24:17.325498 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2774) took 1186 ms. Trace:
I20260709 12:24:17.035606 27707 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:17.325845 20805 rpcz_store.cc:276] 0709 12:24:16.138696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.323956 (+1185260us) service_pool.cc:224] Handling call
0709 12:24:17.325438 (+  1482us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:17.064798 27668 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.994s	user 0.010s	sys 0.003s
I20260709 12:24:17.328655 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2768) took 1221 ms. Trace:
I20260709 12:24:17.329406 20879 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:17.329075 20805 rpcz_store.cc:276] 0709 12:24:16.107521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.326725 (+1219204us) service_pool.cc:224] Handling call
0709 12:24:17.326996 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.328297 (+  1301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.328327 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.328369 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:17.328416 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.328476 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.328601 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:17.305702 27612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.003s	sys 0.036s
I20260709 12:24:17.359678 20802 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:17.117079 27463 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.035692 27672 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.165449 27695 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.122665 27689 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:17.081691 27517 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:17.082695 27681 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.081655 27554 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.021446 27663 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.309634 27503 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:17.361038 27612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.003s	sys 0.036s
I20260709 12:24:17.367476 27463 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.095570 27592 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.371531 27514 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:17.376981 20810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2742) took 1480 ms (client timeout 1600 ms). Trace:
I20260709 12:24:17.380546 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2750) took 1438 ms. Trace:
I20260709 12:24:17.380673 27024 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.355863 27698 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:17.082415 27230 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.387480 20807 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
W20260709 12:24:17.392170 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 401843us
W20260709 12:24:17.409922 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1233) took 2299 ms (client timeout 2059 ms). Trace:
I20260709 12:24:17.400957 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2900) took 1299 ms. Trace:
I20260709 12:24:17.404862 20809 rpcz_store.cc:276] 0709 12:24:15.941888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.957664 (+ 15776us) service_pool.cc:224] Handling call
0709 12:24:17.380243 (+1422579us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b54003c9e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:17.380447 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89293184}
W20260709 12:24:17.404680 20810 rpcz_store.cc:269] 0709 12:24:15.897133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.897248 (+   115us) service_pool.cc:224] Handling call
0709 12:24:17.376633 (+1479385us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:17.376870 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153229696}
I20260709 12:24:17.356616 27668 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.387171 27517 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:17.409189 21399 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.010s	sys 0.022s
I20260709 12:24:17.411960 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.413725 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.411314 20877 rpcz_store.cc:276] 0709 12:24:16.101990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.400020 (+1298030us) service_pool.cc:224] Handling call
0709 12:24:17.400355 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.400541 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.400568 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.400626 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:17.400664 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.400732 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.400880 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.424641 20810 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:17.424948 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.414624 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.401163 27681 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:17.411074 20807 rpcz_store.cc:269] 0709 12:24:15.110808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.110951 (+   143us) service_pool.cc:224] Handling call
0709 12:24:17.039296 (+1928345us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b5400180c20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec14a1d 00007f9e1ebfb1a4 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca327c 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591
0709 12:24:17.409836 (+370540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76085120}
I20260709 12:24:17.430074 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.404645 27373 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.422808 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:17.425943 20809 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:17.426594 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.386983 27689 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.427029 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.427836 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:17.428493 21399 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.011s	sys 0.022s
I20260709 12:24:17.451279 20735 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:17.436673 27230 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.436379 20729 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:24:17.451764 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.16072e+06us
I20260709 12:24:17.413221 27054 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.403810 27554 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.402835 27503 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.455155 20884 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:24:17.456506 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.460738 20731 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:17.456863 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.462105 20810 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:17.463621 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.464411 20731 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:17.464372 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:17.428545 27385 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:17.469069 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.472020 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.473256 20735 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:17.476182 20728 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:17.454909 27344 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.480679 20735 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:17.410475 27698 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.481763 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:17.460484 27612 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:17.487036 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 178090us
I20260709 12:24:17.489033 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.496057 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.407930 27707 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.499266 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.500034 20732 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:24:17.496600 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:17.496845 20805 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:17.451471 27681 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.483137 27583 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.483563 27611 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.498188 27496 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.493553 27430 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.456108 27373 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.479506 27385 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.597743 27514 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:17.528469 27717 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:17.610745 27507 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:17.616926 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:17.535511 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53880.1us
I20260709 12:24:17.526836 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:17.641822 20805 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:17.606377 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:17.606458 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 65103.5us
I20260709 12:24:17.642802 20727 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:24:17.576433 27382 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.064s	user 0.004s	sys 0.009s
I20260709 12:24:17.603942 20730 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:17.609174 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.653056 20732 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:17.653924 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.654654 20727 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:17.615243 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:17.645614 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.642346 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:17.656600 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:17.658388 20731 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:17.600343 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.641641 20728 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:17.668453 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.670028 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:17.670677 20728 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:17.495795 27612 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.680955 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.681614 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:17.684345 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:17.682605 27692 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:17.692058 27692 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.692507 27692 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:17.640720 27507 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.697297 27373 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.698338 27507 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.700284 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.701727 20730 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:17.703116 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:17.703766 27732 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 12:24:17.704864 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:17.705705 27503 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:17.695474 20775 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.553s (SENT)
W20260709 12:24:17.706698 27728 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 12:24:17.706763 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:17.708259 27668 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.709219 27592 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.710632 27554 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.683024 27382 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.712565 27718 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:17.713053 27718 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.713392 27718 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:17.714541 27382 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.712109 27698 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.576069 27611 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.639221 27514 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:17.716616 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.717285 20732 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 13.
I20260709 12:24:17.717989 20727 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:17.718358 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:17.718740 27681 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.716877 27514 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.524325 27583 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.717234 27611 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.762796 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.745337 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1208) took 2598 ms. Trace:
I20260709 12:24:17.719269 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.719132 20732 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:17.724406 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1325) took 1632 ms. Trace:
I20260709 12:24:17.763914 20805 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
W20260709 12:24:17.740376 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 2.018s (SENT) [suppressed 11 similar messages]
I20260709 12:24:17.764755 20727 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 17 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:24:17.764295 20875 rpcz_store.cc:276] 0709 12:24:15.146793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.147608 (+   815us) service_pool.cc:224] Handling call
0709 12:24:15.221608 (+ 74000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.839072 (+617464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.745015 (+1905943us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b5400059e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:17.745259 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418217344}
W20260709 12:24:17.728070 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.767350 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83093.1us
W20260709 12:24:17.742441 20880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1206) took 2855 ms (client timeout 1510 ms). Trace:
I20260709 12:24:17.768484 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:17.768651 20880 rpcz_store.cc:269] 0709 12:24:14.886893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.919742 (+ 32849us) service_pool.cc:224] Handling call
0709 12:24:17.742186 (+2822444us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:17.742322 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620381440}
I20260709 12:24:17.770766 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.675565 20729 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:17.739902 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2828) took 1764 ms. Trace:
I20260709 12:24:17.773710 20849 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:17.773998 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.765022 21432 rpcz_store.cc:276] 0709 12:24:16.091911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.092045 (+   134us) service_pool.cc:224] Handling call
0709 12:24:16.092390 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.131514 (+ 39124us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:16.159387 (+ 27873us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.551172 (+391785us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:16.551267 (+    95us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:16.551335 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.724168 (+1172833us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:17.724217 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.724347 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11343,"prepare.queue_time_us":30097,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":168031,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":16221,"spinlock_wait_cycles":89408768,"thread_start_us":391311,"threads_started":2,"wal-append.queue_time_us":253966}
I20260709 12:24:17.772603 20883 rpcz_store.cc:276] 0709 12:24:15.975580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.322107 (+1346527us) service_pool.cc:224] Handling call
0709 12:24:17.322592 (+   485us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:17.323125 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.323167 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:17.324009 (+   842us) log.cc:844] Serialized 52 byte log entry
0709 12:24:17.491642 (+167633us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:17.491705 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.491805 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:17.739435 (+247630us) raft_consensus.cc:1685] finished
0709 12:24:17.739484 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.739801 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1531,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":167095,"threads_started":1,"wal-append.queue_time_us":413325}
I20260709 12:24:17.777009 20805 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:17.781539 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.776670 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.778154 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1210) took 1580 ms. Trace:
I20260709 12:24:17.778148 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:17.784614 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:17.785771 20880 rpcz_store.cc:276] 0709 12:24:16.197928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.777456 (+1579528us) service_pool.cc:224] Handling call
0709 12:24:17.777723 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.777880 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.777902 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.777936 (+    34us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:17.777964 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.778009 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.778095 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:17.730157 27584 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.002s
I20260709 12:24:17.788051 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.782223 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1212) took 1583 ms. Trace:
I20260709 12:24:17.794034 20883 rpcz_store.cc:276] 0709 12:24:16.198596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.781179 (+1582583us) service_pool.cc:224] Handling call
0709 12:24:17.781537 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.781773 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.781800 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.781838 (+    38us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:17.781883 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.781954 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.782085 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:17.788484 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.782467 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:17.797302 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.683439 27695 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:17.795670 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:17.737295 27744 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:17.721097 27689 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.798306 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.793540 20727 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:17.778616 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:17.822840 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.823704 20728 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:17.801192 27695 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.824817 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.826210 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.827030 20809 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:17.827677 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:17.827863 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.829140 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.824853 27695 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:17.830518 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:17.831153 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.789774 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:17.681447 27496 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:17.728096 27612 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.720137 27672 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.736711 27554 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.737130 27273 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.837693 27554 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.838766 27273 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:17.726613 27385 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.839350 27718 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:17.725925 27663 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:17.797513 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4624957500000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 12:24:17.839424 27273 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.802350 20805 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:17.805629 27717 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:17.728168 27507 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 12:24:17.811594 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:17.733666 27024 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 12:24:17.811929 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:17.813007 27742 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:17.812501 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:17.813442 21434 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:17.842242 21431 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:17.842438 27496 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.814029 27495 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:17.756568 27698 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:17.843925 27698 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.844758 27496 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:17.813948 20803 raft_consensus.cc:2376] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16. Re-sending same reply.
I20260709 12:24:17.815671 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:17.845865 21433 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:17.814811 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:17.835656 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.798116 20730 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:17.775902 27733 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:17.814945 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:17.788512 20849 leader_election.cc:403] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:17.818300 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:17.793568 27382 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:17.820662 27744 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:17.837049 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.730574 27463 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:17.839995 27385 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.719446 27681 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:17.840481 27554 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:17.840541 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4616044540002804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2351s for a response
I20260709 12:24:17.734062 27740 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:17.852025 21431 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:17.936954 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2898) took 2116 ms. Trace:
I20260709 12:24:18.156042 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.144925 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.145311 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:18.293723 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2824) took 2429 ms (client timeout 2018 ms). Trace:
I20260709 12:24:18.120100 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:18.123510 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:18.622620 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.231468 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.363051 27054 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:18.205991 27718 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:18.645892 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:17.876808 27583 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:18.646654 21426 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:17.846426 21430 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:24:17.874684 27727 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.553s	user 0.003s	sys 0.000s
I20260709 12:24:18.281289 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:18.585065 27748 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:24:17.878782 27517 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:18.620563 20807 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:18.647131 27707 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:18.649787 27744 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:18.649893 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:18.650379 27517 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:18.234562 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3015) took 2470 ms. Trace:
I20260709 12:24:18.131981 27672 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:18.652448 27382 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:18.627928 20810 raft_consensus.cc:2368] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:18.652758 27672 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:18.654871 27382 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:17.959786 27679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.963s	user 0.014s	sys 0.018s
I20260709 12:24:18.638548 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2826) took 2729 ms. Trace:
I20260709 12:24:18.589609 27564 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:17.846856 21432 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:24:18.577580 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:18.660964 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 540948us
I20260709 12:24:18.661583 21429 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:18.648121 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.392033 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:18.662688 21430 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 12:24:18.145071 20850 connection.cc:256] RPC call timeout handler was delayed by -1.5644455229999039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
I20260709 12:24:18.617810 21434 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:24:18.345413 27693 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 2.271s	user 0.007s	sys 0.012s
W20260709 12:24:18.588106 27741 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.962s	user 0.007s	sys 0.006s
I20260709 12:24:18.632265 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:18.032445 27702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.952s	user 0.013s	sys 0.029s
I20260709 12:24:18.288889 27535 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:18.632947 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.666776 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.620879 27733 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.656402 27495 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:18.593775 27695 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.000s
I20260709 12:24:18.426288 27750 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.162201 27745 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.655542 21428 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:18.147475 27273 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:18.390020 27746 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:18.703215 27745 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:18.119575 27698 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:18.665006 27679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.678s	user 0.016s	sys 0.043s
I20260709 12:24:18.031829 27692 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:18.693914 27733 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:18.675467 27281 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:18.631606 27385 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:18.682610 27702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.619s	user 0.016s	sys 0.049s
W20260709 12:24:18.702219 27688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.639s	user 0.015s	sys 0.035s
I20260709 12:24:18.706436 27750 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:18.713968 27665 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 3.219s	user 0.005s	sys 0.020s
W20260709 12:24:18.609627 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2765) took 2471 ms (client timeout 2316 ms). Trace:
W20260709 12:24:18.862421 20806 rpcz_store.cc:269] 0709 12:24:16.106671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.107927 (+  1256us) service_pool.cc:224] Handling call
0709 12:24:18.295687 (+2187760us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54002b61d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:18.577980 (+282293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423047424}
I20260709 12:24:18.863952 21433 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:18.650794 20807 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
W20260709 12:24:18.651297 20879 rpcz_store.cc:269] 0709 12:24:15.863907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.899983 (+ 36076us) service_pool.cc:224] Handling call
0709 12:24:18.293444 (+2393461us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400267d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:18.293619 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2904222080}
I20260709 12:24:18.617228 20881 rpcz_store.cc:276] 0709 12:24:15.820971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.830540 (+  9569us) service_pool.cc:224] Handling call
0709 12:24:15.830881 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.831120 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:17.741333 (+1910213us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b1800424c60. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf
0709 12:24:17.741526 (+   193us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.936495 (+194969us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.936561 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:17.936619 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.936697 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.936872 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26432,"prepare.queue_time_us":59308,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3434,"raft.queue_time_us":166,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":37160,"spinlock_wait_cycles":10132736,"thread_start_us":254280,"threads_started":2,"wal-append.queue_time_us":385793}
I20260709 12:24:18.658461 20884 rpcz_store.cc:276] 0709 12:24:15.916357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.333706 (+1417349us) service_pool.cc:224] Handling call
0709 12:24:17.335664 (+  1958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.460378 (+124714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.638211 (+1177833us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540026d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:18.638476 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234031360}
W20260709 12:24:18.690297 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 298332us
I20260709 12:24:18.700778 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.693109 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.697847 21432 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:18.645900 27054 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:18.693477 27495 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:18.709038 27679 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:18.720103 27688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.667s	user 0.015s	sys 0.036s
W20260709 12:24:18.846125 27677 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 2.900s	user 0.000s	sys 0.022s
W20260709 12:24:18.846262 27588 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.531s	user 0.014s	sys 0.015s
I20260709 12:24:18.678390 27564 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:18.294834 20803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2771) took 2157 ms (client timeout 1662 ms). Trace:
I20260709 12:24:18.717692 27672 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:17.847097 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:17.946305 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:18.830576 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2901) took 2721 ms. Trace:
I20260709 12:24:18.831782 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2893) took 3017 ms. Trace:
I20260709 12:24:18.701953 27695 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:18.801952 20732 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.003s	sys 0.003s
I20260709 12:24:17.846541 20728 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:18.126987 27740 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:18.708240 27744 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:18.703797 27746 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:18.914124 27744 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.914498 21430 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:18.917128 21430 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:18.751497 27733 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:18.647970 20809 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
W20260709 12:24:18.867122 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.923369 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3009) took 4237 ms. Trace:
I20260709 12:24:18.923899 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:18.907829 27740 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:18.924736 27514 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.652220 20801 rpcz_store.cc:276] 0709 12:24:15.764053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.764338 (+   285us) service_pool.cc:224] Handling call
0709 12:24:15.782227 (+ 17889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.988509 (+206282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.234202 (+2245693us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b540006ccd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:18.234465 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469123328}
I20260709 12:24:18.900672 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3055) took 1069 ms. Trace:
W20260709 12:24:18.907624 27588 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.592s	user 0.015s	sys 0.019s
I20260709 12:24:18.865952 20806 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:24:18.870090 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2881) took 4374 ms. Trace:
I20260709 12:24:18.904529 20730 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:18.869761 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2713) took 1392 ms. Trace:
I20260709 12:24:18.906725 27695 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:18.368813 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.870901 20802 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2751) took 2928 ms (client timeout 2181 ms). Trace:
I20260709 12:24:18.867960 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:18.870985 27385 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:18.917265 27746 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:18.849615 27707 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
W20260709 12:24:18.210210 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.925649 27514 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:18.923651 27463 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:18.913309 20732 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.010s	sys 0.003s
I20260709 12:24:18.919070 21434 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:24:18.901897 20882 rpcz_store.cc:276] 0709 12:24:15.814427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.814572 (+   145us) service_pool.cc:224] Handling call
0709 12:24:15.814911 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.815290 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.815310 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.815691 (+   381us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.931430 (+115739us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.931484 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.931586 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.344707 (+2413121us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:18.830991 (+486284us) raft_consensus.cc:1685] finished
0709 12:24:18.831044 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.831561 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":729,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4456,"spinlock_wait_cycles":14353664,"thread_start_us":83076,"threads_started":1,"wal-append.queue_time_us":196032}
I20260709 12:24:18.924008 20808 rpcz_store.cc:276] 0709 12:24:14.664052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.665909 (+  1857us) service_pool.cc:224] Handling call
0709 12:24:15.708096 (+1042187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.811602 (+103506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.874977 (+3063375us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002b5cd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:18.901623 (+ 26646us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350078208}
I20260709 12:24:18.914851 27744 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:18.833050 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141720us
W20260709 12:24:17.957250 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 216889us
W20260709 12:24:18.623165 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 499719us
W20260709 12:24:17.949959 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 186409us
W20260709 12:24:18.110592 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 348685us
I20260709 12:24:18.926921 20729 rpcz_store.cc:276] 0709 12:24:17.830761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.833209 (+  2448us) service_pool.cc:224] Handling call
0709 12:24:18.900305 (+1067096us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.930498 21431 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:18.727387 27750 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:18.994519 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3057) took 1140 ms. Trace:
I20260709 12:24:18.995167 20726 rpcz_store.cc:276] 0709 12:24:17.853917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.945954 (+ 92037us) service_pool.cc:224] Handling call
0709 12:24:18.994242 (+1048288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.007964 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:19.008783 27725 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.693s	user 0.000s	sys 0.012s
I20260709 12:24:19.008925 20879 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:19.009006 21400 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:19.009711 21400 leader_election.cc:403] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:19.012128 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2902) took 2858 ms. Trace:
I20260709 12:24:18.999917 20883 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:18.901592 20877 rpcz_store.cc:276] 0709 12:24:16.109299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.424690 (+1315391us) service_pool.cc:224] Handling call
0709 12:24:17.424999 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.425378 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.425409 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.430357 (+  4948us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.479316 (+ 48959us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:17.479353 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.479422 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.389846 (+910424us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:18.829455 (+439609us) raft_consensus.cc:1685] finished
0709 12:24:18.829556 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.830192 (+   636us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15728,"raft.queue_time_us":602,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":9021,"spinlock_wait_cycles":59613568,"thread_start_us":48560,"threads_started":1,"wal-append.queue_time_us":94036}
I20260709 12:24:19.014213 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.014350 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.015333 27495 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.015702 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:19.016552 27724 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.702s	user 0.000s	sys 0.014s
I20260709 12:24:19.016842 20884 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:19.017699 20883 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:19.018267 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.002696 20880 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:24:19.019431 20884 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:24:19.021886 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:19.003958 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4588957500000106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 12:24:19.022305 27463 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.024017 27733 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:19.022670 20803 rpcz_store.cc:269] 0709 12:24:16.137743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.190884 (+ 53141us) service_pool.cc:224] Handling call
0709 12:24:18.294651 (+2103767us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540019afd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:18.294760 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3722513152}
W20260709 12:24:19.023906 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4417957499999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 12:24:19.025125 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4194816839999476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 12:24:19.025610 20700 connection.cc:256] RPC call timeout handler was delayed by -1.3951816839999083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
I20260709 12:24:19.025890 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2820) took 3201 ms. Trace:
I20260709 12:24:19.000718 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.027802 20882 raft_consensus.cc:2368] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:19.029680 27514 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.030076 27463 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:19.031927 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2724) took 1430 ms. Trace:
I20260709 12:24:19.032434 20732 rpcz_store.cc:276] 0709 12:24:17.601658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.775557 (+173899us) service_pool.cc:224] Handling call
0709 12:24:17.797283 (+ 21726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.797622 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.797655 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.798159 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.973956 (+175797us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:19.030725 (+1056769us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.031402 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.031529 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.031609 (+    80us) raft_consensus.cc:1685] finished
0709 12:24:19.031624 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.031859 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":761460,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":2780,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":7346,"spinlock_wait_cycles":34944,"thread_start_us":1128581,"threads_started":2,"wal-append.queue_time_us":288949}
I20260709 12:24:18.930423 27611 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:19.035514 27707 consensus_queue.cc:1243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:24:19.036329 27707 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:19.037134 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1686) took 2833 ms. Trace:
I20260709 12:24:19.006624 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.038513 27723 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.026391 20876 rpcz_store.cc:276] 0709 12:24:15.824066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.863885 (+ 39819us) service_pool.cc:224] Handling call
0709 12:24:15.864224 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.046522 (+182298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.046570 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:16.046630 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:16.046707 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.025342 (+2978635us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400171e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:19.025457 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.025728 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418502016}
I20260709 12:24:19.039155 27688 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:19.037701 21427 rpcz_store.cc:276] 0709 12:24:16.203355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.203547 (+   192us) service_pool.cc:224] Handling call
0709 12:24:16.207909 (+  4362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.306070 (+1098161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.036604 (+1730534us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b5400729a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:19.036926 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095153536}
I20260709 12:24:19.044400 27054 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.041862 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3052) took 1191 ms. Trace:
I20260709 12:24:18.959820 27564 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.047446 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1692) took 1237 ms. Trace:
W20260709 12:24:19.025967 20700 connection.cc:256] RPC call timeout handler was delayed by -1.3900863289995868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 12:24:19.045112 27723 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:19.049309 20700 connection.cc:256] RPC call timeout handler was delayed by -1.0110547579997728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:24:19.050851 27685 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.051736 27685 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.035787 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3046) took 1267 ms. Trace:
I20260709 12:24:19.038877 27713 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.055565 20731 rpcz_store.cc:276] 0709 12:24:17.767935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.846898 (+ 78963us) service_pool.cc:224] Handling call
0709 12:24:18.126815 (+279917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.905328 (+778513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.905384 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.905456 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:18.905518 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.035264 (+129746us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b54003b3b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:19.035327 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.035546 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788976384}
I20260709 12:24:19.056234 27713 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.057574 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.058772 20731 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:19.047232 20728 rpcz_store.cc:276] 0709 12:24:17.801005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.834996 (+ 33991us) service_pool.cc:224] Handling call
0709 12:24:18.988057 (+1153061us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:18.955603 20802 rpcz_store.cc:269] 0709 12:24:15.942178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.978140 (+ 35962us) service_pool.cc:224] Handling call
0709 12:24:18.870333 (+2892193us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540019a5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:18.870625 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3002097920}
I20260709 12:24:19.060549 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.061425 20728 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:19.039670 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 111009us
I20260709 12:24:19.059517 27385 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.064271 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.065572 27203 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:18.728680 27692 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.054961 27685 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:19.048045 21432 rpcz_store.cc:276] 0709 12:24:17.809902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.812272 (+  2370us) service_pool.cc:224] Handling call
0709 12:24:19.046979 (+1234707us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:19.047234 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9838464}
I20260709 12:24:19.069185 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.069809 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3060) took 1067 ms. Trace:
I20260709 12:24:19.012603 20875 rpcz_store.cc:276] 0709 12:24:16.153524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.776280 (+1622756us) service_pool.cc:224] Handling call
0709 12:24:17.776582 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.776805 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:18.121189 (+344384us) log.cc:844] Serialized 14 byte log entry
0709 12:24:18.275862 (+154673us) log.cc:844] Serialized 14 byte log entry
0709 12:24:18.727179 (+451317us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b1800342220. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1
0709 12:24:18.916232 (+189053us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800342220. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:19.011149 (+ 94917us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b1800342220. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:19.011301 (+   152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.011403 (+   102us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:19.011523 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.011645 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.011915 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87427,"prepare.queue_time_us":348872,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":128,"raft.queue_time_us":646903,"raft.run_cpu_time_us":485,"raft.run_wall_time_us":9695,"spinlock_wait_cycles":137706880,"thread_start_us":454494,"threads_started":2,"wal-append.queue_time_us":500844}
I20260709 12:24:19.037096 27707 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:19.015141 20877 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:19.048379 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.071400 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3056) took 1237 ms. Trace:
I20260709 12:24:19.071453 27723 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.075863 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:18.983172 20878 rpcz_store.cc:276] 0709 12:24:14.495178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.495342 (+   164us) service_pool.cc:224] Handling call
0709 12:24:14.514224 (+ 18882us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:14.569658 (+ 55434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.569692 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:14.574532 (+  4840us) log.cc:844] Serialized 52 byte log entry
0709 12:24:15.387167 (+812635us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:15.387623 (+   456us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.387884 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.387985 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.713274 (+3325289us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:18.869436 (+156162us) raft_consensus.cc:1685] finished
0709 12:24:18.869583 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.869864 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":731,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":9995,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":216,"spinlock_wait_cycles":14753152,"thread_start_us":770905,"threads_started":1,"wal-append.queue_time_us":848103}
I20260709 12:24:19.093881 27316 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.072800 27713 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.080394 20731 rpcz_store.cc:276] 0709 12:24:17.992248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.057187 (+1064939us) service_pool.cc:224] Handling call
0709 12:24:19.059147 (+  1960us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.096483 20730 rpcz_store.cc:276] 0709 12:24:17.833666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.846840 (+ 13174us) service_pool.cc:224] Handling call
0709 12:24:19.071318 (+1224478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.955667 20735 rpcz_store.cc:276] 0709 12:24:17.477560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.477689 (+   129us) service_pool.cc:224] Handling call
0709 12:24:17.478602 (+   913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.483790 (+  5188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.868988 (+1385198us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b54001d8654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:18.869457 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3752832}
I20260709 12:24:18.748360 27745 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:18.929436 27583 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:19.114338 20729 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:19.126253 20726 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:19.129418 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2725) took 1527 ms. Trace:
W20260709 12:24:19.133013 27680 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 3.341s	user 0.016s	sys 0.013s
I20260709 12:24:19.136831 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:19.139873 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3010) took 4475 ms. Trace:
I20260709 12:24:19.141840 20802 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:24:19.126967 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3062) took 1134 ms. Trace:
I20260709 12:24:19.115677 27385 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:19.143478 20726 rpcz_store.cc:276] 0709 12:24:17.992930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.005891 (+1012961us) service_pool.cc:224] Handling call
0709 12:24:19.125632 (+119741us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b540036ca20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca2e18 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:19.126869 (+  1237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33787264}
I20260709 12:24:19.144204 27385 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.144788 27385 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:19.147890 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 78968.2us
I20260709 12:24:19.148988 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.149279 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.152180 20726 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 13.
I20260709 12:24:19.150221 27599 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.151592 27707 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.130124 20727 rpcz_store.cc:276] 0709 12:24:17.601905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.833390 (+231485us) service_pool.cc:224] Handling call
0709 12:24:17.833698 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.834081 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.834123 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.834734 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.128436 (+1293702us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b540025f6a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec09090 00007f9e1ec166b8 00007f9e1ec9f92b 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c
0709 12:24:19.128567 (+   131us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:19.128657 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.128795 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.128865 (+    70us) raft_consensus.cc:1685] finished
0709 12:24:19.128879 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.129211 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6803,"spinlock_wait_cycles":1482347264,"thread_start_us":260051,"threads_started":1,"wal-append.queue_time_us":250652}
I20260709 12:24:19.155578 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.156265 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3058) took 1256 ms. Trace:
I20260709 12:24:19.158804 20729 rpcz_store.cc:276] 0709 12:24:17.865480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.047903 (+1182423us) service_pool.cc:224] Handling call
0709 12:24:19.117883 (+ 69980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.140496 20804 rpcz_store.cc:276] 0709 12:24:14.664509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.665994 (+  1485us) service_pool.cc:224] Handling call
0709 12:24:14.666341 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.675653 (+  9312us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:14.692166 (+ 16513us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.787150 (+ 94984us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:24:15.262595 (+475445us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800372280. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:24:15.928572 (+665977us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.930795 (+  2223us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.931823 (+  1028us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.932309 (+   486us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:15.932439 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.933314 (+   875us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.132056 (+3198742us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:19.139348 (+  7292us) raft_consensus.cc:1685] finished
0709 12:24:19.139379 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.139650 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":387261,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":2094,"raft.queue_time_us":7781,"raft.run_cpu_time_us":1765,"raft.run_wall_time_us":138450,"spinlock_wait_cycles":84296064,"thread_start_us":665262,"threads_started":2,"wal-append.queue_time_us":1090625}
W20260709 12:24:19.156659 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 99129.7us
I20260709 12:24:19.176694 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:18.963318 27679 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.178318 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3063) took 1180 ms. Trace:
I20260709 12:24:19.182799 20731 rpcz_store.cc:276] 0709 12:24:17.993196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.100897 (+1107701us) service_pool.cc:224] Handling call
0709 12:24:19.104134 (+  3237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.169372 (+ 65238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.169417 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.172771 (+  3354us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:19.172848 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.172946 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.173125 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.193125 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.197945 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:19.199599 20850 connection.cc:256] RPC call timeout handler was delayed by -1.3232455229999687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
I20260709 12:24:19.198285 27382 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:19.157907 27203 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:24:19.205441 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.205921 20727 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:24:19.207911 27771 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.210336 20850 connection.cc:256] RPC call timeout handler was delayed by -1.316045522999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 12:24:19.208683 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:19.213845 27707 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:19.214058 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 1.39652e+06us
I20260709 12:24:19.219779 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.220943 20733 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:19.223306 27611 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
W20260709 12:24:19.228324 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 143717us
I20260709 12:24:19.228808 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:19.229635 20808 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:19.232822 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52351.8us
I20260709 12:24:19.234779 20879 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:19.206945 27382 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:19.233901 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3044) took 3264 ms. Trace:
I20260709 12:24:19.240387 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.240015 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.234719 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:19.244861 20728 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:19.247771 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:19.241291 20729 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:19.208189 27745 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:24:19.249653 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:19.251479 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:19.251727 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 78057.3us
I20260709 12:24:19.254431 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.251811 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.256261 20877 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:19.256353 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.257905 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.258102 27750 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:19.258877 27510 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:19.259459 27510 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:19.259876 27510 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.191320 27583 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:19.233903 27698 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.237246 27203 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:19.267902 21399 leader_election.cc:336] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.510s (SENT)
I20260709 12:24:19.244065 27382 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.269155 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.273377 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:19.249382 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3061) took 1252 ms. Trace:
I20260709 12:24:19.283399 20728 rpcz_store.cc:276] 0709 12:24:17.992616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.060153 (+1067537us) service_pool.cc:224] Handling call
0709 12:24:19.245285 (+185132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.287526 20877 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:19.287526 20876 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.270462 27698 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:19.292573 27698 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.295363 27611 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.296176 27611 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.296958 27583 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.299144 27771 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.300570 20878 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:19.301880 27771 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:19.307458 27273 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.307934 27273 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:19.308351 27273 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.312996 20876 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
W20260709 12:24:19.313220 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 133006us
I20260709 12:24:19.327477 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:19.327927 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.318394 27744 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.269972 27750 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:19.330206 20806 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:19.331893 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.331658 27750 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.333597 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.334331 20877 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:19.322000 20733 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:19.330338 20875 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:19.342876 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:19.344003 20876 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:19.345382 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:19.346441 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.347329 20879 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:19.350018 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.350979 20729 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:19.353276 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:19.354390 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:19.357437 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:19.360015 20733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2720) took 1759 ms (client timeout 2018 ms). Trace:
I20260709 12:24:19.362221 20729 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:24:19.360450 20733 rpcz_store.cc:269] 0709 12:24:17.600503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.604020 (+  3517us) service_pool.cc:224] Handling call
0709 12:24:19.219336 (+1615316us) spinlock_profiling.cc:243] Waited 909 us on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:24:19.359955 (+140619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089856}
I20260709 12:24:19.369472 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.371059 20876 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:19.372599 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:19.387893 20883 raft_consensus.cc:2376] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22. Re-sending same reply.
I20260709 12:24:19.388134 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:19.374022 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:19.375602 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:19.389077 20807 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:19.389349 20809 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:19.390300 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:19.376281 20735 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.391075 20801 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:19.391790 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.376351 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.376777 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:19.330369 27744 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.378162 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:19.379114 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 138754us
I20260709 12:24:19.396028 20808 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:19.396719 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.380235 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.380601 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.381187 27583 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:19.398411 20806 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:24:19.381705 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 113939us
I20260709 12:24:19.381871 27770 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.382388 27692 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:19.382932 20882 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:24:19.384532 27495 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.240792 20734 rpcz_store.cc:276] 0709 12:24:15.932864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.934013 (+  1149us) service_pool.cc:224] Handling call
0709 12:24:15.934312 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.996963 (+ 62651us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:16.036113 (+ 39150us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.135383 (+ 99270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:16.135463 (+    80us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:16.135525 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.196348 (+3060823us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b5400226fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:19.196447 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.196663 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36571,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":148,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":143441664,"thread_start_us":98803,"threads_started":1,"wal-append.queue_time_us":99186,"wal-append.run_cpu_time_us":16366,"wal-append.run_wall_time_us":1874973}
I20260709 12:24:19.386087 27698 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.417405 27495 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:19.418200 27495 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:19.419696 20804 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 12:24:19.419359 27692 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:19.420003 27385 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:19.362766 27611 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:19.422427 27692 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.429317 27692 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:19.429440 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.424629 27771 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:19.431015 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.423766 27770 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:19.431795 20731 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:19.433715 27770 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.434213 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:19.434890 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.435544 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:19.424407 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43951.4us
I20260709 12:24:19.436801 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.437547 20879 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:19.439409 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:19.429590 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49547us
I20260709 12:24:19.423424 20802 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:19.438926 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:19.449033 20734 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
W20260709 12:24:19.452880 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 199144us
I20260709 12:24:19.386775 27510 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.458102 20735 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:19.460381 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.465546 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.449589 20734 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 12
I20260709 12:24:19.475003 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.487696 20809 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:19.422583 27385 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.493259 20728 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:19.418889 27495 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:24:19.481077 20734 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:24:19.497237 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.495455 20734 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:19.498086 20879 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:19.499763 20733 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:24:19.497047 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:19.500730 20731 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:19.474885 27611 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:19.502658 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62006.1us
I20260709 12:24:19.423532 27583 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:19.507584 20729 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:19.508051 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 193149us
I20260709 12:24:19.508298 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.513048 20726 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:19.509450 20850 leader_election.cc:400] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15.
I20260709 12:24:19.509809 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.514115 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.517519 27510 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:19.519426 20630 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 8 to 11, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 11 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:19.521296 20630 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:19.523543 27776 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.267s	user 0.000s	sys 0.005s
I20260709 12:24:19.524274 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:19.525110 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 337601us
I20260709 12:24:19.527383 27024 raft_consensus.cc:476] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Not starting pre-election -- already a leader
I20260709 12:24:19.483902 27382 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:19.532647 27382 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.532728 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.533715 21429 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:19.534902 20803 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:19.535494 27385 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:19.536396 27385 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:19.517412 27599 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:24:19.542189 27698 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.499861 27685 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:19.474417 27273 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.546319 27273 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:19.550397 27770 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:19.551541 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:19.552235 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66278.9us
I20260709 12:24:19.550477 27707 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:24:19.549049 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:19.553280 27054 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.552471 20809 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:19.553799 27771 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.553834 27054 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.554584 27771 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:19.557669 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3059) took 1607 ms. Trace:
I20260709 12:24:19.558770 27583 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:19.559355 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:19.558060 20732 rpcz_store.cc:276] 0709 12:24:17.950580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.076708 (+1126128us) service_pool.cc:224] Handling call
0709 12:24:19.077343 (+   635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.078392 (+  1049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.078435 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.079353 (+   918us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.498567 (+419214us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:19.498625 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.498735 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.557464 (+ 58729us) raft_consensus.cc:1685] finished
0709 12:24:19.557484 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.557608 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":185,"thread_start_us":412917,"threads_started":1,"wal-append.queue_time_us":459664}
I20260709 12:24:19.522987 20630 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:19.570405 20731 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:19.572889 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:19.573771 20809 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:19.537824 27385 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.502877 27611 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:19.559034 27054 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.578948 20729 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
W20260709 12:24:19.581189 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53982.4us
W20260709 12:24:19.579751 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.573884 27495 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:19.573277 27698 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.588071 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:19.588689 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 192126us
I20260709 12:24:19.580758 27611 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:19.587122 27495 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:19.610462 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:19.608373 27588 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:24:19.608688 27698 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.619710 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.620733 21432 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
W20260709 12:24:19.621141 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 293814us
I20260709 12:24:19.621273 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:19.621896 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.622263 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:19.622534 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:19.622699 21430 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:19.620810 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.623958 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.618567 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.624019 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.621871 21435 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.632113 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:19.633018 20804 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:19.639710 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58017.5us
I20260709 12:24:19.640897 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:19.641748 20699 leader_election.cc:400] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:19.641963 27770 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.644769 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:19.645854 20806 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:19.642231 20699 leader_election.cc:403] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:19.648509 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:19.620787 27698 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:19.653558 20802 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:19.653919 27495 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:19.654726 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47558.7us
I20260709 12:24:19.655706 27495 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.656244 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.658634 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:19.610226 27583 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:19.661294 21435 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:19.669399 20733 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:19.668138 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.631629 27510 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.677386 20802 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:19.672777 20630 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 15, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 15 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:19.635985 27745 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:24:19.716558 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:19.712759 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 214229us
I20260709 12:24:19.673527 27382 raft_consensus.cc:2764] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:24:19.717163 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58622.6us
I20260709 12:24:19.677105 27203 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:19.710958 27744 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:19.679175 27770 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:19.661603 27273 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:24:19.769292 27273 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:19.769912 27273 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:19.774809 27781 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:19.775725 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.777009 27781 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:19.777819 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.781571 27744 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.782271 27744 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:19.785885 27789 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:24:19.735476 20802 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:19.742035 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:19.757937 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:19.757970 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.759594 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:19.761497 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.788550 21433 raft_consensus.cc:2376] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15. Re-sending same reply.
I20260709 12:24:19.791872 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:19.791592 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 57302.6us
I20260709 12:24:19.734395 27679 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:19.790860 27790 raft_consensus.cc:2764] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:24:19.792810 27685 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:19.812448 21430 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 12:24:19.815578 27771 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.816181 27652 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:19.816283 27771 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.816776 27652 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:19.816975 27771 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.817925 27652 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.787897 27273 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:19.819554 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77636.7us
I20260709 12:24:19.820277 27273 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.821220 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.822168 27495 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:19.823010 27495 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:19.824402 21427 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:19.793995 27781 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:19.826045 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 155055us
W20260709 12:24:19.832995 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55358us
I20260709 12:24:19.807893 27054 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:19.825920 27679 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:19.842931 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81536.5us
I20260709 12:24:19.824477 27495 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:19.848325 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:19.851150 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 138353us
I20260709 12:24:19.852674 27786 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:19.852911 27588 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.837647 27791 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:24:19.854676 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:19.858450 20699 leader_election.cc:400] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:19.858819 20699 leader_election.cc:403] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:19.859579 27685 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:19.860756 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:19.863520 21427 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:19.863029 27679 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.855284 27652 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.868963 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:19.869282 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 38d64dd1339c4abfb4ef05669402e801: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:19.869997 27695 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:19.873797 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:19.893052 20850 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
W20260709 12:24:19.893834 20701 connection.cc:256] RPC call timeout handler was delayed by -1.5981326279998029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 12:24:19.895367 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:19.880838 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:19.875155 27583 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:19.906913 20882 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:19.913019 20804 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:19.923133 27792 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:19.896494 27745 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:24:19.939272 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.911769 27583 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.900799 27698 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 12:24:19.951151 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102945us
I20260709 12:24:19.938869 27792 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:19.911537 27786 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:24:19.946772 27745 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:19.947103 27583 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:19.956037 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75311.6us
I20260709 12:24:19.925127 27611 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:19.956899 27786 consensus_queue.cc:1243] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:19.948067 27771 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:19.975363 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 82507.3us
I20260709 12:24:19.976987 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:19.982470 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.996088 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92728.7us
W20260709 12:24:20.004009 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 91786.8us
I20260709 12:24:20.012221 27495 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:20.012157 27785 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:20.014120 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:20.029526 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52652.9us
I20260709 12:24:20.031639 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:20.033556 27770 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:20.034140 27770 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:20.035426 27795 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:20.035099 27770 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.036391 27795 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:20.037593 27795 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:20.051461 20701 connection.cc:256] RPC call timeout handler was delayed by -1.1184326279999368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27440000000000003s for a response
W20260709 12:24:20.054502 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 85152.2us
I20260709 12:24:20.055835 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:20.058823 27791 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:20.059417 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.060313 27791 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:20.060726 27791 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:20.061180 27791 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:20.060827 27745 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.061529 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:20.062971 27781 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:20.063421 27781 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:20.063406 27792 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.064088 27781 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:20.064231 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:20.064661 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:20.070369 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:20.077195 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49296.3us
W20260709 12:24:20.127729 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96229.1us
I20260709 12:24:20.088984 27510 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:20.088665 27695 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:20.087997 27740 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:20.136181 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:20.135177 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:20.141073 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76941.2us
W20260709 12:24:20.142534 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55283.5us
I20260709 12:24:20.143368 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:20.137851 27510 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:20.138202 27695 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:20.153055 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.153057 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:20.164201 27744 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.166531 27744 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:20.151934 27510 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:20.166183 27698 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:20.168002 27698 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:20.168380 27698 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:20.169229 27599 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:20.169703 27599 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.170089 27599 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:20.171862 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:20.169445 27698 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.172942 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.173026 27698 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:20.173450 27273 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:20.173909 27273 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:20.174314 27273 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:20.175515 27495 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:20.175679 27692 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:20.176127 27495 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:20.176227 27692 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:20.177364 27790 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:20.177198 27692 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:20.178721 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 1be2fc0a60b1472a9f537a9cd3bcc133: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:20.179942 27685 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:20.177198 27495 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.156837 27695 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:20.177932 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.166180 27583 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.191488 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.153544 27786 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:20.196871 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50024.2us
I20260709 12:24:20.197134 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:20.198772 20806 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:20.155279 27611 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:20.231961 20881 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:24:20.235805 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.244962 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.205194 27790 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:20.204587 27771 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:20.249200 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 72710.3us
W20260709 12:24:20.251173 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82158.3us
I20260709 12:24:20.259063 20806 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:20.259858 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:20.260691 27745 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:20.261091 27745 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:20.261432 27745 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:20.247550 27611 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:20.270988 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:20.250720 27790 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:20.288514 27583 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:20.297575 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.271921 27750 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:20.262569 27652 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:20.272570 27611 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:20.280138 27698 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:20.331732 27785 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:20.285920 27790 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:20.335615 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 52829.2us
I20260709 12:24:20.272439 27510 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.329171 27611 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:20.355926 27784 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:20.355960 27802 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 12:24:20.356585 27801 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 12:24:20.357281 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75576.4us
I20260709 12:24:20.315856 27273 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.369936 20810 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:20.363322 21435 pending_rounds.cc:85] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 11
I20260709 12:24:20.338793 27785 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.370510 21435 pending_rounds.cc:107] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:24:20.371722 21435 raft_consensus.cc:2889] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:20.336916 27652 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.365563 27203 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:20.354607 27698 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:20.364851 27784 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:20.395452 27784 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:20.392997 27273 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:20.400979 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet e36a8df2d02f44ca965324882aba3ef2: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:20.401125 27652 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:20.382758 27785 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:20.405347 27823 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:20.392812 27791 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.393837 27824 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:20.430028 27510 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:20.439060 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.438881 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.409775 27791 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:20.439682 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 310995us
I20260709 12:24:20.416055 27599 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.442313 27745 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.444005 27599 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:20.444659 27745 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:20.445367 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 152429us
I20260709 12:24:20.446255 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:20.447604 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.448432 20806 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19.
I20260709 12:24:20.449350 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.450138 20806 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:24:20.445340 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:20.450771 20802 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:20.430276 27695 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.459416 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.460470 20806 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 12:24:20.454313 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 325256us
I20260709 12:24:20.459600 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:20.430920 27786 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.462968 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.464768 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:20.467954 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet a48bec319402467881543ac686fb8f0c: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:24:20.468948 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 209220us
I20260709 12:24:20.470551 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:20.474846 20801 raft_consensus.cc:2368] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
I20260709 12:24:20.475051 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.478210 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.478099 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.479154 20806 raft_consensus.cc:2368] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:20.481549 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:20.482352 20808 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:20.484791 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.483124 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:20.467819 27599 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:20.487128 20878 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:20.488442 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.489602 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.454370 27611 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.494011 20801 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:20.496248 20807 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:24:20.481967 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.487535 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:20.506053 20806 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:20.485635 27832 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:20.510377 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.510591 27832 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:20.512822 20879 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:20.515062 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.517673 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.518289 20733 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:20.518983 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.521823 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:20.470634 27786 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.526043 20630 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 21, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 21 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.532629 20878 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:20.537230 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.538151 20879 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:20.539481 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.541579 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.545186 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:20.543840 20630 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 20, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 20 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.550143 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.511834 27599 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:20.545593 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.551900 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.510926 27833 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:20.555858 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.557512 20805 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:20.516084 27832 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:20.550549 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:20.558302 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 97692.9us
I20260709 12:24:20.546029 27770 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:20.560779 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:20.560880 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.561631 20735 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:20.573527 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.576683 20878 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:20.496966 27698 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.579901 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.522907 27781 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.581804 20735 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 23 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:20.582105 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.514675 27679 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 12:24:20.575168 27813 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
I20260709 12:24:20.556823 27583 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:20.584357 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.557432 27833 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:20.596495 27833 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:20.553334 27723 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:24:20.600171 27723 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:20.600715 27723 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:20.601976 27723 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.603518 27599 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.606916 27822 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.607645 27822 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.590596 20729 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:20.609354 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:20.609825 27822 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:20.577955 20630 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 14, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 14 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.612064 20810 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:20.612589 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.613560 20879 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:20.613560 20733 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:20.614593 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.615502 20884 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:20.615592 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.621588 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.622736 20881 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:20.623209 20729 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:20.623577 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:20.624018 20802 raft_consensus.cc:2376] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20. Re-sending same reply.
I20260709 12:24:20.587909 27784 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.632526 21427 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:24:20.637235 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.638375 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.637643 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.643498 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.644255 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:20.645203 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.646046 20881 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:20.649374 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.649904 20732 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:20.650189 20884 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:20.651098 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.651602 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.652307 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.653136 20730 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:20.653540 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:20.654877 20700 connection.cc:256] RPC call timeout handler was delayed by -1.3821430250000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 12:24:20.655339 20700 connection.cc:256] RPC call timeout handler was delayed by -1.2327430249997633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 12:24:20.655629 20700 connection.cc:256] RPC call timeout handler was delayed by -1.1391430250000667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 12:24:20.660204 27781 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:20.637310 20630 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 17, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 17 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.619395 27745 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:24:20.668100 27784 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:20.668771 27744 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:20.670122 27679 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:20.670655 27679 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:20.671937 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.673593 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.673776 27583 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:20.674614 20882 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 17: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:20.649731 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.674837 27583 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:20.675316 27652 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:20.675727 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:20.676357 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:20.676724 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.677066 20801 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 14: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:20.677932 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.679440 20880 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:20.651059 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.679476 20734 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:20.685312 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:20.687862 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:20.646942 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.698429 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:20.701027 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.693284 20881 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:20.700799 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.703855 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.708480 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.709429 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:20.709609 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 173421us
I20260709 12:24:20.621464 27685 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:24:20.634122 27695 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:20.718925 20630 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.727989 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.717106 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:20.730633 20727 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:20.730561 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.725174 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.732127 21430 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:20.727279 20630 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 19, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 19 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:20.734731 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.738693 20880 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:20.740834 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.742278 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.717314 20878 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:20.717044 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.710264 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:20.717648 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.751813 20726 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:20.720042 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.752930 20730 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:20.720686 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.721861 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:20.754457 21427 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:20.722916 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.691346 27723 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:20.769519 20630 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.777717 21427 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:20.777889 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.782373 20884 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:20.783491 20880 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:20.787670 20879 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:20.793857 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104661us
I20260709 12:24:20.796701 20882 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:20.716624 27385 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:24:20.769269 27824 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:20.807668 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.799144 20630 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.781461 27652 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:20.815873 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.816536 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:20.783718 27588 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:20.820673 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.862684 20884 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:20.844604 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 188299us
I20260709 12:24:20.795496 27771 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:20.870666 20630 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.863873 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:20.794895 27611 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:20.890172 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151741us
I20260709 12:24:20.783699 27707 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:24:20.943320 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:20.943509 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:20.819963 27809 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:20.944180 20630 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.945968 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:20.860760 27588 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:20.946677 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 77798.7us
I20260709 12:24:20.873696 27054 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.950980 20734 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:20.951268 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 200) took 27789 ms (client timeout 15000 ms). Trace:
I20260709 12:24:20.869958 27770 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:20.863596 27824 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.799743 27679 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.958709 20877 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:24:21.102669 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 461016us
W20260709 12:24:21.502674 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 556889us
W20260709 12:24:20.960652 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151024us
W20260709 12:24:21.500524 27845 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.324s	user 0.000s	sys 0.000s
W20260709 12:24:21.499013 27838 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.554s	user 0.000s	sys 0.001s
W20260709 12:24:21.211519 27583 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.010s
W20260709 12:24:21.499830 27843 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.310s	user 0.000s	sys 0.000s
W20260709 12:24:20.995028 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 284807us
W20260709 12:24:21.220750 20804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.004s	sys 0.010s
W20260709 12:24:21.632750 20804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.005s	sys 0.023s
W20260709 12:24:21.259388 21432 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.000s	sys 0.016s
W20260709 12:24:21.528962 20728 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.004s	sys 0.037s
W20260709 12:24:21.633953 21432 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.000s	sys 0.029s
W20260709 12:24:21.634125 20728 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.006s	sys 0.038s
W20260709 12:24:21.403463 27792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.003s	sys 0.014s
I20260709 12:24:21.501821 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:20.969170 20630 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 7 to 14, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 14 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:21.569257 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:21.569097 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.021940 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 327936us
I20260709 12:24:21.648365 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:20.945133 27771 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:21.032306 27024 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:21.654817 20881 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:21.655757 20630 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 9 to 19, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 19 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:21.205878 27790 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.845s	user 0.012s	sys 0.000s
I20260709 12:24:21.629623 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:21.675370 20734 raft_consensus.cc:2376] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19. Re-sending same reply.
I20260709 12:24:21.675222 27790 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:21.676295 27833 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.079s	user 0.011s	sys 0.000s
I20260709 12:24:21.676472 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:21.677273 20734 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 14: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:21.676816 27833 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:21.677505 27790 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:21.678119 27790 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.678484 20734 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:24:21.630465 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68724.8us
I20260709 12:24:21.502904 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:21.093235 27786 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:21.680218 27846 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.680356 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:21.643916 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:21.680961 27846 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:20.959582 27781 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:21.681891 27781 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:21.225840 27817 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:24:21.575563 27054 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:21.682309 27781 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:21.266703 27588 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:21.689280 27846 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:21.689694 27588 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:21.659734 27842 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.506s	user 0.000s	sys 0.001s
I20260709 12:24:21.690212 27024 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:21.690773 27024 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:21.660409 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:21.691243 27024 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:21.641446 27583 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.013s	sys 0.019s
I20260709 12:24:21.699775 20734 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:24:21.700824 20630 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 14 to 15, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 15 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.703629 20700 leader_election.cc:400] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19.
I20260709 12:24:21.704030 20700 leader_election.cc:403] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:21.704133 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2738) took 1186 ms. Trace:
W20260709 12:24:21.704849 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48824.9us
W20260709 12:24:21.705766 27832 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.116s	user 0.006s	sys 0.005s
I20260709 12:24:21.706458 27814 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19.
I20260709 12:24:21.689863 27846 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:21.635357 27692 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:21.708144 27846 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:21.707917 27692 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:21.708815 27846 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:21.710551 27692 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:21.709976 27846 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:21.663970 27850 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
W20260709 12:24:21.664379 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 844211us
I20260709 12:24:21.727161 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:21.671645 27770 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:21.672901 27824 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:21.673285 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104476us
I20260709 12:24:21.681381 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:21.683122 20805 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:21.661621 27707 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:21.645725 27792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.004s	sys 0.015s
I20260709 12:24:21.679082 27833 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:21.044090 27745 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:21.713423 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:21.744272 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83943.7us
W20260709 12:24:21.714442 27785 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.311s	user 0.012s	sys 0.000s
I20260709 12:24:21.715112 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2797) took 1122 ms. Trace:
I20260709 12:24:21.664021 27771 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:21.705830 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:21.702852 20630 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:21.704584 20728 rpcz_store.cc:276] 0709 12:24:20.517939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.522480 (+  4541us) service_pool.cc:224] Handling call
0709 12:24:20.522844 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.523234 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.523264 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.523744 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.693066 (+169322us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:21.634542 (+941476us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.634820 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.634936 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.703891 (+ 68955us) raft_consensus.cc:1685] finished
0709 12:24:21.703914 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.704061 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15094,"prepare.queue_time_us":338069,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":9695,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":19394,"thread_start_us":1063145,"threads_started":2,"wal-append.queue_time_us":275412}
I20260709 12:24:21.734412 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:21.689839 27054 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:21.678560 27790 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:21.751526 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:21.757189 20806 rpcz_store.cc:276] 0709 12:24:20.592632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.610236 (+ 17604us) service_pool.cc:224] Handling call
0709 12:24:20.610564 (+   328us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:20.632452 (+ 21888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.632481 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:20.664257 (+ 31776us) log.cc:844] Serialized 78 byte log entry
0709 12:24:20.866082 (+201825us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:21.178540 (+312458us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.179578 (+  1038us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.201203 (+ 21625us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.201407 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.201486 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.659840 (+458354us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:24:21.714810 (+ 54970us) raft_consensus.cc:1685] finished
0709 12:24:21.714836 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.715010 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47496,"prepare.queue_time_us":798420,"raft.queue_time_us":1408260,"raft.run_wall_time_us":1096497,"spinlock_wait_cycles":212589568,"thread_start_us":419148,"threads_started":2,"wal-append.queue_time_us":290247}
W20260709 12:24:21.800848 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 119525us
W20260709 12:24:21.752131 20735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.019s	sys 0.072s
I20260709 12:24:21.766499 21427 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:21.817162 20735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.020s	sys 0.072s
I20260709 12:24:21.735512 27583 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:21.815933 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:21.706542 27832 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:21.732954 27698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.001s	sys 0.017s
W20260709 12:24:21.895203 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 158109us
I20260709 12:24:21.895812 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:21.739326 27692 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.739606 27814 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:21.902163 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:21.917434 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2740) took 1398 ms. Trace:
I20260709 12:24:21.919770 20630 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:21.917898 20731 rpcz_store.cc:276] 0709 12:24:20.518503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.530753 (+ 12250us) service_pool.cc:224] Handling call
0709 12:24:20.531301 (+   548us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:20.557076 (+ 25775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.557106 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:20.575959 (+ 18853us) log.cc:844] Serialized 52 byte log entry
0709 12:24:20.844187 (+268228us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:20.844257 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.844355 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.917172 (+1072817us) raft_consensus.cc:1685] finished
0709 12:24:21.917193 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.917362 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17823,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9843,"thread_start_us":224978,"threads_started":1,"wal-append.queue_time_us":369203}
I20260709 12:24:21.930765 20883 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:24:21.901590 27698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.002s	sys 0.017s
I20260709 12:24:21.926477 21427 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
W20260709 12:24:21.940948 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 125094us
I20260709 12:24:21.944465 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:21.906147 27792 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.896347 27867 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:21.944005 20807 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.004s	sys 0.026s
I20260709 12:24:21.749531 27707 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:21.899413 27784 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:21.906579 27054 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:21.981515 20810 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.022s	user 0.021s	sys 0.015s
W20260709 12:24:21.982224 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 80161.3us
W20260709 12:24:21.982223 20810 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.021s	sys 0.016s
I20260709 12:24:21.906628 27745 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
W20260709 12:24:21.958551 20807 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.005s	sys 0.026s
I20260709 12:24:21.916158 27833 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:21.960676 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 286028us
I20260709 12:24:21.929409 27744 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:21.938839 27685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.009s
I20260709 12:24:21.967672 20630 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 18, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 18 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:21.969565 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 263242us
W20260709 12:24:21.971467 20876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.007s	sys 0.026s
W20260709 12:24:21.998306 20876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.007s	sys 0.027s
I20260709 12:24:21.802171 27785 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:21.956954 27723 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.187s	user 0.012s	sys 0.001s
I20260709 12:24:21.984543 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2799) took 1366 ms. Trace:
W20260709 12:24:21.558360 20628 rpcz_store.cc:269] 0709 12:23:52.918538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.971000 (+ 52462us) service_pool.cc:224] Handling call
0709 12:24:20.698286 (+27727286us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:06.368685 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:06.419936 (+ 51251us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:06.420000 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:24:06.420021 (+    21us) tablet.cc:662] Decoding operations
0709 12:24:06.757464 (+337443us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:06.785938 (+ 28474us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:06.793566 (+  7628us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:24:07.287318 (+493752us) tablet.cc:701] Row locks acquired
0709 12:24:07.287373 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:24:07.306004 (+ 18631us) write_op.cc:270] Start()
0709 12:24:07.338882 (+ 32878us) write_op.cc:276] Timestamp: P: 1783599847289385 usec, L: 0
0709 12:24:07.342955 (+  4073us) op_driver.cc:348] REPLICATION: starting
0709 12:24:07.547530 (+204575us) log.cc:844] Serialized 7877 byte log entry
0709 12:24:08.752095 (+1204565us) spinlock_profiling.cc:243] Waited 60.4 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1bb950ac 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b
0709 12:24:10.304653 (+1552558us) op_driver.cc:464] REPLICATION: finished
0709 12:24:10.539951 (+235298us) write_op.cc:301] APPLY: starting
0709 12:24:10.540096 (+   145us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:11.548486 (+1008390us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:11.548566 (+    80us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:12.730371 (+1181805us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:12.973579 (+243208us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 12:24:12.976981 (+  3402us) write_op.cc:312] APPLY: finished
0709 12:24:13.000411 (+ 23430us) log.cc:844] Serialized 180 byte log entry
0709 12:24:13.252950 (+252539us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:13.398092 (+145142us) write_op.cc:454] Released schema lock
0709 12:24:13.398193 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:13.398876 (+   683us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:13.398910 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1943344,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":193395,"apply.run_cpu_time_us":72041,"apply.run_wall_time_us":2858897,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87727,"num_ops":20,"prepare.queue_time_us":157253,"prepare.run_cpu_time_us":61672,"prepare.run_wall_time_us":2684509,"raft.queue_time_us":786077,"raft.run_cpu_time_us":10123,"raft.run_wall_time_us":271560,"replication_time_us":2919997,"spinlock_wait_cycles":138837760,"thread_start_us":2666722,"threads_started":5,"wal-append.queue_time_us":2010907,"wal-append.run_cpu_time_us":30432,"wal-append.run_wall_time_us":3034984}]]}
W20260709 12:24:21.958004 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.012s	sys 0.020s
W20260709 12:24:22.012063 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.012s	sys 0.020s
I20260709 12:24:21.963950 27867 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.948529 27273 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:21.915750 27692 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.008524 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1223) took 1365 ms. Trace:
W20260709 12:24:22.010887 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.018910 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74510.5us
I20260709 12:24:22.022643 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:22.023494 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.024109 20876 rpcz_store.cc:276] 0709 12:24:20.643314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.759252 (+115938us) service_pool.cc:224] Handling call
0709 12:24:20.759587 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.759858 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:21.105103 (+345245us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.008021 (+902918us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.008090 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:22.008160 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.008279 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.008441 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9123,"prepare.queue_time_us":737514,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":1109673,"threads_started":2}
I20260709 12:24:22.016125 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2750) took 1368 ms. Trace:
I20260709 12:24:22.026317 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:22.026443 20735 rpcz_store.cc:276] 0709 12:24:20.647389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.700391 (+ 53002us) service_pool.cc:224] Handling call
0709 12:24:20.716634 (+ 16243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.735620 (+ 18986us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:20.961855 (+226235us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.015766 (+1053911us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.015839 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:22.015887 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.015946 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.016070 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86108,"prepare.queue_time_us":295751,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":153626,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":2896,"thread_start_us":1010957,"threads_started":3}
I20260709 12:24:22.027657 20875 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:22.007385 27685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.008s	sys 0.009s
W20260709 12:24:22.031107 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47992.8us
W20260709 12:24:22.028781 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 148676us
I20260709 12:24:22.034165 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:22.021848 20810 rpcz_store.cc:276] 0709 12:24:20.618262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.618396 (+   134us) service_pool.cc:224] Handling call
0709 12:24:20.633278 (+ 14882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.664801 (+ 31523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.664836 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.665404 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.876844 (+211440us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:21.983912 (+1107068us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.984195 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.984286 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.984331 (+    45us) raft_consensus.cc:1685] finished
0709 12:24:21.984345 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.984483 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63182,"prepare.queue_time_us":818712,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"spinlock_wait_cycles":88448,"thread_start_us":1170004,"threads_started":2,"wal-append.queue_time_us":504838}
I20260709 12:24:22.032840 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1228) took 1332 ms. Trace:
I20260709 12:24:21.959336 27792 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.007062 27698 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:22.015388 27867 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.121618 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.128465 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.015996 27833 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.984143 27784 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.041018 21432 rpcz_store.cc:276] 0709 12:24:20.691664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.691796 (+   132us) service_pool.cc:224] Handling call
0709 12:24:20.703522 (+ 11726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.703759 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:21.654373 (+950614us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.023069 (+368696us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.023238 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.023281 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:22.023334 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.023398 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.023501 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":961429,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":26410,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":18466,"thread_start_us":1261657,"threads_started":3}
I20260709 12:24:22.135460 21429 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:24:21.988373 27054 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:22.102667 27850 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:24:22.145100 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 122520us
I20260709 12:24:22.015974 20806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
W20260709 12:24:22.212096 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 506304us
W20260709 12:24:22.147095 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140489us
I20260709 12:24:22.231582 20806 raft_consensus.cc:606] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:22.189076 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 198442us
I20260709 12:24:22.031646 27273 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.237028 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:22.212383 21435 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:24:22.183909 27858 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:24:22.134888 27815 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:22.182825 27874 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:24:22.142109 27599 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.235975 20806 raft_consensus.cc:614] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:22.251366 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.184441 27824 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.187127 27889 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:22.183872 27875 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:22.183746 27832 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.244182 27698 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:22.185067 27790 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.184026 27849 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:22.147291 27833 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.262697 27685 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:22.283010 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51007.9us
I20260709 12:24:22.275731 27827 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:22.292565 27879 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:22.292801 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 258746us
I20260709 12:24:22.262287 27815 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.297267 27855 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:22.297622 27054 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:22.297770 27864 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
W20260709 12:24:22.298187 27869 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:24:22.299551 20735 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:22.300714 27833 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.301824 27784 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.301981 27875 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:22.302371 27785 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.303251 27883 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:22.304540 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.305351 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.342916 27883 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:22.306103 27692 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:22.307323 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56056.4us
I20260709 12:24:22.321902 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.319005 20884 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:24:22.325992 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 204538us
W20260709 12:24:22.326393 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89627.2us
I20260709 12:24:22.265694 27599 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.279980 27892 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.355571 27892 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.283118 27698 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:22.268730 27824 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.294411 27827 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:22.348727 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1225) took 1663 ms. Trace:
I20260709 12:24:22.349928 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:22.359949 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 127406us
I20260709 12:24:22.328871 27707 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:22.364475 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:22.368088 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48991.4us
I20260709 12:24:22.363894 21433 rpcz_store.cc:276] 0709 12:24:20.685625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.685800 (+   175us) service_pool.cc:224] Handling call
0709 12:24:20.686213 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.699834 (+ 13621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.699864 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.712370 (+ 12506us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.070655 (+358285us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:21.070713 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.070813 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.348495 (+1277682us) raft_consensus.cc:1685] finished
0709 12:24:22.348515 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.348658 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5812,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":9490,"thread_start_us":287016,"threads_started":1,"wal-append.queue_time_us":972566}
I20260709 12:24:22.369334 20700 leader_election.cc:400] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:22.370594 20700 leader_election.cc:400] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
I20260709 12:24:22.370865 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.372757 27896 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.373018 27846 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:22.374529 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2795) took 1779 ms. Trace:
I20260709 12:24:22.274828 27889 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.370918 20700 leader_election.cc:403] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:24:22.377413 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.372752 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.320999 27744 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:24:22.379714 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.362679 27875 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:22.331133 27685 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.375020 20807 rpcz_store.cc:276] 0709 12:24:20.591894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.616871 (+ 24977us) service_pool.cc:224] Handling call
0709 12:24:20.617214 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.624625 (+  7411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.624652 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.643024 (+ 18372us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.207164 (+564140us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:21.989358 (+782194us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.036513 (+ 47155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.038811 (+  2298us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.370972 (+332161us) raft_consensus.cc:1685] finished
0709 12:24:22.370999 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.371154 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43045,"prepare.queue_time_us":455653,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":308279,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":62891,"thread_start_us":1213062,"threads_started":3,"wal-append.queue_time_us":1107342}
I20260709 12:24:22.395702 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.395993 20879 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:22.396342 20879 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:22.396489 20728 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:22.397238 20879 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.398273 20879 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.400424 27824 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.374594 27692 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:22.401932 27692 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.403522 27894 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:22.404079 27894 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.407990 20807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:22.408705 20807 raft_consensus.cc:606] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:22.409669 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.412818 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.415772 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.358525 27892 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.421571 27827 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.404448 27815 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.422580 27902 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:24:22.422995 27679 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:22.423646 27822 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:22.373728 27896 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.425020 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1290) took 1719 ms. Trace:
I20260709 12:24:22.438468 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:22.438561 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65951.7us
I20260709 12:24:22.428025 20879 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 12:24:22.428910 21435 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:22.439204 20876 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:22.439352 20879 pending_rounds.cc:85] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 15
I20260709 12:24:22.439679 20879 pending_rounds.cc:107] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 12:24:22.439939 20879 raft_consensus.cc:2889] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:22.422071 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.408344 27846 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:22.404465 27692 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.418730 27824 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
W20260709 12:24:22.437124 27781 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.748s	user 0.014s	sys 0.000s
I20260709 12:24:22.427790 27897 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:22.385313 27707 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:24:22.443830 20849 leader_election.cc:400] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:22.444959 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:22.447466 20875 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:22.438889 20809 rpcz_store.cc:276] 0709 12:24:20.705977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.706209 (+   232us) service_pool.cc:224] Handling call
0709 12:24:20.706612 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:20.995930 (+289318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.995981 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:21.575900 (+579919us) log.cc:844] Serialized 52 byte log entry
0709 12:24:21.989909 (+414009us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:21.989960 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.992215 (+  2255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.424818 (+432603us) raft_consensus.cc:1685] finished
0709 12:24:22.424837 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.424965 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40552,"raft.queue_time_us":70761,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":10057,"thread_start_us":573427,"threads_started":2,"wal-append.queue_time_us":722586}
I20260709 12:24:22.449805 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.440373 21432 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 25. Ops: []
W20260709 12:24:22.436839 27024 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.004s
I20260709 12:24:22.441089 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:22.404943 27698 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.536208 20880 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.436977 27892 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:22.535262 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:22.516517 21428 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:22.564752 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.539443 20849 leader_election.cc:403] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:24:22.539798 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 169362us
I20260709 12:24:22.524199 20807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:22.444555 27822 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:22.566591 20807 raft_consensus.cc:606] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:22.584160 20876 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:22.584275 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:22.588421 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 138684us
I20260709 12:24:22.590010 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:22.592271 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:22.593187 21435 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.517894 27827 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:22.506635 27744 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:22.524138 27891 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.564463 27889 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.532327 27685 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.523963 27829 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.602444 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:22.607609 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 174404us
W20260709 12:24:22.609584 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 199997us
I20260709 12:24:22.437122 27883 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.595942 27692 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:22.631377 27893 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:24:22.632856 27893 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.633314 27893 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.605882 27902 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.605818 27824 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:22.636369 20880 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:22.636540 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.637712 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.614806 27903 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.639051 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:22.639848 21431 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:22.639585 27903 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.634022 27846 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.615316 27895 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:22.643749 27895 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.646413 27895 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.615778 27907 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:22.647272 27907 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:22.616163 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.616469 27906 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:22.656900 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:22.617527 27877 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:22.619472 21429 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:22.619814 27829 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.620059 27908 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:22.622081 27822 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.623996 27894 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:22.624467 27905 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:24:22.600847 27744 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:22.627288 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2749) took 1993 ms. Trace:
I20260709 12:24:22.689510 20733 rpcz_store.cc:276] 0709 12:24:20.633788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.633942 (+   154us) service_pool.cc:224] Handling call
0709 12:24:20.634387 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.698399 (+ 64012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.698433 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.722078 (+ 23645us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.026984 (+304906us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:21.027046 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.027152 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.627032 (+1599880us) raft_consensus.cc:1685] finished
0709 12:24:22.627050 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.627216 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43663,"raft.queue_time_us":43816,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":12253,"thread_start_us":249784,"threads_started":1,"wal-append.queue_time_us":1811676}
I20260709 12:24:22.513918 27781 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.601785 27897 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:22.578804 27024 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.668040 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:22.669644 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103505us
I20260709 12:24:22.657148 27692 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:22.696585 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.694615 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.665664 27895 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:24:22.731364 27900 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:24:22.732719 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64787.8us
I20260709 12:24:22.734496 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.735399 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.736145 21432 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 13.
I20260709 12:24:22.737110 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.737013 27832 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:22.735565 27054 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:22.738559 27273 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.739357 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:22.741868 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.699723 27907 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:22.749636 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53159.4us
I20260709 12:24:22.715799 20880 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:22.750625 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.685249 27707 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:22.754257 20884 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 14.
I20260709 12:24:22.752266 21435 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
W20260709 12:24:22.758093 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.658619 27903 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.693401 27685 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.666035 27889 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 12:24:22.693305 27744 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:22.772491 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.759414 21426 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:24:22.751132 27588 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.061s	user 0.006s	sys 0.014s
I20260709 12:24:22.738577 27054 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.824642 27054 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:22.811844 27692 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.830010 27692 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.830878 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2794) took 2239 ms. Trace:
I20260709 12:24:22.831655 27889 consensus_queue.cc:1243] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 25.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:24:22.693645 27896 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:22.604873 27891 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.813405 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.747432 27893 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.759398 27906 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:22.759575 27824 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:22.817451 27829 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.819299 27820 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.831542 20808 rpcz_store.cc:276] 0709 12:24:20.591333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.593197 (+  1864us) service_pool.cc:224] Handling call
0709 12:24:20.637246 (+ 44049us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.995467 (+358221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.995514 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.575289 (+579775us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.946999 (+371710us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:21.947036 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.947113 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.830602 (+883489us) raft_consensus.cc:1685] finished
0709 12:24:22.830624 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.830784 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29730,"raft.queue_time_us":80695,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5674,"thread_start_us":560918,"threads_started":2,"wal-append.queue_time_us":802898}
W20260709 12:24:22.846467 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95690us
I20260709 12:24:22.832407 27903 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:22.832072 27781 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:22.848249 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75865.3us
I20260709 12:24:22.865578 21431 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:22.868934 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:22.849321 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107560us
W20260709 12:24:22.845479 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 246696us
I20260709 12:24:22.850919 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.871734 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.850970 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.843900 27899 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:24:22.882110 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.882954 20733 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:22.884153 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.885290 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.886076 20729 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:22.887691 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:22.888329 20726 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.893469 20733 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:22.894865 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.896381 20733 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:22.886932 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:22.897661 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46779.9us
I20260709 12:24:22.898437 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.872563 21399 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:22.848980 27707 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:22.905468 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.904174 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.908881 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.925974 20726 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:22.833775 27685 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.858496 27903 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:22.941102 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:22.928959 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 189644us
I20260709 12:24:22.939597 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.939436 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:22.940362 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:22.898512 27886 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:22.942919 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:22.903045 27829 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:22.895954 27906 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:22.904568 27867 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.002s
I20260709 12:24:22.945477 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.947155 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:22.948616 20726 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:22.945601 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:22.948887 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.949584 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:22.858071 27820 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:22.956768 27875 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:22.949754 27588 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:24:22.967265 27692 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.968974 27820 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.974015 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:22.944291 27903 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:22.951754 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:22.929186 27698 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:24:22.947721 27867 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:22.944643 27316 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:22.995402 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:22.981166 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77045.9us
I20260709 12:24:23.015686 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:22.982015 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:23.028016 20730 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 12:24:23.028053 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46131.9us
I20260709 12:24:22.979753 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:22.997859 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.017356 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:23.029870 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1226) took 2368 ms (client timeout 1977 ms). Trace:
W20260709 12:24:23.031037 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57090.6us
I20260709 12:24:23.031140 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2796) took 2438 ms. Trace:
I20260709 12:24:23.033666 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.029095 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:23.033066 20883 rpcz_store.cc:269] 0709 12:24:20.644127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.678691 (+ 34564us) service_pool.cc:224] Handling call
0709 12:24:23.012762 (+2334071us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:23.012968 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817753344}
I20260709 12:24:22.959565 27790 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.039050 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:23.045020 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:23.044842 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:22.971320 27024 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:22.982982 27832 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:22.981281 27692 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:23.009521 27809 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:22.981346 27896 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:22.996165 27849 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:22.971350 27685 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.060596 21433 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:23.007820 27913 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:22.982410 27863 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:23.033526 20804 rpcz_store.cc:276] 0709 12:24:20.592267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.617570 (+ 25303us) service_pool.cc:224] Handling call
0709 12:24:20.617859 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.633325 (+1015466us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:21.786797 (+153472us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.794260 (+  7463us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.795019 (+   759us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.795211 (+   192us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:21.795931 (+   720us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.796155 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.796256 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.738910 (+942654us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b180036a060. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:23.030802 (+291892us) raft_consensus.cc:1685] finished
0709 12:24:23.030844 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.031047 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1418127,"prepare.queue_time_us":65976,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":420023,"raft.run_cpu_time_us":517,"raft.run_wall_time_us":1340538,"spinlock_wait_cycles":18628992,"thread_start_us":1099008,"threads_started":3,"wal-append.queue_time_us":153676}
I20260709 12:24:23.040408 27897 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:23.070974 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:22.977277 27875 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.053436 27883 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:23.060364 27692 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.105257 20733 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:22.981782 27921 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 12:24:23.117352 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.064601 27809 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:23.121282 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 171784us
I20260709 12:24:23.122323 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:23.105365 27316 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:23.106415 27897 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.104503 27863 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.059738 27790 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.088327 27849 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.123112 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:23.127954 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88275.9us
I20260709 12:24:23.133543 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.134931 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.136351 27745 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.131595 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.119252 27692 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:23.117854 27832 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:23.119390 27883 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:23.140451 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69601.7us
I20260709 12:24:23.141515 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.141775 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:23.062603 27910 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:24:23.143321 20804 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:23.146768 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.127694 27897 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:23.146008 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.131065 27707 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:23.147492 20804 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:23.128387 27863 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:23.106336 27896 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.139456 27685 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:23.155807 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.138757 27849 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:23.144485 27883 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:23.158063 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:23.152175 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.139683 27781 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:23.144479 27893 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:23.119024 27913 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.145926 27921 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:23.217438 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.144510 27922 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:23.218804 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.221153 20732 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:23.228776 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.143638 27745 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.159956 27909 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:24:23.229907 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71935.7us
I20260709 12:24:23.278486 27909 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.237005 27893 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.241899 27913 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.242057 27896 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:23.242867 27827 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.000s
I20260709 12:24:23.228583 27750 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:23.287774 27750 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.288609 27897 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.244261 27054 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.248521 27790 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:23.243062 27832 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.292086 27054 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:23.231181 27707 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:23.253885 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.294348 20880 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:23.254303 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.295470 20876 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:23.295387 27863 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.256049 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.256224 20803 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:23.297526 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.297626 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.256948 27875 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:23.257628 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.259560 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.259713 27744 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.260883 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.269135 27745 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:23.270748 20728 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:23.271315 20729 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:23.271204 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.309404 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:23.310245 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.311869 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.312583 20801 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:23.314468 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.319450 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.149801 27891 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:23.325979 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:23.326911 20801 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:23.333444 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.334188 20877 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:23.336167 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.335605 20809 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:23.144676 27926 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:23.282181 20732 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:23.283221 27893 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.293730 27750 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.302155 20802 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
I20260709 12:24:23.303223 20804 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:23.309404 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.342792 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:23.343663 21430 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:23.300349 27909 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:23.275761 27903 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.325362 27744 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:23.337134 20879 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:23.360282 27863 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.361403 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:23.362147 27889 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:23.362640 27889 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:23.291002 27897 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:23.341163 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.344055 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:23.364078 20877 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:23.364187 20875 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:23.345134 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.362197 20806 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:23.342028 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:23.348831 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:23.349294 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:23.351258 27934 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:23.325979 21431 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:23.346312 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.364912 20801 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:23.365908 20881 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:24:23.368216 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 229900us
I20260709 12:24:23.366647 20809 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
W20260709 12:24:23.365448 20701 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.573s (SENT) [suppressed 26 similar messages]
I20260709 12:24:23.276002 27809 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.274672 27692 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.380999 20884 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:23.382095 27692 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:23.397328 20801 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:23.378902 27809 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:23.411481 20809 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:23.363814 27926 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:23.435745 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 215893us
I20260709 12:24:23.341291 27849 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.447417 20883 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:23.411316 27934 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.481961 20877 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:23.448937 27849 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.419031 27889 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.513121 20809 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:24:23.484232 27814 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.545s	user 0.008s	sys 0.022s
W20260709 12:24:23.512508 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72738us
W20260709 12:24:23.514504 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 149067us
W20260709 12:24:23.531112 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.525386 27883 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.532943 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.534694 20810 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:23.536521 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.531597 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:23.537221 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:23.537247 20732 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:23.540462 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.536089 27883 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:23.544780 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:23.537400 27814 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.608s	user 0.009s	sys 0.027s
I20260709 12:24:23.550738 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.551445 20809 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:23.546751 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:23.548935 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 180006us
I20260709 12:24:23.553128 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.561446 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:23.563162 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.564253 20809 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:23.565083 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.550443 27889 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:24:23.587764 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.567759 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:23.568399 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.550277 27814 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:23.590299 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.593909 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.600307 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.604197 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:23.567963 27832 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:23.516256 27926 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.604777 27814 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.627530 27832 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:23.628059 27832 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:23.609330 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.565892 27922 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:23.631708 20732 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:23.631534 27922 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.619307 27897 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:23.620976 27934 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.557199 27588 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:23.623361 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.638135 20728 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:23.560788 27692 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:23.592823 27903 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.566094 27920 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:23.648942 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95410.2us
I20260709 12:24:23.649524 27920 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:23.652971 27849 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:23.652971 27943 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:23.660107 27909 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.663975 27909 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.636493 27897 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:23.692113 27922 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:23.700740 20729 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:23.702287 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:23.700886 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50042.8us
I20260709 12:24:23.708560 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.709391 20727 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:23.710975 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.711957 20801 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:23.712735 20728 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:23.712945 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.713501 20809 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:23.713719 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.714319 20728 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:23.714977 20804 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:23.715901 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.717048 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.718161 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.718609 20880 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:24:23.719296 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.718808 20804 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:23.719607 20802 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:23.727269 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.733585 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.734216 20733 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:23.740028 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:23.740693 20801 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:23.740708 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 140532us
I20260709 12:24:23.742347 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.651041 27692 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.751410 20727 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:24:23.743095 27771 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.317s	user 0.002s	sys 0.033s
I20260709 12:24:23.707957 27897 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:23.759768 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.753446 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.775624 20802 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:23.756069 27771 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.344s	user 0.002s	sys 0.033s
I20260709 12:24:23.753858 27889 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:23.772879 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.773293 20728 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:23.752784 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.777629 20809 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
W20260709 12:24:23.761945 27940 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.294s	user 0.000s	sys 0.003s
I20260709 12:24:23.774374 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.778434 20727 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:23.778538 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.781057 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.783696 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:23.774076 27824 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:23.778084 27954 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:24:23.786330 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:23.787164 27926 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:23.793572 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.783690 20733 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:23.794524 20726 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:23.795863 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.795895 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.788062 27771 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:23.796583 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.800515 20809 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:23.801919 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.799566 20804 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:23.806236 20804 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:24:23.809158 20809 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:23.811363 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.797897 27889 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:23.797947 27926 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:23.833014 27926 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.800038 27903 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:23.822517 20732 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:23.826335 20804 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:23.829721 27889 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:23.807520 27922 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:24:23.828006 27962 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:23.844779 27889 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:23.864615 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.825531 27771 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
W20260709 12:24:23.869751 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70658.7us
I20260709 12:24:23.879397 20804 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:23.880569 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:23.882114 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.873945 27692 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.910573 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:23.914372 20809 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:23.916159 20801 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:23.929406 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47353.6us
I20260709 12:24:23.931916 27908 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:23.932340 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:23.869194 27962 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.936887 20802 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:23.939599 27965 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:23.945168 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.933703 27926 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:23.956725 27926 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:23.957696 27926 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.960758 27962 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.960479 27832 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:23.961073 27822 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:23.961349 27962 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:23.962692 27832 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:23.962565 27965 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:23.964843 27965 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:23.950922 27969 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:23.966123 27969 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:23.978381 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57744.9us
W20260709 12:24:23.978981 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59172.4us
I20260709 12:24:23.983705 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:23.987531 20876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:23.988261 20876 raft_consensus.cc:606] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:23.988662 20876 raft_consensus.cc:614] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:23.991865 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:23.996273 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1288) took 3300 ms. Trace:
I20260709 12:24:23.996728 20805 rpcz_store.cc:276] 0709 12:24:20.695620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.704828 (+  9208us) service_pool.cc:224] Handling call
0709 12:24:20.731311 (+ 26483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.739920 (+1008609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.996021 (+2256101us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b54003c9954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:23.996204 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188390272}
I20260709 12:24:23.998698 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:23.998804 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:24.007135 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:24.016034 20803 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:24.017554 27925 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.855s	user 0.004s	sys 0.000s
I20260709 12:24:24.019870 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2762) took 1139 ms. Trace:
I20260709 12:24:24.020298 20734 rpcz_store.cc:276] 0709 12:24:22.880154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.880314 (+   160us) service_pool.cc:224] Handling call
0709 12:24:22.880732 (+   418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:22.903468 (+ 22736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:22.903499 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:22.941478 (+ 37979us) log.cc:844] Serialized 52 byte log entry
0709 12:24:23.106155 (+164677us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:23.147543 (+ 41388us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.327727 (+180184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.327813 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.016892 (+689079us) spinlock_profiling.cc:243] Waited 55 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:24.019505 (+  2613us) raft_consensus.cc:1685] finished
0709 12:24:24.019637 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.019801 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":344730,"prepare.queue_time_us":108400,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":120101,"spinlock_wait_cycles":180206976,"thread_start_us":333703,"threads_started":3,"wal-append.queue_time_us":201267}
I20260709 12:24:24.024075 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:24.026949 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.027030 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:24.029523 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:24.029775 27510 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:24.030508 27385 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:24.031162 27385 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:24.032223 27385 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.034111 27786 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:24.034637 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:24.034861 27786 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:24.035836 27966 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:24:24.036808 27792 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.037744 27908 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:24.038406 27908 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:24.038574 27903 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:24.039445 27908 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.036051 27786 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.047008 20803 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:24:24.048965 20803 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:24.049585 20803 raft_consensus.cc:606] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:24.049957 20803 raft_consensus.cc:614] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:24.051331 27903 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:24.051573 27887 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.051983 27903 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.052389 27887 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.054167 27903 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:24.054940 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:24.055927 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:24.056563 20734 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:24.057045 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:24.057380 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:24.058061 20732 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:24.058341 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:24.030521 27510 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:24.061141 27510 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.067575 27897 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.059144 27966 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:24:24.065171 27903 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.102905 27897 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:24.104569 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1342) took 1692 ms. Trace:
I20260709 12:24:24.105055 21433 rpcz_store.cc:276] 0709 12:24:22.412226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.412439 (+   213us) service_pool.cc:224] Handling call
0709 12:24:24.104356 (+1691917us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:24.104490 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1473046784}
I20260709 12:24:24.087756 27887 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:24.094743 27903 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.130774 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:24.133407 27903 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:24.136045 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:24.138976 27919 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:24.142539 27889 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.143625 27892 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:24:24.145885 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 149217us
I20260709 12:24:24.158347 27892 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:24.148149 27919 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:24.196847 20808 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:24.197980 20734 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:24.200078 20732 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:24.200474 20732 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:24.204530 20732 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.204771 27919 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.205598 20732 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:24.205929 27919 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:24.203729 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 271453us
I20260709 12:24:24.210397 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:24.216682 27973 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:24.217285 20700 leader_election.cc:400] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:24.218338 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:24.218384 27973 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:24.219807 27984 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:24.187932 27792 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:24:24.223892 27792 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:24.224285 27792 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:24.224385 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.225122 27792 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.231771 27792 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:24.242240 20732 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:24.254570 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2937) took 2238 ms. Trace:
W20260709 12:24:24.264277 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46024.6us
I20260709 12:24:24.265594 27984 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:24.266075 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:24.267233 27984 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:24.267509 27936 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:24.267755 27984 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:24.268471 27871 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.853s	user 0.002s	sys 0.004s
I20260709 12:24:24.269313 27936 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:24.269768 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:24.270151 27892 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.271255 27892 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.275417 27588 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:24.275889 27588 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:24.275938 27863 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.276407 27588 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:24.276536 27863 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.278740 27863 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.269742 27936 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:24.268419 27984 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.285140 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.287289 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:24.288048 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.293737 20882 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:24.294265 20877 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:24.255262 20878 rpcz_store.cc:276] 0709 12:24:22.015963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.016835 (+   872us) service_pool.cc:224] Handling call
0709 12:24:22.017139 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.017403 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:22.044183 (+ 26780us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.106853 (+2062670us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18003ea6a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:24:24.253339 (+146486us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.254061 (+   722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.254121 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:24.254191 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.254328 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.254482 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51078,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":115,"raft.queue_time_us":122,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":9577,"spinlock_wait_cycles":32244992,"thread_start_us":145830,"threads_started":2,"wal-append.queue_time_us":2063761}
I20260709 12:24:24.295799 20877 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:24.298291 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.298619 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.300330 20882 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:24.300513 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.301213 20880 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:24.300417 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.302232 20876 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 23 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:24.279963 27889 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:24.302700 27982 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:24:24.294086 27970 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:24:24.388140 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:24.393640 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:24.396325 20801 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:24:24.391408 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.396335 27936 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.361698 27599 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.406307 27867 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.411080 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3056) took 1194 ms. Trace:
I20260709 12:24:24.411572 20806 rpcz_store.cc:276] 0709 12:24:23.216982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.217143 (+   161us) service_pool.cc:224] Handling call
0709 12:24:24.410831 (+1193688us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b540027c254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:24.410949 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106837760}
I20260709 12:24:24.416877 20882 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:24:24.418581 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.423173 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.427800 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:24.434664 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.425837 27868 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:24.439946 27868 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.442495 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:24.445223 27930 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.446936 20875 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:24.309046 27973 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:24.307547 27971 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:24.430683 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:24.450604 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.450956 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.451887 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:24.451622 21432 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:24.456104 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:24.451527 21428 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:24.463042 21435 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:24.465649 21432 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:24.467152 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.470566 21426 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:24.471768 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:24.474275 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:24.474915 21430 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.467939 21434 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:24.476297 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:24.446221 27930 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.447677 27868 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:24.459731 27815 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:24.486347 27887 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:24.480022 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:24.487663 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.479502 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:24.482985 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:24.365438 27588 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.459970 27995 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:24.460686 27024 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:24.485493 20875 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:24.486171 27982 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:24.488677 21429 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:24.489421 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:24.489840 20732 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:24:24.648658 20732 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 10
I20260709 12:24:24.649046 20732 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:24:24.649367 20732 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:24.486737 27815 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:24.651160 27815 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:24.651655 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:24.495605 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:24.495929 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:24.466940 27771 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:24.470069 27891 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.650902 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:24.650820 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:24.655858 20734 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.553153 21434 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:24.564224 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:24.589783 27930 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:24.595858 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2883) took 1533 ms. Trace:
I20260709 12:24:24.591861 27695 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:24.593540 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:24.563284 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.665117 20883 rpcz_store.cc:276] 0709 12:24:23.080734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.080971 (+   237us) service_pool.cc:224] Handling call
0709 12:24:23.116580 (+ 35609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.447925 (+331345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.595553 (+1147628us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:24.595786 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760207616}
I20260709 12:24:24.665643 20804 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:24.492705 27897 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.096s
I20260709 12:24:24.590967 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3057) took 1369 ms. Trace:
I20260709 12:24:24.678067 20810 rpcz_store.cc:276] 0709 12:24:23.221476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.221631 (+   155us) service_pool.cc:224] Handling call
0709 12:24:23.231036 (+  9405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.535430 (+304394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.590641 (+1055211us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400127354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:24.590888 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697714432}
I20260709 12:24:24.569340 27822 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:24.570657 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.460084 27971 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.575397 27024 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.463757 27984 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.494552 27937 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:24.561291 20806 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:24.653256 27982 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:24.463152 27973 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:24.572820 27996 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:24:24.494539 27892 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:24.570668 20805 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:24.494932 27750 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.098s
I20260709 12:24:24.584854 27588 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.562197 21426 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:24.586833 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2882) took 1592 ms. Trace:
I20260709 12:24:24.478554 27809 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 12:24:24.737762 20884 rpcz_store.cc:276] 0709 12:24:22.994787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.995098 (+   311us) service_pool.cc:224] Handling call
0709 12:24:24.586557 (+1591459us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54001ff254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:24.586691 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816839936}
I20260709 12:24:24.494614 27995 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.657394 21428 raft_consensus.cc:2376] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17. Re-sending same reply.
I20260709 12:24:24.661460 21430 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:24.693773 27964 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:24.661974 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166094us
I20260709 12:24:24.683583 21435 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
W20260709 12:24:24.686987 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 191488us
I20260709 12:24:24.675848 27695 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:24.795718 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:24.796111 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.684638 27962 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:24:24.684979 27891 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.723757 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.812064 21434 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:24.723886 20735 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:24.813290 20807 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 12:24:24.725554 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 272471us
W20260709 12:24:24.779721 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.815568 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1728) took 1476 ms. Trace:
W20260709 12:24:24.706586 27900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.922s	user 0.010s	sys 0.016s
I20260709 12:24:24.653880 27887 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:24.712836 27982 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:24.801049 20883 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:24.695118 27824 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:24:24.682695 27771 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.683091 27863 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:24.805029 20875 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:24:24.809839 27868 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.818274 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:24.823863 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:24.826023 20884 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:24.703421 27599 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 12:24:24.796818 27995 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.724710 27960 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 12:24:24.738054 27892 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.811614 27849 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.844230 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.814503 27992 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:24:24.846534 20734 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:24.816111 21427 rpcz_store.cc:276] 0709 12:24:23.339688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.339869 (+   181us) service_pool.cc:224] Handling call
0709 12:24:23.340149 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.340367 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:23.485155 (+144788us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.558776 (+ 73621us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.865738 (+306962us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b18001ea7e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:24.433920 (+568182us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b18001ea7e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:24.690451 (+256531us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.811049 (+120598us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.811106 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:24.811159 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.811248 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.815506 (+  4258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":605827,"prepare.queue_time_us":19211,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":893888,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":198,"spinlock_wait_cycles":16957312,"thread_start_us":528823,"threads_started":4,"wal-append.queue_time_us":123415,"wal-append.run_cpu_time_us":1597,"wal-append.run_wall_time_us":1045037}
W20260709 12:24:24.826639 27900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.046s	user 0.010s	sys 0.016s
W20260709 12:24:24.815112 27940 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:24:24.857088 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:24.837761 27899 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.850847 27930 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:24.873986 27997 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:24.850648 27863 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:24.882586 27998 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:24:24.847597 27791 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
W20260709 12:24:24.885699 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93567.6us
I20260709 12:24:24.887089 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:24.888504 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:24.847482 27695 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.875524 27930 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.847878 27868 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.893738 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:24.894941 20806 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:24.864516 27744 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:24:24.900563 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:24.871546 27910 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:24:24.881388 27997 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:24.883708 27984 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.884370 27937 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.903127 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 227315us
I20260709 12:24:24.905872 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:24.908754 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51733.5us
I20260709 12:24:24.889436 27822 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:24.913436 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.891878 27962 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.895676 27679 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:24.904011 27906 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:24.966261 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2783) took 1072 ms. Trace:
I20260709 12:24:24.978137 20728 rpcz_store.cc:276] 0709 12:24:23.893412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.896125 (+  2713us) service_pool.cc:224] Handling call
0709 12:24:23.896416 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.024652 (+128236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.024685 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.024736 (+    51us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:24.024771 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.966002 (+941231us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540028c154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:24.966050 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.966197 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294332288}
W20260709 12:24:24.970376 21431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1360) took 2027 ms (client timeout 2195 ms). Trace:
I20260709 12:24:24.908322 27867 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:24.917716 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:24.918646 20884 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:24.900341 27930 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:24.872872 27849 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.877516 27927 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:24:24.877545 27899 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:24.911525 27695 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.986162 20883 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:24.904142 27054 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:24.883713 27887 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.996243 27855 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:24.997411 27849 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.000041 20877 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:24.980553 21431 rpcz_store.cc:269] 0709 12:24:22.942534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.942686 (+   152us) service_pool.cc:224] Handling call
0709 12:24:24.970110 (+2027424us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:24.970296 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874829056}
I20260709 12:24:24.932638 27679 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:24.934262 27910 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:24.932332 28014 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:24.963959 27891 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.991533 27867 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.948079 27935 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:24.963235 27024 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:24.943732 27971 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:24.964157 27951 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.143s
I20260709 12:24:24.939846 28003 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.113s
I20260709 12:24:24.924188 27937 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.973302 27962 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:24.925489 27822 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:25.036936 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.095616 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.057864 27935 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:25.057977 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 170942us
I20260709 12:24:25.082602 27679 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.082183 27899 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:25.093750 27892 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.094154 27868 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.054517 28014 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:25.053536 27855 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:25.105152 28014 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.105412 27855 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.106478 28014 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.106650 27855 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:25.057695 27771 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.057936 27867 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:25.103595 27935 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.113060 27935 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:25.115612 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1144) took 2729 ms. Trace:
W20260709 12:24:25.117316 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.100678 27973 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.130445 28010 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:25.131987 28007 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.132540 28007 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.130915 28009 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:25.117219 20809 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:25.005774 20804 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:24:25.159439 21429 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.002s
I20260709 12:24:25.160152 21429 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 14.
W20260709 12:24:25.173627 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.999294 27906 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.116039 20731 rpcz_store.cc:276] 0709 12:24:22.386542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.386697 (+   155us) service_pool.cc:224] Handling call
0709 12:24:22.395570 (+  8873us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:22.422474 (+ 26904us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:22.422805 (+   331us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.652137 (+229332us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b1800321be0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:24:22.761248 (+109111us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.761340 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:22.761762 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 12:24:22.761873 (+   111us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:22.762021 (+   148us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.773460 (+ 11439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.163625 (+390165us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b1800321be0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:24:23.250313 (+ 86688us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b1800321be0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e23bc8328 00007f9e23bbf901 00007f9e23bbf8a7 00007f9e23cb7288 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508
0709 12:24:23.699000 (+448687us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b1800321be0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1
0709 12:24:24.594522 (+895522us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800321be0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:25.115305 (+520783us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b54003b4f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:25.115355 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.115399 (+    44us) raft_consensus.cc:1685] finished
0709 12:24:25.115411 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.115555 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22458,"prepare.queue_time_us":3667304,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":37073,"raft.run_cpu_time_us":2766,"raft.run_wall_time_us":522518,"spinlock_wait_cycles":116599680,"thread_start_us":100999,"threads_started":1,"wal-append.queue_time_us":426313,"wal-append.run_cpu_time_us":3329,"wal-append.run_wall_time_us":1123402}
I20260709 12:24:24.954828 27863 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.151966 28009 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:25.173875 27897 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:25.229355 20701 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.017s
W20260709 12:24:25.231576 20701 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.000s	sys 0.018s
W20260709 12:24:25.231906 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 518635us
W20260709 12:24:25.229410 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 306773us
I20260709 12:24:25.237219 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:25.238289 21429 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:25.241868 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.242472 27723 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:25.243932 28007 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.220214 28009 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:25.246096 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:25.249408 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:25.250164 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:25.251262 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:25.250320 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:25.260269 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:25.265352 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3085) took 1115 ms. Trace:
I20260709 12:24:25.238508 27910 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:25.267323 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:25.268322 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:25.273684 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 367925us
I20260709 12:24:25.275178 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:25.275511 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:25.272342 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:25.277748 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:25.246373 27723 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:25.234735 27897 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.261808 27792 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:25.287456 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:25.265023 27024 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:25.262274 27937 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.291436 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.306173 21431 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:25.265847 20802 rpcz_store.cc:276] 0709 12:24:24.150021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.179953 (+ 29932us) service_pool.cc:224] Handling call
0709 12:24:24.190977 (+ 11024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.191220 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:24.235547 (+ 44327us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.236116 (+   569us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.652245 (+416129us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b1800465ca0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:25.010375 (+358130us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b1800465ca0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1e855d70 00007f9e1e849293 00007f9e1e83ecea 00007f9e1e83f23a 00007f9e1eca9a2a 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:24:25.264681 (+254306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.264904 (+   223us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:25.264979 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.265073 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.265273 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126323,"prepare.queue_time_us":429106,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":8682,"raft.queue_time_us":209406,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":13726,"spinlock_wait_cycles":93116928,"thread_start_us":175364,"threads_started":2}
I20260709 12:24:25.265827 27908 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:25.295084 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.298919 27849 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:25.342697 27849 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.343108 27849 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.344274 27897 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.289238 27891 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:25.299111 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.289085 27971 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 12:24:25.299839 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.349386 21429 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:25.285341 27771 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:25.300086 27054 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.270866 27910 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:25.281409 27692 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:25.319356 21435 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:24:25.293087 28012 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:25.293795 27966 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:24:25.284222 27899 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:25.329551 27930 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.330888 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1151) took 1633 ms. Trace:
I20260709 12:24:25.381392 20726 rpcz_store.cc:276] 0709 12:24:23.697395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.784024 (+ 86629us) service_pool.cc:224] Handling call
0709 12:24:23.794363 (+ 10339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.794939 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.330660 (+1535721us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b54002495d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:25.330822 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22961024}
I20260709 12:24:25.333492 20806 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:25.382264 27723 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.322844 27995 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.335723 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:25.384639 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49054.3us
I20260709 12:24:25.385041 27995 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:25.349113 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 249792us
I20260709 12:24:25.385533 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:25.386143 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:25.309937 27792 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:25.346562 27897 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:24:25.338699 27815 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.000s
I20260709 12:24:25.387387 27926 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:25.387656 27815 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.382457 27910 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.389369 27910 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:25.389796 27809 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:25.390090 27815 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:25.391561 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3055) took 2230 ms (client timeout 2072 ms). Trace:
W20260709 12:24:25.392074 20808 rpcz_store.cc:269] 0709 12:24:23.161264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.161405 (+   141us) service_pool.cc:224] Handling call
0709 12:24:25.391370 (+2229965us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:25.391484 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2122038528}
I20260709 12:24:25.395697 20803 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:24:25.399627 27970 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:25.400193 27970 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:25.401309 27970 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.359663 27771 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.409353 27771 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:25.411120 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.413091 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.413822 20731 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
W20260709 12:24:25.414463 20701 leader_election.cc:336] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.573s (SENT)
I20260709 12:24:25.414731 20726 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:25.347872 27744 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:24:25.416072 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:25.416893 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.416620 27997 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.417812 27997 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:25.361405 27891 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.365373 27692 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:25.420756 27692 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:25.414943 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:25.359819 27971 consensus_queue.cc:1243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:24:25.427855 27937 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:25.427544 28009 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.364971 27054 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.440064 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:25.440739 27930 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.440259 20802 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:25.441519 28023 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:25.442240 28023 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:25.443413 28023 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.445356 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3080) took 1419 ms. Trace:
I20260709 12:24:25.445976 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.446666 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:25.449117 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.402724 27970 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:25.451623 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.453796 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.445747 20803 rpcz_store.cc:276] 0709 12:24:24.025748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.050677 (+ 24929us) service_pool.cc:224] Handling call
0709 12:24:25.395325 (+1344648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.396145 (+   820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.445139 (+ 48994us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54000a0850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:25.445297 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3091609856}
I20260709 12:24:25.452721 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:25.533957 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82772.8us
I20260709 12:24:25.454757 20731 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:24:25.536631 20731 pending_rounds.cc:85] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 12
I20260709 12:24:25.536973 20731 pending_rounds.cc:107] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:24:25.537276 20731 raft_consensus.cc:2889] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:25.451542 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65503.5us
I20260709 12:24:25.532279 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.530553 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:25.534601 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:25.531177 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.543604 20730 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
W20260709 12:24:25.545519 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.547371 20883 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:25.563767 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.563627 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:25.564668 21435 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
W20260709 12:24:25.565167 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.566327 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:25.543046 27982 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.695s	user 0.018s	sys 0.000s
I20260709 12:24:25.573709 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:25.538983 27984 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.459375 27920 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.459750 28012 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:25.539487 27934 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:25.578526 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 288712us
I20260709 12:24:25.579574 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:25.548741 27770 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.539103 27971 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.539243 27969 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:25.610128 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 344549us
W20260709 12:24:25.604648 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.615618 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:25.572505 27024 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:25.619859 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:25.578711 27906 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:25.654505 20731 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:25.577864 27897 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:25.655448 20803 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:25.664338 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44587.8us
I20260709 12:24:25.387394 27792 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.612330 27930 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.657151 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:25.666347 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 86903.8us
I20260709 12:24:25.666471 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.614902 27855 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:24:25.615612 27695 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.602s	user 0.005s	sys 0.010s
I20260709 12:24:25.612668 27910 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.675844 20801 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:25.679597 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.674141 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.679899 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:25.683432 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.680078 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:25.685523 20879 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:25.611845 27936 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:25.625445 27966 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.692051 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:25.582495 27982 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:25.696743 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.669159 28016 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:25.603176 27984 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.672835 27937 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:25.603389 27920 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.616854 28003 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 12:24:25.673511 27897 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:25.695709 27786 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:25.621969 27770 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.705868 20881 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:25.706727 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:25.713065 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.704551 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:25.674634 28022 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:25.679665 27855 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:25.723042 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:25.726354 20878 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:25.748881 20877 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:25.680385 27973 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.620556 27969 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.687052 27930 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.619917 27971 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.722424 27897 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:25.726372 20876 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:25.719619 27937 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:25.727067 20733 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:25.726662 20882 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:25.725150 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:25.707538 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:25.745496 21435 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:25.750816 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:25.749481 21430 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:25.723698 27894 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:24:25.753638 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.754459 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.622537 27934 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:25.753240 28014 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.004s
W20260709 12:24:25.750906 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 456706us
I20260709 12:24:25.753616 27750 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:25.722184 27893 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:25.798216 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:25.798545 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:25.698593 27910 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.726608 27054 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.719965 28009 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.725195 20881 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:25.723042 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.714716 28019 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:24:25.758693 28022 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:25.774268 27785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.808s	user 0.006s	sys 0.019s
I20260709 12:24:25.795369 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3092) took 1209 ms. Trace:
I20260709 12:24:25.799252 21430 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:24:25.802325 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.785673 27926 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:25.712142 27966 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.785180 27937 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:25.807879 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:25.811234 21428 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:25.755110 27855 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.828459 21427 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: [21.16-23.17]
I20260709 12:24:25.719398 28021 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.093s
I20260709 12:24:25.787539 27891 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.803745 28005 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:25.837423 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:25.811455 28020 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:24:25.822050 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:25.840210 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:25.841384 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:25.842067 20878 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:25.790802 27849 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.759057 27973 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.809253 20881 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.848372 20881 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:25.836113 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.815613 27785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.850s	user 0.006s	sys 0.019s
I20260709 12:24:25.837522 21430 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:25.849882 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:25.838183 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:25.838064 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.852403 21435 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:25.812498 20810 rpcz_store.cc:276] 0709 12:24:24.585521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.679079 (+ 93558us) service_pool.cc:224] Handling call
0709 12:24:24.695714 (+ 16635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.696025 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:24.696407 (+   382us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.888198 (+191791us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.128883 (+240685us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b18002aa1e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:25.312004 (+183121us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.788075 (+476071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.788138 (+    63us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:25.788189 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.788271 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.795301 (+  7030us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14421,"prepare.queue_time_us":8213,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":17367,"raft.queue_time_us":530635,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":5226,"spinlock_wait_cycles":2929024,"thread_start_us":844011,"threads_started":3}
I20260709 12:24:25.825664 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:25.843786 28022 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:25.803020 27750 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.818389 27906 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:25.835525 27995 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:25.839294 27953 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:25.861114 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.861610 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:25.863346 20883 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:24:25.870158 21428 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:24:25.871640 21428 pending_rounds.cc:85] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 13
I20260709 12:24:25.872002 21428 pending_rounds.cc:107] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:24:25.872326 21428 raft_consensus.cc:2889] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:25.873672 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.874064 20878 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:25.874693 20850 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:25.859010 27785 raft_consensus.cc:993] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: : Instructing follower 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:24:25.887751 20877 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:25.859243 27891 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:25.856360 28022 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.767396 28012 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.862107 27695 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.895146 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:25.899456 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2829) took 1005 ms. Trace:
I20260709 12:24:25.897789 21428 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
W20260709 12:24:25.911680 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.899920 20805 rpcz_store.cc:276] 0709 12:24:24.893717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.893840 (+   123us) service_pool.cc:224] Handling call
0709 12:24:24.903567 (+  9727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.129658 (+226091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.129709 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.130232 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 12:24:25.264021 (+133789us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:25.264060 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.264160 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.899158 (+634998us) raft_consensus.cc:1685] finished
0709 12:24:25.899252 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.899392 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":225636,"raft.queue_time_us":227792,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":132811,"threads_started":1,"wal-append.queue_time_us":331848}
I20260709 12:24:25.942170 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:25.864923 27814 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:24:25.897671 28019 raft_consensus.cc:1081] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Signalling peer 116023b4c0c94bfea737fc238bda3838 to start an election
I20260709 12:24:25.764703 27815 raft_consensus.cc:2955] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index -1 to 15, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:24:25.895081 28022 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.871783 27906 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.895280 27891 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:25.881886 27953 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:25.883340 27995 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.907320 27920 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.939468 27984 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:25.976398 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.902175 27695 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.957062 27982 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.967288 27891 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:25.962186 27770 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:25.985764 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 250387us
W20260709 12:24:25.991297 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 146500us
W20260709 12:24:25.992004 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 414095us
I20260709 12:24:25.972827 27973 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.898893 27926 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:25.984136 27971 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.012362 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:25.984611 27930 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:25.986375 27984 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.967417 28021 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:24:26.029865 20809 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:24:26.004758 28034 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:24:26.032199 27984 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.034037 28036 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:25.995535 27910 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.034870 27984 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:26.036705 28018 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.653s	user 0.000s	sys 0.005s
I20260709 12:24:26.037231 27995 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.037998 27995 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:26.042239 27962 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:25.986685 27920 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:24:26.038591 27995 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.015451 27969 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.068713 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.070245 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.071069 21432 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:26.072418 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.073697 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.007858 27982 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:24:26.084174 20805 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:26.072249 28019 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.086946 28019 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.068346 27849 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.056541 27971 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:24:26.049821 27855 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:26.058367 28034 consensus_queue.cc:1243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:24:26.087452 27920 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:26.056823 27966 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.070029 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:26.102461 20881 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:26.103969 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.104766 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:26.099252 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 397463us
I20260709 12:24:26.073599 27785 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:26.108398 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:26.111290 20809 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:26.123035 20701 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.099s (SENT)
I20260709 12:24:26.125159 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.116074 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:26.110080 27867 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.000s	user 0.018s	sys 0.003s
I20260709 12:24:26.102083 27695 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.155308 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.156899 20801 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:26.144572 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:26.148062 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.111970 28005 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:26.109827 27954 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:26.173648 20731 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:26.110872 27785 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:26.176425 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 164155us
I20260709 12:24:26.109726 20881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:54488
I20260709 12:24:26.217492 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:26.218281 20881 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:26.218657 20731 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:26.218798 20881 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:26.222690 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 78213.8us
I20260709 12:24:26.143795 28051 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:26.126941 28023 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 12:24:26.175630 27954 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:26.244792 20809 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:24:26.149609 28033 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 12:24:26.244894 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:26.127836 28041 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 12:24:26.154935 27863 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:24:26.223824 28019 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:26.276000 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 270342us
I20260709 12:24:26.248252 20881 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.288825 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.244035 28028 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.249581 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.272433 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.171583 27867 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.292080 21432 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
W20260709 12:24:26.296005 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.296942 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.304165 20881 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:26.248335 27723 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.005s	sys 0.019s
I20260709 12:24:26.306133 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.309122 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:26.276532 27953 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:26.306969 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 137148us
I20260709 12:24:26.250197 27906 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:26.308367 21432 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:26.314833 21431 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:26.316202 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:26.296478 27792 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:26.321811 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:26.308147 27723 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.007s	sys 0.020s
I20260709 12:24:26.325413 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.307639 28036 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:26.316849 28034 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:24:26.323458 27927 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:26.339254 27867 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.366802 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:26.340713 27771 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.931s	user 0.007s	sys 0.006s
I20260709 12:24:26.341115 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1307) took 1066 ms. Trace:
I20260709 12:24:26.368798 20806 rpcz_store.cc:276] 0709 12:24:25.275048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.275254 (+   206us) service_pool.cc:224] Handling call
0709 12:24:26.340918 (+1065664us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b54002a99d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:26.341054 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131833728}
I20260709 12:24:26.370003 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:26.317595 27953 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:26.341732 20804 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:26.343636 28039 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:24:26.323721 27695 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:24:26.314090 27894 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:26.326711 27855 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:24:26.328994 27792 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:26.344666 28046 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:26.247850 27785 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.285305 28051 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:24:26.327178 27973 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.345022 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.303261 28059 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:26.346144 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:26.347646 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102831us
I20260709 12:24:26.350477 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.433302 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:26.433527 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.434454 20881 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:26.435808 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:26.435946 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.361536 28021 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:26.362706 27927 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.318516 27966 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:24:26.389729 27894 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:26.314563 28037 raft_consensus.cc:2764] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:24:26.341652 27982 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.370770 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.302976 28019 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:24:26.461252 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2788) took 1852 ms. Trace:
I20260709 12:24:26.321249 28054 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.342945 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.368839 20809 raft_consensus.cc:2955] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index -1 to 15, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:24:26.462145 28054 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.462865 28015 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:26.464453 28015 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:26.464828 28015 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.465169 28015 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:26.467813 27894 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:26.469444 28027 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.475s	user 0.001s	sys 0.000s
I20260709 12:24:26.470357 28047 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:24:26.473963 28055 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:24:26.441511 27927 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:26.383016 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.301054 28028 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.373222 27971 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:26.479561 27971 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:26.480605 27971 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.461716 20733 rpcz_store.cc:276] 0709 12:24:24.609309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.610756 (+  1447us) service_pool.cc:224] Handling call
0709 12:24:24.611148 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.772640 (+1161492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.461022 (+688382us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003b4a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:26.461192 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2565733760}
W20260709 12:24:26.383569 27929 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:24:26.375722 27920 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:26.422501 20701 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62396122903848d4a49e6d637c5c8b49. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.373775 27771 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:26.363948 27692 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.943s	user 0.005s	sys 0.009s
W20260709 12:24:26.442329 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132110us
I20260709 12:24:26.442584 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:26.439172 27770 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
W20260709 12:24:26.502632 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 137001us
I20260709 12:24:26.460807 27792 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:26.489321 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 111129us
I20260709 12:24:26.506572 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:26.507783 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:26.507965 28051 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:26.508709 28051 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:26.509068 28051 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.509407 28051 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:26.509579 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.461302 28022 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:26.495469 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.441131 27973 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.503961 27692 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.512889 27863 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:26.514482 27692 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.515878 28028 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.516497 27771 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.517261 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.517450 28037 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:26.518151 27849 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 12:24:26.521291 20884 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:26.523892 20884 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:26.526361 20884 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:24:26.527518 20876 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:26.530335 28076 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:26.530834 28076 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:26.538841 27024 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:26.511044 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:26.560247 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49270.9us
I20260709 12:24:26.561147 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:26.562879 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.464843 28054 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.582270 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.597435 20726 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:24:26.598284 20727 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:26.420032 28070 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
W20260709 12:24:26.597905 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 157498us
I20260709 12:24:26.442638 28058 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.146s
I20260709 12:24:26.540410 27953 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:26.622324 28024 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.184s	user 0.000s	sys 0.004s
I20260709 12:24:26.638929 28005 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:26.639611 27785 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:26.640514 28005 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:26.610041 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.622884 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.557291 27920 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:26.619022 27897 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.846s	user 0.006s	sys 0.005s
W20260709 12:24:26.580231 27929 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:26.632870 28074 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:24:26.645650 20729 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:26.546036 27971 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:26.652300 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:26.489384 27982 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:26.654911 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51650.1us
I20260709 12:24:26.558385 27024 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:26.664633 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.670840 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.609540 28056 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:26.612316 27814 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:26.646060 28054 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:26.564203 27891 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.618577 27934 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.644479 27973 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.682874 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:26.683137 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:26.684175 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.685748 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:26.686038 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:26.684176 27970 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.154s	user 0.014s	sys 0.000s
I20260709 12:24:26.714850 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.687788 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:26.711658 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:26.711707 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:26.674027 28036 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:26.678617 28056 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.713760 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:26.691030 27997 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.273s	user 0.016s	sys 0.000s
I20260709 12:24:26.656292 27897 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.687156 27771 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:26.688072 28054 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:26.721055 20732 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:26.665555 27920 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:26.688624 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 252891us
I20260709 12:24:26.739230 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:26.730679 20728 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:26.732882 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:26.728324 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:26.684794 28034 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:24:26.741940 28034 consensus_queue.cc:1243] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 21.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:24:26.734225 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46534us
I20260709 12:24:26.740269 20735 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:26.742252 28071 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.714398 27815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:54916 (request call id 4) took 4396 ms. Trace:
W20260709 12:24:26.745189 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.755846 27982 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.756974 28022 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.733803 28072 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:26.687288 28059 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:24:26.757912 27891 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.677156 28076 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.722230 27814 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 12:24:26.767777 27937 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.931s	user 0.013s	sys 0.000s
I20260709 12:24:26.725054 28056 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:26.768051 28082 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:26.739547 28054 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:26.787030 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.744321 27953 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:26.793778 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:26.794458 27997 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:26.801012 27897 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.801581 27954 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.802956 27920 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:26.803242 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1309) took 1517 ms. Trace:
I20260709 12:24:26.805573 27750 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:26.767252 27815 rpcz_store.cc:276] 0709 12:24:22.293651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.293825 (+   174us) service_pool.cc:224] Handling call
0709 12:24:24.457156 (+2163331us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b2400185ac8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba35cb9 00007f9e1ba33dcb 00007f9e1bb322b1 00007f9e1bb2d848 00007f9e1bb2d7ad 00007f9e1cf6e9a7 00007f9e1ebfb85c 00007f9e1ebfb699 00007f9e1ebf86ad 00007f9e1ebf8527
0709 12:24:24.472651 (+ 15495us) log.cc:844] Serialized 506 byte log entry
0709 12:24:25.121311 (+648660us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b18001ccde0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:26.689530 (+1568219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5255,"raft.queue_time_us":17188,"raft.run_cpu_time_us":1834,"raft.run_wall_time_us":135308,"spinlock_wait_cycles":18856192,"thread_start_us":181406,"threads_started":1,"wal-append.queue_time_us":327710,"wal-append.run_cpu_time_us":2157,"wal-append.run_wall_time_us":1002474}
I20260709 12:24:26.788280 28082 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.772311 28071 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.807437 28038 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:24:26.812479 28040 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.568s	user 0.001s	sys 0.000s
I20260709 12:24:26.770340 27771 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:26.773620 27937 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.733062 27024 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.819021 20735 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:26.848801 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.852947 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.841249 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:26.860128 20802 rpcz_store.cc:276] 0709 12:24:25.285868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.340224 (+ 54356us) service_pool.cc:224] Handling call
0709 12:24:25.340612 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.441342 (+100730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.802926 (+1361584us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b54003cad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:26.803148 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228665600}
I20260709 12:24:26.847354 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:26.863610 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 92758.3us
I20260709 12:24:26.865362 20883 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:26.868644 20877 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:26.878822 20877 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:26.841068 27893 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:26.837564 28070 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:24:26.887001 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:26.881901 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.834797 28065 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:26.883905 21432 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:26.884215 20726 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:24:26.885828 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.886157 21429 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:26.779212 28015 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.881680 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:26.889443 20735 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:26.867087 27953 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:26.891140 21434 raft_consensus.cc:2376] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18. Re-sending same reply.
I20260709 12:24:26.866010 28074 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:24:26.924861 28074 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:26.925228 28074 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:26.926120 28074 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.927364 21430 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:26.927769 21430 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:26.885882 27970 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:26.927759 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.928550 21430 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.929446 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.929666 21430 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:26.930866 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2797) took 1278 ms. Trace:
W20260709 12:24:26.931834 28087 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:26.884594 27893 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:26.930516 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.888962 21427 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:26.870687 28082 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:26.870893 28038 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:26.910537 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:26.835310 27814 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.051s
I20260709 12:24:26.890844 20877 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.915656 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:26.870827 27750 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.866251 27982 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:24:26.923539 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:26.959439 20880 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:26.930810 28074 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:26.831215 27785 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:24:26.948638 20735 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:26.931483 20730 rpcz_store.cc:276] 0709 12:24:25.652640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.652838 (+   198us) service_pool.cc:224] Handling call
0709 12:24:25.675008 (+ 22170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.675498 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:26.634396 (+958898us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b18004a9d40. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:24:26.713939 (+ 79543us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.871313 (+157374us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.871804 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.871934 (+   130us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:26.871969 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.872040 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.930580 (+ 58540us) raft_consensus.cc:1685] finished
0709 12:24:26.930696 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.930813 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4089,"prepare.queue_time_us":1001105,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":1940,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":22246,"spinlock_wait_cycles":5080832,"thread_start_us":218844,"threads_started":2,"wal-append.queue_time_us":214653}
I20260709 12:24:26.955051 20883 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:26.932947 27024 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:26.917623 27868 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:24:26.961437 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:26.962055 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.976034 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:26.964141 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:26.951894 27385 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.976069 21430 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:26.959698 28047 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:26.982336 21427 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:24:26.977324 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 92654.2us
I20260709 12:24:26.984220 21430 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:26.985002 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:26.972554 27785 consensus_queue.cc:1243] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:24:26.988138 20883 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:24:26.988951 27894 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:26.980302 27868 consensus_queue.cc:1243] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:24:26.932991 28025 consensus_queue.cc:1175] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 19 status { last_received { term: 19 index: 16 } last_committed_idx: 15 last_received_current_leader { term: 19 index: 16 } }
I20260709 12:24:27.018246 27937 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.022132 28017 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.023027 28017 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.003927 28038 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.025096 28038 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:27.025833 27953 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.983376 27385 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:26.961864 27750 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:26.958496 27893 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.031618 27893 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:27.033048 27927 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.555s	user 0.005s	sys 0.007s
W20260709 12:24:27.033509 20728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1158) took 1582 ms (client timeout 1724 ms). Trace:
I20260709 12:24:27.033666 27927 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:27.034001 20728 rpcz_store.cc:269] 0709 12:24:25.451289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.451396 (+   107us) service_pool.cc:224] Handling call
0709 12:24:27.033250 (+1581854us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400226ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:27.033425 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2918384896}
I20260709 12:24:27.035323 20876 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:27.035916 28071 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:26.999413 28089 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.090s	user 0.001s	sys 0.001s
I20260709 12:24:27.039150 20876 pending_rounds.cc:85] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 10
I20260709 12:24:27.053499 20876 pending_rounds.cc:107] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:24:27.054889 20876 raft_consensus.cc:2889] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:27.065322 27723 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:27.066076 27723 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.073809 20880 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
W20260709 12:24:27.075139 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.073966 21435 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:27.076602 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1308) took 1798 ms. Trace:
I20260709 12:24:27.022594 28056 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.077071 20807 rpcz_store.cc:276] 0709 12:24:25.278123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.288260 (+ 10137us) service_pool.cc:224] Handling call
0709 12:24:25.288684 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.288902 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:25.294902 (+  6000us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.772712 (+477810us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.882322 (+1109610us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b180034c840. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:27.075055 (+192733us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.075111 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:27.075170 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.076386 (+  1216us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.076538 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18272,"prepare.queue_time_us":11629,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"raft.queue_time_us":142605,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":224,"spinlock_wait_cycles":35868032,"thread_start_us":647582,"threads_started":2,"wal-append.queue_time_us":652921,"wal-append.run_cpu_time_us":1972,"wal-append.run_wall_time_us":1079400}
W20260709 12:24:27.078773 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.084018 20880 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:24:27.071796 28017 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:27.089453 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:27.090209 20733 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:27.091104 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:27.078087 28045 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:27.093605 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:27.094376 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.096951 28015 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.098198 27927 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:27.098264 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 556772us
I20260709 12:24:27.099646 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:27.100759 28065 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:27.102025 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:27.102790 28065 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:27.103288 27707 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:27.093410 28017 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:27.104691 28017 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.105137 28017 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:27.107527 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 335e9cdc3f6745ab9b44f08b6e8b2118: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:27.109073 28026 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:27.109520 28026 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:27.141953 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:27.095021 20880 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
W20260709 12:24:27.147795 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.153940 27785 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:27.165105 28049 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:27.165742 28049 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:27.165534 27785 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:27.166728 27785 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:27.167270 27785 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:27.168237 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.170475 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.168121 27785 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.140327 27934 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:24:27.172318 27785 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:27.176018 28026 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.177557 28026 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:27.181862 28056 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.183230 27723 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:27.192363 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.193935 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.195746 27997 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:27.196168 27997 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:27.197085 27997 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.198464 27997 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:27.199002 21428 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:27.145253 27385 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:27.203545 27973 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:27.166721 28049 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.204026 27973 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:27.204177 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:27.204514 28049 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:27.204825 21430 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:27.205111 27973 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.206622 27973 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:27.208374 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:27.212199 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2795) took 1799 ms. Trace:
I20260709 12:24:27.212611 20732 rpcz_store.cc:276] 0709 12:24:25.413008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.415774 (+  2766us) service_pool.cc:224] Handling call
0709 12:24:27.212032 (+1796258us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:27.212139 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2999451392}
I20260709 12:24:27.165344 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:27.216787 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:27.217743 20732 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:27.216250 28056 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:27.218816 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:27.219089 28056 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.219475 20733 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:27.223047 27883 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:27.223780 27883 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:27.224771 27883 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.225797 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:27.226969 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 61701.4us
I20260709 12:24:27.227872 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.232192 28082 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.234259 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:27.237212 20728 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:27.240087 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:27.247815 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.251555 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:27.255412 20882 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:24:27.191073 28054 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.251943 20876 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:27.263276 20733 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:27.265643 21430 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:27.266422 20733 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:27.264758 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:27.268968 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.269512 20876 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:27.269994 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:27.250120 27809 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.859s	user 0.028s	sys 0.021s
I20260709 12:24:27.275743 20734 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:27.277153 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.255530 28095 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:27.285633 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:27.262547 27887 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.876s	user 0.024s	sys 0.037s
I20260709 12:24:27.261363 27927 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:27.308063 27927 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.309388 28056 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.277025 27750 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.287851 27965 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:27.294811 28082 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:27.296475 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.286751 28095 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.290959 27995 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.300631 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:27.301952 28071 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:27.303628 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:27.345903 20732 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:24:27.347298 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:27.348044 20732 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:27.349050 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:27.352125 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:27.352429 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:27.288897 28051 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.772s	user 0.011s	sys 0.004s
W20260709 12:24:27.315265 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.287634 27809 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.891s	user 0.028s	sys 0.021s
I20260709 12:24:27.327759 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:27.323832 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.377063 20881 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
W20260709 12:24:27.343734 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.304702 28054 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:27.292301 27887 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.906s	user 0.025s	sys 0.037s
W20260709 12:24:27.326803 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.371007 20807 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:27.371655 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:27.372541 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:27.395236 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.327108 28012 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.020s	sys 0.035s
I20260709 12:24:27.418871 20726 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:27.419896 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:27.429589 20803 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:27.426416 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 125868us
I20260709 12:24:27.430970 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:27.368275 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:27.433583 20876 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:27.438549 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:27.312435 27965 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:27.311102 27927 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:27.444650 20810 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:27.342075 28095 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:27.436375 28012 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.094s	user 0.021s	sys 0.036s
I20260709 12:24:27.427721 27937 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.376781 27809 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:27.478410 28037 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:24:27.480963 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.323213 28017 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.418221 27893 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.491493 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.436327 27054 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:27.372861 28051 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.446094 28058 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:27.452023 27891 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.445822 27723 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:27.456321 27385 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:24:27.371670 28071 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.496944 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:27.521513 20881 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
W20260709 12:24:27.576066 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 79274.1us
W20260709 12:24:27.584482 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.587129 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:27.588100 20876 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:27.520365 28012 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:27.456693 28007 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 12:24:27.631572 20733 raft_consensus.cc:2368] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24.
W20260709 12:24:27.534236 28102 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 12:24:27.527245 27893 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.500502 27937 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.633373 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.522394 27971 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 12:24:27.537732 28037 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:27.718771 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 280277us
I20260709 12:24:27.724095 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:27.701822 27785 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.002s
W20260709 12:24:27.712941 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 267944us
I20260709 12:24:27.525346 27887 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 12:24:27.712294 20881 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:27.730084 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:27.629979 27927 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:27.739588 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:27.741629 20806 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
W20260709 12:24:27.723169 28101 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 12:24:27.687922 28012 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:27.781235 20881 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:27.593910 27723 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.745431 27887 consensus_queue.cc:1243] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:24:27.746146 27927 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.752833 27894 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:27.749050 20876 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:27.531530 27054 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.783917 27984 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:27.755508 28037 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.756351 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1733) took 2085 ms. Trace:
I20260709 12:24:27.819399 21431 rpcz_store.cc:276] 0709 12:24:25.670484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.670658 (+   174us) service_pool.cc:224] Handling call
0709 12:24:25.671028 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.315508 (+644480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.756014 (+1440506us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:27.756267 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480086272}
I20260709 12:24:27.750886 27024 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:27.757356 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2806) took 1179 ms. Trace:
I20260709 12:24:27.821328 20729 rpcz_store.cc:276] 0709 12:24:26.577857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.578037 (+   180us) service_pool.cc:224] Handling call
0709 12:24:26.578413 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.656574 (+ 78161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.757055 (+1100481us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b54002990d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:27.757281 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153895296}
I20260709 12:24:27.758143 20735 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:27.761023 28074 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.787s	user 0.008s	sys 0.005s
W20260709 12:24:27.763940 28049 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.001s
I20260709 12:24:27.765529 28071 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:27.721194 27883 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:27.717298 27745 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.570007 27891 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.805284 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:27.807405 20876 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:27.747219 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:27.835964 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 88861.5us
I20260709 12:24:27.837010 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:27.838617 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:27.826647 20729 raft_consensus.cc:1217] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Deduplicated request from leader. Original: 16.8->[18.9-20.10]   Dedup: 18.9->[20.10-20.10]
I20260709 12:24:27.797854 27927 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:27.828439 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.842067 20729 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: [18.9-20.10]
I20260709 12:24:27.761976 27965 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.742996 27809 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.845574 27971 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.845772 27893 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.848820 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:27.847179 20883 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:27.784153 28012 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.860936 20803 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:27.840040 27707 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:24:27.871510 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.853788 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:27.892962 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:27.848272 28051 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:27.901575 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:27.902308 20876 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:27.903724 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:27.906039 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:27.900029 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58486.3us
W20260709 12:24:27.899575 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.824173 27984 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:27.908921 21432 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.000s
I20260709 12:24:27.912254 21432 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:27.908979 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:27.848258 27385 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:27.915859 21431 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:27.832062 27883 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:27.871249 28017 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:27.843847 27745 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:27.935343 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.911415 28106 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
I20260709 12:24:27.910549 28007 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:27.941187 20882 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:27.906282 28115 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:27.855858 27971 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.915769 28116 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:27.920459 27965 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:27.936424 27997 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.737s	user 0.014s	sys 0.000s
I20260709 12:24:27.918872 27750 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
W20260709 12:24:27.941771 27973 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.735s	user 0.016s	sys 0.000s
I20260709 12:24:27.928386 28082 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:27.967298 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.980422 28107 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:24:27.931313 27785 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:24:27.984840 27899 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.001581 20882 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:24:27.983952 28112 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:24:27.915813 27385 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:28.002666 28026 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.820s	user 0.014s	sys 0.000s
I20260709 12:24:27.967238 27937 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:28.025151 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.019299 27385 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:28.033190 28110 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:24:28.057296 27815 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:28.058007 27815 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:28.060178 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2805) took 1482 ms. Trace:
I20260709 12:24:28.060359 27745 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.061805 27984 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:28.065901 27971 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:28.067595 27054 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:28.067893 28108 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 12:24:28.069599 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1761) took 1186 ms. Trace:
W20260709 12:24:28.069977 28063 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.730s	user 0.008s	sys 0.013s
I20260709 12:24:28.071751 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2905) took 2127 ms. Trace:
I20260709 12:24:28.072178 20878 rpcz_store.cc:276] 0709 12:24:25.944006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.945323 (+  1317us) service_pool.cc:224] Handling call
0709 12:24:25.945610 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.009986 (+ 64376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.010026 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.025776 (+ 15750us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.244820 (+219044us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:26.244863 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.343971 (+ 99108us) spinlock_profiling.cc:243] Waited 9.33 ms on lock 0x7b4800240258. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:24:27.216179 (+872208us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b540022fbd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:27.216251 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.017121 (+800870us) spinlock_profiling.cc:243] Waited 6.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:28.071495 (+ 54374us) raft_consensus.cc:1685] finished
0709 12:24:28.071524 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.071673 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40858,"raft.queue_time_us":16283,"raft.run_cpu_time_us":2365,"raft.run_wall_time_us":327242,"spinlock_wait_cycles":145495040,"thread_start_us":145261,"threads_started":1,"wal-append.queue_time_us":243422}
I20260709 12:24:28.082906 27883 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.013487 28082 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.084954 28012 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.035871 28049 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:28.086396 27891 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:28.086908 28111 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.236s	user 0.003s	sys 0.000s
I20260709 12:24:28.020900 27899 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.039008 27723 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.060716 20731 rpcz_store.cc:276] 0709 12:24:26.577535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.577716 (+   181us) service_pool.cc:224] Handling call
0709 12:24:26.578189 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:26.597227 (+ 19038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.597262 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:26.597925 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 12:24:26.732451 (+134526us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:26.732932 (+   481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.059706 (+1326774us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b54001d8654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:28.059768 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.059824 (+    56us) raft_consensus.cc:1685] finished
0709 12:24:28.059841 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.060103 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19336,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":42797824,"thread_start_us":52652,"threads_started":1,"wal-append.queue_time_us":208006}
I20260709 12:24:28.070168 21429 rpcz_store.cc:276] 0709 12:24:26.882867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.981671 (+ 98804us) service_pool.cc:224] Handling call
0709 12:24:26.982037 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.982391 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.982422 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.982866 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:24:27.226650 (+243784us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b18000900a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:24:27.424943 (+198293us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:27.424988 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.425059 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.069167 (+644108us) raft_consensus.cc:1685] finished
0709 12:24:28.069190 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.069493 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4845,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9805,"spinlock_wait_cycles":11941376,"thread_start_us":197594,"threads_started":1,"wal-append.queue_time_us":697447}
I20260709 12:24:28.062682 20877 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:28.101938 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:28.124359 20806 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:28.108516 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:28.125211 21431 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:28.119028 20882 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:24:28.127427 21430 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:28.122181 20876 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:24:28.119642 28047 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.133s	user 0.008s	sys 0.009s
I20260709 12:24:28.122103 28074 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:28.119747 28049 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:28.152545 27966 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.153390 27966 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.141134 28047 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.170773 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:28.174875 20730 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:24:28.175439 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.177659 20801 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:28.178779 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 293132us
I20260709 12:24:28.091406 27815 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:28.183856 20877 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 12:24:28.185168 20877 pending_rounds.cc:85] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 9
W20260709 12:24:28.185328 21435 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.025s	sys 0.041s
I20260709 12:24:28.188421 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.188772 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:28.189920 21426 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1375) took 2896 ms (client timeout 2195 ms). Trace:
I20260709 12:24:28.196812 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.197578 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:28.191330 21433 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:28.193974 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.197110 21426 rpcz_store.cc:269] 0709 12:24:25.293812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.293970 (+   158us) service_pool.cc:224] Handling call
0709 12:24:28.189645 (+2895675us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:28.189816 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4937442688}
I20260709 12:24:28.200112 20727 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:28.202070 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.203766 20806 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:24:28.204676 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.211180 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.211972 20804 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
W20260709 12:24:28.216614 21435 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.025s	sys 0.041s
I20260709 12:24:28.215152 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:28.185886 20877 pending_rounds.cc:107] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 12:24:28.156087 28115 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:24:28.220690 20877 raft_consensus.cc:2889] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.196189 27815 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:28.193846 28123 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:28.224246 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2914) took 1371 ms. Trace:
I20260709 12:24:28.208226 28065 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:28.228626 20877 rpcz_store.cc:276] 0709 12:24:26.853156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.853317 (+   161us) service_pool.cc:224] Handling call
0709 12:24:26.868467 (+ 15150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.062073 (+1193606us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b5800704fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:24:28.223958 (+161885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.224185 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208834944}
W20260709 12:24:28.228649 27849 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:24:28.225570 27973 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:28.249377 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.250963 20804 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:28.187161 28049 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.261377 20875 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:28.263775 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.230122 27893 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:28.266543 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.218608 27953 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:24:28.234493 28078 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:28.273737 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90823.8us
I20260709 12:24:28.280398 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.260546 27894 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.285226 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.285923 20804 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
W20260709 12:24:28.286661 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.272356 28120 ts_tablet_manager.cc:933] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
W20260709 12:24:28.294684 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.263177 28082 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.264228 27966 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:28.296350 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.302937 20808 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:28.263351 28128 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:28.266274 27973 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:28.307116 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.309195 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:28.309482 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:28.309959 21429 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:28.259594 28026 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:24:28.309518 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:28.311149 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 112577us
I20260709 12:24:28.237834 28065 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.233274 28123 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.277660 27899 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:28.318353 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:28.284161 28049 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:28.270961 27893 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.278834 28078 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.323935 27893 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.324273 27973 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.325906 28116 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:28.314870 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:28.321275 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:28.298686 27891 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.328871 21434 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:28.329759 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:28.330346 21429 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.331403 20808 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:28.310308 28128 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:28.328249 28095 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.856s	user 0.007s	sys 0.007s
I20260709 12:24:28.339473 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:28.343348 21426 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:28.342155 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.337267 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:28.323957 28078 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.336282 28128 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.353282 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1165) took 1135 ms. Trace:
I20260709 12:24:28.361804 20735 rpcz_store.cc:276] 0709 12:24:27.218178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.220777 (+  2599us) service_pool.cc:224] Handling call
0709 12:24:27.757893 (+537116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.823061 (+ 65168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.353036 (+529975us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54002990d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:28.353205 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234497792}
I20260709 12:24:28.363867 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.353597 21434 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:28.368081 21429 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:24:28.354483 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:28.355404 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.369024 20883 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:28.334918 27899 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:28.371176 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:28.356573 28125 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:24:28.356792 27679 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:28.360651 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.370949 20806 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:28.383298 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:28.381598 28017 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:28.388823 28017 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:28.382409 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.389297 28017 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:28.389576 20808 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:28.372298 27899 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.391708 27899 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:28.391673 27385 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.393527 20810 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:28.382807 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1172) took 1113 ms. Trace:
I20260709 12:24:28.396106 20733 rpcz_store.cc:276] 0709 12:24:27.269451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.269685 (+   234us) service_pool.cc:224] Handling call
0709 12:24:28.382583 (+1112898us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54001d7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:28.382745 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828902784}
I20260709 12:24:28.336414 27891 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.385520 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:28.386855 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.400390 20735 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:24:28.388065 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:28.388442 28132 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 12:24:28.374521 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.385839 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:28.401904 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.420672 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3123) took 1115 ms. Trace:
I20260709 12:24:28.421393 28031 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:28.421638 28012 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:28.421075 20728 rpcz_store.cc:276] 0709 12:24:27.305422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.547237 (+241815us) service_pool.cc:224] Handling call
0709 12:24:27.548229 (+   992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.999056 (+450827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.999104 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.000827 (+  1723us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.185838 (+185011us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:28.185875 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.348220 (+162345us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54002986d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:28.348263 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.420346 (+ 72083us) raft_consensus.cc:1685] finished
0709 12:24:28.420367 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.420608 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":834,"raft.queue_time_us":23728,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":5614,"spinlock_wait_cycles":458985728,"thread_start_us":421882,"threads_started":2,"wal-append.queue_time_us":217114}
I20260709 12:24:28.421087 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.402701 21434 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.402930 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3113) took 1720 ms. Trace:
I20260709 12:24:28.406322 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:28.416262 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:28.427455 20734 rpcz_store.cc:276] 0709 12:24:26.682408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.682589 (+   181us) service_pool.cc:224] Handling call
0709 12:24:28.402774 (+1720185us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54000c49d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:28.402874 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336767232}
I20260709 12:24:28.429281 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:28.424568 20878 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:28.426673 28116 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:28.417104 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.431473 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.436411 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:28.434449 21434 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:28.440229 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.443464 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.444011 20806 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:28.430092 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:28.438021 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.688s (SENT) [suppressed 17 similar messages]
I20260709 12:24:28.345760 28095 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.441685 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:28.464530 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.452384 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2907) took 2393 ms. Trace:
I20260709 12:24:28.467952 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1269) took 1256 ms. Trace:
I20260709 12:24:28.453100 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3119) took 1152 ms. Trace:
W20260709 12:24:28.460250 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.461587 20805 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:28.384761 27679 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:28.522171 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80600.7us
I20260709 12:24:28.451067 20802 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:28.541402 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.542057 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:28.429559 28012 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:28.544170 20808 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:28.546353 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:28.557358 21428 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:28.558012 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.547950 20730 rpcz_store.cc:276] 0709 12:24:27.300223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.303324 (+  3101us) service_pool.cc:224] Handling call
0709 12:24:28.452932 (+1149608us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b54001d7754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:28.453039 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519311104}
I20260709 12:24:28.549233 20875 rpcz_store.cc:276] 0709 12:24:26.058551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.218564 (+160013us) service_pool.cc:224] Handling call
0709 12:24:26.218882 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.262074 (+2043192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.452142 (+190068us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b5400171e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:28.452321 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4696724352}
I20260709 12:24:28.520968 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:28.539547 20881 rpcz_store.cc:276] 0709 12:24:27.211140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.784352 (+573212us) service_pool.cc:224] Handling call
0709 12:24:27.798008 (+ 13656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.798427 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.798461 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.799032 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.168269 (+369237us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:28.168317 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.168393 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.467704 (+299311us) raft_consensus.cc:1685] finished
0709 12:24:28.467727 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.467887 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3599,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":1803,"thread_start_us":368781,"threads_started":1,"wal-append.queue_time_us":463848}
I20260709 12:24:28.553874 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.561271 20726 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:28.551635 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:28.562212 20727 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:28.459020 28135 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:24:28.559232 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.562776 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.569190 20809 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:28.562803 20732 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:28.570679 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.463817 28130 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:28.586162 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:28.587031 21430 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:28.463418 28051 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:28.464131 28076 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:28.544898 27927 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.695s	user 0.016s	sys 0.000s
I20260709 12:24:28.451402 27855 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:28.574644 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:28.427484 27723 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.134s
I20260709 12:24:28.556980 27679 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:28.562040 27854 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.574682 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.465212 27954 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:28.454125 28034 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:28.469818 27891 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.519878 28126 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.589730 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:28.620863 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.461761 28074 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:28.458402 28065 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.457540 28123 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.591022 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:28.595644 20730 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:28.543545 27969 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:28.601596 28131 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:28.636633 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45684.6us
I20260709 12:24:28.633533 27899 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.556635 28012 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:28.602929 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:28.603533 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.603850 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:28.572652 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 93870.8us
I20260709 12:24:28.575150 28078 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.586922 27054 raft_consensus.cc:2764] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:24:28.589109 28140 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.586066 27997 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.586256 27814 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.608253 28120 tablet_copy_client.cc:323] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.193:34333
I20260709 12:24:28.586427 28130 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:28.665400 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3111) took 1207 ms. Trace:
I20260709 12:24:28.604840 27927 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.623222 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:28.627262 20808 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:24:28.593235 27894 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:24:28.639081 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.671756 20735 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:28.645260 28082 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:28.597512 20727 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:28.679757 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1253) took 2447 ms. Trace:
I20260709 12:24:28.680363 21433 rpcz_store.cc:276] 0709 12:24:26.232409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.232575 (+   166us) service_pool.cc:224] Handling call
0709 12:24:28.191125 (+1958550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.198524 (+  7399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.679579 (+481055us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b5400592250. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:28.679701 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4503317888}
W20260709 12:24:28.649226 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46380.2us
I20260709 12:24:28.684836 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:28.611243 28034 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.686879 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:28.612023 27973 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:28.629338 28074 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.628298 28072 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:28.631093 27750 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:24:28.595139 28051 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.667518 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:28.666285 20810 rpcz_store.cc:276] 0709 12:24:27.458366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.458955 (+   589us) service_pool.cc:224] Handling call
0709 12:24:27.990604 (+531649us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:24:27.990899 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.394008 (+403109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.662357 (+268349us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b540027c254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:28.665327 (+  2970us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937944192}
I20260709 12:24:28.704545 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:28.655252 28045 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:24:28.660115 28123 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:28.698809 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.705817 20730 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:28.705271 20810 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
I20260709 12:24:28.723863 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.728292 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:28.731796 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.738637 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.738818 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:28.696816 28072 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.689952 28034 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.704843 27785 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:28.716984 28051 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:28.702823 28082 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:28.751917 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.700901 28047 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:24:28.756444 27920 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:28.662333 28078 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.669440 27814 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.751763 27970 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:24:28.703810 27997 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 12:24:28.758185 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.720657 28123 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:28.743381 28115 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:28.741715 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:28.786794 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.734066 27973 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:28.742537 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 181834us
W20260709 12:24:28.783577 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 135718us
W20260709 12:24:28.744081 27815 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.004s
W20260709 12:24:28.792125 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.787916 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.612068 27854 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.797533 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.818610 20730 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:28.747530 27785 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.801391 28078 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.644668 28131 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.662209 28140 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.631244 28126 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:28.808152 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:28.808938 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3114) took 1081 ms. Trace:
I20260709 12:24:28.838604 20801 rpcz_store.cc:276] 0709 12:24:27.727908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.728131 (+   223us) service_pool.cc:224] Handling call
0709 12:24:27.728558 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.181154 (+452596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.808628 (+627474us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003c7154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:28.808860 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032272256}
I20260709 12:24:28.745919 27969 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:28.809772 21428 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:24:28.811121 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.740386 27385 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:28.812673 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71069.6us
I20260709 12:24:28.816250 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:28.794322 27965 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.817s	user 0.013s	sys 0.002s
I20260709 12:24:28.828195 27785 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:28.829948 20733 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:28.832077 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:28.756938 27920 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:28.850694 20883 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:28.795445 28115 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.794881 28123 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:28.841058 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 217918us
I20260709 12:24:28.842317 20881 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:28.730036 28012 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.843869 20730 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:28.820693 28121 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.823513 27750 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.847018 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:28.847299 20802 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:24:28.863905 20802 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 12
I20260709 12:24:28.864261 20802 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:24:28.864605 20802 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.828531 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:28.822652 28095 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:28.851744 20878 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:28.840006 28129 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.850972 27920 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:28.870775 28148 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:24:28.883683 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 55245.6us
I20260709 12:24:28.870090 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:28.895676 27679 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.876654 27854 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:28.884600 27965 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:28.883404 28095 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:28.900987 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:28.883692 20884 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:24:28.854436 27936 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:28.868363 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.857491 28115 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:28.880728 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.886456 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:28.922291 20805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1320) took 1554 ms (client timeout 1688 ms). Trace:
I20260709 12:24:28.924062 27920 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.925057 28095 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:28.956471 20805 rpcz_store.cc:269] 0709 12:24:27.368342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.424615 (+ 56273us) service_pool.cc:224] Handling call
0709 12:24:28.188622 (+764007us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:24:28.922082 (+733460us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b540019aad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:28.922229 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619959040}
I20260709 12:24:28.957192 28095 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:28.928212 20884 pending_rounds.cc:85] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 11
I20260709 12:24:28.960328 20884 pending_rounds.cc:107] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:24:28.960690 20884 raft_consensus.cc:2889] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.936401 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:28.962486 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:28.924029 21426 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:28.924153 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:28.940625 20850 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.195s (SENT)
I20260709 12:24:28.964723 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:28.923511 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:28.942497 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.965955 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:28.879819 28147 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:24:28.910976 28152 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:24:28.966097 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:28.945072 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:28.914309 28128 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.547s	user 0.004s	sys 0.004s
W20260709 12:24:28.885011 28146 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 12:24:28.884202 28150 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:24:28.901078 27814 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:28.889535 27927 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.005597 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:29.060528 21429 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:28.854207 27695 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
I20260709 12:24:28.955683 20807 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:24:28.865967 27899 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:28.916082 28131 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:28.920708 28056 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:28.856734 28103 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.130s
I20260709 12:24:29.073555 21435 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:28.908664 28140 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:28.973837 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:28.925312 27965 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:28.884547 28012 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.986349 27966 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.987603 27723 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:29.093921 28034 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.910050 28126 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:28.988404 27815 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:28.939313 27679 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:29.070587 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68093.8us
I20260709 12:24:28.871030 27855 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.926038 28121 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.934422 27936 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.953115 28074 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.888865 27750 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.025601 27970 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:28.961414 28065 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:28.993851 27937 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:29.072058 28056 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:28.861035 28082 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.016621 27785 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.087126 27965 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.112725 27936 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:29.090860 28031 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:24:29.076797 28017 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.004s
I20260709 12:24:28.919833 28123 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.922113 28129 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:28.920835 27971 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.024534 28128 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.090274 28103 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:29.102182 27966 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.115871 27970 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:29.216171 27970 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:29.233441 28128 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.113009 28121 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.120772 28056 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.244442 28123 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.247320 28129 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.262748 28056 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:29.270290 28095 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.267421 28074 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:29.280622 28164 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:29.282166 28051 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.272732 28121 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:29.291435 28072 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 12:24:29.299469 28072 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.310384 28051 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.275080 28095 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.160490 28034 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.326540 28103 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:29.338166 28072 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:29.022186 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56133.8us
W20260709 12:24:29.380910 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.381596 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.070478 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3112) took 1599 ms (client timeout 1938 ms). Trace:
I20260709 12:24:29.396656 20809 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:29.399586 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.568171 28176 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 12:24:29.402786 20877 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:24:29.404323 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.956480 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 103103us
W20260709 12:24:29.412344 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 471516us
W20260709 12:24:29.587826 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.413996 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:29.414844 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.106734 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:29.428316 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1385) took 1112 ms. Trace:
I20260709 12:24:29.429894 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:29.430560 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:29.431674 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:29.110214 27815 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:29.108166 27855 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:29.158890 28031 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:29.605630 28031 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.485220 27750 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.236877 27971 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.607436 28129 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.118360 27937 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:29.608955 27937 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:29.610351 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.331163 28115 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.596464 21432 rpcz_store.cc:276] 0709 12:24:28.316172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.320141 (+  3969us) service_pool.cc:224] Handling call
0709 12:24:28.320478 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:28.321697 (+  1219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.321715 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.391999 (+ 70284us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:24:28.392439 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.632010 (+239571us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:28.632065 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.632156 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.104622 (+472466us) raft_consensus.cc:1685] finished
0709 12:24:29.104644 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.427916 (+323272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":856,"raft.queue_time_us":2812,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"spinlock_wait_cycles":15211776,"thread_start_us":239151,"threads_started":1,"wal-append.queue_time_us":325359}
I20260709 12:24:29.490566 27679 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:29.613075 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:29.613648 27679 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.489571 27883 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:29.615271 21435 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:29.615872 27883 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.617172 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.618319 20877 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:29.490887 28012 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.619906 28012 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.487076 28078 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.490490 27054 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:29.548962 27849 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.540112 28034 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:29.585839 27785 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.339711 28103 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.639844 21434 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:29.612102 20879 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:29.616598 27679 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.490509 27966 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.164341 28017 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:29.560755 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.614693 28115 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.623695 21433 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
W20260709 12:24:29.562054 20807 rpcz_store.cc:269] 0709 12:24:27.470871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.471153 (+   282us) service_pool.cc:224] Handling call
0709 12:24:28.886299 (+1415146us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:24:29.070402 (+184103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9887104}
I20260709 12:24:29.636495 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:29.608587 28129 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.602451 27815 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.630563 27849 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:29.633358 21426 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1389) took 1296 ms (client timeout 1521 ms). Trace:
W20260709 12:24:29.647924 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 234425us
I20260709 12:24:29.639599 27984 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.655951 27883 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:29.651625 21426 rpcz_store.cc:269] 0709 12:24:28.334621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.364966 (+ 30345us) service_pool.cc:224] Handling call
0709 12:24:29.630285 (+1265319us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b54007201d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:29.630598 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238784384}
I20260709 12:24:29.669407 20881 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:29.683526 28103 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:29.687415 28078 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.691555 28072 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.692524 28072 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:29.699425 27679 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:29.700292 27679 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:29.700928 27679 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.691636 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36394 (request call id 7) took 2535 ms. Trace:
I20260709 12:24:29.703663 21428 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:29.703935 20880 rpcz_store.cc:276] 0709 12:24:27.110213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.229418 (+119205us) service_pool.cc:224] Handling call
0709 12:24:29.598936 (+2369518us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca5735 00007f9e23c65d29 00007f9e1d1e41d4 00007f9e1d1e4162 00007f9e1d1e4091 00007f9e1d1e400c 00007f9e1d1e32c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:29.645219 (+ 46283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3701151616}
I20260709 12:24:29.706018 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:29.706365 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:29.707975 27785 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:29.708552 27785 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.709038 27785 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:29.665820 28031 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:29.719707 28012 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.721750 20880 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:29.723384 28034 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:29.725117 28129 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.724838 28034 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.728984 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:29.729993 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.730278 28074 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:29.723523 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:29.733840 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:29.735037 27966 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:29.735972 27966 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:29.737715 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:29.737851 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:29.738689 20805 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:29.738968 20735 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:29.721062 27849 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.743587 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:29.744874 20730 raft_consensus.cc:2376] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22. Re-sending same reply.
I20260709 12:24:29.747020 27984 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.748162 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:29.751657 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:29.751848 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:29.753372 20881 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:29.754665 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.754812 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:29.757530 27855 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.760689 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:29.761127 28017 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.761682 28074 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:29.656517 27054 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.771770 20807 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:29.777269 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.772948 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:29.768428 27984 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.782194 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:29.783010 28130 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:29.784473 28109 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.879s	user 0.046s	sys 0.091s
I20260709 12:24:29.784641 27054 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.785622 20877 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:29.785713 28074 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.786698 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 2988) took 3287 ms. Trace:
I20260709 12:24:29.786651 28074 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.786957 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:29.788071 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.787729 20801 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:29.790457 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56842.4us
I20260709 12:24:29.783758 20728 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:24:29.794186 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:29.790297 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:29.798643 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.791658 20884 rpcz_store.cc:276] 0709 12:24:26.499461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.527110 (+ 27649us) service_pool.cc:224] Handling call
0709 12:24:26.552116 (+ 25006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.961284 (+2409168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.786140 (+824856us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:29.786473 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5360953216}
I20260709 12:24:29.784499 28130 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:29.802446 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:29.803543 20733 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:29.787444 28072 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:29.811556 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51107.9us
I20260709 12:24:29.811827 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:29.813164 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:29.813526 28078 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:29.811950 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.801059 20876 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
W20260709 12:24:29.797269 27970 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.565s	user 0.002s	sys 0.020s
I20260709 12:24:29.826481 21430 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:24:29.823064 27883 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:29.828838 27883 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:29.829263 27883 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:29.830703 27855 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.793484 27849 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:29.837066 20735 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:29.840029 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:29.841459 20734 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:29.849606 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:29.850042 27937 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:29.853113 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63243.1us
I20260709 12:24:29.851737 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.854265 20880 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:29.855005 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.859918 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:29.853695 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:29.861289 20735 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:24:29.864326 27936 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.866385 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:29.868294 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.835873 27970 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.877550 20877 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:29.881578 28031 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:29.882315 28031 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.869262 27849 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.884413 21430 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:29.880406 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.889880 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:29.889000 20880 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:29.882958 27970 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.894920 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:29.892745 20884 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:29.867911 27679 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.897657 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1764) took 3013 ms. Trace:
I20260709 12:24:29.898669 27849 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.898852 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:29.899837 28072 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.898648 20734 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:24:29.900070 21429 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:29.893445 27815 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.902560 27815 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.898149 21427 rpcz_store.cc:276] 0709 12:24:26.884626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.996405 (+111779us) service_pool.cc:224] Handling call
0709 12:24:27.076047 (+ 79642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.076388 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:27.082002 (+  5614us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.175800 (+ 93798us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b18004ac9e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:28.033627 (+857827us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b18004ac9e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:24:28.265037 (+231410us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.267295 (+  2258us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.267456 (+   161us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:28.267495 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.267569 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.364325 (+ 96756us) raft_consensus.cc:1685] finished
0709 12:24:28.364350 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.894698 (+1530348us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b54005a4950. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:29.897441 (+  2743us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6044,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":9906,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":6782,"spinlock_wait_cycles":22552192,"thread_start_us":218418,"threads_started":1,"wal-append.queue_time_us":1227908,"wal-append.run_cpu_time_us":2663,"wal-append.run_wall_time_us":1221139}
I20260709 12:24:29.908082 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:29.908938 28072 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.909915 28034 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.908152 27815 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.911852 27971 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:29.912588 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1178) took 1354 ms. Trace:
W20260709 12:24:29.912726 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 79237.3us
I20260709 12:24:29.913800 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:29.914795 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3017) took 1177 ms. Trace:
I20260709 12:24:29.915241 20878 rpcz_store.cc:276] 0709 12:24:28.737340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.738279 (+   939us) service_pool.cc:224] Handling call
0709 12:24:29.914387 (+1176108us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540019c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:29.914568 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256476544}
I20260709 12:24:29.915624 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:29.917452 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:29.918691 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:29.905678 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:29.920039 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 97759.8us
I20260709 12:24:29.920228 27966 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:29.920519 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:29.921234 27966 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:29.920838 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:29.911473 28074 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:29.923293 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46884.4us
I20260709 12:24:29.930521 28180 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:29.931376 28180 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:29.930557 28072 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.930748 28128 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:29.930665 21426 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:29.913776 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 944245us
I20260709 12:24:29.935968 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:29.936836 21427 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:29.940323 28034 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:29.940413 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:29.940711 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:29.941206 21429 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:29.941880 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:29.942153 27849 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:29.943204 27855 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:29.943517 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:29.945868 28128 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:29.946875 20699 leader_election.cc:336] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.580s (SENT)
I20260709 12:24:29.947161 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:29.947153 28128 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.948256 21429 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:24:29.948638 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 124624us
I20260709 12:24:29.943679 27855 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:29.949395 27855 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:29.950692 28031 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.950932 20745 tablet_copy_service.cc:140] P 49323f66286f4798b268f216bb62b3be: Received BeginTabletCopySession request for tablet 62396122903848d4a49e6d637c5c8b49 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:33906)
I20260709 12:24:29.951824 27971 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:29.951876 20745 tablet_copy_service.cc:161] P 49323f66286f4798b268f216bb62b3be: Beginning new tablet copy session on tablet 62396122903848d4a49e6d637c5c8b49 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:33906: session id = 50c1ab13351d4427b836fbb449c6d7bb-62396122903848d4a49e6d637c5c8b49
I20260709 12:24:29.932559 27679 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.913056 20731 rpcz_store.cc:276] 0709 12:24:28.558507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.558654 (+   147us) service_pool.cc:224] Handling call
0709 12:24:28.559017 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.850094 (+291077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.850117 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:28.850176 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:28.850222 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.912072 (+1061850us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400249fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:29.912141 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.912360 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6740,"raft.queue_time_us":3977,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5116,"spinlock_wait_cycles":659534208}
I20260709 12:24:29.960188 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:29.958854 27771 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:29.969604 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.962486 27971 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:29.969935 27970 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:29.971390 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:29.972556 20734 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:29.973135 20735 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:29.959311 28074 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:29.974646 28074 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:29.977085 28074 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:29.979059 28185 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:24:29.971335 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.962960 28180 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:29.981076 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3124) took 1379 ms. Trace:
I20260709 12:24:29.960361 27815 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:29.981518 20802 rpcz_store.cc:276] 0709 12:24:28.601775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.601927 (+   152us) service_pool.cc:224] Handling call
0709 12:24:28.602246 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.865105 (+262859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.980578 (+1115473us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54003ca854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:29.980895 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562728320}
I20260709 12:24:29.939250 27936 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:29.983484 27679 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.984499 27937 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.984375 27679 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:29.987617 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 67264.1us
W20260709 12:24:29.990311 28184 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:24:29.991263 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:29.992365 20878 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:24:29.983573 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1391) took 1052 ms. Trace:
I20260709 12:24:29.980088 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3122) took 1388 ms. Trace:
I20260709 12:24:29.986850 27679 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:29.998790 21427 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:29.999986 21426 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:29.995368 21428 rpcz_store.cc:276] 0709 12:24:28.932034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.932212 (+   178us) service_pool.cc:224] Handling call
0709 12:24:28.932526 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.704236 (+771710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.983134 (+278898us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:29.983508 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1660973312}
I20260709 12:24:29.996381 20809 rpcz_store.cc:276] 0709 12:24:28.591928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.592204 (+   276us) service_pool.cc:224] Handling call
0709 12:24:28.592618 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.567723 (+975105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.979803 (+412080us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:29.980009 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847511296}
I20260709 12:24:30.007025 20801 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:24:30.007674 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59401.6us
I20260709 12:24:30.014014 27971 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.015270 27785 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.017598 27785 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.024083 28074 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.044121 28074 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:30.047925 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:30.048792 20731 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:24:30.050556 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 100510us
I20260709 12:24:30.053056 27966 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:30.053984 28186 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
I20260709 12:24:30.054172 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:30.054831 27897 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:30.054997 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:30.055420 27897 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.055995 27897 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:30.056135 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 2979) took 3744 ms (client timeout 1580 ms). Trace:
I20260709 12:24:30.056754 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:30.056632 20879 rpcz_store.cc:269] 0709 12:24:26.312004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.342623 (+ 30619us) service_pool.cc:224] Handling call
0709 12:24:30.025130 (+3682507us) spinlock_profiling.cc:243] Waited 3.15 s on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:30.056029 (+ 30899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7243164416}
I20260709 12:24:30.057427 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:30.058225 20728 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.031921 20733 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:30.060649 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:30.032590 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63051.8us
I20260709 12:24:30.062170 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.061856 27883 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.036852 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:30.064570 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.064909 27883 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.038328 20745 tablet_copy_source_session.cc:215] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:30.066998 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.059856 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.055421 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:30.053905 20880 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:30.073791 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:30.043016 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.082504 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.082535 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:30.083704 20807 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 16: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:30.046538 28074 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.088703 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:30.092511 21428 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:30.092345 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.092947 28047 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:24:30.089663 28078 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:30.099700 28078 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.100159 28078 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.100390 21427 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:30.101572 27054 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.102468 27054 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:27.856230 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:30.106026 27054 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.106467 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:30.107326 21434 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:30.109463 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.110517 20877 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:30.028671 27855 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.114691 20809 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:30.115381 27855 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:30.115995 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.116828 20735 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:30.091061 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.004630 28072 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
W20260709 12:24:30.123862 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.126446 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60115.3us
I20260709 12:24:30.125030 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.126044 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.129212 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:30.130437 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.131234 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:30.132529 20878 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:30.133145 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:30.134845 20728 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:30.137050 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:30.137854 21427 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:30.135033 27855 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:30.148706 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.150277 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.153750 28082 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.154465 28082 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.139792 27966 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:30.156991 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70470.4us
I20260709 12:24:30.157795 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.151696 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.157024 20877 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:30.158545 20728 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:30.159911 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.160844 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3012) took 1474 ms. Trace:
I20260709 12:24:30.162045 28082 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.162333 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.162567 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.163552 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.152913 28074 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:30.160751 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.136443 27054 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:24:30.166607 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:30.150517 27855 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.163038 20806 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:30.161926 28131 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:30.167981 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.155833 28128 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:30.171689 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.175375 20804 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:30.182930 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.183157 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.184417 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.170060 27855 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.188347 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:30.189075 20732 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:30.191571 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.191797 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.192632 20726 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:30.192852 20730 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:30.185671 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.161389 20875 rpcz_store.cc:276] 0709 12:24:28.682727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.682915 (+   188us) service_pool.cc:224] Handling call
0709 12:24:28.683458 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:28.854791 (+171333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.854820 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.855170 (+   350us) log.cc:844] Serialized 26 byte log entry
0709 12:24:29.570860 (+715690us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b1800345b80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:24:29.833481 (+262621us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:29.840506 (+  7025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.145443 (+304937us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:30.145525 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:30.145594 (+    69us) raft_consensus.cc:1685] finished
0709 12:24:30.145613 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.157279 (+ 11666us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1818,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2819,"spinlock_wait_cycles":409490048,"thread_start_us":183680,"threads_started":1,"wal-append.queue_time_us":979227}
I20260709 12:24:30.186627 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.188007 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.193534 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.164472 28074 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.195683 20731 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:30.196131 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:30.201004 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.201713 20810 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:30.203723 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.204336 27883 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:30.203774 28131 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:30.205089 27883 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.205868 28193 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.205752 28131 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.206626 28193 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.170730 27815 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.209095 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.172742 28180 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:30.194619 28047 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:30.211483 28193 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:30.180540 28128 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:30.213042 28074 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.187453 27971 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:30.216683 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:30.198859 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.231262 28180 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:30.211014 27883 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.242296 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.243973 20879 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:30.249323 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.244859 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.250558 20806 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:30.247696 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:30.256337 21427 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:30.258905 27937 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.260030 27770 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.260602 27770 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.263756 27054 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:30.256901 28130 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:30.259693 27937 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.240456 28047 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:30.268462 28047 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.269317 27937 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.248580 28193 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:24:30.248026 27815 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.274658 27815 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.277105 20810 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:30.278152 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.278749 20810 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:30.270704 28128 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.283653 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.284616 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:24:30.291615 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 121824us
I20260709 12:24:30.292655 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:30.293699 28017 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:30.294195 28017 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:30.294684 28017 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.295282 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.299676 28130 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.300560 28130 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.304054 28130 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.304276 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.307593 27815 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:30.253389 28074 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.308097 27815 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.308524 27815 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.308589 28074 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.316085 28180 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.316563 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.317557 28180 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:30.318231 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:30.319800 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.262955 27770 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:30.327507 27897 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.328298 27855 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.331681 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.332791 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.336318 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.316933 21434 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:30.362704 20728 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:24:30.307848 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62167.9us
I20260709 12:24:30.349506 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:30.351586 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95533.9us
I20260709 12:24:30.354671 20809 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:24:30.326530 20732 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:24:30.357653 28193 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:30.377619 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.379782 20733 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:30.381863 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:30.382391 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:30.382885 20731 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:30.383113 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 2938) took 1537 ms. Trace:
I20260709 12:24:30.383481 21428 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:30.387293 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.378227 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.383751 20883 rpcz_store.cc:276] 0709 12:24:28.845244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.939217 (+ 93973us) service_pool.cc:224] Handling call
0709 12:24:29.916704 (+977487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:29.917415 (+   711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:29.917452 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:29.918122 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:24:29.939327 (+ 21205us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:30.127971 (+188644us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.134709 (+  6738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.138958 (+  4249us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:30.139996 (+  1038us) raft_consensus.cc:1685] finished
0709 12:24:30.140024 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.382772 (+242748us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b540019c8d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:30.382911 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18390,"prepare.queue_time_us":417158,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":10511,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":1836,"spinlock_wait_cycles":2244695296,"thread_start_us":139132,"threads_started":2,"wal-append.queue_time_us":73071}
I20260709 12:24:30.410988 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:30.395289 27855 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.419378 27679 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.420218 28019 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:30.420184 27679 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:30.421458 20681 auto_rebalancer.cc:629] Failed to schedule move for tablet 13cfeabebc54415b87ab9ce1d8a4caab: Illegal state: Replica 116023b4c0c94bfea737fc238bda3838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:30.422464 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:30.352156 28074 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.428388 20803 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:30.428699 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.426822 27883 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:30.431533 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.433632 20807 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:30.427652 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:30.435541 27883 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:30.436033 20732 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:30.436698 27883 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.439087 27970 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:30.440188 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:30.441601 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66074.4us
I20260709 12:24:30.441143 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:30.442920 21426 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:30.445024 20733 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:30.390970 28130 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.423156 27679 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.446349 27970 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.388626 28078 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.459229 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.424368 28017 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.457516 20728 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
W20260709 12:24:30.465602 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 87447us
W20260709 12:24:30.475412 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88396us
I20260709 12:24:30.479293 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.470969 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:30.483322 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 393823us
I20260709 12:24:30.494613 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:30.504648 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73354.4us
I20260709 12:24:30.463825 28082 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:30.471263 27679 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:30.511639 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:30.512907 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:30.482223 28074 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:30.540560 27969 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:30.501843 28019 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:30.507149 27770 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:30.497089 27966 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:30.544922 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 105577us
I20260709 12:24:30.553943 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:30.476475 28130 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.583619 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.585381 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:30.541285 27969 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.589677 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.547297 28019 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.589223 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:30.589413 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57304.8us
I20260709 12:24:30.535369 28074 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.541211 27897 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.594728 20883 raft_consensus.cc:2410] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:24:30.595971 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:30.596583 20732 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:30.600405 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118671us
W20260709 12:24:30.600466 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88952.3us
I20260709 12:24:30.558611 27770 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.597694 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.607962 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:30.608202 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:30.613874 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:30.593803 28019 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:30.623343 20806 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:30.623579 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:30.624244 20802 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:24:30.624488 21426 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:24:30.625022 20733 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:30.625818 21430 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:30.595525 28074 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.612465 27770 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.634099 27893 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:30.634341 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.586911 27966 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.645995 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:30.668835 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.634968 27785 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:30.634670 27893 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.664536 20732 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:30.684372 20881 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:30.684250 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.685357 20883 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
W20260709 12:24:30.693382 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95894.2us
I20260709 12:24:30.678429 28192 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:30.686553 27893 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.698559 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:30.702044 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 138474us
I20260709 12:24:30.703756 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:30.704610 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 78327.7us
I20260709 12:24:30.707005 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.706188 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:30.712471 20879 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:30.717597 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:30.707791 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:30.725414 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.732713 20883 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:30.735992 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.741792 20804 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:24:30.731135 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.704404 27970 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.700057 28078 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.703992 28017 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.744711 20876 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
W20260709 12:24:30.755786 28205 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:30.758255 28193 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.758581 28130 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.762223 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.731470 28215 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.762373 28207 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:24:30.776530 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.702602 28206 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:24:30.759579 28017 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:30.795426 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57365.7us
I20260709 12:24:30.738125 27815 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.793586 20882 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:30.758971 28193 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.819053 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:30.824179 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:30.801823 28017 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:30.835475 20732 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:24:30.812376 28207 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:30.864866 20810 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:30.868881 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.827756 27750 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:30.833475 28017 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:30.883527 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70990.3us
W20260709 12:24:30.875263 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 57312.9us
I20260709 12:24:30.839031 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:30.876643 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.885259 20879 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:30.895891 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:30.897418 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.869451 28206 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:30.880219 28212 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:30.900125 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.830531 28215 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.917812 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:30.919006 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.915490 28206 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:30.920792 28229 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:30.920745 28206 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:30.921730 28203 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.205s	user 0.001s	sys 0.001s
I20260709 12:24:30.922251 28206 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:30.923329 28226 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:30.925338 28193 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:30.927665 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88718us
I20260709 12:24:30.927825 28228 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.928566 28228 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.939832 28207 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.940724 28128 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.941478 28128 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.944020 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:30.945170 28213 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:24:30.944212 28230 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:30.879276 27966 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:30.945459 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.945936 27966 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.945822 28230 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:30.946128 20879 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:30.918001 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:30.948203 20883 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:24:30.931108 28228 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:30.914876 28223 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:24:30.876041 28214 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:30.915242 28212 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.942749 27893 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:30.976624 28128 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:30.978278 28215 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:30.999475 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 291720us
I20260709 12:24:30.969347 28217 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:24:30.983933 28180 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.666s	user 0.012s	sys 0.003s
I20260709 12:24:30.927842 28078 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:30.984834 28230 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:30.969798 28193 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:30.946455 27966 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:30.985002 27815 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.005338 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:31.000053 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.001490 27893 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:30.995118 28212 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.982955 27770 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:31.012619 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:31.014432 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.002278 28120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62396122903848d4a49e6d637c5c8b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:31.024391 28232 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:31.029270 28217 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:31.029615 27966 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.030649 21428 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:31.013159 28019 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.021135 28214 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:31.035627 28214 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:31.036087 28214 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:31.016125 28180 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:31.039376 27770 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:30.999683 28017 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.042642 28074 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:31.047948 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:31.060200 28074 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:31.061614 28196 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.638s	user 0.001s	sys 0.001s
I20260709 12:24:31.066197 28237 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:31.059330 28019 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.070461 28193 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:31.070194 28237 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:31.080399 28212 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:31.081024 28237 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.083352 28212 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.085215 21427 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:31.075459 28215 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.085659 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:31.087759 21434 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:31.088268 28232 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:31.071898 28017 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:31.088721 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:31.100530 21430 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:31.088738 28215 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.102486 28214 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.105085 28215 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:31.110677 28214 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.159739 21430 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:31.240334 28120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44466 (request call id 1267) took 3901 ms. Trace:
I20260709 12:24:31.240900 28120 rpcz_store.cc:276] 0709 12:24:27.350990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.372706 (+ 21716us) service_pool.cc:224] Handling call
0709 12:24:28.310147 (+937441us) ts_tablet_manager.cc:934] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
0709 12:24:28.924482 (+614335us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b4c001bd980. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1cf6ba20 00007f9e1cf7ca29 00007f9e1cf7d13f 00007f9e1cf412ab 00007f9e1cf6ef3c 00007f9e1cf6f461 00007f9e1cf6f768 00007f9e22cf52eb 00007f9e23c1ad3b 00007f9e23c15008 00007f9e23c14405
0709 12:24:31.000249 (+2075767us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:31.095992 (+ 95743us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:31.096162 (+   170us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:31.239297 (+143135us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:31.239675 (+   378us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:31.240011 (+   336us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:31.240203 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":49118,"spinlock_wait_cycles":21532544,"tablet-copy.queue_time_us":726350,"thread_start_us":903460,"threads_started":2}
I20260709 12:24:31.243232 28120 tablet_copy_client.cc:806] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:31.243940 28120 tablet_copy_client.cc:670] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
W20260709 12:24:31.276897 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.008s	sys 0.003s
W20260709 12:24:31.277372 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.010s	sys 0.004s
W20260709 12:24:31.277683 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 668905us
I20260709 12:24:31.278888 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:31.285233 27969 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.338524 20882 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:24:31.381948 28209 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.645s	user 0.002s	sys 0.000s
W20260709 12:24:31.417999 28192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.003s	sys 0.000s
W20260709 12:24:31.418615 28192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.004s	sys 0.000s
I20260709 12:24:31.418927 28192 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.471748 28215 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:31.472441 28215 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.473295 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:31.474290 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2840) took 1285 ms. Trace:
I20260709 12:24:31.474691 28215 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:31.475920 28192 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.479871 28215 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:31.480507 28215 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.482771 28215 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.484369 28215 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:31.484906 28215 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.486868 28131 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:31.487066 28215 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:31.488029 28131 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.496878 28131 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.474844 20730 rpcz_store.cc:276] 0709 12:24:30.188760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.397550 (+1208790us) service_pool.cc:224] Handling call
0709 12:24:31.474222 (+ 76672us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:31.530027 20881 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.001s
I20260709 12:24:31.530570 20881 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
W20260709 12:24:31.593442 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 666269us
I20260709 12:24:31.595574 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.599363 28228 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:31.600129 28228 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.600384 28236 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:31.600596 28230 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:31.601660 28230 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:31.601310 28228 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:31.599489 28231 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:24:31.602505 28230 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:31.601919 28236 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:31.603477 28236 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.602967 28231 consensus_queue.cc:1243] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:31.602957 28230 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.608520 20804 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:31.609266 28230 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.611353 28230 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:31.604519 28236 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.614708 28236 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:31.614775 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 684731us
I20260709 12:24:31.616588 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:31.624850 28251 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:31.635514 28049 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:24:31.645781 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50327.2us
I20260709 12:24:31.644846 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:31.648242 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:31.649013 28251 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:31.651501 28251 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:31.651994 28251 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:31.658532 28252 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:31.659320 28252 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.659914 28252 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:31.665622 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.005s	sys 0.004s
W20260709 12:24:31.666072 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.006s	sys 0.005s
W20260709 12:24:31.666715 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 962502us
I20260709 12:24:31.690157 28251 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:31.674906 28252 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:31.697103 28241 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:31.697628 28252 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.697793 28241 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.699990 28251 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:31.700893 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49109.9us
I20260709 12:24:31.699064 28241 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.709228 28255 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:31.710001 28255 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.713011 28255 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.710817 28251 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:31.714660 28251 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:31.715415 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1794) took 1306 ms (client timeout 1506 ms). Trace:
W20260709 12:24:31.715893 21428 rpcz_store.cc:269] 0709 12:24:30.409380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.409518 (+   138us) service_pool.cc:224] Handling call
0709 12:24:31.715222 (+1305704us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:31.715349 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305795328}
I20260709 12:24:31.716878 28251 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.722873 28251 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.723564 28251 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.725973 28251 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.731652 28252 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:31.732107 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3137) took 1555 ms. Trace:
I20260709 12:24:31.732329 28252 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.732565 20808 rpcz_store.cc:276] 0709 12:24:30.176148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.203862 (+ 27714us) service_pool.cc:224] Handling call
0709 12:24:30.204213 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.227665 (+ 23452us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:30.259384 (+ 31719us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.499679 (+240295us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.499767 (+    88us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:30.499840 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.421769 (+921929us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b18000892c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:24:31.731754 (+309985us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b540019a5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:31.731801 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.731936 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21118,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":175,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5343,"spinlock_wait_cycles":93612544,"thread_start_us":239671,"threads_started":1,"wal-append.queue_time_us":317369}
I20260709 12:24:31.735723 28252 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.736704 28252 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:31.738914 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:31.739689 20804 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:31.738817 28252 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.741650 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:31.741977 20803 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.311s	user 0.017s	sys 0.000s
I20260709 12:24:31.742321 28252 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:31.743130 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:31.743844 20807 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
W20260709 12:24:31.745918 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.007s	sys 0.012s
W20260709 12:24:31.746253 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.007s	sys 0.012s
I20260709 12:24:31.746484 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:31.746522 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 801154us
I20260709 12:24:31.748870 20808 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:31.749133 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.745792 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:31.749320 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:31.742394 20803 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
W20260709 12:24:31.816180 21399 leader_election.cc:336] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.658s (SENT)
I20260709 12:24:31.860761 28252 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:31.861838 28252 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:31.873436 28214 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.874145 28214 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.877841 28214 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.873613 28252 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:31.909174 28234 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:24:31.909703 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.02447e+06us
I20260709 12:24:31.910573 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.911721 28225 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:31.912194 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.923996 27971 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:31.924562 27971 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.925666 28214 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:31.926342 28214 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.928656 28214 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:31.930166 28204 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.221s	user 0.004s	sys 0.004s
I20260709 12:24:31.936427 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1287) took 1323 ms. Trace:
I20260709 12:24:31.936906 20875 rpcz_store.cc:276] 0709 12:24:30.613023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.613173 (+   150us) service_pool.cc:224] Handling call
0709 12:24:30.627916 (+ 14743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.652857 (+ 24941us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:30.869047 (+216190us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.965176 (+ 96129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.965239 (+    63us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:31.936079 (+970840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.936178 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.936349 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16653,"prepare.queue_time_us":28790,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":651564,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":240299,"threads_started":2,"wal-append.queue_time_us":137757}
I20260709 12:24:31.941376 27971 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.942662 27971 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:31.949012 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3136) took 1773 ms. Trace:
I20260709 12:24:31.949519 20803 rpcz_store.cc:276] 0709 12:24:30.175826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.175933 (+   107us) service_pool.cc:224] Handling call
0709 12:24:31.948772 (+1772839us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:31.948914 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562716032}
I20260709 12:24:31.968330 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:31.969540 20803 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:31.970691 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:31.972765 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 326237us
I20260709 12:24:31.973305 28249 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:31.973620 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:31.974684 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 225642us
I20260709 12:24:31.974725 28228 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:31.975944 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.977272 28228 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:31.977267 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:31.978783 28230 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:31.978856 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.979401 28230 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.981019 28249 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:31.981563 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:31.983234 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:31.983453 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:31.983863 21430 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:31.984148 28228 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:31.984803 28228 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:31.985224 28228 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:31.985355 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:31.986096 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:31.986115 28228 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:31.987823 28230 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:31.989771 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:31.990665 21434 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:31.991588 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:31.992345 21433 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:31.993391 21430 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:31.993448 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:31.994607 27855 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:24:31.995806 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:31.996122 27855 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:31.991725 21427 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 12:24:31.986765 21428 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:31.999446 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.002556 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:31.996057 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:32.003399 20806 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:32.004417 27855 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:32.004997 27855 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:32.005893 27855 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.007519 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:32.008441 28031 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:31.996500 21426 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:32.008841 28031 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.009249 28031 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:32.010619 28049 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.011404 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:32.011261 28049 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.012213 28214 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:32.012648 28214 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.013036 28214 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:32.016987 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2845) took 1687 ms. Trace:
I20260709 12:24:32.017799 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.026818 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.027347 27054 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:32.017398 20731 rpcz_store.cc:276] 0709 12:24:30.329276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.383884 (+ 54608us) service_pool.cc:224] Handling call
0709 12:24:30.384354 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.386186 (+  1832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.386217 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:30.440957 (+ 54740us) log.cc:844] Serialized 26 byte log entry
0709 12:24:30.888488 (+447531us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:30.888548 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.888647 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.016757 (+1128110us) raft_consensus.cc:1685] finished
0709 12:24:32.016783 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.016925 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15653,"raft.queue_time_us":1418,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":32498,"thread_start_us":426528,"threads_started":1,"wal-append.queue_time_us":617254}
I20260709 12:24:32.018553 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:32.024801 28049 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.031561 28241 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.032181 27785 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:32.032533 27054 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.033116 27054 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.035595 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.036264 20875 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:32.027385 20882 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:24:32.037623 28249 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.038110 28249 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.041044 28249 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.041188 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.041822 20877 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:32.043256 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:32.044976 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.045513 28249 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.046039 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.045519 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.046206 28249 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.047010 20876 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:32.047523 20884 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:32.047603 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.048305 20880 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:32.048450 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.044998 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.049016 20875 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.050189 21426 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:24:32.053026 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 385349us
W20260709 12:24:32.053390 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:32.054683 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 142598us
I20260709 12:24:32.055496 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:32.056635 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:32.057528 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.057646 20876 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:32.058364 28234 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.058881 28234 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.059626 28239 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.060222 28239 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.061398 28234 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.061506 28249 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:32.062088 28234 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:32.062274 28239 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.063411 28218 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:32.063889 28218 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.064015 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.064271 28218 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:32.064291 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:32.059756 27054 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.065698 27937 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:32.066323 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.066983 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:32.067340 28240 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.067694 28212 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:32.068044 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.067839 28240 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.068761 20879 raft_consensus.cc:2376] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16. Re-sending same reply.
I20260709 12:24:32.069036 28212 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:32.069690 28208 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:24:32.069886 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 790980us
I20260709 12:24:32.070163 28208 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:32.070784 28240 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.071328 20879 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:32.071946 28226 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:32.072824 28226 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:32.073201 28226 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:32.074026 28226 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.075675 20882 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:32.075944 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.076558 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:32.077001 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.072137 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2837) took 1919 ms. Trace:
W20260709 12:24:32.078974 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.078441 20734 rpcz_store.cc:276] 0709 12:24:30.153008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.153130 (+   122us) service_pool.cc:224] Handling call
0709 12:24:30.154858 (+  1728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.155141 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:30.434942 (+279801us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.381253 (+946311us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:32.016054 (+634801us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b180009fe80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a
0709 12:24:32.071785 (+ 55731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.071843 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:32.071905 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.071972 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.072082 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55393,"prepare.queue_time_us":369805,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3090,"raft.queue_time_us":56326,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":50292864,"thread_start_us":395623,"threads_started":2,"wal-append.queue_time_us":241941}
I20260709 12:24:32.081312 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:32.071410 28207 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:32.087167 28207 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.087384 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.087689 28207 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:32.088307 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.088819 28226 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:32.089442 20730 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:32.089496 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.090870 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.091437 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.091639 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:32.092025 20806 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 14.
I20260709 12:24:32.077193 20808 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:32.081955 20805 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:32.093744 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57982.3us
I20260709 12:24:32.036151 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:32.099335 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 65551.7us
I20260709 12:24:32.062896 28239 raft_consensus.cc:2764] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:24:32.107424 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.079416 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.101743 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.085069 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:32.095595 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:32.102411 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:32.107048 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.108266 20879 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:32.110261 20732 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:32.111352 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.113096 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.082347 21428 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:24:32.112524 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.120497 20876 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:32.121779 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.123656 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.123805 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.124459 20733 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:32.126740 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.122923 20877 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:32.115617 20727 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
W20260709 12:24:32.132356 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47393.9us
W20260709 12:24:32.134424 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51938.1us
I20260709 12:24:32.125686 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.134801 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.135522 20731 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:32.135563 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:32.136474 20728 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:24:32.143811 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52025.8us
I20260709 12:24:32.146073 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.146373 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.148216 20806 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:32.155804 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.106828 28225 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.165272 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.107048 28256 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:32.155964 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.157312 21427 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:32.157457 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.164973 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.110826 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.156594 20733 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:32.124686 27937 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.109531 28240 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:32.172434 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:32.190768 20726 raft_consensus.cc:2368] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:32.163687 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:32.154677 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 52379.8us
I20260709 12:24:32.182863 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.184983 20809 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:24:32.186002 20728 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:32.192108 20809 pending_rounds.cc:85] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 12
I20260709 12:24:32.054339 28214 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.188663 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.189396 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:32.201308 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:32.143558 27971 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:32.071290 28208 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.134495 28249 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:32.202910 28249 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.204041 28251 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:32.205853 28249 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.134382 27969 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:32.194301 28239 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:32.196223 20804 raft_consensus.cc:2368] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:32.192359 20809 pending_rounds.cc:107] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:24:32.200156 20734 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:32.211169 20729 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 12:24:32.187314 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.198030 21428 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:32.235541 20809 raft_consensus.cc:2889] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:32.190922 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:32.199343 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88638.7us
I20260709 12:24:32.200383 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.186540 20732 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:32.208179 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.241384 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.176194 28049 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:32.182725 28072 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:32.243929 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.253983 21427 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:32.232018 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:32.231899 20850 consensus_peers.cc:597] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1047135f71834dbab6b869eb3c081b81. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:32.247684 20727 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
W20260709 12:24:32.256043 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 122908us
I20260709 12:24:32.252620 21433 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:32.265501 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.268677 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:32.267179 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.274473 21435 raft_consensus.cc:2368] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24.
I20260709 12:24:32.276265 20727 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:32.210958 27054 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:32.211764 27785 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:32.284186 20734 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:32.206665 27971 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.241639 28234 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.286345 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.237233 28239 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.303961 20875 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:32.305186 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.246382 28256 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.251801 28072 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.292956 20631 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:32.257037 27936 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.316284 20729 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:32.195322 27937 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.330521 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.336894 20883 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:24:32.332651 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.294551 27785 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.293219 28261 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.275281 28212 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.288120 27054 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.338186 20884 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.333422 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.344343 20878 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:32.210490 28240 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.338778 20849 leader_election.cc:400] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:32.339746 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.247849 28255 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:24:32.297402 27971 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:32.184123 28225 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.308131 28239 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:32.301566 28252 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:32.211123 27815 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:24:32.352959 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:32.352640 27785 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:32.357543 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 150575us
I20260709 12:24:32.279799 28231 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:24:32.246922 27969 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.311229 28256 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:32.366598 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 176199us
I20260709 12:24:32.365391 27855 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:32.326537 28072 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:32.386021 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 195057us
I20260709 12:24:32.355605 20849 leader_election.cc:403] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:32.336948 28236 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.352950 27054 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.308683 28234 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.400504 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:32.402770 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134160us
W20260709 12:24:32.400993 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80601.5us
I20260709 12:24:32.386179 27970 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:32.342543 27936 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.392266 28251 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:32.388491 28252 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.420460 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:32.420836 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 182580us
I20260709 12:24:32.460608 20877 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:32.383284 28276 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:32.407642 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 140455us
I20260709 12:24:32.424075 28224 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:32.424412 28251 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.428591 20801 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:32.428489 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:32.430265 28271 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:32.431536 28259 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:24:32.431844 28264 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:24:32.398169 28273 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:24:32.447990 28257 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:32.454209 27937 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.379542 28261 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:32.457513 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 104621us
I20260709 12:24:32.377629 28212 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.467646 28276 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:32.392210 28031 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.499645 20883 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:32.469866 28252 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:32.526645 20727 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:32.548766 20880 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:24:32.472347 28131 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:32.415864 28236 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.477108 28251 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:32.560014 20882 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 12:24:32.561405 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.532474 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:32.533282 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.408030 27855 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.534279 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.534847 20801 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:32.535616 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.538880 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.548154 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.396433 28049 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:32.519434 27970 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:32.559708 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.501904 28224 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.562311 20735 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:32.766494 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.766861 27969 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.767897 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.769619 27054 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.637132 28279 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.136s
I20260709 12:24:32.769892 28072 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.582628 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.776327 20731 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:32.622819 20884 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:32.777205 20881 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:32.777741 20881 raft_consensus.cc:614] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:32.778081 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 245647us
W20260709 12:24:32.682852 20808 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.001s
W20260709 12:24:32.602593 20805 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 12:24:32.780771 20808 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:32.780997 20805 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:32.782794 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:32.783026 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 371568us
I20260709 12:24:32.783761 21426 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:32.706300 20878 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:32.780089 20727 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:24:32.737130 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 276255us
I20260709 12:24:32.788179 20733 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
W20260709 12:24:32.683943 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 280256us
I20260709 12:24:32.792918 20809 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:24:32.793417 20729 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:32.611505 20631 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:32.779875 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:32.736866 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 338003us
I20260709 12:24:32.790973 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:32.796454 27054 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.798689 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.799376 20727 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:32.774863 28225 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:32.622045 28241 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.003s
W20260709 12:24:32.713922 28218 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.006s
I20260709 12:24:32.549763 28214 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:32.548312 28273 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.595394 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800251b60 after lost signal to thread 19606
I20260709 12:24:32.805068 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.555338 27937 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.711109 28226 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.001s
I20260709 12:24:32.806988 20730 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:32.793537 28031 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.823872 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.824509 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.824815 20731 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.809656 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.636297 28282 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.810892 27785 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.537252 28276 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:32.811018 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.830765 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:32.812669 20877 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:24:32.831962 20877 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 12
I20260709 12:24:32.832212 20877 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:24:32.832440 20877 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:32.590088 27815 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:32.548732 28249 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
W20260709 12:24:32.817765 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08002c2ee0 after lost signal to thread 20667
W20260709 12:24:32.856555 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800194de0 after lost signal to thread 20681
W20260709 12:24:32.858240 20905 debug-util.cc:398] Leaking SignalData structure 0x7b080035fee0 after lost signal to thread 20759
W20260709 12:24:32.858621 20905 debug-util.cc:398] Leaking SignalData structure 0x7b080035b620 after lost signal to thread 20760
W20260709 12:24:32.860131 20905 debug-util.cc:398] Leaking SignalData structure 0x7b080035b6a0 after lost signal to thread 20834
W20260709 12:24:32.861707 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800385f40 after lost signal to thread 20908
W20260709 12:24:32.862066 20905 debug-util.cc:398] Leaking SignalData structure 0x7b080034a1a0 after lost signal to thread 20909
I20260709 12:24:32.819447 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.819238 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.616472 28212 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.779737 28234 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.534675 28228 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.270s
I20260709 12:24:32.815688 28278 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:32.853791 20631 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 15 to 16, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 16 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:24:32.807626 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 274562us
I20260709 12:24:32.788925 28131 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:32.809911 27054 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:32.767316 28239 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:32.833606 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53810.1us
I20260709 12:24:32.871636 27855 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:32.871374 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49368.9us
I20260709 12:24:32.821470 28273 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.873340 28239 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.812184 28256 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.875036 28273 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.831136 28282 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.875440 28275 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:32.876058 28275 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:32.863945 27971 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.876878 28275 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.878260 28275 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:32.878527 28236 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:32.819312 28208 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:32.843288 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.840194 28255 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:32.841383 28240 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.840461 28049 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.884016 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.831399 28214 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:32.843845 28287 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:32.846277 27815 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.847522 20806 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:32.819244 28218 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.840284 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:32.913602 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73400us
I20260709 12:24:32.914326 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:32.914916 28218 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:32.915354 28255 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:32.851464 28289 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.916154 28278 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:32.916329 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:32.842146 28281 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:24:32.843950 28285 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:32.916616 28255 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.616374 28280 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.261s
I20260709 12:24:32.918563 28280 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:32.918919 28280 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:32.919478 28207 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.831s	user 0.008s	sys 0.004s
I20260709 12:24:32.919946 28207 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.823240 28272 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:32.920845 28272 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.921034 28239 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:32.921962 28072 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.922026 28239 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:32.922590 28239 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.922665 28072 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:32.923095 28072 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:32.922953 28239 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:32.868217 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:32.823688 27937 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:32.923954 28072 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:32.924916 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56832us
I20260709 12:24:32.925144 28072 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:32.926038 27937 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.926592 27937 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:32.926927 27937 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.871347 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.879880 20730 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.880818 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.886097 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:32.886814 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:32.893966 20809 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 12:24:32.894560 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:32.867606 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800217060 after lost signal to thread 21459
I20260709 12:24:32.893594 28290 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:32.899422 27966 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:24:32.896938 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:32.941454 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.915580 28218 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:32.919767 28280 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.798583 28279 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:32.840158 28261 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.923368 20732 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:32.870589 28131 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.927433 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:32.927357 28240 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:32.929035 28272 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.930029 20726 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26.
I20260709 12:24:32.932585 20728 raft_consensus.cc:2368] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24.
I20260709 12:24:32.870484 27936 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:32.919519 20734 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: [22.19-22.19]
I20260709 12:24:32.936725 20727 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:24:32.932884 20905 debug-util.cc:398] Leaking SignalData structure 0x7b080037b8e0 after lost signal to thread 28019
I20260709 12:24:32.937762 27054 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:32.938246 20735 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:24:32.985529 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800345bc0 after lost signal to thread 28144
I20260709 12:24:32.985770 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.986420 20727 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:32.878376 27971 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:32.950171 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66225.1us
I20260709 12:24:32.988435 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:32.950191 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53313.4us
I20260709 12:24:32.951665 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:32.951996 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65422.5us
I20260709 12:24:32.959491 20801 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:24:32.963872 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.966293 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.967038 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:32.956161 20631 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:32.992348 20884 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:32.967994 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.968746 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.962299 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:32.970022 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89274.7us
I20260709 12:24:32.994403 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:32.970788 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.937551 27937 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.916620 28278 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.959367 27969 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:32.986341 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.922037 28207 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:32.939380 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:32.965391 28214 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.965866 28290 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:32.998170 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.998354 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:32.998700 20729 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:33.020849 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:32.958303 28231 raft_consensus.cc:2764] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:24:33.002601 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:32.959139 27855 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:32.956552 28279 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:32.937541 28212 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:24:32.937819 28180 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:24:33.013533 21426 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:32.996024 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.013690 20879 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 12:24:32.979561 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 175821us
I20260709 12:24:32.971897 28261 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:33.036270 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47253.4us
I20260709 12:24:32.997771 20733 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:33.008966 27054 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.019359 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:32.997265 28031 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.030820 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:33.073758 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72818.1us
I20260709 12:24:33.066824 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:33.104643 21429 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:33.080206 20884 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
W20260709 12:24:33.092656 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71890.1us
I20260709 12:24:33.105476 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:33.016212 28049 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:33.047492 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:33.094513 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:33.032601 20631 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 12 to 14, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 14 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:33.108619 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.113642 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:33.113901 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.114897 20876 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 26 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:24:33.116032 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.116804 20876 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:33.117810 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:33.119061 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.105777 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:32.964366 28251 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.017323 28272 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:32.938562 28287 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.006682 28278 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.007680 28252 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:32.916193 28289 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:33.190764 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61816.3us
I20260709 12:24:33.191692 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.191817 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:33.192790 20883 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:33.187266 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:33.188122 20776 leader_election.cc:400] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24.
I20260709 12:24:33.193668 20776 leader_election.cc:403] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:33.129561 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 180875us
W20260709 12:24:33.162329 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 164651us
I20260709 12:24:33.031978 27815 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.096803 28279 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:33.033083 28282 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:33.128527 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.103955 28256 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:33.194655 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:33.107733 28249 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:33.203258 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:33.124553 28272 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:33.131649 20733 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:33.237741 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:33.214468 28228 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.112712 28236 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:33.243268 20733 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:24:33.243649 20733 pending_rounds.cc:85] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 14
I20260709 12:24:33.243865 20733 pending_rounds.cc:107] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:24:33.244088 20733 raft_consensus.cc:2889] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:33.200852 27971 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:24:33.215263 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.122793 27785 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.229225 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:33.214450 20631 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:33.242697 28236 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:33.253058 28289 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:33.259687 28236 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:33.259891 20631 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 18 to 20, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 20 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.127969 28226 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:33.033645 27936 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.161s
I20260709 12:24:33.272433 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:33.273363 20735 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 12:24:33.273880 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:33.274699 20875 raft_consensus.cc:606] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:33.216034 20804 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:33.240187 20808 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:33.239223 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:33.261549 20631 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 16 to 18, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 18 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.246309 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:33.242470 28272 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:33.112562 28049 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:33.249464 20630 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 302) took 33582 ms (client timeout 15000 ms). Trace:
I20260709 12:24:33.249701 28286 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:33.250671 28261 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.111625 28031 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.251286 27855 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:33.252224 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.245085 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:33.240926 28282 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:24:33.260612 28281 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:33.232589 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.262058 28297 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:33.263207 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:33.239532 28271 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.712s	user 0.004s	sys 0.006s
W20260709 12:24:33.263588 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159122us
I20260709 12:24:33.233428 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:33.263983 20732 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:33.266705 20730 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:33.106830 27969 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.281278 28251 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:33.036554 28296 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.210s
I20260709 12:24:33.287031 20883 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.047572 28214 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.288374 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:33.263088 28231 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:24:33.314739 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69764.8us
I20260709 12:24:33.314472 20631 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 16 to 19, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 19 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:33.320220 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:33.353662 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:33.323349 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 84264.1us
I20260709 12:24:33.324684 27054 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:33.325690 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:33.355258 27054 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:33.355741 27054 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:33.313633 28286 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.254189 28228 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.328568 27937 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.287806 28226 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:33.321986 28230 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:33.361433 20631 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 10 to 15, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 15 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:33.318073 28261 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:33.330873 20731 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.002s
I20260709 12:24:33.304152 28049 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:33.331595 28031 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.303438 27966 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:33.346685 20884 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:24:33.354717 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 160140us
I20260709 12:24:33.313916 27936 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.359977 28214 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:33.360245 28317 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24.
I20260709 12:24:33.360610 28297 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:33.380218 27936 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.381013 28297 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:33.381369 28297 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:33.382637 27936 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:33.382278 28297 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.383267 28316 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:33.383371 20878 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:33.384410 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:33.385084 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:33.380254 28301 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:33.385948 28301 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:33.386778 28301 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.387809 28301 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:33.388906 28276 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 12:24:33.389339 28276 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.377790 27966 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:33.393000 27966 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.361271 28252 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:33.361480 28303 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:33.362085 27971 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:33.365980 28273 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:33.282003 28290 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.364337 28131 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.402911 20631 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 14 to 17, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 17 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:33.404820 28131 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.365514 20879 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:33.366503 20881 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:33.404647 20631 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 13 to 15, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 15 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.358579 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:33.406716 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48212.2us
I20260709 12:24:33.367841 20883 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:33.368844 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:33.371502 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
W20260709 12:24:33.371531 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45886.5us
I20260709 12:24:33.372699 21432 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:24:33.345420 28271 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:33.377945 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:33.398962 20878 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:33.400020 20727 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:33.413975 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:33.414256 20881 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:33.414395 27855 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:33.414899 27855 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:33.415321 27855 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:33.415351 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:33.416131 20801 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:24:33.262506 28298 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.149s
W20260709 12:24:33.401224 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 213274us
I20260709 12:24:33.401799 21429 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:33.401870 20877 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:24:33.373586 28230 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:33.353679 28281 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:33.413816 27936 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:24:33.401451 28231 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.412221 20631 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 17 to 18, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 18 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.428833 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:33.427418 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58694us
I20260709 12:24:33.429705 20803 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:33.427299 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:33.430099 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:33.433154 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:33.434370 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:33.407027 28273 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.438426 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:33.425211 28239 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.501s	user 0.004s	sys 0.006s
I20260709 12:24:33.423702 28230 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:33.433975 20631 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 18 to 19, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 19 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:24:33.442956 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48338us
I20260709 12:24:33.411741 28316 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:33.444893 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.424857 27966 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.425488 28297 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:33.449434 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:33.455052 28294 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:33.456125 28261 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.455674 28226 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:33.456271 28281 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.451326 27936 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:24:33.424685 27969 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.440130 28273 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:33.459298 27785 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:24:33.459959 28208 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 12:24:33.460371 20631 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 12 to 18, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 18 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:33.458943 27937 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:33.462393 28309 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:33.462728 21429 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:33.442638 28287 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:33.302503 20630 rpcz_store.cc:269] 0709 12:23:59.636201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.636499 (+   298us) service_pool.cc:224] Handling call
0709 12:24:33.202941 (+33566442us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:22.946737 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:22.987851 (+ 41114us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:22.987909 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:24:22.987929 (+    20us) tablet.cc:662] Decoding operations
0709 12:24:22.990383 (+  2454us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:22.990514 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:22.990546 (+    32us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:24:23.240591 (+250045us) tablet.cc:701] Row locks acquired
0709 12:24:23.240638 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:24:23.240825 (+   187us) write_op.cc:270] Start()
0709 12:24:23.240980 (+   155us) write_op.cc:276] Timestamp: P: 1783599863240753 usec, L: 0
0709 12:24:23.241000 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:24:23.251826 (+ 10826us) log.cc:844] Serialized 7953 byte log entry
0709 12:24:24.061315 (+809489us) op_driver.cc:464] REPLICATION: finished
0709 12:24:24.185404 (+124089us) write_op.cc:301] APPLY: starting
0709 12:24:24.185516 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:24.385526 (+200010us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:24.385649 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:24.578238 (+192589us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:24.578677 (+   439us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 12:24:24.578737 (+    60us) write_op.cc:312] APPLY: finished
0709 12:24:24.684117 (+105380us) log.cc:844] Serialized 180 byte log entry
0709 12:24:24.706562 (+ 22445us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:24.792075 (+ 85513us) write_op.cc:454] Released schema lock
0709 12:24:24.819267 (+ 27192us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:24.872708 (+ 53441us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:24.875874 (+  3166us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2400408,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":116897,"apply.run_cpu_time_us":31795,"apply.run_wall_time_us":662548,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87727,"num_ops":20,"prepare.queue_time_us":334905,"prepare.run_cpu_time_us":29490,"prepare.run_wall_time_us":532314,"raft.queue_time_us":193240,"raft.run_cpu_time_us":32712,"raft.run_wall_time_us":68682,"replication_time_us":820150,"thread_start_us":511656,"threads_started":4,"wal-append.queue_time_us":215469,"wal-append.run_cpu_time_us":52835,"wal-append.run_wall_time_us":2402724}]]}
I20260709 12:24:33.463526 28228 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:33.453567 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:33.464007 28312 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:33.464404 28269 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.448514 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:33.469228 21400 leader_election.cc:400] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:33.465780 28131 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:24:33.447609 28252 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.061s
I20260709 12:24:33.468706 28271 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:33.469460 28289 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:33.555747 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106377us
I20260709 12:24:33.647342 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:33.538110 27054 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.511950 28290 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:24:33.602617 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:33.652704 21430 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:33.433712 28231 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.621600 20803 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:33.469718 28236 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.625083 21432 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:33.557082 28251 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:33.468995 28239 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:33.630525 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 177045us
I20260709 12:24:33.634028 21400 leader_election.cc:403] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:24:33.639377 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 190930us
W20260709 12:24:33.645735 28324 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.004s
I20260709 12:24:33.650279 21434 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:33.654726 28323 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:24:33.656708 28049 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.632707 28276 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.653401 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:33.658978 28268 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:33.636121 28287 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:33.633574 28312 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:33.732659 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60157.9us
I20260709 12:24:33.639145 28309 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:33.823702 28312 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.653201 20631 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 10 to 14, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 14 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:33.825111 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:33.734916 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 312004us
I20260709 12:24:33.826108 27969 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:33.826959 27969 consensus_queue.cc:1243] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:33.827380 28315 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:33.827570 28208 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:33.825169 20631 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 13 to 15, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 15 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:33.828377 28315 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:33.781373 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:33.642701 28271 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:33.829149 28315 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:33.733865 28292 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:24:33.782295 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 313198us
I20260709 12:24:33.829912 28292 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:33.626518 28279 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.832154 28120 tablet_copy_client.cc:538] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:33.833519 28279 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:33.728865 28313 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:33.734658 27937 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:33.815130 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 161783us
I20260709 12:24:33.645179 28317 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:33.819303 20728 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:33.827498 27969 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:33.828404 28208 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:33.829288 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.757s (SENT) [suppressed 22 similar messages]
I20260709 12:24:33.854830 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:33.829537 28271 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:33.659356 28321 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.057s	user 0.002s	sys 0.003s
W20260709 12:24:33.839655 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58389.7us
I20260709 12:24:33.841212 21432 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:33.737671 27054 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:33.846266 20775 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:24:33.878084 20775 leader_election.cc:403] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:33.646881 28289 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.818794 28275 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:33.872134 20631 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 16 to 20, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 20 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:24:33.885178 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.882381 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:33.885979 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:33.886530 21432 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:33.846246 28308 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:33.805508 28131 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:24:33.842969 28261 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22.
I20260709 12:24:33.843483 28287 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:33.889804 20775 leader_election.cc:400] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24.
I20260709 12:24:33.895399 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:33.896227 21433 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:24:33.899644 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:33.905542 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:33.912652 21430 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:33.913709 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:33.914482 20631 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:33.887660 28289 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.919438 21428 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: [22.19-22.19]
I20260709 12:24:33.900643 27969 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:33.922734 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:33.925539 20727 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:33.889430 28276 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:33.900894 28131 consensus_queue.cc:1243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:24:33.925462 20631 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 12 to 15, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 15 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:33.892138 28293 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:24:33.909663 28308 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:33.936520 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:33.974556 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.974774 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:33.981515 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76102.2us
I20260709 12:24:33.874861 27937 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:33.983559 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:33.983810 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:33.986816 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:33.987782 20803 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:33.984802 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:33.986768 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:33.984840 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:33.898491 28279 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:33.934418 28272 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.002s
I20260709 12:24:33.992702 20730 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 15.
I20260709 12:24:33.993763 27969 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:33.994693 20760 heartbeater.cc:646] Failed to heartbeat to 127.19.36.254:46039 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.36.254:46039 timed out after 15.000s (SENT)
I20260709 12:24:33.996249 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.032136 20732 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:33.997958 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:33.998077 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:33.998667 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:33.999410 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.034695 20805 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 14.
W20260709 12:24:34.002681 27855 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.587s	user 0.005s	sys 0.002s
W20260709 12:24:34.002993 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.037905 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:34.008354 28336 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:24:34.009482 28331 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:24:34.010457 28332 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 12:24:34.012630 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 657bc62d20374e539b23641715f22745 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:24:34.046226 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59996us
I20260709 12:24:34.013278 20631 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:33.989107 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.018814 28049 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:34.054363 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 65354.3us
I20260709 12:24:34.055497 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:33.990684 28072 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:34.022357 28295 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:34.022800 28320 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.038839 21428 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:33.934283 28276 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:33.990409 28322 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:24:34.042342 21432 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:34.043450 27855 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.041635 20734 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:34.052495 20732 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:24:34.051051 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53543.2us
I20260709 12:24:34.021903 28249 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:33.990252 28270 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.007366 28318 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.152s
I20260709 12:24:34.063107 28031 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:33.831112 28292 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.082764 27855 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.083456 28292 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:34.088902 28289 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:34.045122 28283 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.027s	sys 0.037s
W20260709 12:24:34.091423 28283 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.027s	sys 0.037s
I20260709 12:24:34.090952 28289 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.063655 20631 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 14 to 15, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 15 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:24:34.040993 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 211707us
I20260709 12:24:33.644169 28269 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:34.102636 28334 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:24:33.879434 28317 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.106838 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3045) took 1135 ms. Trace:
W20260709 12:24:34.054543 28278 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.830s	user 0.011s	sys 0.001s
I20260709 12:24:34.108103 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1377) took 1222 ms. Trace:
I20260709 12:24:34.108433 28278 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.109700 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:34.110237 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:34.107312 20875 rpcz_store.cc:276] 0709 12:24:32.971626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.301252 (+329626us) service_pool.cc:224] Handling call
0709 12:24:33.301707 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.303065 (+  1358us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:33.487290 (+184225us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.785225 (+297935us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.789256 (+  4031us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.789421 (+   165us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:33.789643 (+   222us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.789848 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.789941 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.106622 (+316681us) raft_consensus.cc:1685] finished
0709 12:24:34.106644 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.106777 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148,"prepare.queue_time_us":235942,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":287,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":47662,"spinlock_wait_cycles":73775744,"thread_start_us":329535,"threads_started":2,"wal-append.queue_time_us":327478}
I20260709 12:24:34.110976 28278 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:34.075734 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92949.6us
I20260709 12:24:33.931212 28208 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.115947 28208 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.116892 28031 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:34.117671 28031 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.045531 28239 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.010998 28287 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.076333 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:34.120191 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43959.6us
I20260709 12:24:34.120877 28286 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.122308 28308 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:33.659642 28316 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.071449 28261 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:34.125645 28261 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:34.126317 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.126609 28261 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.128245 28314 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:34.128681 28314 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.081001 28326 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:34.129020 28314 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.132148 28239 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.132740 28239 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.133785 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:34.134089 28285 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.134498 21432 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.134632 28285 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.135566 28212 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:34.136958 28212 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:34.137075 28285 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.137372 28212 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:34.139459 28328 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.395s	user 0.001s	sys 0.001s
I20260709 12:24:33.931870 28309 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.096892 28231 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.141991 28270 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.108596 20808 rpcz_store.cc:276] 0709 12:24:32.885698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.885830 (+   132us) service_pool.cc:224] Handling call
0709 12:24:34.107885 (+1222055us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b540027b354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:34.108030 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709080960}
I20260709 12:24:34.075480 28249 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:34.143211 21432 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:34.146700 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:34.147954 20631 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 15 to 16, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 16 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:34.147085 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:34.138255 28212 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.042464 28272 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:34.156193 28273 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 12:24:34.156873 28272 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.156687 28273 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.157879 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:34.158565 21428 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:34.074611 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:34.152375 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:34.134958 28239 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.154908 28212 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.159399 20631 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 7 to 17, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 17 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.160319 28120 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:24:34.147109 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.161775 20631 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 13 to 14, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 14 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:34.161800 21435 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:34.149644 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:34.066026 28313 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:34.166326 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:34.175657 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.175747 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.158926 28273 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:34.190719 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 116184us
I20260709 12:24:34.181766 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:34.189601 21430 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:34.196130 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.197470 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.198325 20810 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:34.071002 28320 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.096433 28236 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.199155 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.207583 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.207779 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:34.204496 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54970.2us
I20260709 12:24:34.212508 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.201980 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.215161 20802 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:24:34.216308 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.201980 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:34.204205 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51908.1us
I20260709 12:24:34.203315 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.227002 21427 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:34.207123 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.228076 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.229241 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.234769 20735 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:34.242647 20804 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:34.242563 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.236202 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:34.247100 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52933.3us
I20260709 12:24:34.242597 21428 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:34.231658 28072 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:34.255800 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:34.201756 28320 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.204735 28278 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:34.235934 28281 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.259614 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.241700 28285 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:34.267933 20810 raft_consensus.cc:2376] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22. Re-sending same reply.
I20260709 12:24:34.260437 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:34.261029 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:24:34.247296 27966 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.003s
I20260709 12:24:34.204892 28342 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:34.268584 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:34.247578 28226 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:34.270818 20735 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26.
I20260709 12:24:34.273200 20807 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:34.269694 28337 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:34.275677 20775 leader_election.cc:400] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26.
I20260709 12:24:34.276129 20775 leader_election.cc:403] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:24:34.273191 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:34.278759 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48214.7us
I20260709 12:24:34.279215 28355 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.205875 28314 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.279546 21429 pending_rounds.cc:85] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:24:34.285959 21429 pending_rounds.cc:107] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:24:34.286358 21429 raft_consensus.cc:2889] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:34.270998 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.274922 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:34.279863 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:34.274063 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.288141 20735 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:34.293015 20734 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:34.295241 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.297472 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.297755 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:34.298293 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.298689 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.256050 28272 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:24:34.300009 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:34.271968 28278 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:34.276043 28337 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.305022 21426 raft_consensus.cc:2368] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16.
I20260709 12:24:34.301397 20732 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:34.308579 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.281533 28289 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:34.309530 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.301383 28255 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:34.306895 28251 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:34.313524 28300 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26.
I20260709 12:24:34.313753 28251 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:34.314582 28345 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:34.314894 28251 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.304400 28273 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:34.319650 28273 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.320008 28273 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:34.320524 28314 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.328351 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.299866 28280 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.277850 28072 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:24:34.331028 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:34.199776 28313 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.273957 28281 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.334889 20732 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.337776 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.304948 28290 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:34.343591 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:34.344312 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.347714 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.311656 28337 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.312266 28285 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:34.361984 20810 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.268621 28249 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.366576 28359 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:34.337127 28239 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:34.330297 27969 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:34.370399 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:34.338615 28289 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:34.383688 28270 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.361203 28290 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:34.385746 28312 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.000s
I20260709 12:24:34.385834 28281 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.297137 28355 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.386448 28249 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:34.390161 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.363406 28342 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:34.396154 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.405460 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1813) took 1049 ms. Trace:
I20260709 12:24:34.413204 20731 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:24:34.304291 28315 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.305245 28326 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:24:34.408815 20726 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
W20260709 12:24:34.394928 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.397585 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.409878 20810 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:34.400532 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.422149 20732 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:34.400588 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:34.423478 20732 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:34.423677 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:34.403005 20802 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:34.427479 20802 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:24:34.364655 28224 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.404321 20808 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:24:34.413617 20805 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:34.399026 28344 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:24:34.413100 21435 rpcz_store.cc:276] 0709 12:24:33.356270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.357201 (+   931us) service_pool.cc:224] Handling call
0709 12:24:33.360341 (+  3140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.176756 (+816415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.405228 (+228472us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b5400690fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:34.405399 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1842633472}
I20260709 12:24:34.406365 27937 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.420949 20729 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:34.401885 20809 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:34.397379 28276 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.398806 28356 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:34.403388 28290 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:34.540767 28230 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.028s	user 0.013s	sys 0.000s
I20260709 12:24:34.541527 28230 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.406525 28278 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.395206 28359 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:34.406919 28355 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.405094 28312 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.531520 28299 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:34.535441 27937 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.537762 28273 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.563016 28273 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.523041 21435 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:34.554546 20730 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:24:34.520746 28275 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.000s
I20260709 12:24:34.565061 28275 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:34.543656 28294 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.006s	user 0.013s	sys 0.000s
I20260709 12:24:34.404313 28289 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.566022 28294 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:34.570178 20803 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:34.571090 28345 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:34.571434 28239 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.572146 28239 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:34.572661 28299 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:34.573175 28299 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:34.565593 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3173) took 1754 ms. Trace:
I20260709 12:24:34.580425 20728 rpcz_store.cc:276] 0709 12:24:32.811396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.033045 (+221649us) service_pool.cc:224] Handling call
0709 12:24:33.033363 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.846896 (+813533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.565342 (+718446us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54003b3b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:34.565534 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806889728}
I20260709 12:24:34.588220 21435 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:34.519709 28326 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.592941 21432 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.18-20.18]
W20260709 12:24:34.577798 28271 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.002s
I20260709 12:24:34.577589 28031 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.575414 27937 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.574219 28299 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.579066 28315 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:34.621412 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 277892us
I20260709 12:24:34.622813 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.624282 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.629722 28072 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:34.631119 28072 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.631534 28285 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.631621 28072 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.632071 28285 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.633481 28072 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.634192 28285 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:34.642468 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 814967us
I20260709 12:24:34.660655 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:34.625820 28299 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.625566 28240 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:34.647148 20881 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:24:34.647770 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.648483 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.648980 28287 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.649156 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:34.626578 28355 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:34.649819 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.687230 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:34.650725 28212 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:34.603801 28275 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:24:34.604976 28280 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:24:34.697650 20877 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:34.654424 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.687964 20808 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:34.687876 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:34.688987 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.690425 20879 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:34.691776 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.642704 28249 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:24:34.628710 27937 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.695331 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.695699 28312 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.595551 28337 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.721441 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:34.576673 28359 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.600646 28271 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.696028 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.603989 28326 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:34.753769 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.696296 28317 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:34.685586 28241 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:34.704130 20803 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:34.607419 28342 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.713793 20805 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:34.723672 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:34.725548 21434 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:34.705875 28297 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.727821 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:34.663780 28292 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:34.706393 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:34.778460 21432 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:34.714954 28224 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:24:34.761487 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.722254 28230 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.712240 28280 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:24:34.764014 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47824.7us
W20260709 12:24:34.770504 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46940.2us
I20260709 12:24:34.720185 28236 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:34.770895 21428 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:34.634747 28276 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.121s
I20260709 12:24:34.766055 28241 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.770175 28297 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.780387 28317 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:34.760404 27936 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:34.760790 27966 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:34.762504 28367 ts_tablet_manager.cc:933] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:34.780602 20701 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:34.823709 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45861us
I20260709 12:24:34.824751 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:34.784468 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:34.711138 28239 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.787549 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:34.789541 20808 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:24:34.783289 28280 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:34.794975 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:34.706364 28320 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.076s
I20260709 12:24:34.697988 28287 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.752008 27855 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:24:34.786875 28292 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:34.785983 28236 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.813344 28297 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.808108 28317 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.816812 28241 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.770466 28249 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.831440 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47067.3us
W20260709 12:24:34.840341 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52916.8us
W20260709 12:24:34.829993 28363 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:24:34.855228 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:34.824118 28276 consensus_queue.cc:1243] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:34.782047 28368 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
W20260709 12:24:34.866748 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73014.6us
I20260709 12:24:34.823362 28296 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:34.871686 20882 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
W20260709 12:24:34.872329 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 81432.8us
I20260709 12:24:34.906596 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.899004 20884 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:34.806396 28231 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:24:34.843611 28131 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:34.845494 28261 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:34.843439 28207 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:34.879621 21432 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:24:34.843371 27966 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:34.828250 27936 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:34.871937 28236 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:34.912984 20776 leader_election.cc:400] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16.
I20260709 12:24:35.079327 20776 leader_election.cc:403] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:34.895125 21433 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:34.903007 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:34.920938 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:34.922547 20735 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:34.924937 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:34.929263 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:34.929512 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.873799 28317 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:35.029279 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:35.034245 28297 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:34.911002 28212 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.881135 28320 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:35.041626 28337 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:35.042505 27855 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.043473 28287 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:34.909330 28269 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:35.093669 28231 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:24:34.979002 28296 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.163913 28296 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:34.886981 28249 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.059151 28230 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:34.988974 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:35.165508 20882 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:35.166599 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:34.986795 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:35.167944 28377 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 12:24:35.084455 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 181555us
I20260709 12:24:35.091747 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:35.092231 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:35.169929 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77788.7us
I20260709 12:24:35.168006 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:35.171420 28261 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:35.172030 28261 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:35.173017 28261 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.119681 20884 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:35.176371 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:35.178512 20882 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:35.180341 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:35.043499 28380 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:35.098415 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:35.191861 20728 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:35.134629 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 221790us
I20260709 12:24:34.967268 28207 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.160878 20729 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:35.193011 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.115689 27937 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.905987 28280 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.116268 21427 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:24:35.134451 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:35.117472 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:35.060608 28292 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:35.090178 28289 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.197590 20726 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:35.204722 28280 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:35.206089 20729 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
W20260709 12:24:35.206332 28375 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:24:35.206538 27937 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.207180 28269 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:35.207698 28131 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:24:35.209043 28315 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:35.166911 27936 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.135870 28212 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.195019 28207 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:35.215894 28373 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:35.215473 28231 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:35.192770 27855 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:35.217638 28372 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 12:24:35.089982 28241 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:24:35.096529 28251 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:35.156167 28320 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:35.183023 28316 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16.
I20260709 12:24:35.226795 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:35.242460 20807 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:24:35.243638 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.242759 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.199076 28281 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:24:35.136343 28297 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.278378 28315 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.220631 28380 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.246832 27966 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.289007 20728 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
W20260709 12:24:35.561439 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 488773us
I20260709 12:24:35.562861 20808 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:35.282811 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:35.279744 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:35.504323 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.564921 20878 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:35.565285 20801 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 17: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:35.565840 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:35.451685 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.561759 20880 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:35.566937 20878 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 23 because replica has last-logged OpId of term: 22 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 18.
I20260709 12:24:35.562027 20802 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:35.565709 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 2904) took 1386 ms (client timeout 1650 ms). Trace:
I20260709 12:24:35.504730 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 2875) took 1256 ms. Trace:
I20260709 12:24:35.572276 20734 rpcz_store.cc:276] 0709 12:24:34.219239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.231481 (+ 12242us) service_pool.cc:224] Handling call
0709 12:24:35.475070 (+1243589us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b54000c3fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:35.475303 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57253632}
I20260709 12:24:35.331483 20882 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:35.278813 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:35.578490 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.565402 20883 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 23 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
W20260709 12:24:35.571628 20801 rpcz_store.cc:269] 0709 12:24:34.179982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.180167 (+   185us) service_pool.cc:224] Handling call
0709 12:24:35.504103 (+1323936us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:24:35.565647 (+ 61544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3356032}
I20260709 12:24:35.589695 20882 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:35.435762 28299 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.006s
I20260709 12:24:35.593665 20879 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:35.593811 20876 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:35.284740 28378 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:24:35.454131 28381 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:24:35.376195 27785 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.484385 27936 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.286387 28366 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:35.606596 20881 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:35.597922 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:35.415874 28312 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.374147 28239 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.502530 28345 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:35.279729 28212 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:35.611279 20876 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:35.568245 28251 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.625874 20805 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:35.261087 28289 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.332742 28367 tablet_copy_client.cc:323] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:24:35.592453 28326 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.273880 28269 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.283881 28382 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.413s
I20260709 12:24:35.627730 28366 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:35.652619 28383 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:24:35.636124 28251 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:35.628471 27785 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.674271 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3063) took 1015 ms. Trace:
I20260709 12:24:35.628896 27936 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.674896 20881 rpcz_store.cc:276] 0709 12:24:34.658579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.096723 (+438144us) service_pool.cc:224] Handling call
0709 12:24:35.674032 (+577309us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540005b254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:35.674192 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939551616}
I20260709 12:24:35.654398 28249 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:35.637238 28345 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.633322 28312 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:35.677490 28236 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.003s
W20260709 12:24:35.697489 28364 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.945s	user 0.003s	sys 0.000s
I20260709 12:24:35.698401 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1223) took 1713 ms. Trace:
W20260709 12:24:35.703723 28384 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:24:35.716490 20730 rpcz_store.cc:276] 0709 12:24:33.984538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.029304 (+ 44766us) service_pool.cc:224] Handling call
0709 12:24:34.029822 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.564835 (+535013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.698133 (+1133298us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54002490d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:35.698336 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205455744}
W20260709 12:24:35.703917 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.719412 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.704331 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:35.704697 20875 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:35.705190 28239 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:35.705484 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 2995) took 1059 ms. Trace:
I20260709 12:24:35.708019 20882 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:35.714092 20881 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:35.721887 20884 rpcz_store.cc:276] 0709 12:24:34.646045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.649608 (+  3563us) service_pool.cc:224] Handling call
0709 12:24:35.705321 (+1055713us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:35.705425 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457550208}
W20260709 12:24:35.706059 28393 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:24:35.729557 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.732589 20802 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:35.688220 27937 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.734750 20884 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:24:35.736052 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1833) took 1017 ms. Trace:
I20260709 12:24:35.709100 28249 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.736547 21429 rpcz_store.cc:276] 0709 12:24:34.719164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.719316 (+   152us) service_pool.cc:224] Handling call
0709 12:24:34.719772 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:34.720880 (+  1108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.720911 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:34.741016 (+ 20105us) log.cc:844] Serialized 52 byte log entry
0709 12:24:34.856541 (+115525us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:34.856578 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.856643 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.735795 (+879152us) raft_consensus.cc:1685] finished
0709 12:24:35.735821 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.735972 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3143,"raft.queue_time_us":7183,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":4266,"thread_start_us":77557,"threads_started":1,"wal-append.queue_time_us":221957}
I20260709 12:24:35.709824 28212 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.745257 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:35.756538 20726 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.003s
I20260709 12:24:35.757686 20726 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:35.763149 20802 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:24:35.766116 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:35.753726 28249 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.772485 27936 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.773339 27855 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:35.773972 28320 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.775449 28240 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:24:35.775854 28388 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:24:35.775952 28240 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.776504 28240 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:35.779050 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:35.761371 28212 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.780407 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3059) took 1122 ms. Trace:
I20260709 12:24:35.780849 28288 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:35.780793 20882 rpcz_store.cc:276] 0709 12:24:34.657969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.591635 (+933666us) service_pool.cc:224] Handling call
0709 12:24:35.707000 (+115365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.722270 (+ 15270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.780201 (+ 57931us) spinlock_profiling.cc:243] Waited 80.6 ms on lock 0x7b5400306950. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:35.780346 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185415936}
I20260709 12:24:35.781395 28345 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:35.781594 28212 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.782210 28212 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:35.782687 28212 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:35.783823 28391 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:35.729187 28342 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:35.785858 21426 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:35.787317 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:35.788228 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.006s	sys 0.005s
W20260709 12:24:35.788870 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.006s	sys 0.005s
I20260709 12:24:35.716074 28236 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:35.789149 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 728579us
I20260709 12:24:35.798658 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:35.799341 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:35.793848 20733 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:35.795810 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:35.803938 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:35.806399 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:35.812619 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:35.743088 28368 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:24:35.814150 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:35.815996 20726 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:24:35.805430 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:35.819625 20733 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:35.792215 27936 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.791337 28299 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:35.820711 28389 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:35.823483 21433 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:35.826177 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.739614 28239 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.820854 20876 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:35.761336 27937 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:24:35.822193 20802 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:24:35.802727 28297 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.834509 20735 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [17.14-18.15]
I20260709 12:24:35.825098 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:35.836377 20880 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:35.847596 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1316) took 1124 ms. Trace:
I20260709 12:24:35.848090 20880 rpcz_store.cc:276] 0709 12:24:34.723055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.196111 (+473056us) service_pool.cc:224] Handling call
0709 12:24:35.847534 (+651423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:35.817152 28031 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:35.840046 20726 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:35.734993 28366 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.844725 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:35.863443 28249 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.864424 28366 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:35.814561 28345 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.030s
W20260709 12:24:35.867127 28390 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 12:24:35.868044 28290 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.328s	user 0.014s	sys 0.000s
I20260709 12:24:35.828528 27936 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.868600 28290 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.820143 28337 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:24:35.872539 28337 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:35.873202 28337 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:35.857709 28297 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.876888 28297 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.803247 28261 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:24:35.858191 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:35.805666 28231 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:24:35.878795 21430 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:35.874281 28337 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:35.840127 28315 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:35.819350 28207 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:35.895622 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 452287us
I20260709 12:24:35.850244 28389 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.850528 28224 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:35.847280 28230 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:24:35.907872 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.908394 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:35.909788 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:35.910727 20808 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.12-26.12]
I20260709 12:24:35.912791 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:35.895336 27936 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.892854 28289 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:35.897257 28290 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:35.900362 28269 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:35.935977 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:35.842684 28212 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:35.919585 28317 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.761s	user 0.010s	sys 0.002s
W20260709 12:24:35.947224 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 166704us
I20260709 12:24:35.951603 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:35.952425 20877 raft_consensus.cc:606] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:35.952829 20877 raft_consensus.cc:614] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:35.953119 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:35.958339 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:35.920912 28380 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.920363 27815 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:35.921907 27785 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:35.981614 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:35.982331 20805 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:35.983235 20727 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:35.983716 20727 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:35.983929 28296 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.984422 28296 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.984557 20727 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.985469 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1323) took 1206 ms. Trace:
I20260709 12:24:35.986018 20875 rpcz_store.cc:276] 0709 12:24:34.779157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.721103 (+941946us) service_pool.cc:224] Handling call
0709 12:24:35.721406 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.721630 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:35.744847 (+ 23217us) log.cc:844] Serialized 14 byte log entry
0709 12:24:35.984993 (+240146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.985068 (+    75us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:35.985133 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.985228 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.985390 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79307,"prepare.queue_time_us":23974,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"thread_start_us":160401,"threads_started":1}
I20260709 12:24:35.901788 28389 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:35.941397 27937 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:35.920575 28283 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.070s
I20260709 12:24:35.969832 27936 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.945284 28315 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:24:36.025516 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1314) took 1298 ms. Trace:
I20260709 12:24:35.963361 28269 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:24:36.028790 21430 rpcz_store.cc:276] 0709 12:24:34.727329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.728184 (+   855us) service_pool.cc:224] Handling call
0709 12:24:34.766344 (+ 38160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.890573 (+1124229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.025129 (+134556us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540050c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:36.025418 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2557768064}
I20260709 12:24:35.963790 28317 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.071856 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.022522 28276 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:24:35.992586 27815 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.016588 28236 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:36.087796 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.006649 27937 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.075222 28297 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:36.076457 28312 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:36.102375 28402 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:24:36.103088 28396 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:24:36.104691 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3066) took 1445 ms. Trace:
I20260709 12:24:36.104966 28296 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:36.101150 27815 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.078876 28269 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.109279 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 153033us
I20260709 12:24:36.035327 27936 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.110402 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:36.090158 28276 consensus_queue.cc:1243] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:36.113225 20733 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:36.081370 28405 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:36.117630 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:36.154922 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:36.155573 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45260.8us
I20260709 12:24:36.087267 28407 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.124732 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:36.120944 20879 rpcz_store.cc:276] 0709 12:24:34.659004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.606846 (+947842us) service_pool.cc:224] Handling call
0709 12:24:35.607207 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:35.625188 (+ 17981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.625237 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:35.630336 (+  5099us) log.cc:844] Serialized 26 byte log entry
0709 12:24:35.915659 (+285323us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:35.915732 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.915874 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.104432 (+188558us) raft_consensus.cc:1685] finished
0709 12:24:36.104461 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.104622 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121681,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":21725,"thread_start_us":261974,"threads_started":1,"wal-append.queue_time_us":338112}
I20260709 12:24:36.160605 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:36.160991 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:36.161772 20879 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:36.155092 20875 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:36.164587 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1825) took 1977 ms. Trace:
I20260709 12:24:36.164951 20875 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:24:36.165344 20875 pending_rounds.cc:85] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 11
I20260709 12:24:36.165843 20875 pending_rounds.cc:107] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:24:36.166286 20875 raft_consensus.cc:2889] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:36.155633 20880 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:36.165297 21428 rpcz_store.cc:276] 0709 12:24:34.187499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.252870 (+ 65371us) service_pool.cc:224] Handling call
0709 12:24:34.253348 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.786092 (+532744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.164255 (+1378163us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:36.164506 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1175470464}
I20260709 12:24:36.108625 28256 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:36.118587 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:36.120196 20882 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 19. (term mismatch)
I20260709 12:24:36.140507 28031 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:36.168249 20882 pending_rounds.cc:85] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 18
I20260709 12:24:36.171351 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:36.170285 20882 pending_rounds.cc:107] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 22.19
I20260709 12:24:36.187722 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:36.145375 28297 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:36.212719 28031 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.213388 27937 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.214704 28389 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:36.151685 28207 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.215695 28239 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.216444 27936 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:36.170310 28256 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:36.191253 28335 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:36.219815 28335 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:36.220679 28335 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.221424 20727 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:36.169718 28276 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:36.223492 28276 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:36.224570 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3071) took 1536 ms. Trace:
I20260709 12:24:36.224965 28405 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:36.225692 28276 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.216486 20882 raft_consensus.cc:2889] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:36.226183 28333 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:36.226601 28333 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:36.227499 28333 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.228926 28333 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:36.229936 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:36.223050 28312 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.218860 28290 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:36.231603 28290 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:36.231552 28312 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.232914 28290 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.233789 28312 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.229756 28240 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.236052 28240 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.161522 27815 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 12:24:36.238538 28297 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.240258 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.241384 20726 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:36.241477 28297 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:36.167089 28241 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.003s
I20260709 12:24:36.189222 28286 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:36.243219 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.243585 28286 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.154424 28359 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:36.245525 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.170358 28407 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.154114 28269 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:36.217819 28289 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:24:36.198352 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 228050us
I20260709 12:24:36.259990 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:36.265676 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 94401.5us
I20260709 12:24:36.252748 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:36.170899 20884 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:24:36.225162 20878 rpcz_store.cc:276] 0709 12:24:34.688268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.625311 (+937043us) service_pool.cc:224] Handling call
0709 12:24:35.636303 (+ 10992us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:35.731381 (+ 95078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.731428 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:35.742164 (+ 10736us) log.cc:844] Serialized 52 byte log entry
0709 12:24:35.902155 (+159991us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:35.902221 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.224025 (+321804us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b54001fde54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:36.224090 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.224168 (+    78us) raft_consensus.cc:1685] finished
0709 12:24:36.224190 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.224487 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22856,"raft.queue_time_us":72373,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":5217,"spinlock_wait_cycles":179799936,"thread_start_us":134562,"threads_started":1,"wal-append.queue_time_us":231684}
I20260709 12:24:36.268707 20875 raft_consensus.cc:606] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:36.269348 20875 raft_consensus.cc:614] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:36.250687 28290 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.276849 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.279663 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.280611 20802 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:36.270205 27815 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 12:24:36.286170 27815 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:36.286562 27815 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:36.287413 27815 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.288642 27815 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:36.290858 20801 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:36.291302 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.292100 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:36.215042 28313 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.292114 20807 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:36.337903 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43881us
I20260709 12:24:36.290872 20805 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:24:36.338940 20735 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:36.339167 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:36.270350 28404 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.113s	user 0.000s	sys 0.003s
I20260709 12:24:36.255160 28289 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:36.342145 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.271130 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:36.339013 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:36.353482 21428 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:36.295038 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 491123us
I20260709 12:24:36.314373 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.356832 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.341653 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.315090 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:36.358759 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 213934us
I20260709 12:24:36.358888 20732 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:36.299237 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.364043 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.365494 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.367511 20734 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:24:36.367671 20808 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:36.357517 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:36.372639 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.373123 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:36.293486 28342 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.010s
I20260709 12:24:36.377334 20726 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:24:36.383092 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.374817 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.379832 20731 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:36.384097 20809 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:24:36.375032 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.191710 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:36.308650 20727 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:36.299033 28312 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:24:36.390434 28415 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.391090 28415 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.393844 28415 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.394994 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:36.395658 21432 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:36.393580 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.416462 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.416370 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:36.422828 21428 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
W20260709 12:24:36.395509 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.397094 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:36.337580 28313 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.343112 28326 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 12:24:36.386461 28409 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:24:36.428905 28313 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:36.290571 28414 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:36.421031 20727 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:24:36.395818 28342 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.017s	sys 0.014s
I20260709 12:24:36.444499 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.445641 20730 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:36.445998 28296 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.439692 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:36.385689 28131 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:36.446776 28296 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.447312 28131 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:36.448675 28131 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.449640 28296 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.453259 28230 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:36.454860 28382 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:36.455606 28382 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:36.455824 28318 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.456590 28318 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.458666 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:36.442233 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45221.1us
W20260709 12:24:36.422134 28278 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:24:36.468457 20733 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.12-26.12]
I20260709 12:24:36.468775 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.492340 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.492728 20731 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:36.270519 28299 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.153s
I20260709 12:24:36.290361 28301 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 12:24:36.431650 20729 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:24:36.454806 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.468022 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:36.475152 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 120461us
I20260709 12:24:36.489876 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.497231 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.505632 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:36.396006 28416 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.432348 28312 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:24:36.513265 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45358.5us
I20260709 12:24:36.513866 28240 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:36.298925 28268 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.198s
I20260709 12:24:36.442987 28297 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.552994 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:36.498684 28389 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:36.490669 28317 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.567731 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.568674 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:36.570448 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.456665 28382 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.468858 27936 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.468353 27855 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.544032 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:36.491021 28418 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:36.563050 28421 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:36.564080 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.008s	sys 0.020s
I20260709 12:24:36.571683 20734 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:24:36.587158 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.008s	sys 0.020s
W20260709 12:24:36.587822 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 629541us
I20260709 12:24:36.500837 28380 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 12:24:36.524508 28241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36394 (request call id 8) took 5675 ms. Trace:
I20260709 12:24:36.578110 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:36.557459 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 265451us
W20260709 12:24:36.570468 28406 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.323s	user 0.003s	sys 0.000s
I20260709 12:24:36.596801 28359 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:36.573159 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 154820us
I20260709 12:24:36.604068 28212 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:36.560583 28312 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:36.562054 28240 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:36.564497 28389 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.608862 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:36.542908 27966 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:24:36.541002 28416 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:36.666132 28423 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:36.651456 28236 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:36.592783 27936 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.669673 21429 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 12:24:36.587397 27855 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:36.579736 28418 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.588557 28417 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:36.667889 21433 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:24:36.680490 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.694869 28240 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.679253 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:36.672509 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:36.696763 28312 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.641140 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:36.698604 28422 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:36.628885 28382 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:36.699153 28422 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:36.628523 28287 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:36.695837 27855 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:36.701126 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:36.606212 28421 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:36.652534 28280 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.402s	user 0.008s	sys 0.007s
I20260709 12:24:36.700781 28287 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:36.689721 28359 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.704067 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:36.714923 20810 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:36.702908 28276 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:36.716821 28276 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:36.717538 28421 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.699961 28422 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:36.701751 28417 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:36.710364 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 140452us
I20260709 12:24:36.722059 28276 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:36.701340 28212 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.725653 28417 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:36.727044 28212 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.665200 21434 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:36.690918 27966 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:36.729288 27966 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:36.730031 20805 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:24:36.730484 20809 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:36.730664 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.731256 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.731655 20803 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:24:36.732239 27936 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.722229 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.714088 28418 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.735881 20809 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:36.631254 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53255.8us
I20260709 12:24:36.739897 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.737849 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.702661 28389 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.739892 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.732692 27966 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.568674 28297 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.763303 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:36.741595 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.745930 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.741138 27936 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:36.768519 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.543704 28289 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.758975 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.605206 28296 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.772696 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:36.780195 27855 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:36.779444 28428 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:36.780891 28240 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:36.781747 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65237us
I20260709 12:24:36.810768 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:36.810878 28240 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.812515 28425 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:36.812958 28425 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.813367 28425 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:36.813514 28240 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.743297 28271 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:36.815768 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44400.1us
W20260709 12:24:36.816264 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.817116 28428 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:36.782910 28440 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:36.783737 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.819473 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.764549 28407 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.819962 28428 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:36.764276 28268 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:36.820614 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:36.819962 28251 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.144s	user 0.005s	sys 0.008s
I20260709 12:24:36.821352 20806 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:24:36.784192 28429 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:24:36.786375 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.790057 28212 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:36.790817 27966 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.793908 27936 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:36.794637 28236 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:36.795657 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.797655 28416 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:36.798717 28120 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:24:36.800686 28289 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:36.801407 20735 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:36.802871 28380 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:36.809118 28297 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.741019 28359 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.845424 28320 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:36.734949 28342 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:24:36.824491 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:36.834187 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.835340 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.838533 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:36.838835 21431 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:24:36.752091 28269 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:36.839763 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.844372 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:36.849068 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:36.849303 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:36.852818 21435 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 12:24:36.885833 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 61427us
I20260709 12:24:36.856914 20735 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:36.852003 27855 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:36.821681 28251 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.859635 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:36.809849 28296 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.873760 20733 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:24:36.890235 21433 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:36.864957 28380 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:36.965498 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 2934) took 1720 ms. Trace:
I20260709 12:24:36.967554 21435 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 12:24:36.969137 27815 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.001s
I20260709 12:24:36.970124 20732 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:24:36.970573 28380 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:36.892081 28359 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:36.950541 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.892324 27855 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:36.855042 28434 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 12:24:36.920253 28430 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:36.988678 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.877380 28290 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.001s
I20260709 12:24:36.838773 28031 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:24:36.607604 28241 rpcz_store.cc:276] 0709 12:24:30.821064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.827455 (+  6391us) service_pool.cc:224] Handling call
0709 12:24:31.337322 (+509867us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a7a0a 00007f9e1d5a7693 00007f9e1d5a7631 00007f9e1ecda902 00007f9e1ecda6dd 00007f9e1eca5f3f 00007f9e1eca56b3 00007f9e23c65d29 00007f9e1d1e41d4 00007f9e1d1e4162
0709 12:24:31.342338 (+  5016us) log.cc:844] Serialized 506 byte log entry
0709 12:24:36.496785 (+5154447us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":250332,"raft.queue_time_us":254109,"raft.run_cpu_time_us":1958,"raft.run_wall_time_us":1955,"spinlock_wait_cycles":143588096,"thread_start_us":10290,"threads_started":1,"wal-append.queue_time_us":9407,"wal-append.run_cpu_time_us":1796,"wal-append.run_wall_time_us":1013336}
I20260709 12:24:36.914057 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:36.858726 28448 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:24:36.958333 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:36.959360 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:36.865300 28236 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:36.975596 28269 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:36.844658 28268 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 12:24:36.973145 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 113635us
I20260709 12:24:36.966104 20810 rpcz_store.cc:276] 0709 12:24:35.245432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.245643 (+   211us) service_pool.cc:224] Handling call
0709 12:24:35.245906 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.715790 (+1469884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.965152 (+249362us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540027ae54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:36.965410 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3377568512}
W20260709 12:24:36.974500 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 162523us
I20260709 12:24:37.031994 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:37.032801 28359 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:37.033201 28359 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.033392 20810 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:37.033576 28359 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:37.033885 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:36.925204 28251 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:37.034178 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:37.035007 27815 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:37.035362 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:37.035739 28212 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:37.036429 28212 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:24:37.037082 28276 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:37.037515 27815 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:37.038203 28212 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:24:36.978830 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.918241 28425 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.009502 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.039427 20810 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 27 index: 12. (index mismatch)
I20260709 12:24:37.039522 20805 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:37.040633 28425 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:37.041615 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.030376 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.042392 28317 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:36.977816 20809 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:37.043164 28317 consensus_queue.cc:1243] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:24:37.044613 28317 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:37.035218 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:37.046404 21435 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:37.049036 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:37.051155 28212 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:37.051996 20805 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:37.053859 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:37.056701 27815 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:24:37.058300 28281 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:37.061375 28281 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:37.062143 28281 consensus_queue.cc:1243] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:37.063453 28359 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.066349 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:37.066907 20806 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:37.067893 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 218761us
I20260709 12:24:37.070315 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.073800 21435 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:37.009502 28430 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:37.030913 28333 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.801s	user 0.012s	sys 0.002s
I20260709 12:24:37.089406 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:37.028316 28355 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:37.091162 28430 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:37.139632 20806 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:37.099918 28440 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:37.147382 28359 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:37.098866 28313 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:37.154960 28457 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.161324 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:37.164397 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 368799us
I20260709 12:24:37.180531 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:37.183300 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:37.189251 28457 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.180001 28313 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.190698 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3078) took 1066 ms. Trace:
I20260709 12:24:37.192276 20884 rpcz_store.cc:276] 0709 12:24:36.124285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.124438 (+   153us) service_pool.cc:224] Handling call
0709 12:24:37.190470 (+1066032us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54000a32d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:37.190632 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23586688}
I20260709 12:24:37.176358 28440 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:37.180326 28416 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:37.190404 28333 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:37.190128 28241 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:37.121702 27855 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 26.12, Last appended by leader: 12, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:37.248243 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54387.3us
I20260709 12:24:37.263062 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:37.249419 28417 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.008s
W20260709 12:24:37.264212 28454 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:24:37.269714 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:37.262079 28422 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.012s
I20260709 12:24:37.268095 21429 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:37.234898 28333 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.248193 28416 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:37.244997 27785 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:37.231333 28418 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:37.279739 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.283502 28453 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:37.264750 28380 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:24:37.285776 28380 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:37.286201 28380 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:37.287097 28380 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.289815 28380 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:37.289513 27785 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.272625 28241 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:37.295274 28440 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.298188 27785 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:37.300448 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.274624 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:37.294963 20803 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:24:37.311849 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.312628 20805 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:37.314810 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:37.315569 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:37.322132 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:37.274734 27855 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:37.274478 28460 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:37.317250 28236 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:37.344738 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.345942 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.346478 20805 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
W20260709 12:24:37.329734 28382 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.629s	user 0.003s	sys 0.006s
I20260709 12:24:37.332042 28268 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:37.356019 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.336067 28287 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:37.338040 28389 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.007s
I20260709 12:24:37.338444 27936 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:37.338663 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:37.274441 28333 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:37.342172 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:37.366139 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.344691 20734 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:24:37.345067 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:37.367153 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:24:37.349118 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.350039 20731 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:37.350742 28416 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.350787 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:37.368974 20726 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:24:37.313133 28251 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.313833 27815 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.325783 28249 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:37.357789 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1314) took 2635 ms. Trace:
I20260709 12:24:37.272140 28417 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.276711 28422 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.357501 28400 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:37.365178 28460 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.369504 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.366070 28292 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.372356 20876 rpcz_store.cc:276] 0709 12:24:34.722442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.634231 (+911789us) service_pool.cc:224] Handling call
0709 12:24:35.634453 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.839109 (+204656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.353535 (+1514426us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:37.357726 (+  4191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428125056}
I20260709 12:24:37.355247 28268 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.343797 20810 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:24:37.369220 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:37.392511 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.374727 28296 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:37.392373 28224 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:37.370507 28389 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.379740 28287 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:37.379401 28400 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:37.426187 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87585.8us
I20260709 12:24:37.412225 28236 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:37.415727 20876 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:37.397287 28425 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:37.406046 28268 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:37.406574 28292 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.374603 28251 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.374892 27815 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.410540 28418 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:37.421188 28342 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:37.437496 27966 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.000s
I20260709 12:24:37.420467 28241 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:37.457295 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.452978 20801 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:37.458981 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.460920 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.464206 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:37.464593 20875 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:37.423082 28224 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:37.472095 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:37.475864 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.476361 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:37.476588 20878 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:37.477741 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:37.478518 20883 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:37.456096 28469 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.404937 28460 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.492194 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:37.490183 28469 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.483767 28476 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:37.475528 27966 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:37.505833 28469 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:37.507162 28483 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:37.508795 28483 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.509199 28483 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:37.503402 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 187934us
I20260709 12:24:37.479944 28286 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:37.511844 28288 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:37.511991 28417 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:37.512305 28288 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:37.512677 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.507611 27966 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.513840 27966 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:37.513303 28288 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.514664 28288 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:37.509523 28474 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:37.519685 20884 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:24:37.532102 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:37.537953 20883 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:37.466984 28342 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.531515 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:37.541911 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.540417 28480 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.545475 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.507792 20893 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 1047135f71834dbab6b869eb3c081b81 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:24:37.555660 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:37.546751 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:37.548766 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.548930 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:37.546228 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.557802 20883 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:37.558322 28480 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.650146 20893 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 1047135f71834dbab6b869eb3c081b81 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-1047135f71834dbab6b869eb3c081b81
I20260709 12:24:37.559376 28460 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:37.560240 28475 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:37.560534 28342 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:37.560796 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:37.561074 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:37.682845 20809 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:37.561420 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.683535 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:37.565099 28398 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:37.565434 28433 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:37.570775 28271 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:37.573704 27855 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:37.714126 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.582693 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:37.714941 21433 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:37.584184 28249 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:37.549654 28417 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:37.715382 28249 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:37.507423 28224 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:37.715821 28249 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:37.715941 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.716329 28224 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:37.716929 21433 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:37.717898 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.718853 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.637831 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.639377 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.640298 28337 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:24:37.642283 28236 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.642863 28422 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:37.485193 28400 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:37.643121 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:37.644361 28469 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:37.547753 28241 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:37.597882 20877 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:24:37.645051 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:37.565110 28368 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:37.645380 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:37.701012 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:37.649127 28279 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.707902 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:37.599645 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.707041 28433 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:37.704547 28398 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:37.720041 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1325) took 1460 ms. Trace:
I20260709 12:24:37.720548 28421 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:37.715091 20880 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:37.722216 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:38.012238 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
W20260709 12:24:37.724366 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 152705us
W20260709 12:24:37.962896 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 444601us
I20260709 12:24:38.022265 20878 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:37.881441 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.031107 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1323) took 1772 ms. Trace:
W20260709 12:24:38.016644 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 364264us
W20260709 12:24:38.040580 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 339638us
W20260709 12:24:37.734373 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.032372 21430 rpcz_store.cc:276] 0709 12:24:36.258389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.259474 (+  1085us) service_pool.cc:224] Handling call
0709 12:24:36.299659 (+ 40185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.470546 (+170887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.470593 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.508379 (+ 37786us) log.cc:844] Serialized 26 byte log entry
0709 12:24:36.711135 (+202756us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:24:36.711314 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.011973 (+1300659us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400776854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:38.012048 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.012124 (+    76us) raft_consensus.cc:1685] finished
0709 12:24:38.012148 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.030986 (+ 18838us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17782,"raft.queue_time_us":105,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6167,"spinlock_wait_cycles":391785344,"thread_start_us":146715,"threads_started":1,"wal-append.queue_time_us":250329,"wal-append.run_cpu_time_us":1728,"wal-append.run_wall_time_us":1026391}
W20260709 12:24:38.045006 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.710207 28271 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:37.804003 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.071017 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.071162 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1429) took 1230 ms. Trace:
I20260709 12:24:38.019006 21427 rpcz_store.cc:276] 0709 12:24:36.259225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.261655 (+  2430us) service_pool.cc:224] Handling call
0709 12:24:36.294490 (+ 32835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.294930 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.294958 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.336860 (+ 41902us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.719419 (+1382559us) spinlock_profiling.cc:243] Waited 924 us on lock 0x7b180027b7a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1
0709 12:24:37.719608 (+   189us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:37.719653 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.719746 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.719787 (+    41us) raft_consensus.cc:1685] finished
0709 12:24:37.719798 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.719987 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121868,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":16475,"spinlock_wait_cycles":2125568,"thread_start_us":104765,"threads_started":1,"wal-append.queue_time_us":251329,"wal-append.run_cpu_time_us":3389,"wal-append.run_wall_time_us":1313581}
I20260709 12:24:38.071659 21429 rpcz_store.cc:276] 0709 12:24:36.840821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.844036 (+  3215us) service_pool.cc:224] Handling call
0709 12:24:38.070975 (+1226939us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:38.071103 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880408320}
W20260709 12:24:38.076201 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 433115us
W20260709 12:24:38.037822 20774 connection.cc:256] RPC call timeout handler was delayed by -1.3987262480000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24860000000000004s for a response
W20260709 12:24:38.042706 28288 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.528s	user 0.006s	sys 0.007s
W20260709 12:24:38.080822 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 435479us
I20260709 12:24:37.867116 27815 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.069411 28279 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.070025 28368 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:38.088846 28400 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:38.088748 28296 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:37.723210 28389 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:38.054193 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.095366 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:38.088510 28486 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 12:24:38.102442 20880 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:38.104414 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:38.102948 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.105144 20883 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:37.851140 28483 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.111220 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:38.105836 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:38.081964 28276 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.110636 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.037282 28241 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.091564 28480 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.119596 20883 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:38.125104 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:38.126607 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.129082 28241 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.129906 28483 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.132647 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1845) took 1371 ms. Trace:
I20260709 12:24:38.133117 21434 rpcz_store.cc:276] 0709 12:24:36.760873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.761011 (+   138us) service_pool.cc:224] Handling call
0709 12:24:36.773968 (+ 12957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.821312 (+ 47344us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:36.821626 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.265910 (+444284us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.266105 (+   195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:37.266152 (+    47us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:37.266203 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.131559 (+865356us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b1800173ce0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:24:38.132387 (+   828us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:38.132437 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.132578 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27318,"prepare.queue_time_us":5771,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":5999,"raft.queue_time_us":55833,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":117651712,"thread_start_us":418988,"threads_started":2,"wal-append.queue_time_us":386892}
I20260709 12:24:38.108579 28433 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:38.115868 28430 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.952s	user 0.012s	sys 0.003s
I20260709 12:24:38.136441 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.136987 28430 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.137923 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.139454 28368 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.140295 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:38.140995 20875 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:38.141047 28368 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:38.143569 21431 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:38.144831 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1847) took 1298 ms. Trace:
I20260709 12:24:38.145190 21431 rpcz_store.cc:276] 0709 12:24:36.845975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.913632 (+ 67657us) service_pool.cc:224] Handling call
0709 12:24:38.144602 (+1230970us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400729a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:38.144765 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577411456}
W20260709 12:24:38.145895 28287 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.003s
I20260709 12:24:38.146967 28491 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:38.147084 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.147692 28491 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:38.147954 20877 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:38.148212 28491 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.148938 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.069478 28236 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.150938 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:38.151000 20802 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:38.105527 28380 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.152585 20883 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:38.034505 28398 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.154086 28279 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:38.169839 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64109.1us
I20260709 12:24:38.171231 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.172138 20883 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:38.173418 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.174939 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.177033 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:38.178077 20883 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:38.159157 20878 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:38.121157 28490 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.163335 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.184397 20879 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:38.185657 20878 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:38.163393 20875 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:38.189786 20880 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:38.152405 28487 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.179090 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.188226 28460 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:38.183904 28480 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.187407 28483 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.295078 20878 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:38.189385 28288 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.304096 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 217518us
I20260709 12:24:38.163985 28380 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:38.307484 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 266468us
I20260709 12:24:38.297581 28241 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.152546 28271 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:38.304144 28345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.009s	sys 0.032s
I20260709 12:24:38.317334 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.299666 28405 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.379688 20877 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:24:38.298699 28460 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:38.377900 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.193161 28490 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:38.380226 28345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.009s	sys 0.037s
I20260709 12:24:38.315168 28430 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.389878 20878 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:38.308413 28480 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.313205 28288 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.400923 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:38.296437 28487 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.383662 28460 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.311360 28483 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.401702 20807 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:24:38.403795 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:38.406795 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:38.407042 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:38.410671 20801 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:38.395757 28230 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:38.407509 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:38.386691 27815 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.419392 27815 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.422307 27815 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:38.424038 27966 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.910s	user 0.015s	sys 0.002s
I20260709 12:24:38.424621 27966 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.428408 28288 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.390803 28337 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:38.430318 28380 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.383438 28405 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.430858 28337 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.431491 28236 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.433087 28405 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:38.433928 28337 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:38.438721 28249 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.722s	user 0.008s	sys 0.003s
I20260709 12:24:38.378823 28241 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:38.402504 28268 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.939s	user 0.012s	sys 0.000s
I20260709 12:24:38.462824 20878 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:38.437440 20893 tablet_copy_source_session.cc:215] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:24:38.466475 28493 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:24:38.466365 28483 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:38.466434 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.482599 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.473691 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.475458 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.462852 28485 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 12:24:38.487551 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:38.488499 20807 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.482952 20801 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:38.480891 28405 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.458392 28249 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.485057 27815 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:38.488991 28422 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.505333 28333 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.128s	user 0.016s	sys 0.000s
I20260709 12:24:38.529904 20807 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:38.531836 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.472316 28268 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.555394 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.555788 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.557369 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.554419 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.602996 21433 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:38.603061 21429 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:38.566182 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1334) took 1106 ms. Trace:
I20260709 12:24:38.603367 21431 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 15.
I20260709 12:24:38.537678 28389 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.571750 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.591511 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.596378 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.596148 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:38.613210 21435 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:38.565197 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:38.611708 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 126396us
I20260709 12:24:38.524557 28405 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.603992 20881 rpcz_store.cc:276] 0709 12:24:37.460075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.467294 (+  7219us) service_pool.cc:224] Handling call
0709 12:24:37.484349 (+ 17055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.530866 (+ 46517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.530898 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.547176 (+ 16278us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.103354 (+556178us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:38.103416 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.103519 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.565875 (+462356us) raft_consensus.cc:1685] finished
0709 12:24:38.565908 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.566104 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46049,"raft.queue_time_us":46883,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":458389,"threads_started":1,"wal-append.queue_time_us":689347}
I20260709 12:24:38.537578 28333 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:38.594498 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 195827us
I20260709 12:24:38.609985 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.535651 28500 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.057s
W20260709 12:24:38.610065 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 78349.6us
I20260709 12:24:38.591563 20879 raft_consensus.cc:2955] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Committing config change with OpId 20.15: config changed from index -1 to 15, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:24:38.537866 27815 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.611128 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1335) took 1150 ms. Trace:
I20260709 12:24:38.612008 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.604641 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.538029 28422 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.615545 21433 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:38.616250 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.616813 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.620482 20882 rpcz_store.cc:276] 0709 12:24:37.460430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.467536 (+  7106us) service_pool.cc:224] Handling call
0709 12:24:37.467871 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:37.475727 (+  7856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.475761 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:37.484207 (+  8446us) log.cc:844] Serialized 52 byte log entry
0709 12:24:37.706017 (+221810us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:37.706070 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.706166 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.610899 (+904733us) raft_consensus.cc:1685] finished
0709 12:24:38.610925 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.611067 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17180,"raft.queue_time_us":4734,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":8134,"thread_start_us":198063,"threads_started":1,"wal-append.queue_time_us":540244}
I20260709 12:24:38.621292 21432 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:24:38.595664 28431 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.593319 28288 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:38.641448 21426 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:38.643399 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.614117 28276 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:38.659353 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3095) took 1114 ms. Trace:
I20260709 12:24:38.628029 28494 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:38.671307 20875 rpcz_store.cc:276] 0709 12:24:37.544776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.188597 (+643821us) service_pool.cc:224] Handling call
0709 12:24:38.189036 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.189341 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:38.407585 (+218244us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.658106 (+250521us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.658188 (+    82us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:38.658263 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.659073 (+   810us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.659272 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":348638,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":3363,"raft.queue_time_us":11093,"thread_start_us":354394,"threads_started":2}
I20260709 12:24:38.594911 28491 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.627966 20893 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:53814 (request call id 1328) took 1195 ms. Trace:
W20260709 12:24:38.648769 28224 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.932s	user 0.013s	sys 0.000s
I20260709 12:24:38.648769 20880 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:24:38.675769 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.620644 28389 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.740242 20884 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
W20260709 12:24:38.674868 28400 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.002s
I20260709 12:24:38.739972 21429 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:38.742743 20876 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:38.743525 28389 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.747295 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.672977 28490 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.749420 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.721134 28276 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.719858 28480 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:38.643301 28431 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:38.756104 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134262us
I20260709 12:24:38.758072 20881 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:38.748502 28279 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:38.764557 28279 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:38.766520 20875 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:38.740321 28487 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:38.745126 28494 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.768700 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6fe576036c04c149240cd6107285eca"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:24:38.768759 28494 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.769469 20875 raft_consensus.cc:606] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:38.738541 28241 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:38.761013 28398 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 12:24:38.755489 28510 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:38.746045 28503 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:24:38.752489 28256 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:38.763875 28502 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:24:38.784682 28521 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.767787 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.785229 28521 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.771114 28494 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.786104 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3098) took 1228 ms. Trace:
I20260709 12:24:38.786528 20883 rpcz_store.cc:276] 0709 12:24:37.557618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.294507 (+736889us) service_pool.cc:224] Handling call
0709 12:24:38.294874 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.296509 (+  1635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.296571 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.297210 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.475584 (+178374us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:24:38.475660 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.475734 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.785889 (+310155us) raft_consensus.cc:1685] finished
0709 12:24:38.785911 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.786047 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1174,"raft.queue_time_us":5636,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5954,"thread_start_us":177868,"threads_started":1,"wal-append.queue_time_us":293724}
I20260709 12:24:38.644134 28288 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.767010 28279 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:38.776286 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:38.797194 28417 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:38.797626 28417 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:38.797966 28417 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:38.719801 20893 rpcz_store.cc:276] 0709 12:24:37.438172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.466351 (+ 28179us) service_pool.cc:224] Handling call
0709 12:24:38.609999 (+1143648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:38.779003 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:38.783095 20730 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:38.800488 20804 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:38.784322 28256 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.805370 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:38.807461 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.808493 28241 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:38.808907 28241 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.814790 28241 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.817909 28212 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:38.818526 28457 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:38.819289 28457 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.818575 28212 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.819828 28457 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.822516 28519 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:38.822398 28212 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.822880 20806 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:24:38.823357 20806 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 12
I20260709 12:24:38.823645 20806 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:24:38.823872 20806 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:38.781560 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 164800us
I20260709 12:24:38.826051 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:38.825757 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.827287 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:38.828053 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:38.828321 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:38.828691 20732 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:38.841524 20881 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:38.841392 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:38.824477 28286 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.848052 20732 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:38.772701 28480 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:38.856119 20804 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:38.842881 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:38.853675 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.853696 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3009) took 1062 ms. Trace:
I20260709 12:24:38.857605 20881 rpcz_store.cc:276] 0709 12:24:37.790955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.852614 (+1061659us) service_pool.cc:224] Handling call
0709 12:24:38.853018 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.853261 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.853302 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.853365 (+    63us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:38.853415 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.853491 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.853625 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:38.855321 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.860275 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:38.860752 20731 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:38.862188 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:38.865468 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:38.866299 20803 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:38.866521 20875 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 27 index: 12. (term mismatch)
I20260709 12:24:38.867461 20875 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 11
I20260709 12:24:38.841203 27815 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.824846 28249 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.839437 27855 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:38.868520 20875 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 26.12
I20260709 12:24:38.870203 20875 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.871054 20881 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:38.873138 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3010) took 1034 ms. Trace:
I20260709 12:24:38.873518 20881 rpcz_store.cc:276] 0709 12:24:37.838892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.867532 (+1028640us) service_pool.cc:224] Handling call
0709 12:24:38.870873 (+  3341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.872868 (+  1995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.873080 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:38.851432 28212 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:38.850945 28490 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:38.875488 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3102) took 1210 ms. Trace:
I20260709 12:24:38.856354 28389 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:38.781288 28398 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.877264 20881 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:38.863377 28318 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:38.900574 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.935484 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 484124us
I20260709 12:24:38.876065 20875 rpcz_store.cc:276] 0709 12:24:37.664833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.782778 (+1117945us) service_pool.cc:224] Handling call
0709 12:24:38.866176 (+ 83398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.870873 (+  4697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.874381 (+  3508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:38.868016 28224 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:38.938913 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:38.947772 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3011) took 1021 ms. Trace:
I20260709 12:24:38.948429 20881 rpcz_store.cc:276] 0709 12:24:37.859703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.875122 (+1015419us) service_pool.cc:224] Handling call
0709 12:24:38.875456 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.880625 (+  5169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.880867 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:38.966693 20733 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:24:38.878511 28212 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:38.864792 28509 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:38.984645 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47456.5us
I20260709 12:24:38.986932 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:38.987579 20731 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.902158 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:38.988102 20729 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:38.876750 27815 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:38.979286 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.861937 28286 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.884990 20732 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:38.978945 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:38.823096 28519 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:38.878132 28389 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.881444 28268 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:38.999646 20735 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:38.934630 28521 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.008826 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:39.020306 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1329) took 1563 ms. Trace:
I20260709 12:24:39.020905 20876 rpcz_store.cc:276] 0709 12:24:37.456715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.456945 (+   230us) service_pool.cc:224] Handling call
0709 12:24:39.020043 (+1563098us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400366854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:39.020223 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2940907776}
I20260709 12:24:39.022881 20876 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:38.876914 28480 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.026875 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3090) took 1483 ms. Trace:
I20260709 12:24:39.029261 20875 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:39.029460 20884 rpcz_store.cc:276] 0709 12:24:37.543529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.138929 (+595400us) service_pool.cc:224] Handling call
0709 12:24:39.018914 (+879985us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:39.026807 (+  7893us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350543744}
I20260709 12:24:39.029863 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3012) took 1153 ms. Trace:
I20260709 12:24:39.030172 20875 rpcz_store.cc:276] 0709 12:24:37.876137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.995866 (+1119729us) service_pool.cc:224] Handling call
0709 12:24:39.029068 (+ 33202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.029635 (+   567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.029812 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:39.027281 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3013) took 1134 ms. Trace:
I20260709 12:24:39.031491 20876 rpcz_store.cc:276] 0709 12:24:37.893147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.022209 (+1129062us) service_pool.cc:224] Handling call
0709 12:24:39.022634 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.026939 (+  4305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.027209 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:39.034133 20876 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:38.988431 28405 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:38.992100 28333 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:38.879913 28520 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.138s
W20260709 12:24:39.046231 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 184089us
I20260709 12:24:39.028223 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:39.039445 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.040747 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.042500 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:39.052996 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:39.038975 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.044061 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.054692 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:39.011781 28389 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.056034 20731 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:39.061429 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:39.062397 20731 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.015492 28422 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.016206 28487 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:39.009729 28518 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:39.019243 28509 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:39.068936 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:39.026000 28236 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.067058 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:38.997534 28279 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:24:39.074162 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:39.015302 28498 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:24:39.043500 28508 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:39.074085 28131 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:39.046592 28491 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.065711 28525 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:39.072503 28531 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:39.076974 28518 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:39.137410 28416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.003s	sys 0.006s
W20260709 12:24:39.138657 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 278455us
I20260709 12:24:39.138816 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.087567 28508 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:39.147652 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:39.147082 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 278289us
I20260709 12:24:39.149536 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:39.151836 20731 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:39.155810 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:39.130748 28131 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:39.082733 28236 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:39.137956 28416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.262s	user 0.003s	sys 0.006s
I20260709 12:24:39.131505 28531 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:39.169919 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:39.170324 28531 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:39.170718 20731 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:39.171627 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:39.179456 28513 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.415s	user 0.003s	sys 0.000s
I20260709 12:24:39.164065 28415 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:39.184079 28415 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.184576 28415 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:39.189384 28241 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.167783 28288 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:39.196113 28460 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.738s	user 0.007s	sys 0.005s
I20260709 12:24:39.086881 28212 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.199946 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.198773 28236 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:39.168423 28131 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:39.131839 28518 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:39.227334 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.205685 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 136811us
I20260709 12:24:39.134696 28431 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.222772 21430 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:39.140813 28417 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.152540 28508 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.216001 28276 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:39.227968 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62039.8us
I20260709 12:24:39.245396 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:39.230695 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:39.228645 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 158155us
I20260709 12:24:39.228446 28131 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:39.244812 28535 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:39.249476 28535 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:39.250379 28535 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:39.248245 28529 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.132s	user 0.001s	sys 0.001s
I20260709 12:24:39.237443 28271 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.166038 28492 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.270951 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:39.270385 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:39.224824 28460 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.252975 28398 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.284821 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:39.256498 28251 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.237377 28530 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:24:39.204566 28509 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.298247 28241 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.302191 28431 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:39.303874 28541 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:24:39.304823 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44451.8us
W20260709 12:24:39.339443 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.305761 28537 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:39.307351 28507 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:39.308849 28428 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:39.313161 20735 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:39.362974 20735 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [17.14-18.15]
I20260709 12:24:39.318456 20726 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:39.297104 28480 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.325850 28313 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:39.373500 20699 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 2.227s (SENT) [suppressed 18 similar messages]
W20260709 12:24:39.328147 28504 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.583s	user 0.002s	sys 0.000s
W20260709 12:24:39.328558 28526 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
I20260709 12:24:39.281292 28492 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.285934 28491 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:39.388288 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.389155 21430 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:39.389575 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.288232 28519 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:39.345902 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.356351 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.352499 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:39.398098 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45724.1us
W20260709 12:24:39.356959 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 110534us
I20260709 12:24:39.316646 28276 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:39.349522 28527 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 12:24:39.386929 28523 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.245s	user 0.001s	sys 0.001s
I20260709 12:24:39.323518 20727 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:39.371418 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:39.371644 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.335233 28457 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.356850 28537 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.342888 28417 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.384725 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.341897 28543 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:39.391745 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1419) took 2114 ms. Trace:
I20260709 12:24:39.413622 20802 rpcz_store.cc:276] 0709 12:24:37.272875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.273317 (+   442us) service_pool.cc:224] Handling call
0709 12:24:37.273600 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.151774 (+878174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.378028 (+1226254us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54003a0554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:39.387215 (+  9187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016993664}
I20260709 12:24:39.361763 28428 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:39.335177 28531 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.407423 28292 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.407310 28131 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:39.409564 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.410396 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:39.408115 28536 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:24:39.418220 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.393337 28491 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.426201 20876 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:39.426453 28491 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:39.427306 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.409281 28241 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.427610 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:39.428269 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:39.423395 20729 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
W20260709 12:24:39.428969 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55521.2us
I20260709 12:24:39.382470 28430 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:24:39.438136 28355 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:39.438730 28355 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:39.441102 28531 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.430667 20875 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:39.433151 27966 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:39.435763 21430 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:39.443885 20735 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 17: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:39.445900 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.420013 28368 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.447700 28368 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.448622 28431 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:39.449328 28544 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:39.449227 28428 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.449843 28507 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:39.450346 28544 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:39.450385 28368 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.450886 28544 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:39.451778 28544 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.454762 28430 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:39.456559 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3093) took 1912 ms. Trace:
I20260709 12:24:39.445991 28417 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.458283 28431 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:39.458720 28431 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:39.458989 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.459389 28546 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:24:39.459614 28431 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.460750 28431 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.462627 28500 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:39.463063 28500 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:39.463236 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:39.464031 28500 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.465006 28448 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:39.465592 28448 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.457605 28417 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.466586 28430 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:39.469957 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:39.421248 28313 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:39.414966 28537 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.470396 28240 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:39.457000 20879 rpcz_store.cc:276] 0709 12:24:37.544294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.185465 (+641171us) service_pool.cc:224] Handling call
0709 12:24:38.186285 (+   820us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:38.190180 (+  3895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.190230 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:38.321249 (+131019us) log.cc:844] Serialized 532 byte log entry
0709 12:24:38.591416 (+270167us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:38.909361 (+317945us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.934359 (+ 24998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.934469 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.326968 (+392499us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:39.456243 (+129275us) raft_consensus.cc:1685] finished
0709 12:24:39.456348 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.456497 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47796,"raft.queue_time_us":25541,"raft.run_cpu_time_us":9140,"raft.run_wall_time_us":146559,"spinlock_wait_cycles":8749312,"thread_start_us":187589,"threads_started":1,"wal-append.queue_time_us":276676}
I20260709 12:24:39.465418 28500 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.424656 28519 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:24:39.446084 28530 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:24:39.482009 28279 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.481793 28541 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:39.456749 28543 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:24:39.546954 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:39.547835 20802 raft_consensus.cc:606] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:39.548290 20802 raft_consensus.cc:614] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:39.489168 28448 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.445878 28398 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.411809 28276 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.444010 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:39.495427 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:39.558120 20728 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.511468 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.559219 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:39.560258 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.467870 28389 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.465356 28492 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.520736 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:39.525846 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 121207us
I20260709 12:24:39.562377 20879 raft_consensus.cc:2376] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25. Re-sending same reply.
I20260709 12:24:39.563257 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.563896 20879 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:39.450497 28428 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:39.452896 28544 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.492919 28508 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.496377 28313 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:39.516970 28256 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:39.516191 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:39.558145 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 114211us
I20260709 12:24:39.567539 21432 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:39.572625 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.575699 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:39.576122 20876 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:39.576932 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.531859 28355 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.579453 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:39.581738 20728 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:39.582717 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.584120 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.587304 20879 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:39.566679 28520 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:39.590224 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:39.573596 28538 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
W20260709 12:24:39.579030 28510 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.787s	user 0.010s	sys 0.002s
I20260709 12:24:39.574924 28313 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:39.588929 28288 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:39.574501 28286 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.560163 28460 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.595752 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.615850 21432 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:39.616812 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.617564 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:39.583352 28256 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.555375 28541 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.583715 28509 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:39.550477 28279 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.609982 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.560433 28398 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.597129 28510 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:39.592729 28520 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:39.600620 28212 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:39.599746 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:24:39.612746 28416 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.629464 20875 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:39.635730 20733 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:39.633030 28520 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.635116 28519 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:39.639256 28519 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.640350 28389 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.641387 28519 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:39.643301 28398 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:39.645525 28541 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.641580 28120 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:39.648674 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.620118 28553 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:39.656252 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.657256 20879 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16.
I20260709 12:24:39.649101 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.650874 20876 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:39.658859 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.659649 20876 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:39.660910 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.662026 20876 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:24:39.658417 20881 raft_consensus.cc:2376] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20. Re-sending same reply.
I20260709 12:24:39.677641 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:39.679848 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.680416 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.680785 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.678500 20881 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:39.685982 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.686225 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.619344 28286 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:24:39.687808 20879 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:24:39.697184 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.697857 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:39.655856 28553 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.647132 28241 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:39.684548 28212 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.702245 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.684022 28131 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.699770 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.647109 28491 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.691511 28398 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:39.706790 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.708798 20881 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.648185 28416 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.695896 28508 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.709507 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.711390 20879 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:24:39.696542 28417 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:39.688057 28415 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.711602 28398 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.715095 28415 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:39.715391 28557 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:24:39.715698 28355 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.716845 28256 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.717761 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1436) took 2077 ms. Trace:
I20260709 12:24:39.701716 28553 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.718115 21426 rpcz_store.cc:276] 0709 12:24:37.640261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.641609 (+  1348us) service_pool.cc:224] Handling call
0709 12:24:37.641942 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.648827 (+1006885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.717527 (+1068700us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54005f4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:39.717701 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270160640}
I20260709 12:24:39.720304 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:39.717537 28550 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:24:39.721709 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.725270 28533 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:39.725677 28533 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:39.729382 28525 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:39.724113 21433 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:24:39.721211 28120 tablet_bootstrap.cc:492] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:24:39.720172 28276 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:39.707466 28212 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.735628 28433 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.726475 28533 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.738880 28533 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.740620 28520 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.729825 28525 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:39.742337 28518 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.000s
I20260709 12:24:39.744486 28518 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.745204 28520 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.745993 28525 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.747246 28525 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.750315 28318 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:39.750746 28318 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:39.751593 28318 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.752496 28416 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.753978 28430 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.756387 28430 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:39.758831 28543 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:39.759296 28543 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:39.760126 28543 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.761512 28543 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.762889 28553 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.765383 28553 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.766912 28537 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:39.795864 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:24:39.798722 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.800908 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:39.804235 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:39.806242 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:39.813321 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.814862 20879 raft_consensus.cc:2410] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 22 because replica has last-logged OpId of term: 20 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 12:24:39.816921 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:39.817888 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.818632 20879 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.834684 20727 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:39.838689 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.840978 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:39.841894 20882 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:39.842568 20735 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:39.843014 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.843950 20882 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:39.846575 20730 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:39.847738 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1257) took 1013 ms. Trace:
I20260709 12:24:39.848268 20730 rpcz_store.cc:276] 0709 12:24:38.834023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.845404 (+1011381us) service_pool.cc:224] Handling call
0709 12:24:39.845993 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.847234 (+  1241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.847675 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:39.852651 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1418) took 2580 ms. Trace:
I20260709 12:24:39.853235 20806 rpcz_store.cc:276] 0709 12:24:37.272549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.272978 (+   429us) service_pool.cc:224] Handling call
0709 12:24:37.273298 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.824302 (+1551004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.851918 (+1027616us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54003ca854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:39.852377 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3562810624}
I20260709 12:24:39.852831 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1340) took 2216 ms. Trace:
I20260709 12:24:39.854979 20883 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:24:39.854902 20878 rpcz_store.cc:276] 0709 12:24:37.636630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.424344 (+787714us) service_pool.cc:224] Handling call
0709 12:24:38.424745 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.463440 (+ 38695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.852213 (+1388773us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:39.852611 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86417152}
I20260709 12:24:39.856956 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.857751 20878 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:39.860028 20884 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:39.861440 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.861789 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3014) took 1945 ms. Trace:
W20260709 12:24:39.862421 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.862201 20884 rpcz_store.cc:276] 0709 12:24:37.916698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.030307 (+1113609us) service_pool.cc:224] Handling call
0709 12:24:39.859400 (+829093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.860555 (+  1155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.861562 (+  1007us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b540021cad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:39.861696 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905760512}
W20260709 12:24:39.865784 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.866953 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:39.867873 20884 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.872792 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:39.873855 21428 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:39.874109 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:39.876021 21434 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:39.877369 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:39.877867 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:39.875790 21432 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:24:39.878825 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.885673 28287 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:39.884696 28120 ts_tablet_manager.cc:1403] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 5.725s	user 0.090s	sys 0.069s
I20260709 12:24:39.870817 20875 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:39.880342 28279 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:39.879946 28241 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:39.884549 28410 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:39.894104 28367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047135f71834dbab6b869eb3c081b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:39.895062 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1258) took 1060 ms. Trace:
I20260709 12:24:39.887863 21426 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:24:39.859032 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1343) took 2221 ms (client timeout 2365 ms). Trace:
W20260709 12:24:39.900187 20877 rpcz_store.cc:269] 0709 12:24:37.638571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.148704 (+510133us) service_pool.cc:224] Handling call
0709 12:24:39.858601 (+1709897us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:39.858842 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528317312}
I20260709 12:24:39.902163 20877 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:39.906294 20876 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
I20260709 12:24:39.909819 28496 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:39.899317 28410 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:39.931907 28410 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:39.899951 20734 rpcz_store.cc:276] 0709 12:24:38.837984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.893357 (+1055373us) service_pool.cc:224] Handling call
0709 12:24:39.893947 (+   590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.894314 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.894345 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:39.894405 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:39.894501 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.894571 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.894737 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:39.935659 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.940653 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3096) took 2395 ms. Trace:
I20260709 12:24:39.941169 20880 rpcz_store.cc:276] 0709 12:24:37.545002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.191068 (+646066us) service_pool.cc:224] Handling call
0709 12:24:38.191808 (+   740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.726152 (+534344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.940098 (+1213946us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54001fed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:39.940475 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050038016}
I20260709 12:24:39.942672 28496 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:39.944715 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:39.944221 28496 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:39.946201 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:39.924180 20774 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16.
I20260709 12:24:39.752523 28318 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.927513 28251 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:39.947377 20774 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:39.960457 20884 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:39.969144 20882 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:24:39.960495 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:39.966223 21433 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:39.984656 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:39.987423 20881 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:39.987056 21430 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:24:39.991098 20881 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [18.12-20.13]
I20260709 12:24:39.998407 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:40.000855 28279 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:40.003122 20882 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:40.004753 20882 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:40.005108 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3005) took 2303 ms. Trace:
I20260709 12:24:40.005501 20883 rpcz_store.cc:276] 0709 12:24:37.701307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.788059 (+1086752us) service_pool.cc:224] Handling call
0709 12:24:39.854323 (+1066264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.855577 (+  1254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.004920 (+149343us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400367250. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:40.005048 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450959616}
I20260709 12:24:40.008714 28457 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.009506 28457 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:40.011907 20881 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:40.014621 21426 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:24:40.015312 28335 raft_consensus.cc:2764] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election decision vote started in defunct term 25: won
I20260709 12:24:39.983139 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:39.967021 28241 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:40.037647 20878 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:24:40.037971 28559 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.038251 28212 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:40.038726 28559 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:40.001520 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 123861us
I20260709 12:24:39.994760 28120 raft_consensus.cc:359] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:40.051709 20805 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:40.052335 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2917) took 1035 ms. Trace:
I20260709 12:24:40.052851 20733 rpcz_store.cc:276] 0709 12:24:39.017029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.017284 (+   255us) service_pool.cc:224] Handling call
0709 12:24:39.017668 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.636430 (+618762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.051755 (+415325us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:40.052137 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420909824}
I20260709 12:24:40.059813 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:40.059864 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64158.1us
I20260709 12:24:40.061892 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:40.062145 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:40.060446 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 136570us
I20260709 12:24:40.060623 21429 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:40.078963 20880 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:40.050699 28120 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
W20260709 12:24:40.086860 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.089516 28212 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:40.090106 28212 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.090519 28212 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:40.094084 28537 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:40.100308 21433 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:40.075575 28559 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:40.024533 28287 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:40.121279 21426 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:40.124598 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:40.136207 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:40.139948 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:40.139083 28398 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:40.152693 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:40.159286 28271 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.806s	user 0.010s	sys 0.001s
I20260709 12:24:40.160828 21427 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:40.170233 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83078.4us
I20260709 12:24:40.166662 28212 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:40.189849 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 92815.5us
I20260709 12:24:40.373781 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1328) took 1334 ms. Trace:
W20260709 12:24:40.380014 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.365522 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.526855 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.570493 20807 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:40.472608 20876 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
W20260709 12:24:40.374426 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.597174 20879 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.778s	user 0.019s	sys 0.002s
I20260709 12:24:40.598517 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:40.353768 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.588379 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:40.375030 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:40.552033 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:40.623476 20879 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:41.105458 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3118) took 1484 ms. Trace:
W20260709 12:24:41.005806 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.746372 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.112911 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3038) took 1485 ms. Trace:
I20260709 12:24:40.711668 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:40.612960 21427 rpcz_store.cc:276] 0709 12:24:39.038957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.039111 (+   154us) service_pool.cc:224] Handling call
0709 12:24:40.373520 (+1334409us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:40.373680 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548366080}
I20260709 12:24:40.622054 20876 pending_rounds.cc:85] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 14
W20260709 12:24:41.112991 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3042) took 1296 ms (client timeout 1696 ms). Trace:
W20260709 12:24:40.370922 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 242099us
W20260709 12:24:40.947631 20884 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.011s	sys 0.007s
W20260709 12:24:41.117512 20884 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.014s	sys 0.010s
I20260709 12:24:40.719059 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:40.907080 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3032) took 1435 ms. Trace:
I20260709 12:24:40.874594 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1269) took 1260 ms. Trace:
I20260709 12:24:41.120918 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:41.116606 20879 rpcz_store.cc:269] 0709 12:24:39.816864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.817622 (+   758us) service_pool.cc:224] Handling call
0709 12:24:41.112933 (+1295311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.705399 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:40.458328 28417 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.186807 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:41.128475 20733 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:41.129063 20734 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.170745 28457 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:40.587405 28559 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.628609 28410 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.696s	user 0.013s	sys 0.002s
W20260709 12:24:40.740185 28318 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.002s
I20260709 12:24:40.536827 28287 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:40.712332 28518 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:40.589366 28313 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.979s	user 0.018s	sys 0.000s
I20260709 12:24:40.282853 28433 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:40.291435 28271 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:24:40.714815 28430 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.958s	user 0.009s	sys 0.006s
W20260709 12:24:41.136461 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.611855 28241 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:40.720304 28212 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:40.371392 28520 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:41.140709 20731 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:41.115463 20876 pending_rounds.cc:107] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:24:41.157505 20876 raft_consensus.cc:2889] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:41.158490 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3114) took 1684 ms. Trace:
I20260709 12:24:41.158862 20876 rpcz_store.cc:276] 0709 12:24:39.474394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.442384 (+967990us) service_pool.cc:224] Handling call
0709 12:24:40.471870 (+ 29486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.158214 (+686344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.158427 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:41.163705 28417 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.166651 28417 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:41.167598 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:40.186266 28491 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.143381 28541 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:41.143929 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:41.145292 28583 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:40.717095 28431 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:41.137421 28279 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.134s	user 0.014s	sys 0.001s
I20260709 12:24:41.113389 20882 rpcz_store.cc:276] 0709 12:24:39.621090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.005474 (+384384us) service_pool.cc:224] Handling call
0709 12:24:40.005950 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.063383 (+ 57433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.063414 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.064061 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.300550 (+236489us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:40.666171 (+365621us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.666411 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.666513 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.105169 (+438656us) raft_consensus.cc:1685] finished
0709 12:24:41.105193 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.105374 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":298647,"prepare.queue_time_us":380403,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":10311,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":20595,"thread_start_us":235959,"threads_started":1,"wal-append.queue_time_us":843634}
I20260709 12:24:41.114488 20881 rpcz_store.cc:276] 0709 12:24:39.628293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.014342 (+386049us) service_pool.cc:224] Handling call
0709 12:24:40.016491 (+  2149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.017659 (+  1168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.017731 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.076138 (+ 58407us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.322461 (+246323us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:40.322519 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.322631 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.905840 (+583209us) raft_consensus.cc:1685] finished
0709 12:24:40.905870 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.112848 (+206978us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":148221,"threads_started":1,"wal-append.queue_time_us":784450}
I20260709 12:24:41.119290 20877 rpcz_store.cc:276] 0709 12:24:39.471873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.903161 (+431288us) service_pool.cc:224] Handling call
0709 12:24:39.903918 (+   757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.905020 (+  1102us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:39.981210 (+ 76190us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.108783 (+127573us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.148632 (+ 39849us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.148782 (+   150us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:40.179976 (+ 31194us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.339622 (+159646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.339713 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.906790 (+567077us) raft_consensus.cc:1685] finished
0709 12:24:40.906819 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.906998 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40284,"prepare.queue_time_us":724980,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":8241,"raft.queue_time_us":122503,"raft.run_cpu_time_us":542,"raft.run_wall_time_us":47279,"spinlock_wait_cycles":36736,"thread_start_us":324468,"threads_started":3,"wal-append.queue_time_us":891336}
I20260709 12:24:41.153784 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:41.122218 20729 rpcz_store.cc:276] 0709 12:24:39.614056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.844720 (+230664us) service_pool.cc:224] Handling call
0709 12:24:39.845368 (+   648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.846156 (+   788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.846205 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.847045 (+   840us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.959416 (+112371us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:39.959456 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.959547 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.034461 (+ 74914us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:24:40.874033 (+839572us) raft_consensus.cc:1685] finished
0709 12:24:40.874070 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.874383 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66137,"raft.run_cpu_time_us":1738,"raft.run_wall_time_us":107322,"spinlock_wait_cycles":106189056,"thread_start_us":94907,"threads_started":1,"wal-append.queue_time_us":1012392}
W20260709 12:24:41.201649 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 826673us
I20260709 12:24:40.175915 28120 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:41.188252 20805 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:41.205475 28398 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:24:41.141971 28410 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.210708 28410 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.211714 28431 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.153749 28559 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.219378 28279 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16.
I20260709 12:24:41.220520 28541 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.148756 28433 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:41.237995 20734 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:41.154266 28271 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:41.184976 28457 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:41.246018 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3225) took 1056 ms. Trace:
I20260709 12:24:41.246657 20734 rpcz_store.cc:276] 0709 12:24:40.184189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.186500 (+  2311us) service_pool.cc:224] Handling call
0709 12:24:41.240589 (+1054089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:41.236536 28496 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:41.236948 28559 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.241524 28433 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:41.156145 28313 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:41.289551 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 115230us
W20260709 12:24:41.289841 28367 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.581s	user 0.013s	sys 0.000s
I20260709 12:24:41.251750 28457 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:41.303691 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.305527 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:41.296087 28520 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:41.321362 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.295136 28279 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.301185 28457 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:41.294732 28541 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.308197 28494 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.301254 28496 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:41.332129 20733 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:41.243140 28583 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.337025 20807 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:41.317762 28520 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:41.342839 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 623925us
I20260709 12:24:41.339408 20730 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:41.325771 28417 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:41.346319 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:41.344317 28584 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:41.340112 28559 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:41.333668 28494 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.336710 28518 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:24:41.383469 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:41.382478 20700 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.013s	sys 0.056s
W20260709 12:24:41.389209 20700 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.013s	sys 0.056s
W20260709 12:24:41.389631 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 652487us
W20260709 12:24:41.392388 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.386397 21430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.024s	sys 0.055s
W20260709 12:24:41.393231 21430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.024s	sys 0.055s
I20260709 12:24:41.395985 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.346858 28279 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.402833 20805 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:24:41.406390 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.377765 28584 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:41.407543 21427 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:41.407990 21432 raft_consensus.cc:2368] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17.
I20260709 12:24:41.404724 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1345) took 1791 ms. Trace:
W20260709 12:24:41.413131 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.401937 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:41.345525 28520 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:41.413328 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:41.420677 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.421720 20802 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:24:41.424661 28494 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.425920 28584 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.400884 28313 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.427559 28120 ts_tablet_manager.cc:1434] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 1.517s	user 0.020s	sys 0.020s
W20260709 12:24:41.427937 28120 ts_tablet_manager.cc:1467] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Tablet startup took 1531ms
I20260709 12:24:41.412288 21430 rpcz_store.cc:276] 0709 12:24:39.610347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.046863 (+436516us) service_pool.cc:224] Handling call
0709 12:24:40.115145 (+ 68282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.454798 (+339653us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:40.548836 (+ 94038us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.401024 (+852188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:41.401099 (+    75us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:41.401185 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.401276 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.401453 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":224,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":16004,"thread_start_us":844298,"threads_started":1}
I20260709 12:24:41.429464 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:41.442369 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:41.413625 28491 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.402319 28431 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.445679 21431 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:41.380106 28417 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.447876 21428 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:41.393580 28559 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.458163 21427 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEARNER]: Advancing to term 22
W20260709 12:24:41.459041 28576 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.318s	user 0.005s	sys 0.000s
I20260709 12:24:41.460712 28559 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.454620 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:41.465350 28541 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.466094 21435 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:41.471879 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.472993 20808 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:24:41.474068 28431 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.474344 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.448400 21433 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.369081 28271 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.447438 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:41.475692 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:41.479446 21426 raft_consensus.cc:2410] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:41.466526 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.480255 28494 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:41.455662 28410 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:41.492035 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.492880 20808 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
W20260709 12:24:41.492468 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63129us
I20260709 12:24:41.496964 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.497583 21434 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:41.480494 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44462 (request call id 1421) took 7174 ms. Trace:
I20260709 12:24:41.474728 28431 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.491751 28494 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.508181 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.509337 20803 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 27 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 12.
I20260709 12:24:41.518538 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.518915 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:41.519549 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:41.520388 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.520571 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.521126 20804 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:41.491804 28318 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
W20260709 12:24:41.522166 28503 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.030s	user 0.053s	sys 0.091s
I20260709 12:24:41.522257 28318 consensus_queue.cc:1243] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:24:41.522763 28503 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.054s	sys 0.092s
I20260709 12:24:41.523046 28318 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:41.523520 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:41.524290 20734 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:41.518693 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:41.525749 20733 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:41.466110 28541 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.511550 28212 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.523454 28318 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:41.512506 28559 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:41.485764 28491 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.476999 28496 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.546350 20807 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:41.552035 20805 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:41.549741 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.539940 28518 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:41.557201 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.559716 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:41.560549 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:41.571300 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.546454 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.573956 20808 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:41.558851 28241 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:41.574262 20809 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:41.541469 28313 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:24:41.558511 28433 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:41.571321 28579 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.364s	user 0.002s	sys 0.004s
W20260709 12:24:41.580436 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.580886 21431 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:41.553987 28559 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:41.590651 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.594165 28318 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.595827 28318 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.591483 28583 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.599002 20628 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 13 to 14, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 14 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.600917 28431 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.601663 28431 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:41.594899 28491 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.547822 28212 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.613307 20802 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:41.616350 20804 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:24:41.601028 20628 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 21, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 21 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:41.620522 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.625787 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.626776 20806 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:41.588369 28279 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.627570 28417 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.629408 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:41.587415 28241 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.630414 20809 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:41.631939 28241 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:41.632625 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:41.634689 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:41.636849 28518 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:41.637301 28518 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:41.638237 28518 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.504002 28367 rpcz_store.cc:276] 0709 12:24:34.262949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.425047 (+162098us) service_pool.cc:224] Handling call
0709 12:24:34.840908 (+415861us) ts_tablet_manager.cc:934] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
0709 12:24:39.886356 (+5045448us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:40.708318 (+821962us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:40.708461 (+   143us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:41.306686 (+598225us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:41.362461 (+ 55775us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:41.397408 (+ 34947us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:41.431118 (+ 33710us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":50326,"tablet-copy.queue_time_us":118325,"thread_start_us":253123,"threads_started":2}
I20260709 12:24:41.639927 28367 tablet_copy_client.cc:806] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:41.640712 28367 tablet_copy_client.cc:670] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:41.639712 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:41.428290 28120 ts_tablet_manager.cc:1469] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Trace:
0709 12:24:27.350990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.372706 (+ 21716us) service_pool.cc:224] Handling call
0709 12:24:28.310147 (+937441us) ts_tablet_manager.cc:934] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333)
0709 12:24:28.924482 (+614335us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b4c001bd980. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1cf6ba20 00007f9e1cf7ca29 00007f9e1cf7d13f 00007f9e1cf412ab 00007f9e1cf6ef3c 00007f9e1cf6f461 00007f9e1cf6f768 00007f9e22cf52eb 00007f9e23c1ad3b 00007f9e23c15008 00007f9e23c14405
0709 12:24:31.000249 (+2075767us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:31.095992 (+ 95743us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:31.096162 (+   170us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:31.239297 (+143135us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:31.239675 (+   378us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:31.240011 (+   336us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:31.240203 (+   192us) inbound_call.cc:177] Queueing success response
0709 12:24:34.159923 (+2919720us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:39.919336 (+5759413us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:39.991742 (+ 72406us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:39.992518 (+   776us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":49118,"mutex_wait_us":26296,"raft.queue_time_us":46083,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":23218,"spinlock_wait_cycles":21532544,"tablet-copy.queue_time_us":726350,"tablet-download-pool.queue_time_us":207,"tablet-download-pool.run_cpu_time_us":1541,"tablet-download-pool.run_wall_time_us":2587156,"thread_start_us":903460,"threads_started":2}
I20260709 12:24:41.644033 28433 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.651664 28494 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.653249 28541 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.657670 28251 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:41.658175 28251 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:41.626946 28212 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.645346 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:41.664256 20808 raft_consensus.cc:2368] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:41.574150 28287 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 12:24:41.667801 20803 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16.
I20260709 12:24:41.669567 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:41.680501 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.684504 20628 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.668030 28417 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:41.678076 28241 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:41.696826 28541 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:41.701068 28559 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.659053 28251 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.701727 28518 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:41.674788 28212 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:41.706144 28541 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:41.720760 20628 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 21, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 21 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.725234 28541 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.726740 28541 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:41.726168 20628 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:41.710923 21433 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:24:41.653276 28457 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.711333 28583 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:41.710335 28287 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:41.750999 20775 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.026s	sys 0.038s
W20260709 12:24:41.752658 20775 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.027s	sys 0.038s
W20260709 12:24:41.754108 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 612782us
W20260709 12:24:41.756661 20775 connection.cc:256] RPC call timeout handler was delayed by -1.0984536979999575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
W20260709 12:24:41.767829 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.768334 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.768657 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.771126 20805 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:24:41.771239 20628 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 24, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:41.731122 28583 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:41.731050 28287 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.792213 20628 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.731842 28212 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.795305 20628 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 17, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 17 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.796042 28583 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.798345 20628 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 20, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 20 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:24:41.801623 20875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.020s	sys 0.016s
I20260709 12:24:41.777616 28318 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:41.804572 20884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.014s	sys 0.020s
W20260709 12:24:41.807732 20884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.014s	sys 0.020s
I20260709 12:24:41.793654 28433 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.805157 20875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.021s	sys 0.016s
I20260709 12:24:41.818635 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3111) took 2345 ms. Trace:
I20260709 12:24:41.819095 20884 rpcz_store.cc:276] 0709 12:24:39.473465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.970271 (+496806us) service_pool.cc:224] Handling call
0709 12:24:39.970752 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.124551 (+153799us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:24:41.121544 (+996993us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.809474 (+687930us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:41.809551 (+    77us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:41.809625 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.818382 (+  8757us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:41.818428 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.818573 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56105,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"spinlock_wait_cycles":247401216,"thread_start_us":1633444,"threads_started":2}
I20260709 12:24:41.804800 20628 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 13 to 16, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 16 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.828114 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3110) took 2354 ms. Trace:
I20260709 12:24:41.825659 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:41.867908 20628 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 19, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 19 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:24:41.850421 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.007s	sys 0.015s
W20260709 12:24:41.870458 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.007s	sys 0.015s
W20260709 12:24:41.870810 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 728531us
W20260709 12:24:41.851298 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.009s	sys 0.013s
W20260709 12:24:41.871902 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.009s	sys 0.013s
I20260709 12:24:41.872896 20628 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:41.873179 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:41.872460 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 667422us
I20260709 12:24:41.813992 28520 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.852523 28457 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.815495 28318 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.828715 20875 rpcz_store.cc:276] 0709 12:24:39.473209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.943084 (+469875us) service_pool.cc:224] Handling call
0709 12:24:39.943675 (+   591us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:39.944364 (+   689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.944395 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:39.945124 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 12:24:40.019713 (+ 74589us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:40.148796 (+129083us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.691648 (+542852us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b18002ed5c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:24:40.692017 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.814175 (+1122158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.814248 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.827791 (+ 13543us) raft_consensus.cc:1685] finished
0709 12:24:41.827822 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.828042 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80773,"prepare.queue_time_us":220126,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":650766,"raft.run_cpu_time_us":576,"raft.run_wall_time_us":51261,"spinlock_wait_cycles":41538432,"thread_start_us":1010504,"threads_started":4,"wal-append.queue_time_us":187021,"wal-append.run_cpu_time_us":4004,"wal-append.run_wall_time_us":1556521}
I20260709 12:24:41.877182 28541 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:41.916440 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 316729us
I20260709 12:24:41.918992 28559 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:41.919605 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:41.920347 28559 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:41.920882 28559 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:41.922170 28559 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:41.885653 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.010s	sys 0.012s
W20260709 12:24:41.925426 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.010s	sys 0.012s
W20260709 12:24:41.886109 20850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.002s	sys 0.018s
W20260709 12:24:41.926139 20850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.002s	sys 0.019s
W20260709 12:24:41.926537 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.19419e+06us
W20260709 12:24:41.887055 28588 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:24:41.888031 28212 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:41.905732 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 102171us
W20260709 12:24:41.905977 20631 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 295) took 30293 ms (client timeout 15000 ms). Trace:
I20260709 12:24:41.889174 28287 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.938166 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:41.882069 20628 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 13 to 18, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 18 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.867662 28433 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:41.947412 28598 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:41.936998 28241 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:41.948901 28241 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.948962 28433 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.949339 28241 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:41.951270 28212 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:41.951669 28593 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:41.952291 28593 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.953143 28318 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:41.954471 28593 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.954975 28593 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.955724 28593 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:41.946005 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 23, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 23 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.961576 28593 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:41.931691 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47388.7us
W20260709 12:24:41.933514 20631 rpcz_store.cc:269] 0709 12:24:11.612198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.613959 (+  1761us) service_pool.cc:224] Handling call
0709 12:24:41.905727 (+30291768us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:34.850472 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:34.918122 (+ 67650us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:34.987518 (+ 69396us) write_op.cc:435] Acquired schema lock
0709 12:24:35.030896 (+ 43378us) tablet.cc:662] Decoding operations
0709 12:24:35.213569 (+182673us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:35.213680 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:35.213722 (+    42us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:24:35.782282 (+568560us) tablet.cc:701] Row locks acquired
0709 12:24:35.782321 (+    39us) write_op.cc:260] PREPARE: finished
0709 12:24:35.782493 (+   172us) write_op.cc:270] Start()
0709 12:24:35.782657 (+   164us) write_op.cc:276] Timestamp: P: 1783599875782427 usec, L: 0
0709 12:24:35.782679 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:24:35.783505 (+   826us) log.cc:844] Serialized 9847 byte log entry
0709 12:24:37.146547 (+1363042us) op_driver.cc:464] REPLICATION: finished
0709 12:24:37.351530 (+204983us) write_op.cc:301] APPLY: starting
0709 12:24:37.351618 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:37.573987 (+222369us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:37.574113 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:38.802889 (+1228776us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b1800175d80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e9de0ec 00007f9e1e9e0af7 00007f9e1e9e1fb0 00007f9e226f61bb 00007f9e2273d5d4 00007f9e225e98da 00007f9e225ec3cb 00007f9e225ebfd3 00007f9e226eae34
0709 12:24:39.485621 (+682732us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:39.486079 (+   458us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 12:24:39.486133 (+    54us) write_op.cc:312] APPLY: finished
0709 12:24:39.493536 (+  7403us) log.cc:844] Serialized 220 byte log entry
0709 12:24:39.554096 (+ 60560us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:39.573410 (+ 19314us) write_op.cc:454] Released schema lock
0709 12:24:39.592175 (+ 18765us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:39.630183 (+ 38008us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:39.632787 (+  2604us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2301077,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":204638,"apply.run_cpu_time_us":58683,"apply.run_wall_time_us":2263550,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108627,"num_ops":25,"prepare.queue_time_us":202948,"prepare.run_cpu_time_us":66475,"prepare.run_wall_time_us":1234418,"raft.queue_time_us":292559,"raft.run_cpu_time_us":28642,"raft.run_wall_time_us":131165,"replication_time_us":1363623,"spinlock_wait_cycles":61974528,"thread_start_us":690529,"threads_started":5,"wal-append.queue_time_us":497204,"wal-append.run_cpu_time_us":30506,"wal-append.run_wall_time_us":2981978}]]}
I20260709 12:24:41.963029 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:41.928262 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 357717us
I20260709 12:24:41.964553 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:41.964988 28520 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:41.966212 28603 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:41.971442 28594 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:41.960958 20628 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.972373 28594 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:41.974112 28251 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:41.973628 20628 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 16 to 20, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 20 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:41.977361 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.929585 20730 raft_consensus.cc:2368] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:24:41.975447 28212 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:41.985244 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:41.988667 20628 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:42.000677 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.002596 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.004695 28212 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.009366 21433 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:41.975168 28594 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.010767 20803 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:42.042555 28592 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:42.043324 28592 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.045449 28592 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:42.010964 28601 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.002s	sys 0.004s
I20260709 12:24:42.011514 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.011821 20628 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.012626 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.048379 21433 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:42.049517 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.048882 20628 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 14, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 14 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:42.050282 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.050729 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.013372 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.050580 20628 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 13 to 17, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 17 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.051451 28496 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:42.052104 28496 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:42.014612 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.015247 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:42.017307 21426 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:42.053382 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:42.019021 28596 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:42.020004 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.020603 28417 raft_consensus.cc:993] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 49323f66286f4798b268f216bb62b3be to start an election
W20260709 12:24:42.030313 20701 connection.cc:256] RPC call timeout handler was delayed by -1.8851289590002125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 12:24:42.035401 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44788.4us
I20260709 12:24:42.035904 28318 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:42.035631 28597 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:42.014302 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.052112 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:42.052451 20628 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 19, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 19 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:42.017913 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:42.054735 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.054833 21433 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:42.058784 21430 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:42.053552 21431 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:42.055838 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 125120us
I20260709 12:24:42.053381 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:42.058918 21400 connection.cc:256] RPC call timeout handler was delayed by -1.0886487000002489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 12:24:42.061094 21432 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:24:42.068110 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:42.069073 21426 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:42.070106 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.078148 28584 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:42.078835 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.071399 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:42.053300 28541 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:42.082077 28417 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.082428 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.082852 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:42.082629 28417 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.085199 28417 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.060164 28318 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:42.088225 28318 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.091410 28398 raft_consensus.cc:1081] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Signalling peer 49323f66286f4798b268f216bb62b3be to start an election
W20260709 12:24:42.095536 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 110515us
I20260709 12:24:42.096680 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:42.099313 20701 connection.cc:256] RPC call timeout handler was delayed by -1.025818604000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1996s for a response
I20260709 12:24:42.099752 28559 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:42.063026 28583 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:42.102187 28583 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:42.103226 28583 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.104617 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.082494 21435 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 12:24:42.105962 21435 pending_rounds.cc:85] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:24:42.106175 21435 pending_rounds.cc:107] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:24:42.106412 21435 raft_consensus.cc:2889] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:42.113066 21426 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:42.105238 20731 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:42.118134 21431 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:42.121097 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.120891 20628 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 21, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 21 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:42.132190 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.051472 28433 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:24:42.121708 20882 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:42.140516 20730 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:42.141821 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:42.148558 20733 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:42.147079 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.052472 28287 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:24:42.158458 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:42.160254 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.114228 28598 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.156015 28607 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:42.164760 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.171257 28607 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:42.171518 20877 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:42.168819 20628 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:42.170408 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.173071 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:42.173995 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.114120 28520 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:42.175732 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.176589 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:42.176630 20876 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:42.178354 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.179180 20726 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:42.177996 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:42.175561 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.180379 20880 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:42.053097 28496 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.115434 28518 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:24:42.172257 28607 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.142694 28597 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:42.213778 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:42.216243 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.180794 20883 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:42.114672 28241 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:42.216136 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 119630us
W20260709 12:24:42.208334 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.187453 20628 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 12 to 16, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 16 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.179397 28541 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:42.223337 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.224184 20884 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:24:42.221289 20879 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:24:42.187546 28592 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:42.236105 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.239667 20878 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:42.181048 28520 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:42.242225 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.216739 28593 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:42.262698 28608 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.227380 28398 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.232852 28318 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:42.266615 20628 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 21, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 21 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:24:42.174688 28598 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.239909 28613 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:42.220950 28597 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.247555 28279 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:42.328665 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 268852us
W20260709 12:24:42.329660 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90001.9us
I20260709 12:24:42.326399 20628 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 23, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 23 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.338759 20745 tablet_copy_service.cc:342] P 49323f66286f4798b268f216bb62b3be: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-62396122903848d4a49e6d637c5c8b49 received from {username='slave'} at 127.0.0.1:33906
I20260709 12:24:42.339859 20745 tablet_copy_service.cc:434] P 49323f66286f4798b268f216bb62b3be: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-62396122903848d4a49e6d637c5c8b49 on tablet 62396122903848d4a49e6d637c5c8b49 with peer 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:42.343523 28611 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:24:42.325906 28318 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.311317 28608 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.347543 28608 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.348291 28608 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:42.348680 28608 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.349058 28608 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:42.350389 28541 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.351472 28433 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:42.351701 28279 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.352203 28433 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.354005 28616 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:42.354694 28616 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:42.356098 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.356945 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.357599 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 177537us
I20260709 12:24:42.358031 28457 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.358659 28457 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.310676 28594 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:42.368363 20733 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:42.310063 28430 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:42.346825 28318 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.394479 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:42.398028 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49542.1us
I20260709 12:24:42.399324 20734 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:42.375533 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 43813.2us
W20260709 12:24:42.401618 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 339229us
W20260709 12:24:42.402006 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52324.3us
I20260709 12:24:42.403743 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:42.412945 20731 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:42.260546 28592 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.414479 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:42.316121 28398 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.418006 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.393308 28518 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.374989 28594 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:42.356353 28431 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.311116 28593 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:42.397689 28212 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:24:42.419894 28241 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:42.423333 28625 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.413708 28607 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:24:42.412282 28613 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:24:42.463500 28520 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.425352 28518 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.421216 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:42.474444 28431 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.464008 20734 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:42.533922 28431 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.480121 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.535240 28520 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:42.489390 27785 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:24:42.495919 28615 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:42.537108 27785 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:42.500442 28593 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:42.538424 27785 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.500901 28594 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.539997 28594 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:42.540129 27785 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:42.536803 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 118204us
I20260709 12:24:42.505903 28318 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:24:42.494947 28592 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.544718 28318 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:42.474112 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.486115 20879 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:42.509099 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:42.555770 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.578176 20880 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:24:42.580541 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71500.5us
W20260709 12:24:42.602463 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47035.5us
W20260709 12:24:42.603474 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129541us
I20260709 12:24:42.604641 20803 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:42.488021 28602 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:42.504788 28620 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:42.608834 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 206353us
I20260709 12:24:42.526834 28398 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:42.520273 28597 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.609766 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.518601 28613 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.574542 28318 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:42.490458 28616 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.481374 28313 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:42.521049 28457 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:42.556507 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.531221 28518 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.493141 28279 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.609328 28618 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:42.577405 28241 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:42.504240 28598 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.614966 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:42.485556 28433 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:42.629881 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 213201us
I20260709 12:24:42.580412 28417 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.526100 28608 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.505965 28251 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.109s
I20260709 12:24:42.628947 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.509545 28607 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 12:24:42.506695 28496 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.109s
W20260709 12:24:42.630851 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.608507 28602 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:42.627022 28287 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:42.632232 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.624672 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.485813 28625 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.632328 28618 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:42.642202 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 238567us
I20260709 12:24:42.658636 28618 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.648206 28433 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:42.659308 28433 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.659770 28433 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:42.650197 28625 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:42.645722 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.661859 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:42.662541 20880 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:42.662838 28251 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:42.663408 28251 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.663775 28251 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:42.665836 28625 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:42.662596 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.668550 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.669374 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.669664 28279 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:42.670123 28279 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:42.670377 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.671309 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:42.671854 20801 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:42.672858 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.673178 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:42.674088 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.674639 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:42.675709 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.618767 28620 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.676739 28496 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:42.670483 28279 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:42.677850 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.678722 20805 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:24:42.675880 28608 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:42.677964 28620 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:42.658103 28584 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 12:24:42.634189 28613 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.668761 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.676896 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.647483 20803 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
W20260709 12:24:42.654636 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 230180us
I20260709 12:24:42.679617 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.682560 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:42.683099 20849 leader_election.cc:400] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:24:42.687867 20849 leader_election.cc:403] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:24:42.678824 28602 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.689284 20810 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:42.681072 28632 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:42.702543 20805 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:42.702685 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:42.703356 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:42.705005 20875 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:42.707072 20884 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:42.707866 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:42.685191 28212 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:42.710108 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.710999 20876 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:42.711994 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.712558 20876 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 16.
I20260709 12:24:42.713420 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.713946 20876 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:42.714718 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.643586 28287 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:42.715816 28287 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:42.716549 20810 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:42.717562 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.718436 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.719051 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.719349 28584 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.719620 28520 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:42.719915 28584 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.720419 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.721015 20809 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:42.721761 28630 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:42.721903 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.722097 28584 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:42.722425 28630 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:42.723846 20881 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:42.723441 28630 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.725426 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.725773 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:42.726354 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.710202 28212 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:42.726549 20731 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 15: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:42.729501 20806 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:42.726963 20733 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:42.731024 20726 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:42.689253 28619 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:42.723071 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.756338 20876 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:42.709836 20880 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:42.736709 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.727046 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.767416 20801 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:42.767701 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 121158us
I20260709 12:24:42.765334 28279 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:42.768030 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60912.9us
W20260709 12:24:42.762995 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 156605us
I20260709 12:24:42.769835 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.696676 28625 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:42.727473 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:42.777427 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.777762 20727 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:24:42.773209 20880 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:42.775759 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:42.641009 28417 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.778028 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:42.761348 28212 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:42.704273 28632 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.778611 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.778111 20882 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:24:42.793323 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:42.795895 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:42.797458 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:42.798952 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:42.760774 28619 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.799738 20803 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:42.729058 28613 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.771111 28620 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:42.804626 21429 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:42.805686 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.811305 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.812495 20809 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:42.813867 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.823648 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.816494 20801 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:42.825529 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.903990 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.904834 20881 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:42.905630 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:42.804961 28619 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:42.705258 28318 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:42.803551 28632 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:42.861316 28618 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.910600 28618 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.892331 28433 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.913369 28433 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.806568 28625 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:42.825430 28636 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:24:42.914592 28625 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.915561 20802 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:42.905445 28287 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.921510 20807 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
W20260709 12:24:42.851195 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55390.4us
I20260709 12:24:42.925565 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:42.813074 28613 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:42.852978 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.814368 28602 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:42.858074 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.926844 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:42.929155 28603 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:42.929759 28603 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:42.930305 28603 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:42.859236 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58989.7us
I20260709 12:24:42.860630 20880 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:42.931774 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:42.932230 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.932888 20880 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 16.
I20260709 12:24:42.933063 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:42.933337 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.933831 28608 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:42.934600 28608 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:42.930493 28620 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:24:42.937474 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:42.938388 28620 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:42.828429 28251 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:42.939759 20804 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:42.879640 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:42.940835 20875 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:42.880633 28241 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.942539 28241 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:42.882162 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:42.941170 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61621.4us
I20260709 12:24:42.884585 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.885780 28417 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:42.886277 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.960801 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:42.961855 20877 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:42.899987 28279 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.926671 28613 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:42.933756 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.850142 28598 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:42.929394 28287 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.880205 28614 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:42.941311 28251 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:42.941370 28620 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:42.948046 20884 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:42.984603 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.949923 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.827121 20805 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:24:42.961179 20882 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:42.962008 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:42.985921 20875 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
W20260709 12:24:42.908169 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 138104us
I20260709 12:24:42.963732 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.976922 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.979606 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:42.988073 20878 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 15.
I20260709 12:24:42.987497 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.989961 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:42.989161 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:42.995010 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.996562 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.998091 20808 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:43.001787 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.990679 28398 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:43.003914 28398 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.001859 28313 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:43.004498 28398 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:43.004878 28313 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.005395 28313 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:42.982436 28417 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:24:43.011050 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:42.976267 28613 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:43.001446 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:43.011804 20805 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:43.014587 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.014791 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:43.012768 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:43.016662 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.017802 20801 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:43.019811 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.022918 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.027521 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.022500 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:42.990094 28598 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:43.028048 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:43.032945 28457 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:43.032976 28518 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:43.035336 20809 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:24:43.036407 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.043870 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.049384 20881 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:43.049284 20875 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:24:43.050560 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89874.5us
I20260709 12:24:43.032366 28431 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:43.052366 28431 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.052733 28431 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:43.057885 20801 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:24:43.058331 20801 pending_rounds.cc:85] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 12
I20260709 12:24:43.058586 20801 pending_rounds.cc:107] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:24:43.058833 20801 raft_consensus.cc:2889] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.059325 28313 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.062230 28313 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:43.022189 28251 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:43.062536 28241 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.069386 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:43.079509 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64868.5us
I20260709 12:24:43.078804 28431 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.078809 28614 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:43.082100 28614 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:43.082499 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.083678 28367 tablet_copy_client.cc:538] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:43.085704 28628 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:43.086828 28520 raft_consensus.cc:1064] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:24:43.087761 28619 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.090073 28619 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:43.091666 28541 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:43.092106 28541 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:43.083624 28431 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:43.094300 20879 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:24:43.071979 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:43.100916 20881 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:43.073115 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:43.109318 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:43.111585 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:43.043907 28627 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:43.030009 28279 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:24:43.067263 28251 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.122319 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:43.123202 20876 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:43.068441 28457 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:43.126153 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45776.2us
I20260709 12:24:43.105180 28431 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:43.131716 21429 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:43.131860 28431 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:43.119536 28518 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:43.137157 28656 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:43.137537 28656 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.137943 28656 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:43.121970 28313 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:43.125027 28627 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.139631 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:43.131312 28251 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:43.152189 28241 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:43.143044 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.153680 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.154242 20875 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:43.138903 28431 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:43.133558 28632 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.155131 28367 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:24:43.160964 28632 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:43.142376 28598 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.162308 28614 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.128724 28287 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:43.163578 28287 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.140166 28518 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:43.164518 28518 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.165135 28597 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.165673 28597 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.135285 28457 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.142280 28627 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:43.167315 28212 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.168059 28597 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:43.168959 20810 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:43.169890 20734 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:43.162943 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:43.146688 28603 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.185760 28287 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:43.170851 28613 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:43.200165 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.222584 20875 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
W20260709 12:24:43.224095 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 112621us
I20260709 12:24:43.229369 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.230042 20875 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:43.230873 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.243502 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.228461 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:43.273744 28658 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:43.276690 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.277199 28664 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.277413 20802 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:43.277835 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.278620 20807 raft_consensus.cc:2368] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22.
I20260709 12:24:43.281527 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.282195 20730 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:43.283533 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:43.283763 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.283993 28663 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:43.284556 28663 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:43.284989 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56920.7us
I20260709 12:24:43.286712 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.287328 20733 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:43.288091 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.288880 20883 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:43.285080 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.285440 28417 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:43.285951 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.297686 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:43.219935 28654 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.296537 20727 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:43.302510 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.303303 20884 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:43.310966 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.311636 20875 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:43.311885 20882 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:43.312955 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.314148 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.314837 20875 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:43.313059 21431 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:43.319716 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.320531 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.321444 20729 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:43.317763 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:43.320581 20733 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:43.218364 28541 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.293476 28417 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:43.335640 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 266371us
I20260709 12:24:43.323971 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.343158 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:43.343860 20879 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:43.345155 20879 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:24:43.301349 28313 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:43.319463 28592 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.348634 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.352710 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.297742 28651 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
W20260709 12:24:43.349375 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62048.4us
I20260709 12:24:43.288774 28663 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.353027 20730 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:43.308100 28658 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:43.367589 20879 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:24:43.370059 20882 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:43.371510 28665 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:24:43.347311 28619 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.373584 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:43.379155 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.307960 28654 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.356647 28541 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:43.381619 27785 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:43.382325 20775 connection.cc:256] RPC call timeout handler was delayed by -1.8498381069998686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 12:24:43.413000 20775 connection.cc:256] RPC call timeout handler was delayed by -1.8417381069998555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 12:24:43.383987 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.360663 28592 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:43.413398 20775 connection.cc:256] RPC call timeout handler was delayed by -1.5483381069998359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
I20260709 12:24:43.387807 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:43.387733 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70057us
W20260709 12:24:43.420082 20775 connection.cc:256] RPC call timeout handler was delayed by -1.4961381069997515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
I20260709 12:24:43.390033 20732 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
W20260709 12:24:43.420492 20775 connection.cc:256] RPC call timeout handler was delayed by -1.4439381069996671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
I20260709 12:24:43.380645 28313 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.413411 20727 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:24:43.419916 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:43.378698 20875 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
W20260709 12:24:43.420789 20775 connection.cc:256] RPC call timeout handler was delayed by -1.2117381069997464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 12:24:43.424319 20735 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:24:43.423995 27785 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.420701 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.429441 20884 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:43.429783 20875 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
W20260709 12:24:43.432144 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58665.4us
I20260709 12:24:43.373816 28496 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:43.396016 28619 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.435847 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:43.440104 28655 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:24:43.440286 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:43.440937 20732 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:24:43.441016 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.442341 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:43.443533 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:43.444051 20728 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:43.445747 28646 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:24:43.446957 28417 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.452446 20876 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:43.436949 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:43.447947 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85207.2us
I20260709 12:24:43.449663 20732 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:43.475486 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:43.454072 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:43.470999 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 308083us
I20260709 12:24:43.426419 28620 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:43.467496 21435 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:43.479099 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:43.488193 20876 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:43.488229 20810 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:24:43.490792 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:43.490958 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:43.491489 21400 connection.cc:256] RPC call timeout handler was delayed by -1.3304407860000538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 12:24:43.491896 21400 connection.cc:256] RPC call timeout handler was delayed by -1.2809407859999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
I20260709 12:24:43.492195 20732 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:43.493718 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:43.494876 20879 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:43.495222 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.494854 20876 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:43.498118 20876 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:24:43.498827 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62012.2us
I20260709 12:24:43.495877 20726 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:43.504565 21433 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:24:43.507650 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:43.511840 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:43.512979 21426 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:43.505987 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:43.519135 20726 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:43.519361 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:43.503574 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:43.528714 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58981.4us
I20260709 12:24:43.529086 20805 pending_rounds.cc:85] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 14
I20260709 12:24:43.529383 20805 pending_rounds.cc:107] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:24:43.529623 20805 raft_consensus.cc:2889] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.532572 20804 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:43.533586 21435 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:43.536214 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:43.539028 28603 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:43.495520 28658 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:43.542317 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 105113us
I20260709 12:24:43.543143 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:43.514386 28663 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.547433 20728 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:43.516732 28614 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:43.527959 28632 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:43.476882 28496 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.496619 28398 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.579700 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.548071 28592 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:43.560230 28417 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:43.578132 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:43.560881 28584 raft_consensus.cc:2764] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:24:43.600183 20880 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:43.545764 28602 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:24:43.569789 28433 raft_consensus.cc:2764] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:24:43.573462 28559 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:43.547976 28251 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.617048 20849 leader_election.cc:400] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:43.619828 20849 leader_election.cc:403] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.620867 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.623436 28677 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:43.625660 28668 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:43.626199 28668 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.626560 28668 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:43.593972 28614 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:43.638550 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.607198 27785 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:43.618474 28559 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.566404 28663 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.668437 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:43.652103 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83785.9us
W20260709 12:24:43.683291 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63646.4us
I20260709 12:24:43.669916 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:43.693434 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 88748.3us
I20260709 12:24:43.670173 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:43.651702 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52822.1us
I20260709 12:24:43.618367 28287 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:24:43.653261 28584 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:43.616467 28672 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.099s
I20260709 12:24:43.718636 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:43.719614 28680 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:43.681396 28559 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:43.720675 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:43.667552 28650 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:43.717670 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.750813 20730 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:43.751015 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:43.746317 20726 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:24:43.739311 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54057us
I20260709 12:24:43.752246 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:43.755416 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:43.757390 28627 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
W20260709 12:24:43.760001 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74989us
I20260709 12:24:43.755789 20883 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:24:43.761013 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.762020 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:43.743501 28632 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:43.744108 28619 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:43.757560 28675 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:43.694087 28677 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:24:43.760555 28602 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:43.770236 28630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.000s	sys 0.012s
W20260709 12:24:43.820020 28630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.001s	sys 0.012s
I20260709 12:24:43.823689 28496 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:43.802341 28602 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:43.825353 28602 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:43.794637 28241 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:43.763640 28664 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:43.766368 28690 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 12:24:43.828953 28685 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:43.829326 28685 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:43.829587 28661 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:43.830266 28685 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.763931 28627 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.832720 28627 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:43.798164 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.834486 20726 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:43.835532 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.835916 28685 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:43.760303 28680 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:43.837054 28680 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.766582 28594 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:43.801146 28679 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:43.840909 28679 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:43.841399 28679 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:43.805037 28676 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:43.805761 28662 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
W20260709 12:24:43.807003 28518 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.000s
W20260709 12:24:43.844846 28656 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.000s
W20260709 12:24:43.807721 28659 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.294s	user 0.000s	sys 0.003s
I20260709 12:24:43.844619 28518 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.846029 21432 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:43.812170 28663 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:43.812518 28686 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:43.847363 28656 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.849764 28656 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:43.797335 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.802292 28632 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:43.856951 21429 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:43.857730 28632 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.859853 28676 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:43.860880 28676 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:43.861223 28676 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:43.813078 28677 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:43.794719 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:43.804692 28681 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:24:43.795672 28431 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.000s
W20260709 12:24:43.806734 28673 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:24:43.851518 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96377.9us
I20260709 12:24:43.869535 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.870906 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:43.853514 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.874068 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.856066 28318 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:43.814299 28398 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:43.830742 28661 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:43.862191 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.872637 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.862541 20876 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
W20260709 12:24:43.864573 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67350.1us
I20260709 12:24:43.796072 28614 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:43.876572 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.896796 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:43.875631 28287 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:43.898428 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.899380 20804 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:43.899125 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:43.877525 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.903676 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:43.906005 20734 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.877763 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.919474 28594 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:43.919929 20877 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:43.920197 28666 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:43.920955 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.921641 21434 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:43.899969 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:43.924353 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51809.7us
I20260709 12:24:43.926671 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.863032 28677 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:43.868588 28672 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:43.948348 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.944967 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:43.959676 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59778.5us
I20260709 12:24:43.877616 28496 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:43.964208 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.967993 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:43.969308 20810 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:43.973809 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:43.877657 28661 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:43.975037 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.989562 20734 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:43.992291 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.995291 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:43.920855 28666 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.984210 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:43.971562 28619 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:43.973003 28584 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:43.969067 28594 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:43.956974 28672 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:43.964737 28614 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:43.947134 28287 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:43.976092 28650 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:43.949206 28398 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.026530 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 226725us
I20260709 12:24:43.963485 28630 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:24:44.012187 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:43.876523 28668 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:43.991007 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:44.000938 28541 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
W20260709 12:24:44.040107 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49178.8us
I20260709 12:24:44.011619 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.017261 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.021816 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.042917 20804 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:24:44.022735 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:43.971868 28496 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.878015 28431 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.040225 28417 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:43.985376 27785 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:44.010815 28616 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:43.997680 28433 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:24:44.014109 28594 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:44.022650 28287 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:44.059020 20806 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:44.011780 28584 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:44.019858 28672 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:44.016733 28619 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:44.074472 28584 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:44.072115 28287 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:44.043354 28398 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:43.980172 28678 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 12:24:44.059751 28496 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:44.073017 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.073508 20732 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:44.026427 28650 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:44.072412 28417 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:44.073841 28614 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
W20260709 12:24:44.086841 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 166003us
I20260709 12:24:44.053186 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:44.060958 28313 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.969082 28676 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.064677 28541 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:44.006563 28559 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.163493 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:44.170645 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 198214us
I20260709 12:24:44.172235 20803 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:44.174726 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3009) took 1295 ms. Trace:
W20260709 12:24:44.173863 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 132464us
I20260709 12:24:44.107611 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:44.109616 28696 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.296s	user 0.001s	sys 0.002s
I20260709 12:24:44.073707 28594 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.184556 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.189258 20806 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.190358 20803 rpcz_store.cc:276] 0709 12:24:42.879580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.006343 (+1126763us) service_pool.cc:224] Handling call
0709 12:24:44.174644 (+168301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.181143 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:44.190301 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 137243us
I20260709 12:24:44.197679 28616 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:44.199108 20730 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:44.170396 28650 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:44.202375 20810 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:44.137228 28518 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.106190 28433 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:44.213910 28433 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:44.215245 28433 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.167893 28541 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:44.189338 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.204741 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:44.217741 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:44.196441 28692 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.111s	user 0.000s	sys 0.005s
I20260709 12:24:44.038368 28677 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.221427 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.224272 21434 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:24:44.227485 28700 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:24:44.227818 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.228770 28619 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.229344 28712 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:44.230580 28518 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:44.227548 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.167145 28651 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:24:44.203118 28658 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:44.233610 20806 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
W20260709 12:24:44.241264 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69692us
W20260709 12:24:44.242584 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.242457 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.237483 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.173395 28313 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.221069 28541 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:44.246254 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.247764 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:44.250857 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.251461 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.256927 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.256935 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.254786 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.263572 21434 raft_consensus.cc:2368] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:44.268402 20804 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:44.217831 28602 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:44.270885 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.243741 28433 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.271049 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.076331 28661 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.275292 20809 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:44.247383 28625 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:44.277990 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.279748 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.283960 20808 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:24:44.284421 21431 raft_consensus.cc:2368] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:44.286108 28602 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.287495 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:44.287364 28616 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.288184 20875 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:44.251386 28658 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:44.281337 21432 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:44.250538 28431 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.256160 28668 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.299715 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.296998 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.305130 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:44.291927 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.305934 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.307071 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.307566 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.307853 20803 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:24:44.308506 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.309623 21433 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 12:24:44.310043 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.298177 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.271162 28677 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:44.286998 28602 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:44.320412 20806 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:44.321633 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.322285 28714 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:44.312359 28712 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:44.324257 28712 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.324680 28650 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.327756 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.330049 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.316972 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.318849 21434 raft_consensus.cc:2376] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 16. Re-sending same reply.
I20260709 12:24:44.323072 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.283643 28625 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.337137 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.326468 28584 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.342110 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:44.357414 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 17.
I20260709 12:24:44.358127 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.359033 20883 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.343215 20875 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:44.312793 28632 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:44.329092 28602 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.354907 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.363512 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.364140 21432 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:44.355341 28711 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:24:44.328873 28702 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:24:44.351018 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:44.364778 21426 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:44.332831 28661 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:44.368341 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.356436 28625 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:44.381239 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.378086 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.284343 28417 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.363150 28632 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.367270 28602 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.356400 28666 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.406847 20883 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:44.410327 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.411057 20883 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:44.413188 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107982us
I20260709 12:24:44.366492 28711 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:44.410444 28632 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:24:44.432344 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 81484.8us
I20260709 12:24:44.395826 28608 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:44.460975 20883 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:44.423857 28699 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:24:44.414105 28559 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.475160 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.477982 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:44.486603 20727 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:44.423354 28398 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:44.440980 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.012s (SENT) [suppressed 34 similar messages]
W20260709 12:24:44.492377 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51670.5us
I20260709 12:24:44.493980 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:44.519338 20883 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:44.525494 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:44.531106 20850 leader_election.cc:400] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:44.534935 28726 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:24:44.543376 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.511823 28608 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:44.505496 28706 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.537425 28710 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:24:44.490602 28541 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.424862 28658 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.559312 20850 leader_election.cc:400] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
I20260709 12:24:44.561115 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.590828 20850 leader_election.cc:403] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.561414 28584 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:44.525525 28709 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:24:44.563723 28698 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:24:44.564450 28287 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.548272 28721 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:44.568243 28694 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.755s	user 0.002s	sys 0.000s
I20260709 12:24:44.569192 28520 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 15, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.628106 28632 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.629473 20877 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:44.631649 28541 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.559319 28706 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.634295 28706 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.635411 28718 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.635996 28718 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.638223 28718 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.638518 28625 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.639813 20810 raft_consensus.cc:2368] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
W20260709 12:24:44.642627 28707 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.367s	user 0.011s	sys 0.001s
I20260709 12:24:44.574148 28687 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:44.590698 28650 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.580484 28630 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:44.643771 28630 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:44.644496 28625 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.644737 28630 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.645094 28687 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:44.645783 28687 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:44.582298 28313 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.648507 28632 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.541564 28398 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:24:44.650156 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1288) took 1073 ms. Trace:
I20260709 12:24:44.579052 28251 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.591041 20729 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:44.563273 28654 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.606805 28241 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18.
W20260709 12:24:44.657126 28685 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.818s	user 0.009s	sys 0.003s
I20260709 12:24:44.657943 20726 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:44.595865 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.574180 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1881) took 1095 ms. Trace:
W20260709 12:24:44.607746 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 104273us
I20260709 12:24:44.660398 20727 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:44.609957 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50805.3us
I20260709 12:24:44.568202 20883 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:24:44.611649 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 194306us
W20260709 12:24:44.662786 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.616587 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:44.618347 28318 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.000s
I20260709 12:24:44.663884 20805 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:44.661358 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.619822 28708 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.600931 28614 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.601619 28616 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.620385 28724 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:44.642361 28496 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.645819 28630 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:44.571692 28212 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.603688 20808 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:24:44.672338 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:44.619086 28705 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
I20260709 12:24:44.677830 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:44.665109 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48638.7us
I20260709 12:24:44.678961 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.650724 20728 rpcz_store.cc:276] 0709 12:24:43.577001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.667496 (+ 90495us) service_pool.cc:224] Handling call
0709 12:24:44.649857 (+982361us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400226ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:44.650059 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725774080}
I20260709 12:24:44.680354 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.681667 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.666821 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:44.682787 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.660606 21430 rpcz_store.cc:276] 0709 12:24:43.478919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.480784 (+  1865us) service_pool.cc:224] Handling call
0709 12:24:43.503966 (+ 23182us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.510144 (+  6178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.510192 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.510807 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:24:44.573503 (+1062696us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18003fa960. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1
0709 12:24:44.573679 (+   176us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:44.573721 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.573822 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.573866 (+    44us) raft_consensus.cc:1685] finished
0709 12:24:44.573882 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.574108 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14948,"raft.queue_time_us":29533,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":12686,"spinlock_wait_cycles":2986752,"thread_start_us":17951,"threads_started":1,"wal-append.queue_time_us":233184}
I20260709 12:24:44.683012 28731 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:44.683806 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.684914 28731 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:44.685319 28731 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:44.689088 28496 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:44.689878 28670 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:44.690351 28670 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.690725 28670 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.670053 28318 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:44.671998 28614 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.685235 28287 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.712062 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:44.709889 28592 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:24:44.673204 28616 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.715448 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.715632 28616 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.716410 28616 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:44.699952 28613 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.532650 28711 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 27.12, Last appended by leader: 12, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.718506 28711 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:44.719743 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:44.720729 28724 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:44.703219 28720 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:24:44.721493 28724 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:44.692221 28679 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.850s	user 0.007s	sys 0.007s
I20260709 12:24:44.702876 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.723731 20729 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:44.692014 20807 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:44.705642 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.696269 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:44.737504 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.724138 21429 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:44.685907 28731 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:44.727037 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.746470 20809 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:44.742774 20808 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:24:44.744043 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48874.5us
I20260709 12:24:44.750553 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.754027 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:44.709787 28672 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.001s
I20260709 12:24:44.767410 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.758450 20803 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:44.764827 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.770102 28433 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:44.772043 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.772673 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.773367 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:44.727839 28496 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:44.699319 28717 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.776183 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.793687 21426 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:44.794070 20804 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:44.788384 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.788488 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.795137 21431 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:44.775153 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:44.789002 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.752609 28613 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:44.803557 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.742549 28287 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.790470 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.790829 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.812012 21435 raft_consensus.cc:2368] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27.
I20260709 12:24:44.812041 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.805284 28602 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.812932 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:44.766417 28603 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:44.828840 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.792718 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:44.800429 20728 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:44.791231 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.810190 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.780377 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:44.776859 28313 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:44.829704 20884 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:44.839596 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3248) took 1030 ms. Trace:
I20260709 12:24:44.840306 20729 rpcz_store.cc:276] 0709 12:24:43.808801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.792454 (+983653us) service_pool.cc:224] Handling call
0709 12:24:44.839499 (+ 47045us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:44.833779 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53474.9us
I20260709 12:24:44.824826 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.834352 20727 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:24:44.842506 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3255) took 1031 ms. Trace:
I20260709 12:24:44.837028 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.842895 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:44.843468 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:44.845017 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:44.789997 28671 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.823998 28594 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 12:24:44.842865 20727 rpcz_store.cc:276] 0709 12:24:43.810811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.589610 (+778799us) service_pool.cc:224] Handling call
0709 12:24:44.842443 (+252833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.777034 28656 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:24:44.858487 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.850389 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.805845 28592 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:44.820154 28650 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:44.863250 20729 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.739801 28679 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.765550 28616 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 12:24:44.866525 20776 leader_election.cc:400] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27.
I20260709 12:24:44.822275 28431 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:44.820891 28620 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:44.836123 28241 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:44.788408 28251 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:24:44.827144 28724 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:44.672493 28708 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.826606 28625 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.828200 28603 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.857246 28398 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:44.895939 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:44.845218 28727 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:24:44.881248 20776 leader_election.cc:403] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.812893 28717 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.882630 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:44.882782 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:44.856854 28732 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:44.838752 28658 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:24:44.922282 28725 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:44.922951 28725 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.923381 28725 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.893730 28663 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:24:44.927290 28732 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.845727 28632 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.872159 28594 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.928815 28632 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.930363 28732 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:44.931280 28632 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.892632 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:44.932865 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.911531 28541 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:44.934226 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.933995 28541 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.934949 20727 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:44.936589 28541 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.936746 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.938082 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:44.938721 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:44.939471 28676 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.862268 28417 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:24:44.941196 28417 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:44.941583 28417 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:44.941977 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:44.837044 28613 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:44.943279 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:44.878717 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.944120 28613 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.944794 20733 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.846134 28496 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:44.949397 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.858357 28721 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:44.896795 28627 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:44.874961 28650 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.873200 28584 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:44.788239 28672 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.912494 21435 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:44.827569 28287 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.914031 20884 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
W20260709 12:24:44.887135 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.008s	sys 0.032s
I20260709 12:24:44.928133 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1457) took 1446 ms. Trace:
W20260709 12:24:44.978338 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.009s	sys 0.032s
I20260709 12:24:44.952507 20883 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:44.889487 20728 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:44.902880 20729 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:44.982604 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:44.983795 20729 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:44.904129 28603 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:44.970511 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 104200us
I20260709 12:24:44.912236 28431 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:44.971760 20809 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:24:44.912639 28614 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.897606 28724 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:44.974404 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:44.978382 20802 rpcz_store.cc:276] 0709 12:24:43.481248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.481399 (+   151us) service_pool.cc:224] Handling call
0709 12:24:43.495817 (+ 14418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.622858 (+127041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.622891 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:43.622942 (+    51us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:43.622985 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.927858 (+1304873us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540027b354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:44.927914 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.928040 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257211264}
I20260709 12:24:44.878026 28592 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:45.009656 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 846182us
I20260709 12:24:44.948036 28496 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:44.905366 28602 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:45.018618 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:44.874392 28608 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:24:44.985734 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:44.976709 28597 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:44.920352 28620 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.003109 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:44.905145 28656 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 12:24:44.989845 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:44.967687 28594 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:44.870296 28671 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:45.044399 28734 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 12:24:45.045270 28671 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:45.046036 28620 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:45.048653 28620 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:45.049986 28602 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:45.051144 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2955) took 1706 ms. Trace:
I20260709 12:24:45.051615 20732 rpcz_store.cc:276] 0709 12:24:43.344544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.502479 (+157935us) service_pool.cc:224] Handling call
0709 12:24:43.502767 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.164477 (+661710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.050829 (+886352us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54001d8154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:45.051044 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311969536}
I20260709 12:24:45.052521 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3145) took 1744 ms. Trace:
I20260709 12:24:45.053061 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:45.052946 20876 rpcz_store.cc:276] 0709 12:24:43.308327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.500580 (+192253us) service_pool.cc:224] Handling call
0709 12:24:43.769646 (+269066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.877554 (+107908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.051412 (+1173858us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b5400267854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:45.052315 (+   903us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400267850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:45.052458 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771863296}
I20260709 12:24:45.054561 20732 raft_consensus.cc:2368] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:24:45.054622 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:45.055333 28661 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.677s	user 0.008s	sys 0.004s
I20260709 12:24:45.001873 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:44.969321 28627 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:45.056879 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:45.058791 28672 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.006516 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:45.073088 20804 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:24:45.073324 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66967.1us
I20260709 12:24:45.068063 21427 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:24:44.967362 28721 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:45.017408 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:45.058986 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:45.075703 20728 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:45.075747 20729 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.007359 28724 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:45.096203 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.010433 20727 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:44.942687 28417 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:44.980844 28520 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:44.977900 28632 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.982236 28712 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:45.022464 20881 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:45.024010 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:44.996925 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.036580 28616 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:45.026392 21435 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:45.026167 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:45.019048 20733 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:45.004431 28668 raft_consensus.cc:1027] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: attempt to promote peer 50c1ab13351d4427b836fbb449c6d7bb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:24:45.032874 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:45.002112 28584 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:45.036755 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47032.6us
I20260709 12:24:45.217728 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:45.010489 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.057053 28708 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.057307 28627 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.058794 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3252) took 1249 ms. Trace:
I20260709 12:24:45.061331 28592 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.062463 28431 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 27.12, Last appended by leader: 12, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:45.063561 20726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3253) took 1253 ms (client timeout 1555 ms). Trace:
I20260709 12:24:44.990960 28541 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27.
I20260709 12:24:45.014983 28614 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:44.907963 28625 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:44.976819 28650 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:45.071157 20802 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:45.058969 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.132469 20804 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 12:24:45.148772 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:45.156190 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3238) took 1295 ms. Trace:
I20260709 12:24:45.239463 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3150) took 1330 ms. Trace:
I20260709 12:24:45.167526 20878 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:45.171799 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3152) took 1260 ms. Trace:
I20260709 12:24:45.239319 20805 rpcz_store.cc:276] 0709 12:24:43.861090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.861817 (+   727us) service_pool.cc:224] Handling call
0709 12:24:45.155873 (+1294056us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b540027b854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:45.156069 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752277760}
I20260709 12:24:45.239223 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:45.240073 20882 rpcz_store.cc:276] 0709 12:24:43.908927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.148342 (+1239415us) service_pool.cc:224] Handling call
0709 12:24:45.239405 (+ 91063us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.174362 28717 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:45.176330 20730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3238) took 1448 ms (client timeout 1612 ms). Trace:
I20260709 12:24:45.120275 28398 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:45.178610 28679 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.179829 28663 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:45.199551 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106596us
I20260709 12:24:45.024686 20735 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:45.028005 28597 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:45.207428 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3254) took 1397 ms. Trace:
I20260709 12:24:45.371810 20733 rpcz_store.cc:276] 0709 12:24:43.810533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.847498 (+1036965us) service_pool.cc:224] Handling call
0709 12:24:45.207367 (+359869us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.206542 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:45.212432 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 364930us
I20260709 12:24:45.373144 20875 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
W20260709 12:24:45.212745 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 188776us
W20260709 12:24:45.378032 20735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3239) took 1648 ms (client timeout 1676 ms). Trace:
I20260709 12:24:45.378512 20775 leader_election.cc:400] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:24:45.215099 20727 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:45.379762 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3244) took 1572 ms. Trace:
I20260709 12:24:45.380239 20727 rpcz_store.cc:276] 0709 12:24:43.807808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.102544 (+1294736us) service_pool.cc:224] Handling call
0709 12:24:45.214802 (+112258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.379427 (+164625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.379701 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.216267 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.084038 28726 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:45.072572 28287 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.219138 20732 rpcz_store.cc:276] 0709 12:24:43.809809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.056628 (+1246819us) service_pool.cc:224] Handling call
0709 12:24:45.058736 (+  2108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.096279 28721 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:45.023403 28687 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.228122 21429 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEARNER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:24:45.227869 20726 rpcz_store.cc:269] 0709 12:24:43.810043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.835178 (+ 25135us) service_pool.cc:224] Handling call
0709 12:24:45.063387 (+1228209us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b540028d554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:45.063505 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889595648}
I20260709 12:24:45.160555 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3243) took 1172 ms. Trace:
I20260709 12:24:45.245344 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:45.245579 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.247488 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.390337 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:45.247783 28417 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:45.251591 28584 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:45.254575 28625 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:45.237888 28627 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:45.222451 20728 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6fe576036c04c149240cd6107285eca"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:37276
I20260709 12:24:45.396531 20728 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:45.396961 20728 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:45.398181 20727 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:45.267692 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 257251us
I20260709 12:24:45.399772 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:45.223004 28433 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:45.270347 28520 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:45.270287 20878 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:45.271080 28607 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:45.273517 28677 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.937s	user 0.010s	sys 0.003s
I20260709 12:24:45.213466 28632 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:45.281848 28630 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.610s	user 0.005s	sys 0.005s
I20260709 12:24:45.223304 28616 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:45.391692 28241 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:45.295344 28672 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:45.299120 20805 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:45.439762 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.440338 20805 raft_consensus.cc:606] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:45.440816 20805 raft_consensus.cc:614] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:45.240607 28708 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:45.303393 28674 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:45.303926 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66102.8us
I20260709 12:24:45.251627 20729 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:45.454133 20729 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:45.454308 20732 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:45.205415 28313 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:45.240460 20880 rpcz_store.cc:276] 0709 12:24:43.911177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.911341 (+   164us) service_pool.cc:224] Handling call
0709 12:24:43.955274 (+ 43933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.989155 (+ 33881us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:44.014770 (+ 25615us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.288716 (+273946us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b18003ca9c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:24:44.473016 (+184300us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.661616 (+188600us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:44.669960 (+  8344us) log.cc:844] Serialized 26 byte log entry
0709 12:24:44.670111 (+   151us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:44.670143 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.670210 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:45.154335 (+484125us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b18003ca9c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:45.171504 (+ 17169us) raft_consensus.cc:1685] finished
0709 12:24:45.171534 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.171715 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8075,"prepare.queue_time_us":14388,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":122,"raft.queue_time_us":226252,"raft.run_cpu_time_us":571,"raft.run_wall_time_us":39913,"spinlock_wait_cycles":24940800,"thread_start_us":371873,"threads_started":3,"wal-append.queue_time_us":445980}
I20260709 12:24:45.454710 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2966) took 1194 ms. Trace:
W20260709 12:24:45.328182 20730 rpcz_store.cc:269] 0709 12:24:43.727519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.759722 (+ 32203us) service_pool.cc:224] Handling call
0709 12:24:45.176052 (+1416330us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:45.176246 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771476864}
I20260709 12:24:45.238719 28592 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.368237 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:45.378741 20735 rpcz_store.cc:269] 0709 12:24:43.730138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.862803 (+132665us) service_pool.cc:224] Handling call
0709 12:24:45.377767 (+1514964us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002279d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:45.377951 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2512493952}
I20260709 12:24:45.422256 28749 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.386092 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.379098 20775 leader_election.cc:403] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:45.392024 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.032s	sys 0.028s
I20260709 12:24:45.389336 20807 rpcz_store.cc:276] 0709 12:24:43.988105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.988843 (+   738us) service_pool.cc:224] Handling call
0709 12:24:45.160245 (+1171402us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54003d68d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:45.160459 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525050112}
I20260709 12:24:45.375970 28597 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.429000 28433 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:45.406951 21431 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:45.407164 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:45.434772 28686 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:45.409307 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3256) took 1489 ms. Trace:
I20260709 12:24:45.429904 28748 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:24:45.424669 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.414026 20728 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:45.287480 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 278996us
I20260709 12:24:45.483372 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:45.484668 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:45.438506 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.372638 28663 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.450627 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:45.488030 20735 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:45.489118 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3257) took 1080 ms. Trace:
I20260709 12:24:45.489594 20735 rpcz_store.cc:276] 0709 12:24:44.408538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.460392 (+1051854us) service_pool.cc:224] Handling call
0709 12:24:45.487562 (+ 27170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.488773 (+  1211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.489041 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:45.491242 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 105233us
I20260709 12:24:45.495486 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:45.497882 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.499025 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60626.3us
I20260709 12:24:45.499876 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.455156 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2964) took 1195 ms. Trace:
I20260709 12:24:45.500842 20732 rpcz_store.cc:276] 0709 12:24:44.259588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.453478 (+1193890us) service_pool.cc:224] Handling call
0709 12:24:45.453885 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.454861 (+   976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.455089 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.502264 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:45.503283 20732 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:45.455607 20729 rpcz_store.cc:276] 0709 12:24:44.260236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.453683 (+1193447us) service_pool.cc:224] Handling call
0709 12:24:45.453943 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.454493 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.454662 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.456681 20733 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:45.388517 28602 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:24:45.382189 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.522722 21428 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:45.523813 20807 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:45.524024 28687 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:45.524747 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.525890 20807 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:45.526329 28749 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:45.527788 20728 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:45.528311 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:37276 (request call id 3243) took 1762 ms. Trace:
I20260709 12:24:45.528491 28749 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:45.528636 20728 rpcz_store.cc:276] 0709 12:24:43.798675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.125714 (+1327039us) service_pool.cc:224] Handling call
0709 12:24:45.528205 (+402491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.504765 28708 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:24:45.530578 28677 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:45.532034 28632 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.468075 28674 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.535918 28674 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.535989 28607 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:45.536428 28607 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:45.537324 28607 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.538578 28607 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:45.540378 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:45.505151 28541 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:45.541042 20882 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:45.505304 28620 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:45.541793 28625 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:45.542070 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:45.542328 28712 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:24:45.541955 28721 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.460016 28398 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:45.544440 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:45.544587 28721 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:45.545341 28608 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:24:45.545884 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46077.9us
I20260709 12:24:45.546645 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.546895 28613 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:45.505167 28671 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.548321 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:45.548699 28671 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.549485 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.389603 28603 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.550520 28663 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:45.551451 20881 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:45.551856 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:45.552968 28714 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:45.553097 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:45.553695 20881 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:45.553854 28690 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:45.554473 28690 raft_consensus.cc:2764] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:24:45.465207 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:45.472548 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.033s	sys 0.029s
I20260709 12:24:45.555517 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:45.555853 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.557538 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:45.430975 28651 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:24:45.407874 20806 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.011s	sys 0.033s
W20260709 12:24:45.566701 20806 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.011s	sys 0.033s
W20260709 12:24:45.475777 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.476402 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 98082.7us
I20260709 12:24:45.569411 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:45.477072 20727 rpcz_store.cc:276] 0709 12:24:43.909889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.393831 (+1483942us) service_pool.cc:224] Handling call
0709 12:24:45.398015 (+  4184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.398572 (+   557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.398820 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:45.383221 28717 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:45.488180 20809 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:45.450861 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.521754 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.573262 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3249) took 1764 ms. Trace:
I20260709 12:24:45.573405 21430 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:45.554798 28672 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:45.385452 28726 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.542891 20882 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:45.577958 20882 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:45.577617 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:45.578981 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:45.550778 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.560592 21427 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:45.555868 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:45.581305 20878 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:45.498116 28661 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:24:45.581655 20878 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:45.506052 28676 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.004s
W20260709 12:24:45.558553 28711 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.839s	user 0.010s	sys 0.004s
I20260709 12:24:45.561956 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:45.573729 20733 rpcz_store.cc:276] 0709 12:24:43.809051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.381435 (+1572384us) service_pool.cc:224] Handling call
0709 12:24:45.456440 (+ 75005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.509539 (+ 53099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.573062 (+ 63523us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54000c3fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:45.573205 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144121216}
I20260709 12:24:45.603230 21431 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:45.563858 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:45.541085 28541 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.584339 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:45.585546 20729 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:45.579769 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:45.476598 28241 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:45.553714 28714 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:45.607312 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2971) took 1021 ms. Trace:
I20260709 12:24:45.628657 20729 rpcz_store.cc:276] 0709 12:24:44.585702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.575362 (+989660us) service_pool.cc:224] Handling call
0709 12:24:45.575774 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.606993 (+ 31219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.607242 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:45.608373 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.451133 28727 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.126s
I20260709 12:24:45.550796 28671 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:45.563488 28598 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:45.563462 28721 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:45.606498 20878 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:45.645447 20878 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:24:45.476344 28630 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.109s
I20260709 12:24:45.601739 28748 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:45.613772 28541 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:45.582964 28287 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:45.660912 28287 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.661438 28287 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:45.664239 28613 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:45.664968 28613 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:45.614441 28674 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:45.613641 28616 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:45.613826 28520 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:45.582634 28717 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.636437 28241 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:45.616968 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51469.1us
I20260709 12:24:45.566577 28651 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:45.582748 28620 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:45.582404 28726 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:45.473011 28592 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:45.642661 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73336.3us
I20260709 12:24:45.719238 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:45.720687 20810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.028s	sys 0.058s
W20260709 12:24:45.650269 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 99541.9us
I20260709 12:24:45.721748 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.651427 28708 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:45.651980 28433 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:45.609849 28679 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:45.498987 28686 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.607663 28625 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:45.614316 28704 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:45.659400 28598 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.660091 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:45.841859 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 181855us
W20260709 12:24:45.667785 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 81965.5us
I20260709 12:24:45.668951 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:45.655129 28721 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:45.646889 28318 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.002s	sys 0.010s
I20260709 12:24:45.680675 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:45.644333 28608 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:45.673135 28670 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.982s	user 0.007s	sys 0.006s
W20260709 12:24:45.667622 28731 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.884s	user 0.009s	sys 0.000s
W20260709 12:24:45.706092 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.715986 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:45.882090 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:45.631852 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:45.722862 20810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.029s	sys 0.058s
I20260709 12:24:45.644794 28730 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:24:45.672372 28674 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.737553 20727 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:45.888370 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1299) took 1174 ms. Trace:
I20260709 12:24:45.889134 20727 rpcz_store.cc:276] 0709 12:24:44.713561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.689747 (+976186us) service_pool.cc:224] Handling call
0709 12:24:45.737072 (+ 47325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.887975 (+150903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.888287 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:45.717362 28725 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.793s	user 0.012s	sys 0.000s
I20260709 12:24:45.630213 28680 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.858207 20878 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:24:45.687673 28651 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:45.725754 28708 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:45.833660 28668 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.013s	sys 0.009s
I20260709 12:24:45.741024 28620 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:45.894809 28748 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.832297 28212 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:45.895902 28730 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:45.714717 28711 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.199s
W20260709 12:24:45.896014 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.976154 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.865186 28672 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.871104 28687 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:45.865581 28676 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.644146 28313 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.725893 28671 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.179s
W20260709 12:24:45.874367 28318 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.002s	sys 0.010s
W20260709 12:24:45.949767 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 392249us
I20260709 12:24:45.714581 28714 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.172s
W20260709 12:24:45.953759 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 232091us
W20260709 12:24:45.963155 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.896839 28674 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:45.887553 28608 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:45.950084 28668 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.014s	sys 0.009s
I20260709 12:24:45.739545 28712 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.198s
I20260709 12:24:45.977427 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:45.827376 28594 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.011s
W20260709 12:24:46.028111 28594 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.011s	sys 0.011s
I20260709 12:24:45.941911 28708 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:45.959877 28251 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.013s	sys 0.043s
I20260709 12:24:45.996434 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:46.036015 20878 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:46.037731 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:45.972651 28603 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:45.982954 28650 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.000s
I20260709 12:24:45.897290 28747 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.122s
I20260709 12:24:45.897147 28731 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.023947 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:45.982743 28687 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.897032 28670 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:46.033684 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56348.7us
W20260709 12:24:46.113921 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.942354 28680 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:45.944738 28725 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.014190 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:46.121305 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 107221us
W20260709 12:24:46.046150 20731 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.004s
I20260709 12:24:46.122188 20731 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:45.983354 28431 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:46.082867 20878 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:24:45.927063 28613 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:46.034467 28496 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.026s	sys 0.051s
I20260709 12:24:46.028944 28746 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.039891 28251 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.013s	sys 0.043s
I20260709 12:24:45.991503 28672 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.039592 28690 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:24:46.103343 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 260021us
I20260709 12:24:46.097136 28318 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.023607 28212 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:24:45.999696 28676 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.083974 28663 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:46.127956 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:46.111145 28724 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.895s	user 0.012s	sys 0.002s
W20260709 12:24:46.120062 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 540295us
I20260709 12:24:46.132896 20878 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:24:46.095410 28650 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.116998 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3077) took 1557 ms. Trace:
I20260709 12:24:46.190096 20877 rpcz_store.cc:276] 0709 12:24:44.559881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.197091 (+637210us) service_pool.cc:224] Handling call
0709 12:24:45.197530 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.197854 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:45.666874 (+469020us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.116506 (+449632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.116584 (+    78us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:46.116649 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.116729 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.116889 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12959,"prepare.queue_time_us":834605,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":2557,"raft.queue_time_us":2552,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":10756,"thread_start_us":896477,"threads_started":2}
I20260709 12:24:46.140383 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3080) took 1579 ms. Trace:
I20260709 12:24:46.193876 20884 rpcz_store.cc:276] 0709 12:24:44.560636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.005444 (+444808us) service_pool.cc:224] Handling call
0709 12:24:46.140169 (+1134725us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:46.140308 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542299392}
I20260709 12:24:46.113749 28687 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:46.138437 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:46.200500 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62170.1us
I20260709 12:24:46.153151 20734 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:24:46.190900 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1475) took 1437 ms. Trace:
I20260709 12:24:46.153647 20876 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:24:46.155239 21434 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:24:46.131491 28496 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.026s	sys 0.051s
I20260709 12:24:46.129047 28417 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:46.157415 28690 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:46.159262 20726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.012s	sys 0.008s
W20260709 12:24:46.225183 20726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.013s	sys 0.008s
I20260709 12:24:46.126633 28431 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:46.158212 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144451us
W20260709 12:24:46.158780 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 349263us
I20260709 12:24:46.125205 28630 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:46.153553 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 465473us
I20260709 12:24:46.119557 28632 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:46.136354 20727 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:46.150971 28241 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:46.152117 28663 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:46.154098 28717 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:46.207474 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.179869 28750 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
I20260709 12:24:46.020762 28625 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.201642 20810 rpcz_store.cc:276] 0709 12:24:44.753900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.162005 (+408105us) service_pool.cc:224] Handling call
0709 12:24:45.162433 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.162704 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:45.163102 (+   398us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.190424 (+1027322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.190501 (+    77us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:46.190562 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.190660 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.190812 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5570,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":294017,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":9682,"thread_start_us":1026412,"threads_started":2,"wal-append.queue_time_us":800733}
I20260709 12:24:46.221235 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3092) took 1288 ms. Trace:
I20260709 12:24:46.173069 28721 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:46.303922 20881 rpcz_store.cc:276] 0709 12:24:44.932655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.556391 (+623736us) service_pool.cc:224] Handling call
0709 12:24:45.556669 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.556882 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:46.042648 (+485766us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.208739 (+166091us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.220881 (+ 12142us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:46.220938 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.221019 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.221156 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":559370,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1042,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":15011,"thread_start_us":608319,"threads_started":2}
I20260709 12:24:46.086758 28620 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.224931 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:46.180559 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52672.7us
I20260709 12:24:46.226195 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:46.183456 28712 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.231973 20803 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:46.235891 28677 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.232079 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:46.213328 28765 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:24:46.235445 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1301) took 1521 ms. Trace:
I20260709 12:24:46.157701 28318 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.290881 27785 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:46.173621 28724 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.305001 28584 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:46.352982 28724 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:46.353344 28584 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:46.355134 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3156) took 2025 ms. Trace:
I20260709 12:24:46.355625 20876 rpcz_store.cc:276] 0709 12:24:44.330203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.055922 (+725719us) service_pool.cc:224] Handling call
0709 12:24:45.056307 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.201861 (+1145554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.354840 (+152979us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540026cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:46.355058 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523029760}
I20260709 12:24:46.357052 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:46.137121 28746 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.357632 28559 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:46.358345 28760 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
I20260709 12:24:46.359853 28746 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:46.360504 28767 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:24:46.360690 20882 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:46.234650 28771 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.161662 28251 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
W20260709 12:24:46.228061 28751 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 12:24:46.362700 28704 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:46.363279 28704 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:46.362913 28771 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:46.364375 28704 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.365921 28771 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:46.290100 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3089) took 1358 ms. Trace:
I20260709 12:24:46.366894 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2968) took 2106 ms. Trace:
I20260709 12:24:46.367347 20730 rpcz_store.cc:276] 0709 12:24:44.260916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.455980 (+1195064us) service_pool.cc:224] Handling call
0709 12:24:45.475657 (+ 19677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.495104 (+ 19447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.495144 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.529373 (+ 34229us) log.cc:844] Serialized 26 byte log entry
0709 12:24:45.879795 (+350422us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:45.879858 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.879960 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.359831 (+479871us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:46.366531 (+  6700us) raft_consensus.cc:1685] finished
0709 12:24:46.366556 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.366827 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24715,"raft.queue_time_us":3252,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":34375,"spinlock_wait_cycles":24763648,"thread_start_us":349924,"threads_started":1,"wal-append.queue_time_us":580527}
I20260709 12:24:46.370399 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1302) took 1633 ms. Trace:
I20260709 12:24:46.209427 28747 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:46.371405 28747 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.371804 28747 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:46.374686 28597 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.896s	user 0.008s	sys 0.000s
I20260709 12:24:46.202422 28773 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:46.240674 28417 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:46.294015 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.229176 28431 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:46.312181 20884 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:46.306782 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:46.176689 28686 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:46.315824 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 89725.9us
I20260709 12:24:46.319597 20801 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:46.322861 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1292) took 1736 ms. Trace:
I20260709 12:24:46.235672 28592 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:46.267536 28690 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.289412 28602 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:46.337924 20727 rpcz_store.cc:276] 0709 12:24:44.714094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.135752 (+1421658us) service_pool.cc:224] Handling call
0709 12:24:46.136142 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.235130 (+ 98988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.235388 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:46.301286 28603 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:24:46.327817 28770 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:24:46.347853 27785 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:46.412786 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.379009 28769 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:24:46.305372 28656 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:24:46.290705 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57360.8us
I20260709 12:24:46.366000 28704 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:46.291601 28598 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:46.366633 20883 rpcz_store.cc:276] 0709 12:24:44.931608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.002827 (+ 71219us) service_pool.cc:224] Handling call
0709 12:24:46.269498 (+1266671us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b540019c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:46.290021 (+ 20523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224977664}
W20260709 12:24:46.428226 20734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3241) took 2673 ms (client timeout 2012 ms). Trace:
I20260709 12:24:46.316002 28630 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 12:24:46.437296 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:46.370764 20730 rpcz_store.cc:276] 0709 12:24:44.736682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.369483 (+1632801us) service_pool.cc:224] Handling call
0709 12:24:46.369847 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.370054 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.370091 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.370132 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:46.370165 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.370218 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.370334 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:46.306716 28496 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:24:46.321820 28763 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
I20260709 12:24:46.383533 20733 rpcz_store.cc:276] 0709 12:24:44.586673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.602984 (+1016311us) service_pool.cc:224] Handling call
0709 12:24:45.603431 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.603702 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:45.959946 (+356244us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.313875 (+353929us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.313959 (+    84us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:46.314027 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.314155 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.322735 (+  8580us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4821,"prepare.queue_time_us":429701,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":644,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":12448,"thread_start_us":708688,"threads_started":2}
I20260709 12:24:46.386309 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:46.314980 28718 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.102s
W20260709 12:24:46.391032 20731 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1291) took 2813 ms (client timeout 2240 ms). Trace:
I20260709 12:24:46.380487 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:46.399122 28251 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:46.341125 28677 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.400583 28784 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:46.377816 28755 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
W20260709 12:24:46.412818 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.484638 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.443361 20731 rpcz_store.cc:269] 0709 12:24:43.577846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.423470 (+845624us) service_pool.cc:224] Handling call
0709 12:24:46.390765 (+1967295us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b54002495d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:46.390928 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194474880}
I20260709 12:24:46.486806 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1470) took 1757 ms. Trace:
I20260709 12:24:46.487360 20803 rpcz_store.cc:276] 0709 12:24:44.729042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.868295 (+139253us) service_pool.cc:224] Handling call
0709 12:24:46.486332 (+1618037us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003cad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:46.486735 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3061351808}
I20260709 12:24:46.478094 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1468) took 1749 ms. Trace:
I20260709 12:24:46.348948 28318 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:46.376434 28768 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:24:46.426522 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1397) took 1220 ms. Trace:
I20260709 12:24:46.310938 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:46.489868 20882 rpcz_store.cc:276] 0709 12:24:45.206338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.584849 (+378511us) service_pool.cc:224] Handling call
0709 12:24:45.585324 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.361134 (+775810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.426241 (+ 65107us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54000a41d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:46.426449 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782650112}
I20260709 12:24:46.429075 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1892) took 1664 ms. Trace:
I20260709 12:24:46.375422 28773 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:46.428783 20734 rpcz_store.cc:269] 0709 12:24:43.751133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.994925 (+243792us) service_pool.cc:224] Handling call
0709 12:24:46.424349 (+2429424us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b5400226fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:46.424488 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4942054784}
I20260709 12:24:46.376295 28676 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:46.432998 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1392) took 1503 ms. Trace:
I20260709 12:24:46.433944 21432 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:46.438197 20727 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:24:46.430423 28758 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.318s	user 0.000s	sys 0.002s
I20260709 12:24:46.328405 28712 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.402957 28602 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:46.403329 28603 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:46.376271 28417 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:46.400141 28650 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:46.442888 20733 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:46.809085 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1304) took 2035 ms. Trace:
I20260709 12:24:46.809495 20733 rpcz_store.cc:276] 0709 12:24:44.774060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.442371 (+1668311us) service_pool.cc:224] Handling call
0709 12:24:46.442709 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.808784 (+366075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.809011 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:46.810262 28759 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.677s	user 0.004s	sys 0.005s
I20260709 12:24:46.810776 20733 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:46.442070 20876 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:24:46.812073 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:46.813643 20876 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:46.375171 28597 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.814301 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1377) took 1614 ms. Trace:
I20260709 12:24:46.814690 21432 rpcz_store.cc:276] 0709 12:24:45.199687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.199920 (+   233us) service_pool.cc:224] Handling call
0709 12:24:46.429110 (+1229190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.547296 (+118186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.814100 (+266804us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540047ead0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:46.814243 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2826116480}
I20260709 12:24:46.443810 20884 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:46.816787 20884 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:46.817350 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3164) took 1455 ms. Trace:
I20260709 12:24:46.817639 20884 rpcz_store.cc:276] 0709 12:24:45.362267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.816136 (+1453869us) service_pool.cc:224] Handling call
0709 12:24:46.816441 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.817107 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.817296 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:46.818055 28607 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.279s	user 0.012s	sys 0.004s
W20260709 12:24:46.819103 28674 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.786s	user 0.000s	sys 0.011s
I20260709 12:24:46.819375 20728 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:46.424089 28656 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:46.819779 28674 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.821069 28656 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.822177 28656 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:46.471446 28772 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:24:46.431888 28608 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:46.824445 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1393) took 1894 ms. Trace:
I20260709 12:24:46.824870 20880 rpcz_store.cc:276] 0709 12:24:44.930332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.476091 (+545759us) service_pool.cc:224] Handling call
0709 12:24:45.557263 (+ 81172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.557597 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.557627 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.558071 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:24:45.980368 (+422297us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:45.980421 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.980499 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.824213 (+843714us) raft_consensus.cc:1685] finished
0709 12:24:46.824234 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.824384 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9921,"raft.queue_time_us":5884,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":10532,"thread_start_us":421992,"threads_started":1,"wal-append.queue_time_us":652526}
W20260709 12:24:46.825814 28651 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.875s	user 0.010s	sys 0.002s
I20260709 12:24:46.472836 28680 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:46.473834 28731 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:46.475894 28627 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.071s	user 0.013s	sys 0.000s
I20260709 12:24:46.478088 21427 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 12:24:46.478554 28541 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.800s	user 0.008s	sys 0.000s
I20260709 12:24:46.827857 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1297) took 2115 ms. Trace:
I20260709 12:24:46.479781 28785 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.481375 28687 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.482553 28677 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:46.404062 28663 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.491178 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:46.536098 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:46.540149 20883 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:46.543682 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:46.545724 20879 rpcz_store.cc:276] 0709 12:24:44.929998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.293080 (+363082us) service_pool.cc:224] Handling call
0709 12:24:45.293432 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.293789 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.293813 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.294293 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:24:45.679029 (+384736us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:45.679095 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.679258 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.357680 (+678422us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:46.415448 (+ 57768us) raft_consensus.cc:1685] finished
0709 12:24:46.415480 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.432919 (+ 17439us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1442,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":29165312,"thread_start_us":383382,"threads_started":1,"wal-append.queue_time_us":679069}
I20260709 12:24:46.488822 20801 rpcz_store.cc:276] 0709 12:24:44.728572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.937505 (+208933us) service_pool.cc:224] Handling call
0709 12:24:44.937834 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.382872 (+1445038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.477796 (+ 94924us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003c9954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:46.478022 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3014370048}
W20260709 12:24:46.549193 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 238367us
I20260709 12:24:46.491180 21434 rpcz_store.cc:276] 0709 12:24:44.764422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.764540 (+   118us) service_pool.cc:224] Handling call
0709 12:24:46.428869 (+1664329us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:46.429015 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3173311872}
W20260709 12:24:46.603623 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.639662 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:46.445169 28287 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.783s	user 0.007s	sys 0.007s
W20260709 12:24:46.662992 20773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.000s	sys 0.023s
I20260709 12:24:46.467931 28251 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:24:46.645260 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 416457us
W20260709 12:24:46.645517 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 421071us
I20260709 12:24:46.440778 28496 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:24:46.753624 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.033s	sys 0.029s
I20260709 12:24:46.539417 28670 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:46.536528 28672 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:46.554905 28725 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:46.554888 28773 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:46.448288 28559 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:46.443786 28780 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.554487 20734 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:24:46.783567 20804 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:46.540390 28661 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.228s
I20260709 12:24:46.787765 28602 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:46.792975 28726 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.962s	user 0.010s	sys 0.000s
I20260709 12:24:46.795567 28431 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.122608 20773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.005s	sys 0.034s
I20260709 12:24:46.797375 28318 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:24:46.798521 28766 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.467s	user 0.000s	sys 0.001s
I20260709 12:24:46.799861 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3263) took 1970 ms. Trace:
W20260709 12:24:46.800416 28708 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.719s	user 0.014s	sys 0.005s
W20260709 12:24:46.752934 21428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.009s	sys 0.003s
I20260709 12:24:46.798448 28757 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.804538 28584 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:46.743796 20878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.016s	sys 0.014s
I20260709 12:24:46.545567 28771 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:46.686092 20735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.014s	sys 0.016s
W20260709 12:24:47.256826 28799 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.468s	user 0.001s	sys 0.001s
I20260709 12:24:46.822837 28674 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:46.828255 20728 rpcz_store.cc:276] 0709 12:24:44.712972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.652470 (+939498us) service_pool.cc:224] Handling call
0709 12:24:45.652793 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.819875 (+1167082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.827621 (+  7746us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000e92d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:46.827802 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682419968}
I20260709 12:24:47.123765 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1307) took 1654 ms. Trace:
I20260709 12:24:47.123564 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:46.826951 20880 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
W20260709 12:24:47.120846 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.957477 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2980) took 2052 ms. Trace:
W20260709 12:24:47.124166 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.033s	sys 0.039s
I20260709 12:24:47.123379 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3161) took 1763 ms. Trace:
W20260709 12:24:46.886732 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.901690 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.121946 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1305) took 1666 ms. Trace:
W20260709 12:24:47.120832 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.944021 20727 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:46.875355 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3160) took 1516 ms. Trace:
I20260709 12:24:47.126801 20882 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:24:47.122895 20850 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.012s (SENT)
I20260709 12:24:47.127861 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:47.027364 20729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.012s	sys 0.007s
I20260709 12:24:46.830581 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:46.796232 28783 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
W20260709 12:24:47.121389 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.125627 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.123346 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3163) took 1761 ms. Trace:
W20260709 12:24:47.117403 28789 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.318s	user 0.003s	sys 0.000s
W20260709 12:24:46.928136 28625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.015s	sys 0.004s
I20260709 12:24:46.968488 28731 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.120952 28632 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.235371 28795 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:46.830210 28687 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:46.876477 28718 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:47.119560 28790 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.331s	user 0.002s	sys 0.002s
W20260709 12:24:46.928539 28313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.009s	sys 0.009s
W20260709 12:24:47.429641 28313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.013s	sys 0.013s
I20260709 12:24:46.893240 28721 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:47.305995 20878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.018s	sys 0.014s
I20260709 12:24:47.430507 28721 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.431070 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3159) took 2072 ms. Trace:
I20260709 12:24:47.235931 27785 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.303440 20802 rpcz_store.cc:276] 0709 12:24:44.829599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.309576 (+479977us) service_pool.cc:224] Handling call
0709 12:24:45.309941 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.310333 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.310370 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.310851 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:24:45.735629 (+424778us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:45.735711 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.735828 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.591469 (+855641us) raft_consensus.cc:1685] finished
0709 12:24:46.799623 (+208154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.799794 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3403,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":424303,"threads_started":1,"wal-append.queue_time_us":636469}
I20260709 12:24:47.302829 28708 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.308919 20735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.014s	sys 0.021s
I20260709 12:24:47.390588 28779 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:46.826442 28651 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.255036 28798 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.467s	user 0.000s	sys 0.002s
W20260709 12:24:47.253414 28791 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.359s	user 0.000s	sys 0.000s
W20260709 12:24:47.420328 28748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.020s	sys 0.057s
W20260709 12:24:47.244056 28781 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 12:24:47.249799 28776 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.406s	user 0.000s	sys 0.001s
W20260709 12:24:47.251225 28788 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.450s	user 0.000s	sys 0.001s
I20260709 12:24:46.969305 28592 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.246353 28792 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.330s	user 0.000s	sys 0.000s
I20260709 12:24:47.136080 28607 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.128893 28559 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:47.134526 28251 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:47.263840 28602 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:47.381594 20729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.012s	sys 0.013s
W20260709 12:24:47.381428 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.382383 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 259500us
I20260709 12:24:47.235721 28777 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.234000 28672 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:47.234982 28613 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.235906 28614 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.535882 21426 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1484) took 2682 ms (client timeout 2012 ms). Trace:
I20260709 12:24:47.485646 28677 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.125141 28608 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.607129 28608 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.523737 28771 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:47.608469 28771 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.610054 28584 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.610710 28584 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.236377 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3262) took 2407 ms. Trace:
I20260709 12:24:47.383080 20733 rpcz_store.cc:276] 0709 12:24:45.469097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.810268 (+1341171us) service_pool.cc:224] Handling call
0709 12:24:46.810582 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.838832 (+ 28250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.123713 (+284881us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.386536 20883 rpcz_store.cc:276] 0709 12:24:45.359887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.467960 (+1108073us) service_pool.cc:224] Handling call
0709 12:24:46.539972 (+ 72012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.123153 (+583181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.123335 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.387378 20731 rpcz_store.cc:276] 0709 12:24:44.904901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.487770 (+1582869us) service_pool.cc:224] Handling call
0709 12:24:46.785440 (+297670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.946925 (+161485us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:46.946989 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.947065 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:46.947124 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.947242 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.947434 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":148,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104}
I20260709 12:24:47.388033 20734 rpcz_store.cc:276] 0709 12:24:45.455615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.547886 (+1092271us) service_pool.cc:224] Handling call
0709 12:24:46.548286 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.121675 (+573389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.121888 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.381505 20879 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:24:47.389178 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3168) took 2024 ms. Trace:
W20260709 12:24:47.349845 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.389268 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 261484us
I20260709 12:24:47.389647 20876 rpcz_store.cc:276] 0709 12:24:45.361561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.813035 (+1451474us) service_pool.cc:224] Handling call
0709 12:24:46.813320 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.813951 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.123291 (+309340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.139995 28318 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:47.140637 28616 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.278388 28671 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:47.393985 20728 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:24:47.232617 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 741450us
I20260709 12:24:47.258692 28431 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:47.117653 28796 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.399894 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:47.245463 28712 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:47.236151 28597 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:47.236618 28663 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:47.410708 20884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.029s	sys 0.044s
I20260709 12:24:47.386411 20877 rpcz_store.cc:276] 0709 12:24:45.359128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.201332 (+842204us) service_pool.cc:224] Handling call
0709 12:24:46.201623 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.201840 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:46.291865 (+ 90025us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.874925 (+583060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.874989 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:46.875043 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.875110 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.875291 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68083,"prepare.queue_time_us":113208,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":99797,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":482931,"threads_started":1,"wal-append.queue_time_us":990291}
I20260709 12:24:47.410550 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:47.140229 28726 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.384688 28625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.015s	sys 0.012s
I20260709 12:24:47.303172 28725 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:24:46.845009 28747 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.399894 28656 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:47.429039 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:47.132894 28367 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.353s	user 0.008s	sys 0.009s
I20260709 12:24:47.123144 28603 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:46.958873 28704 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.527s	user 0.022s	sys 0.000s
I20260709 12:24:47.390738 28650 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.390398 28717 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.410844 28661 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.431639 20878 rpcz_store.cc:276] 0709 12:24:45.358450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.200600 (+842150us) service_pool.cc:224] Handling call
0709 12:24:46.200931 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.201154 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:46.230351 (+ 29197us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.254219 (+1023868us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:47.430682 (+176463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.430745 (+    63us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:47.430806 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.430877 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.431010 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":109404,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":5613,"spinlock_wait_cycles":227336576,"thread_start_us":1199985,"threads_started":1,"wal-append.queue_time_us":557055}
I20260709 12:24:47.305835 28757 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.135097 28287 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:46.828586 28785 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.248217 28793 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.448s	user 0.003s	sys 0.000s
I20260709 12:24:47.141743 28780 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.417621 28632 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.460846 28748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.021s	sys 0.057s
I20260709 12:24:47.414268 28731 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.421131 28627 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:47.484751 20730 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: []
W20260709 12:24:47.480980 28686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.027s	sys 0.066s
I20260709 12:24:47.453588 28433 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.497540 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1300) took 2783 ms. Trace:
I20260709 12:24:47.514664 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3169) took 2030 ms. Trace:
I20260709 12:24:47.527700 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1293) took 2940 ms. Trace:
I20260709 12:24:47.531461 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1306) took 2062 ms. Trace:
I20260709 12:24:47.432619 28718 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.535565 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3091) took 2603 ms. Trace:
I20260709 12:24:46.828269 28541 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.546413 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 417008us
W20260709 12:24:47.572570 21426 rpcz_store.cc:269] 0709 12:24:44.842452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.842598 (+   146us) service_pool.cc:224] Handling call
0709 12:24:47.524365 (+2681767us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:47.524540 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5951255296}
I20260709 12:24:47.426359 28496 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.594548 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2973) took 3008 ms. Trace:
I20260709 12:24:47.523739 28607 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.421969 28795 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.485853 28784 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.457561 28779 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:47.600534 28613 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.491542 28592 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.606360 28794 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:47.725941 20884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.029s	sys 0.049s
I20260709 12:24:47.723888 20882 rpcz_store.cc:276] 0709 12:24:45.483752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.395060 (+1911308us) service_pool.cc:224] Handling call
0709 12:24:47.395375 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.441170 (+ 45795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.441215 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.441275 (+    60us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:47.441330 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.514324 (+ 72994us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:47.514388 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.514580 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101856512}
I20260709 12:24:47.729351 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:47.732304 20877 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:47.732187 20875 rpcz_store.cc:276] 0709 12:24:44.932368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.477129 (+544761us) service_pool.cc:224] Handling call
0709 12:24:45.521239 (+ 44110us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.980759 (+459520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.980790 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.981319 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.432951 (+451632us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:46.433007 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.433104 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.118839 (+685735us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:47.523291 (+404452us) raft_consensus.cc:1685] finished
0709 12:24:47.523325 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.535486 (+ 12161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61271,"raft.queue_time_us":52425,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":24751,"spinlock_wait_cycles":368296320,"thread_start_us":824919,"threads_started":2,"wal-append.queue_time_us":760468}
I20260709 12:24:47.733762 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:47.734331 20877 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:47.730206 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3167) took 2362 ms. Trace:
I20260709 12:24:47.735081 20884 rpcz_store.cc:276] 0709 12:24:45.364394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.818325 (+1453931us) service_pool.cc:224] Handling call
0709 12:24:46.818630 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.818850 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.818874 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.851169 (+ 32295us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.726683 (+875514us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:47.726739 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.726833 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.726893 (+    60us) raft_consensus.cc:1685] finished
0709 12:24:47.726911 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.727114 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":874973,"threads_started":1,"wal-append.queue_time_us":764981}
I20260709 12:24:47.736898 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2984) took 2266 ms. Trace:
I20260709 12:24:47.737342 20728 rpcz_store.cc:276] 0709 12:24:45.470034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.393357 (+1923323us) service_pool.cc:224] Handling call
0709 12:24:47.393660 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.736594 (+342934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.736841 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.739604 20727 rpcz_store.cc:276] 0709 12:24:45.468728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.639433 (+1170705us) service_pool.cc:224] Handling call
0709 12:24:46.808408 (+168975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.389715 (+581307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.531135 (+141420us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54003b5454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:47.531382 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311244160}
I20260709 12:24:47.743435 20700 leader_election.cc:400] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22.
I20260709 12:24:47.744009 20700 leader_election.cc:403] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:47.727228 20732 rpcz_store.cc:276] 0709 12:24:44.586173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.580271 (+994098us) service_pool.cc:224] Handling call
0709 12:24:45.580781 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.612547 (+ 31766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.612596 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.643606 (+ 31010us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.222067 (+578461us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:46.222199 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.222914 (+   715us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.593754 (+1370840us) raft_consensus.cc:1685] finished
0709 12:24:47.593785 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.594459 (+   674us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4541,"raft.queue_time_us":2144,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":22883,"thread_start_us":484877,"threads_started":1,"wal-append.queue_time_us":832592}
I20260709 12:24:47.725611 20880 rpcz_store.cc:276] 0709 12:24:45.365102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.826395 (+1461293us) service_pool.cc:224] Handling call
0709 12:24:46.826763 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.388924 (+562161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.389116 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.726027 20804 rpcz_store.cc:276] 0709 12:24:44.829292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.301369 (+472077us) service_pool.cc:224] Handling call
0709 12:24:45.301771 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.124869 (+1823098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.236132 (+111263us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54003c9e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:47.236315 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3404554624}
I20260709 12:24:47.725637 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:47.748799 20876 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:47.535079 28672 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:47.734695 20729 rpcz_store.cc:276] 0709 12:24:44.586899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.629566 (+1042667us) service_pool.cc:224] Handling call
0709 12:24:45.629871 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.866648 (+236777us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:46.237897 (+371249us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.504183 (+1266286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.504256 (+    73us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:47.504338 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.527364 (+ 23026us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540028c154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:47.527428 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.527605 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":216670,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5659,"raft.queue_time_us":4867,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":9277,"spinlock_wait_cycles":543954688,"thread_start_us":1607132,"threads_started":2,"wal-append.queue_time_us":1200488}
W20260709 12:24:47.751704 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 348709us
I20260709 12:24:47.739020 20728 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:24:47.611088 28771 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:47.621541 28802 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:24:47.618770 28804 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:24:47.614008 28810 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:24:47.613257 28806 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:24:47.612351 28651 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:47.780037 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1303) took 3027 ms. Trace:
W20260709 12:24:47.742828 28686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.035s	sys 0.074s
I20260709 12:24:47.724742 20735 rpcz_store.cc:276] 0709 12:24:44.713812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.939377 (+1225565us) service_pool.cc:224] Handling call
0709 12:24:45.939897 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.940519 (+   622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.940567 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.941180 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.255899 (+1314719us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:47.438076 (+182177us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:47.438135 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.438236 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.480679 (+ 42443us) raft_consensus.cc:1685] finished
0709 12:24:47.480705 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.497449 (+ 16744us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1521,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":12930,"spinlock_wait_cycles":280856448,"thread_start_us":1496395,"threads_started":1,"wal-append.queue_time_us":846620}
I20260709 12:24:47.615262 28608 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:47.786162 21433 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23.
I20260709 12:24:47.787725 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:47.788388 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.017s	sys 0.042s
W20260709 12:24:47.822180 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.018s	sys 0.042s
I20260709 12:24:47.620781 27785 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:47.786321 28597 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22.
I20260709 12:24:47.736428 28671 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:47.763856 28776 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:24:47.743517 28794 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.534575 28777 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.808418 20730 rpcz_store.cc:276] 0709 12:24:44.752510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.438326 (+1685816us) service_pool.cc:224] Handling call
0709 12:24:46.461701 (+ 23375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.743141 (+1281440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.779781 (+ 36640us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003b4f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:47.779968 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2352094976}
I20260709 12:24:47.810393 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:47.874104 20875 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:47.759639 28672 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:47.764969 28674 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:47.766052 28496 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:47.818132 20884 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:47.883692 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3099) took 2340 ms. Trace:
I20260709 12:24:47.884079 20884 rpcz_store.cc:276] 0709 12:24:45.543572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.817250 (+2273678us) service_pool.cc:224] Handling call
0709 12:24:47.817653 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.883352 (+ 65699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.883621 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:47.885789 21428 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.027s	sys 0.038s
W20260709 12:24:47.886163 21428 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.027s	sys 0.039s
I20260709 12:24:47.825642 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:47.789175 20805 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 12:24:47.828337 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 303) took 18112 ms (client timeout 15000 ms). Trace:
I20260709 12:24:47.830442 20882 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:47.889213 20879 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:47.822659 20629 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 3 to 14. New cstate: current_term: 14 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.754431 28650 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.752852 28784 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.755741 28627 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.753692 28661 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.840569 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80749.9us
I20260709 12:24:47.893772 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:47.723453 28614 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.742861 28677 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:47.842039 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1902) took 1636 ms (client timeout 1791 ms). Trace:
I20260709 12:24:47.756739 28433 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.845932 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1387) took 1538 ms. Trace:
I20260709 12:24:47.743374 28559 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.829672 28608 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:24:47.857801 28603 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:24:47.863888 28807 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 12:24:47.865644 28632 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:47.867143 28779 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:47.760718 28717 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.801571 28704 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
W20260709 12:24:47.835244 28417 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.026s	sys 0.030s
I20260709 12:24:47.877048 20876 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:47.754245 28796 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.837414 28794 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:47.761176 28616 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.880505 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2969) took 3610 ms. Trace:
I20260709 12:24:47.886027 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1405) took 1523 ms. Trace:
I20260709 12:24:47.936184 20884 rpcz_store.cc:276] 0709 12:24:46.362466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.885149 (+1522683us) service_pool.cc:224] Handling call
0709 12:24:47.885506 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.885681 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.885712 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.885755 (+    43us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:47.885795 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.885850 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.885977 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29568}
I20260709 12:24:47.938165 20884 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:47.780632 28771 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.889597 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3100) took 2345 ms. Trace:
W20260709 12:24:47.892083 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66577.8us
I20260709 12:24:47.957585 20882 rpcz_store.cc:276] 0709 12:24:45.543837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.829638 (+2285801us) service_pool.cc:224] Handling call
0709 12:24:47.830018 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.889359 (+ 59341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.889545 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.959983 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1407) took 1464 ms. Trace:
I20260709 12:24:47.960676 20884 rpcz_store.cc:276] 0709 12:24:46.495161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.937631 (+1442470us) service_pool.cc:224] Handling call
0709 12:24:47.937955 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.959617 (+ 21662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.959905 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.848356 28725 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:47.872853 28795 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:47.898114 21433 rpcz_store.cc:269] 0709 12:24:46.205985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.207779 (+  1794us) service_pool.cc:224] Handling call
0709 12:24:47.841782 (+1634003us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540013d154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:47.841962 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3355884928}
I20260709 12:24:47.871644 28721 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:47.883003 28674 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.901300 21431 rpcz_store.cc:276] 0709 12:24:46.307328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.307494 (+   166us) service_pool.cc:224] Handling call
0709 12:24:46.328529 (+ 21035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.329085 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.329126 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.399444 (+ 70318us) log.cc:844] Serialized 564 byte log entry
0709 12:24:46.779537 (+380093us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:46.779590 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.779701 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.845729 (+1066028us) raft_consensus.cc:1685] finished
0709 12:24:47.845748 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.845876 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":379694,"threads_started":1,"wal-append.queue_time_us":1059847}
I20260709 12:24:47.908115 20700 leader_election.cc:400] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:47.967229 20884 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 12. Preceding OpId from leader: term: 28 index: 13. (index mismatch)
W20260709 12:24:47.967392 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59472.2us
I20260709 12:24:47.969396 20884 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:24:47.881681 28708 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:47.831102 28318 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.972532 28318 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:47.973563 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 274049us
W20260709 12:24:47.887065 20628 rpcz_store.cc:269] 0709 12:24:29.715915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.716380 (+   465us) service_pool.cc:224] Handling call
0709 12:24:47.828241 (+18111861us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:42.889949 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:42.890572 (+   623us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:42.890627 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:24:42.890646 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:42.942692 (+ 52046us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:42.942782 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:42.942811 (+    29us) tablet.cc:685] Acquiring locks for 26 operations
0709 12:24:42.945041 (+  2230us) tablet.cc:701] Row locks acquired
0709 12:24:42.945095 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:24:42.945264 (+   169us) write_op.cc:270] Start()
0709 12:24:42.945411 (+   147us) write_op.cc:276] Timestamp: P: 1783599882945199 usec, L: 0
0709 12:24:42.945432 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:24:43.103764 (+158332us) log.cc:844] Serialized 10298 byte log entry
0709 12:24:44.177404 (+1073640us) op_driver.cc:464] REPLICATION: finished
0709 12:24:44.501809 (+324405us) write_op.cc:301] APPLY: starting
0709 12:24:44.522033 (+ 20224us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:44.683757 (+161724us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:44.683878 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:45.532605 (+848727us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:45.532989 (+   384us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=52,mrs_lookups=0
0709 12:24:45.533037 (+    48us) write_op.cc:312] APPLY: finished
0709 12:24:45.533588 (+   551us) log.cc:844] Serialized 228 byte log entry
0709 12:24:45.894488 (+360900us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.894761 (+   273us) write_op.cc:454] Released schema lock
0709 12:24:45.894852 (+    91us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:45.938011 (+ 43159us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.938060 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":387852,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":246853,"apply.run_cpu_time_us":72800,"apply.run_wall_time_us":1457648,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112807,"num_ops":26,"prepare.queue_time_us":154090,"prepare.run_cpu_time_us":54421,"prepare.run_wall_time_us":483223,"raft.queue_time_us":226802,"raft.run_cpu_time_us":11942,"raft.run_wall_time_us":289664,"replication_time_us":1186405,"thread_start_us":941445,"threads_started":5,"wal-append.queue_time_us":837724,"wal-append.run_cpu_time_us":31936,"wal-append.run_wall_time_us":2941417}]]}
W20260709 12:24:47.915853 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.030s	sys 0.039s
W20260709 12:24:47.975842 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.031s	sys 0.039s
I20260709 12:24:47.927541 28632 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:47.930500 28620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.028s	sys 0.039s
I20260709 12:24:47.898944 28625 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:24:47.923310 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.923844 20883 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:47.980137 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:47.924039 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.025s	sys 0.025s
I20260709 12:24:47.929414 28251 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:47.905300 28597 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:47.896786 28671 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:47.935961 20629 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 8 to 15, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 15 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:47.922749 28417 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.026s	sys 0.031s
W20260709 12:24:47.963009 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.963905 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.000s	sys 0.031s
I20260709 12:24:47.964646 28559 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:47.918017 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.019361 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.032s
I20260709 12:24:47.963335 28779 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.920724 28614 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:47.981047 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.026s	sys 0.025s
I20260709 12:24:47.939172 20876 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:47.934104 20726 rpcz_store.cc:276] 0709 12:24:44.261261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.456702 (+1195441us) service_pool.cc:224] Handling call
0709 12:24:45.479591 (+ 22889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.479901 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.479941 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.507349 (+ 27408us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.225558 (+718209us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:46.289051 (+ 63493us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.289266 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.289337 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.806234 (+516897us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b1800441040. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:46.829190 (+ 22956us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b1800441040. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:47.248968 (+419778us) spinlock_profiling.cc:243] Waited 86.6 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:47.871475 (+622507us) raft_consensus.cc:1685] finished
0709 12:24:47.871507 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.871687 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":73414,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":8944,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":10571,"spinlock_wait_cycles":233301376,"thread_start_us":608824,"threads_started":1,"wal-append.queue_time_us":1266506}
I20260709 12:24:47.954918 28608 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:48.020910 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3173) took 1535 ms. Trace:
I20260709 12:24:48.007138 20804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "761ea542665e419c8a63392e3014efa7"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:24:48.012262 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:48.020079 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.18949e+06us
I20260709 12:24:48.026769 20882 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:24:48.021441 20876 rpcz_store.cc:276] 0709 12:24:46.485292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.937099 (+1451807us) service_pool.cc:224] Handling call
0709 12:24:47.938739 (+  1640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.020553 (+ 81814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.020839 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:48.007023 28620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.029s	sys 0.039s
I20260709 12:24:47.982599 28747 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:47.983129 28287 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.057078 20875 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
W20260709 12:24:48.038542 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.976197 28584 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:48.049551 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.978880 28674 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:48.020299 28607 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.075347 20877 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:48.077198 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.080237 20733 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:24:48.019140 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.020540 28541 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.024746 28625 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:48.033507 28671 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:47.976168 28771 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.027827 28718 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.022373 28608 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:48.075428 20732 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:48.082785 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:48.032258 28597 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:47.976577 28725 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:48.051826 28780 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:48.058897 28708 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:48.117583 28607 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.132175 28613 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.132222 28607 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:48.134963 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1408) took 1012 ms. Trace:
I20260709 12:24:48.135825 28592 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:48.137064 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.138029 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1399) took 2551 ms. Trace:
I20260709 12:24:48.138603 28785 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.139407 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3102) took 1916 ms. Trace:
I20260709 12:24:48.139829 20875 rpcz_store.cc:276] 0709 12:24:46.222818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.882902 (+1660084us) service_pool.cc:224] Handling call
0709 12:24:47.885512 (+  2610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.058230 (+172718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.139136 (+ 80906us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400267d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:48.139345 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394127360}
I20260709 12:24:48.140240 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:48.141253 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:48.137068 28607 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:48.098379 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 339256us
I20260709 12:24:48.110499 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.143829 20735 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:24:48.115293 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.024s	sys 0.037s
W20260709 12:24:48.145104 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.024s	sys 0.038s
I20260709 12:24:48.068456 28795 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.145973 28795 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.146504 28592 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:48.147382 28592 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.148281 28795 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.149485 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:48.150311 28433 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:48.152623 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.156399 28796 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.160372 28661 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:48.163216 28616 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.163861 28616 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:48.120442 20726 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:24:48.165165 28616 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:48.166565 28616 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:48.033063 28779 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.167488 28627 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:48.168684 28779 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.128799 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1482) took 2740 ms. Trace:
I20260709 12:24:48.169286 20805 rpcz_store.cc:276] 0709 12:24:45.387952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.617596 (+229644us) service_pool.cc:224] Handling call
0709 12:24:45.618028 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.886759 (+2268731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.128430 (+241671us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b54003b7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:48.128715 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4992366336}
I20260709 12:24:48.174721 28597 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.083587 28625 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.177620 28625 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:48.115604 28784 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:48.116340 28650 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.179839 28717 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.125147 28608 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:48.143915 28726 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:48.138420 20883 rpcz_store.cc:276] 0709 12:24:45.586680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.837695 (+2251015us) service_pool.cc:224] Handling call
0709 12:24:47.923652 (+ 85957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.980580 (+ 56928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.137847 (+157267us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b540019d7d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:48.137969 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185664256}
I20260709 12:24:48.124284 28725 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:48.135416 20882 rpcz_store.cc:276] 0709 12:24:47.122296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.958717 (+836421us) service_pool.cc:224] Handling call
0709 12:24:47.982253 (+ 23536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.027254 (+ 45001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.134717 (+107463us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b540005a854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:48.134900 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101477760}
I20260709 12:24:48.166754 28251 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:48.030519 28651 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:24:48.181504 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:48.194351 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 446224us
I20260709 12:24:48.113730 20733 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:24:48.199805 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.198328 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:48.201406 20883 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:48.204187 20810 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:48.098618 20728 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.104492 20730 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:48.206758 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:48.206873 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57515.1us
I20260709 12:24:48.200498 20805 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:48.208032 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.208580 20883 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:48.208932 20877 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:24:48.210016 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.210318 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:48.210870 20801 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:48.210935 20807 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:48.193889 28814 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.211896 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:48.212464 20801 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:48.210053 28820 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:24:48.211661 20808 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:48.213903 28796 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:48.214468 28796 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:48.214672 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:48.214859 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.210084 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:48.215662 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.215337 28796 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.216017 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:48.216364 20802 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:48.216603 20808 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:48.216920 20808 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:48.217598 20808 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:48.218483 20808 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:48.212111 28814 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:48.215421 20806 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:48.087218 28815 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.221349 20773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.068s	sys 0.114s
I20260709 12:24:48.099860 28731 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.219826 20807 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:48.232591 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.233346 20809 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:48.223008 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.234555 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.234746 20877 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:48.235201 20875 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:24:48.225111 20773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.068s	sys 0.115s
I20260709 12:24:48.235968 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.232609 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:48.236951 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:48.237846 20807 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:24:48.238435 20805 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:48.239658 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:48.240603 20805 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:48.240650 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.241530 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.241568 20733 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:48.236510 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:48.248013 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.248975 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.230219 21433 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:48.230720 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:48.233887 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:48.253542 21434 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:48.253829 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:48.254635 20807 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:48.254519 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:48.254865 20808 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:24:48.256124 20884 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:48.256904 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:48.265512 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:48.266363 20883 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:48.273470 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:48.221993 28608 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:48.275686 21435 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:24:48.299656 21435 pending_rounds.cc:85] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:24:48.277324 21431 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:48.300109 21435 pending_rounds.cc:107] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:24:48.300853 21435 raft_consensus.cc:2889] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:48.286738 20809 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:48.281776 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:48.281860 28690 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:48.281978 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:48.282845 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.284240 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:48.286408 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:48.303292 20807 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:48.282052 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:48.233700 28757 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:48.304527 28656 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:48.305178 28690 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.305614 28690 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.242972 28777 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.305626 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:48.305876 28663 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:48.306085 28748 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:48.306751 28677 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.306953 28663 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:48.307421 28663 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.307538 28677 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.307894 28663 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:48.307315 28747 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:48.308096 28677 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:48.308780 28747 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:48.254019 28822 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.309597 28822 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.309681 28747 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:48.288444 28704 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:48.289249 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:48.289829 28592 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:48.291006 20806 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: [19.15-21.16]
W20260709 12:24:48.292317 28816 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:24:48.311118 28747 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:48.312837 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.313208 28822 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:48.314459 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.315284 20884 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:24:48.294196 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.296954 28814 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:48.297870 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.298684 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.298804 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.300066 20882 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:48.326649 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.327742 20884 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:24:48.329027 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.302390 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.303267 20877 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:48.314883 28677 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:48.316542 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.317996 20734 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.006s	sys 0.017s
W20260709 12:24:48.358758 20734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.007s	sys 0.017s
W20260709 12:24:48.324215 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.009s	sys 0.015s
W20260709 12:24:48.359946 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.010s	sys 0.015s
W20260709 12:24:48.360682 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 573026us
I20260709 12:24:48.364140 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.364991 20733 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:48.302593 20875 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:48.333921 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.292815 20801 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:48.371485 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.371789 20805 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:48.324792 20726 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:48.373821 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.374961 20726 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:48.329792 20884 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:48.374053 20877 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:24:48.385625 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.387490 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.357405 20728 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:48.244684 28815 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:48.808928 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.026s	sys 0.032s
I20260709 12:24:48.869390 20875 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:24:48.512312 21400 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.032s	sys 0.028s
I20260709 12:24:48.765224 20883 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:48.492422 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:48.712191 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.967849 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.482614 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:48.553771 20776 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.006s	sys 0.006s
W20260709 12:24:48.972324 20776 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.017s	sys 0.021s
W20260709 12:24:48.967896 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.970227 20884 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
W20260709 12:24:48.973338 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.973694 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.680075 21428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.013s	sys 0.010s
W20260709 12:24:48.974241 21428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.085s	user 0.017s	sys 0.018s
W20260709 12:24:48.972123 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.405474 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:48.667114 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.477747 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:48.975894 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:48.475845 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:48.977722 20735 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 12:24:48.490293 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.978770 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.394810 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:48.391433 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.411614 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.980221 20727 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
W20260709 12:24:48.968883 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.029s	sys 0.035s
W20260709 12:24:48.967422 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.982226 20730 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:24:48.982837 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 935569us
I20260709 12:24:48.972186 20806 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:24:48.969600 21400 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.159s	user 0.042s	sys 0.055s
W20260709 12:24:48.985723 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.21295e+06us
I20260709 12:24:48.971509 20809 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:48.967945 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:48.383526 28367 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:24:49.016337 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:48.995525 20805 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:48.305178 28656 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:48.376287 28313 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.025s	sys 0.011s
I20260709 12:24:49.001248 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:49.014191 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.021399 20726 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:24:48.680442 28614 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.011s	sys 0.004s
I20260709 12:24:49.038260 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:49.041090 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:49.044114 20802 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.004s	sys 0.015s
W20260709 12:24:49.045934 20802 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.004s	sys 0.016s
I20260709 12:24:49.064038 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.051172 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:49.056291 20879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.008s	sys 0.045s
W20260709 12:24:49.081773 20879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.008s	sys 0.045s
I20260709 12:24:49.094281 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.095052 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:24:49.095348 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:49.097640 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.098483 20728 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 21 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:49.098423 20809 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:49.099318 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.100170 20728 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
W20260709 12:24:49.038992 28313 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.068s	sys 0.029s
W20260709 12:24:49.031572 28559 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.002s	user 0.026s	sys 0.008s
I20260709 12:24:49.101838 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.099423 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.119980 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.111516 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:49.125249 20730 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:24:49.112854 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.132226 20805 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:49.041373 28614 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.011s	sys 0.006s
I20260709 12:24:49.143343 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:49.058178 28773 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.899s	user 0.011s	sys 0.023s
W20260709 12:24:49.155049 20729 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.013s	sys 0.017s
W20260709 12:24:49.157226 20882 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.020s	sys 0.076s
W20260709 12:24:49.166580 20729 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.013s	sys 0.017s
W20260709 12:24:49.166651 20882 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.021s	sys 0.077s
I20260709 12:24:49.167213 20806 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:24:49.146724 20726 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:49.158248 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:49.168821 20728 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:24:49.171312 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:49.172339 20727 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:49.189321 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:49.190047 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:49.119951 28602 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.592s	user 0.020s	sys 0.016s
W20260709 12:24:49.103055 28651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.023s	sys 0.052s
I20260709 12:24:49.051586 28771 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:49.219055 20849 connection.cc:256] RPC call timeout handler was delayed by -2.6121744339998259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
W20260709 12:24:49.219699 20849 connection.cc:256] RPC call timeout handler was delayed by -2.5752744339997662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2288s for a response
I20260709 12:24:49.195314 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.195767 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.198359 28773 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:49.223265 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:49.224103 20883 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:49.221046 20804 raft_consensus.cc:606] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:24:49.225041 20804 raft_consensus.cc:614] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:49.155000 28796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.011s	sys 0.013s
I20260709 12:24:49.197283 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.226991 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:49.227262 20808 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.967s	user 0.003s	sys 0.046s
W20260709 12:24:49.227733 20808 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.003s	sys 0.046s
W20260709 12:24:49.228588 20849 connection.cc:256] RPC call timeout handler was delayed by -1.9398670860000493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 12:24:49.251545 20849 connection.cc:256] RPC call timeout handler was delayed by -1.740067086000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 12:24:49.220549 28651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.023s	sys 0.052s
I20260709 12:24:49.256042 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.257064 20733 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 16: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:49.256067 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.263399 28651 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:49.263933 28651 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:49.264333 28651 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:49.240211 28771 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:24:49.227864 28796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.011s	sys 0.014s
I20260709 12:24:49.267791 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:49.272333 20881 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:49.272269 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32862 (request call id 8) took 1289 ms. Trace:
I20260709 12:24:49.273476 20804 rpcz_store.cc:276] 0709 12:24:47.979740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.006955 (+ 27215us) service_pool.cc:224] Handling call
0709 12:24:49.268670 (+1261715us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec99853 00007f9e23c6760a 00007f9e1d1ed7d4 00007f9e1d1ed762 00007f9e1d1ed691 00007f9e1d1ed60c 00007f9e1d1ec8c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:49.268796 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2757475712}
W20260709 12:24:49.276016 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.068s	user 0.009s	sys 0.028s
W20260709 12:24:49.276436 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.009s	sys 0.028s
I20260709 12:24:49.172924 28559 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
W20260709 12:24:49.276921 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.0692e+06us
I20260709 12:24:49.281198 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.294097 20804 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:24:49.294608 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:49.301755 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.303366 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.304523 20807 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:24:49.259269 28821 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.021s	sys 0.031s
I20260709 12:24:49.322844 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:49.329584 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.374517 20726 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
W20260709 12:24:49.381983 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.394798 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.285585 28687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.013s	sys 0.038s
I20260709 12:24:49.392366 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1901) took 3175 ms. Trace:
W20260709 12:24:49.321448 28663 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.013s	user 0.019s	sys 0.009s
I20260709 12:24:49.405480 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.314481 28814 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:24:49.315361 28318 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.342s	user 0.020s	sys 0.005s
W20260709 12:24:49.391896 28821 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.021s	sys 0.032s
W20260709 12:24:49.315871 28708 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.178s	user 0.014s	sys 0.005s
I20260709 12:24:49.439952 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:49.408485 21428 rpcz_store.cc:276] 0709 12:24:46.205622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.220557 (+ 14935us) service_pool.cc:224] Handling call
0709 12:24:46.222460 (+  1903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.222668 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:24:46.247100 (+ 24432us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.889425 (+1642325us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.974853 (+1085428us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.380418 (+405565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.380498 (+    80us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:49.380570 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.380671 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.380836 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":793154,"prepare.run_cpu_time_us":3792,"prepare.run_wall_time_us":31161,"raft.queue_time_us":397488,"raft.run_cpu_time_us":575,"raft.run_wall_time_us":66315,"spinlock_wait_cycles":28714752,"thread_start_us":3131842,"threads_started":4,"wal-append.queue_time_us":1483238,"wal-append.run_cpu_time_us":1282,"wal-append.run_wall_time_us":1147456}
I20260709 12:24:49.449769 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.450611 20727 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:49.456341 28607 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:49.456930 28607 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:49.457922 28607 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:49.459939 28834 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:24:49.416666 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:49.267791 28771 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.258236 28656 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.431865 28845 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:49.408377 28687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.013s	sys 0.039s
I20260709 12:24:49.383637 28670 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:49.490098 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:49.493371 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.533s (SENT) [suppressed 7 similar messages]
W20260709 12:24:49.494611 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.508479 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 465837us
I20260709 12:24:49.509148 20805 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:24:49.509462 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:49.468719 28830 raft_consensus.cc:2764] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:24:49.511091 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3268) took 1131 ms. Trace:
I20260709 12:24:49.471414 28708 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:49.525293 20806 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:24:49.525918 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3272) took 1145 ms. Trace:
I20260709 12:24:49.526219 20806 rpcz_store.cc:276] 0709 12:24:48.380857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.495715 (+1114858us) service_pool.cc:224] Handling call
0709 12:24:49.524860 (+ 29145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.525681 (+   821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.525864 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:49.523165 20805 rpcz_store.cc:276] 0709 12:24:48.379691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.494137 (+1114446us) service_pool.cc:224] Handling call
0709 12:24:49.494525 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.509585 (+ 15060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.511019 (+  1434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:49.427983 28663 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:49.532312 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.529767 20805 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:49.481805 28815 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:49.523222 28496 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.632s	user 0.014s	sys 0.000s
I20260709 12:24:49.545035 20806 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 18. (term mismatch)
I20260709 12:24:49.548067 20806 pending_rounds.cc:85] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 17
I20260709 12:24:49.548643 20806 pending_rounds.cc:107] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:24:49.536559 20807 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:24:49.546721 20803 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:24:49.549319 20806 raft_consensus.cc:2889] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.492597 28670 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.490866 28687 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:49.551070 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:49.550186 28603 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.636s	user 0.008s	sys 0.005s
I20260709 12:24:49.547317 20726 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 12:24:49.524242 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:49.568102 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 498018us
W20260709 12:24:49.568841 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.570654 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.562170 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 459488us
I20260709 12:24:49.559677 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:49.573176 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.531330 28845 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:24:49.576933 28845 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:49.566972 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:49.578619 21429 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
W20260709 12:24:49.574091 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.020s	sys 0.026s
I20260709 12:24:49.552047 28616 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.549777 28838 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:49.580806 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.020s	sys 0.026s
I20260709 12:24:49.551566 28773 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:49.582878 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:49.579406 28708 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:24:49.587770 28708 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:49.588157 28708 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:49.588503 28708 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:49.588928 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:49.591427 28796 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:49.592731 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2985) took 3190 ms. Trace:
I20260709 12:24:49.593195 20734 rpcz_store.cc:276] 0709 12:24:46.401982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.727673 (+1325691us) service_pool.cc:224] Handling call
0709 12:24:47.727994 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.752635 (+ 24641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.752683 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.753301 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.359393 (+606092us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:48.359454 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.359584 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.592478 (+1232894us) raft_consensus.cc:1685] finished
0709 12:24:49.592504 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.592662 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41151,"raft.queue_time_us":30617,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":548554,"threads_started":1,"wal-append.queue_time_us":1733150}
W20260709 12:24:49.595783 28833 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:49.597208 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3095) took 4098 ms. Trace:
I20260709 12:24:49.597640 20880 rpcz_store.cc:276] 0709 12:24:45.498783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.820231 (+2321448us) service_pool.cc:224] Handling call
0709 12:24:47.834283 (+ 14052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.841153 (+  6870us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:47.841185 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.846923 (+  5738us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.356903 (+509980us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:48.356968 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.357064 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.596972 (+1239908us) raft_consensus.cc:1685] finished
0709 12:24:49.596999 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.597140 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99869,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":38469,"thread_start_us":497250,"threads_started":1,"wal-append.queue_time_us":1583947}
I20260709 12:24:49.579125 28687 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.579559 28726 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.605598 28726 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:49.595817 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3175) took 3102 ms (client timeout 1618 ms). Trace:
I20260709 12:24:49.586310 28318 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:49.580331 28616 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:49.584226 28838 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:49.641855 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.644405 20810 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
W20260709 12:24:49.607100 20881 rpcz_store.cc:269] 0709 12:24:46.492973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.639335 (+146362us) service_pool.cc:224] Handling call
0709 12:24:49.595533 (+2956198us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b54000965d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:49.595705 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4936854912}
I20260709 12:24:49.641104 20878 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:24:49.580605 28815 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
W20260709 12:24:49.648653 28837 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.083s	user 0.000s	sys 0.003s
I20260709 12:24:49.642591 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:49.678844 21430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.012s	sys 0.024s
I20260709 12:24:49.646463 28496 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:49.681586 20882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.013s	sys 0.023s
W20260709 12:24:49.667330 28672 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.772s	user 0.017s	sys 0.000s
W20260709 12:24:49.682039 21430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.012s	sys 0.024s
I20260709 12:24:49.696972 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1912) took 1415 ms. Trace:
I20260709 12:24:49.641348 28843 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:49.691520 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:24:49.704046 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:49.705176 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:49.668449 28663 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:49.691506 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:49.692601 20882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.014s	sys 0.023s
I20260709 12:24:49.666291 28318 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:49.697482 21429 rpcz_store.cc:276] 0709 12:24:48.281657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.281782 (+   125us) service_pool.cc:224] Handling call
0709 12:24:49.696801 (+1415019us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:49.696916 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2931088256}
W20260709 12:24:49.675872 28841 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.198s	user 0.000s	sys 0.003s
W20260709 12:24:49.700757 20810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3028) took 1496 ms (client timeout 1535 ms). Trace:
I20260709 12:24:49.755084 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3020) took 1553 ms. Trace:
I20260709 12:24:49.758637 20880 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:49.709470 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:49.763643 20810 rpcz_store.cc:269] 0709 12:24:48.204325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.206543 (+  2218us) service_pool.cc:224] Handling call
0709 12:24:49.700547 (+1494004us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003d6dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:49.700672 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3063968000}
I20260709 12:24:49.764497 20881 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:49.763129 20803 rpcz_store.cc:276] 0709 12:24:48.201774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.201890 (+   116us) service_pool.cc:224] Handling call
0709 12:24:49.754782 (+1552892us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003a0f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:49.754995 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3059220352}
I20260709 12:24:49.695708 28796 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:49.785895 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1416) took 1477 ms. Trace:
I20260709 12:24:49.788949 20882 rpcz_store.cc:276] 0709 12:24:48.308894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.482919 (+174025us) service_pool.cc:224] Handling call
0709 12:24:48.510329 (+ 27410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.510639 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:49.167384 (+656745us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.784264 (+616880us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.785461 (+  1197us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:49.785558 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.785650 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.785811 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":969594,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":21622,"thread_start_us":1251419,"threads_started":2}
I20260709 12:24:49.696183 28496 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:49.793865 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:49.765393 28367 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:24:49.756870 28318 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:49.800173 20880 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:49.785843 28824 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.806949 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1406) took 3402 ms. Trace:
I20260709 12:24:49.811643 20879 rpcz_store.cc:276] 0709 12:24:46.405068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.890138 (+1485070us) service_pool.cc:224] Handling call
0709 12:24:47.890556 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.175605 (+285049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.175638 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.176144 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.119250 (+943106us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:49.119305 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.199050 (+ 79745us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:49.199124 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.590750 (+391626us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:49.806715 (+215965us) raft_consensus.cc:1685] finished
0709 12:24:49.806746 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.806885 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1613,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":660801152,"thread_start_us":905668,"threads_started":1,"wal-append.queue_time_us":1351294}
I20260709 12:24:49.707818 28672 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.791383 28796 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.799343 28496 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:49.800853 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 506289us
W20260709 12:24:49.834303 20729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.022s	sys 0.010s
W20260709 12:24:49.834767 20729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.022s	sys 0.010s
W20260709 12:24:49.808851 28614 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.006s	sys 0.022s
I20260709 12:24:49.840812 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:49.848805 28849 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.849397 28849 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:49.844265 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2993) took 1769 ms. Trace:
I20260709 12:24:49.844864 20732 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:49.809983 28815 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.854822 20729 rpcz_store.cc:276] 0709 12:24:48.074453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.074598 (+   145us) service_pool.cc:224] Handling call
0709 12:24:48.098522 (+ 23924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.098809 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:49.188634 (+1089825us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.843920 (+655286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.843971 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:49.844018 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.844086 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.844205 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8673,"prepare.queue_time_us":1350077,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"thread_start_us":1604576,"threads_started":2}
W20260709 12:24:49.838929 28671 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.735s	user 0.012s	sys 0.001s
I20260709 12:24:49.867304 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:49.835795 28796 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:49.845363 28614 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.006s	sys 0.023s
W20260709 12:24:49.894989 28651 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.630s	user 0.013s	sys 0.000s
I20260709 12:24:49.895709 28651 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.896464 28670 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:49.869369 28851 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:49.908049 28838 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:49.910414 28651 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:49.918529 28851 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:49.911626 28849 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:49.919941 28851 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:49.923840 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1417) took 1611 ms. Trace:
I20260709 12:24:49.924337 20875 rpcz_store.cc:276] 0709 12:24:48.312502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.970282 (+657780us) service_pool.cc:224] Handling call
0709 12:24:48.970639 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:48.972340 (+  1701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.972373 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:49.008227 (+ 35854us) log.cc:844] Serialized 52 byte log entry
0709 12:24:49.543655 (+535428us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:49.543697 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.543772 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.923561 (+379789us) raft_consensus.cc:1685] finished
0709 12:24:49.923589 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.923765 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1214,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":39142,"thread_start_us":382318,"threads_started":1,"wal-append.queue_time_us":753375}
W20260709 12:24:49.938330 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.941628 28726 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:49.943730 28846 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.290s	user 0.000s	sys 0.002s
W20260709 12:24:49.945060 28794 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.981s	user 0.013s	sys 0.012s
I20260709 12:24:49.945717 28794 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:49.947661 28771 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:49.948424 28771 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:49.955358 28625 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.777s	user 0.014s	sys 0.011s
I20260709 12:24:49.956050 28625 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:49.959316 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.959916 28747 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.646s	user 0.010s	sys 0.004s
I20260709 12:24:49.964427 28687 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:49.966068 28690 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.591s	user 0.010s	sys 0.005s
I20260709 12:24:49.966789 28690 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.970062 28690 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:49.972014 28674 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.859s	user 0.020s	sys 0.016s
I20260709 12:24:49.972514 28674 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:49.974670 28708 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.877099 28815 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:49.975522 20809 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:24:49.977149 28708 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:49.977766 28608 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.664s	user 0.022s	sys 0.016s
I20260709 12:24:49.977873 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3267) took 1598 ms. Trace:
I20260709 12:24:49.833164 28824 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:49.978483 20809 rpcz_store.cc:276] 0709 12:24:48.379326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.489857 (+1110531us) service_pool.cc:224] Handling call
0709 12:24:49.522692 (+ 32835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.976116 (+453424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.977503 (+  1387us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400127854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:49.977794 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040500480}
I20260709 12:24:49.985774 20735 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:49.989265 20801 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19.
I20260709 12:24:49.990504 20731 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:49.991645 20876 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:24:49.992172 20876 pending_rounds.cc:85] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 10
I20260709 12:24:49.992444 20876 pending_rounds.cc:107] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:24:49.992717 20876 raft_consensus.cc:2889] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.993084 20876 pending_rounds.cc:107] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:24:49.993348 20876 raft_consensus.cc:2889] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.978643 28608 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:49.880820 28313 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.974947 28674 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:50.012126 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.019s	sys 0.043s
W20260709 12:24:50.012809 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.019s	sys 0.043s
W20260709 12:24:50.014707 20629 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.015s	sys 0.034s
W20260709 12:24:50.020434 20629 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.253s	user 0.016s	sys 0.035s
W20260709 12:24:50.060210 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.994059 28814 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:50.000283 28367 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:24:49.989934 28663 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:50.030181 20808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.024s	sys 0.039s
I20260709 12:24:49.977164 28815 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:50.082285 20808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.025s	sys 0.041s
W20260709 12:24:50.071485 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.027s	sys 0.025s
W20260709 12:24:50.077152 20877 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.020s	sys 0.033s
W20260709 12:24:50.083593 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.028s	sys 0.026s
W20260709 12:24:50.083705 20877 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.020s	sys 0.033s
I20260709 12:24:49.999598 28625 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:50.092607 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3113) took 1902 ms. Trace:
W20260709 12:24:50.090526 20884 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.001s	sys 0.027s
W20260709 12:24:50.093421 20884 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.001s	sys 0.027s
W20260709 12:24:50.090942 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3264) took 1714 ms (client timeout 1792 ms). Trace:
W20260709 12:24:50.067927 28821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.015s	sys 0.048s
I20260709 12:24:50.093139 20876 rpcz_store.cc:276] 0709 12:24:48.190059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.199206 (+  9147us) service_pool.cc:224] Handling call
0709 12:24:48.199509 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.993865 (+1794356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.092344 (+ 98479us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54001d01d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:50.092538 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4120423680}
I20260709 12:24:50.010289 28608 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:50.096390 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.096910 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:50.093979 20801 rpcz_store.cc:269] 0709 12:24:48.376486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.383003 (+  6517us) service_pool.cc:224] Handling call
0709 12:24:50.090698 (+1707695us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540006ccd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:50.090874 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3679152512}
I20260709 12:24:50.091800 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 2987) took 3689 ms. Trace:
I20260709 12:24:50.098223 20731 rpcz_store.cc:276] 0709 12:24:46.402605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.734195 (+1331590us) service_pool.cc:224] Handling call
0709 12:24:47.782810 (+ 48615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.991007 (+2208197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.091500 (+100493us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b54000e6ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:50.091728 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5076534016}
I20260709 12:24:50.098635 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.099315 21430 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:50.101253 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.102272 21430 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:50.083421 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1495) took 1835 ms. Trace:
I20260709 12:24:49.983166 28773 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.102s
I20260709 12:24:50.110540 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:50.111738 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:50.107318 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.001s	sys 0.026s
W20260709 12:24:50.108163 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.043s
W20260709 12:24:50.113735 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.001s	sys 0.026s
W20260709 12:24:50.113814 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.000s	sys 0.043s
I20260709 12:24:50.114650 28607 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:50.113566 28672 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:50.115571 28671 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.115657 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:50.116472 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 590592us
I20260709 12:24:50.116281 28671 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.118546 28671 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:50.108466 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.029s
W20260709 12:24:50.119402 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.012s	sys 0.029s
W20260709 12:24:50.119830 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 546014us
I20260709 12:24:50.111078 20808 rpcz_store.cc:276] 0709 12:24:48.248115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.255656 (+  7541us) service_pool.cc:224] Handling call
0709 12:24:48.256016 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.256268 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:49.250602 (+994334us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.082917 (+832315us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:50.082997 (+    80us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:50.083064 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.083173 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.083344 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1203530,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":99,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":1802386,"threads_started":2}
I20260709 12:24:50.121155 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:50.121534 28827 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:50.123072 28672 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:50.126061 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.085158 28687 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.127867 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:50.136569 20732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3003) took 1839 ms (client timeout 2044 ms). Trace:
W20260709 12:24:50.105715 28821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.015s	sys 0.049s
I20260709 12:24:50.002902 28313 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:50.137065 20732 rpcz_store.cc:269] 0709 12:24:48.298230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.298385 (+   155us) service_pool.cc:224] Handling call
0709 12:24:50.136339 (+1837954us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002274d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:50.136497 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3493628288}
I20260709 12:24:50.137830 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.171873 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
W20260709 12:24:50.172228 20698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.018s	sys 0.029s
I20260709 12:24:50.166689 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3101) took 4564 ms. Trace:
W20260709 12:24:50.172580 20698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.018s	sys 0.029s
I20260709 12:24:50.172724 20878 rpcz_store.cc:276] 0709 12:24:45.560933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.834589 (+2273656us) service_pool.cc:224] Handling call
0709 12:24:49.602157 (+1767568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.672204 (+ 70047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.124845 (+452641us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54000965d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:50.124974 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4064001920}
I20260709 12:24:50.118938 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:50.167914 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.004s	sys 0.029s
W20260709 12:24:50.174085 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.004s	sys 0.029s
W20260709 12:24:50.174273 20850 leader_election.cc:336] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.815s (SENT)
W20260709 12:24:50.168944 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53366.6us
I20260709 12:24:50.169130 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:50.175104 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.175783 20735 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:50.176862 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.178570 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:50.179204 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.180068 21428 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:24:50.181833 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.019s	sys 0.013s
W20260709 12:24:50.182199 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.019s	sys 0.014s
I20260709 12:24:50.186352 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:50.086109 28814 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.187158 20731 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:50.094866 28663 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:50.189893 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.014s	sys 0.033s
W20260709 12:24:50.227342 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.015s	sys 0.033s
W20260709 12:24:50.227746 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 638826us
W20260709 12:24:50.200759 20776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.012s
I20260709 12:24:50.130522 28672 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:50.224730 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:50.230573 20776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.013s	sys 0.013s
I20260709 12:24:50.235071 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:50.229008 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:50.237985 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:50.241534 28708 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:24:50.260108 28855 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:24:50.170528 28687 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.242044 28625 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.241856 28815 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:50.275411 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.187233 28747 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:24:50.249172 28614 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:50.261469 28616 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.223430 28726 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:24:50.254153 28670 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:24:50.248620 28849 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:24:50.258590 28773 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:24:50.278923 28794 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.282811 28614 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.278625 28625 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.268309 28608 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.283763 28870 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:24:50.341662 28614 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:50.368053 28857 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
W20260709 12:24:50.369120 28859 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 12:24:50.369949 28860 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:24:50.374598 28862 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:24:50.377279 28872 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:50.427448 28872 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:50.428923 28872 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:50.378077 28858 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:24:50.302438 28866 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:24:50.352293 28824 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:50.406809 28864 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:24:50.434437 28625 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.436261 28687 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.439312 28814 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:50.444267 28771 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.437497 28851 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:50.431124 28872 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:50.174547 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 875831us
I20260709 12:24:50.500903 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:50.504915 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:50.514211 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 395089us
I20260709 12:24:50.522285 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:50.533510 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:50.534549 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:50.535399 20733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.018s	sys 0.036s
I20260709 12:24:50.536075 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:50.536428 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:50.537364 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.016s	sys 0.019s
W20260709 12:24:50.538108 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.008s	sys 0.020s
W20260709 12:24:50.538371 20880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.024s	sys 0.044s
W20260709 12:24:50.538872 20882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.020s
I20260709 12:24:50.540457 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3127) took 1267 ms. Trace:
W20260709 12:24:50.541116 20850 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.831s (SENT)
W20260709 12:24:50.545770 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 304954us
W20260709 12:24:50.546707 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 306064us
I20260709 12:24:50.548038 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3031) took 2342 ms. Trace:
W20260709 12:24:50.555701 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 314673us
I20260709 12:24:50.555135 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:50.557292 21429 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:24:50.558677 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:50.559404 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:50.559686 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:50.559895 20881 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:50.565553 20875 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:24:50.566893 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.568707 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:50.594358 28559 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:50.598920 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.617957 28313 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.422487 28663 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.393138 28671 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:50.312963 28672 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:50.635002 20880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.025s	sys 0.045s
I20260709 12:24:50.667588 28607 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.668505 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3132) took 1125 ms. Trace:
I20260709 12:24:50.669198 20880 rpcz_store.cc:276] 0709 12:24:49.543172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.830705 (+287533us) service_pool.cc:224] Handling call
0709 12:24:49.846209 (+ 15504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.846578 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.846616 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.868676 (+ 22060us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.982600 (+113924us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b4800240258. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:24:50.667838 (+685238us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:50.667895 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.668005 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.668078 (+    73us) raft_consensus.cc:1685] finished
0709 12:24:50.668088 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.668418 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3440,"raft.run_cpu_time_us":3735,"raft.run_wall_time_us":132862,"spinlock_wait_cycles":11233536,"thread_start_us":798608,"threads_started":1,"wal-append.queue_time_us":564279}
W20260709 12:24:50.635505 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.017s	sys 0.019s
W20260709 12:24:50.671571 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 977130us
I20260709 12:24:50.671818 28671 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:50.636008 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.009s	sys 0.021s
I20260709 12:24:50.675084 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3129) took 1262 ms. Trace:
W20260709 12:24:50.675500 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 696117us
I20260709 12:24:50.675588 28876 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.625454 20802 rpcz_store.cc:276] 0709 12:24:48.205046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.216932 (+ 11886us) service_pool.cc:224] Handling call
0709 12:24:48.217230 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.217600 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:49.047877 (+830277us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.689513 (+641636us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.690202 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.690397 (+   195us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:49.699767 (+  9370us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.188453 (+488686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.188561 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.547368 (+358807us) raft_consensus.cc:1685] finished
0709 12:24:50.547411 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.547704 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1242164,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":74,"raft.queue_time_us":354055,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":129291,"thread_start_us":1794875,"threads_started":3,"wal-append.queue_time_us":954915}
I20260709 12:24:50.676657 28876 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.680979 20731 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:24:50.681421 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:50.675614 20884 rpcz_store.cc:276] 0709 12:24:49.412201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.412445 (+   244us) service_pool.cc:224] Handling call
0709 12:24:49.412852 (+   407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:50.093883 (+681031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.093916 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:50.118889 (+ 24973us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:24:50.119440 (+   551us) log.cc:844] Serialized 52 byte log entry
0709 12:24:50.542794 (+423354us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:50.542899 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.543079 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.674506 (+131427us) raft_consensus.cc:1685] finished
0709 12:24:50.674531 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.674846 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127655,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3912,"spinlock_wait_cycles":54055808,"thread_start_us":1102794,"threads_started":2,"wal-append.queue_time_us":536158}
I20260709 12:24:50.683285 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:50.684098 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.684664 20808 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:50.633277 20733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.019s	sys 0.036s
W20260709 12:24:50.632263 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 91460us
W20260709 12:24:50.635496 20882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.000s	sys 0.021s
I20260709 12:24:50.640213 20734 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:50.640789 28851 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:24:50.687628 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.641255 20732 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:50.640905 28854 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:24:50.455008 28796 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.002s
I20260709 12:24:50.620265 28625 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:24:50.620685 28687 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.018s
W20260709 12:24:50.640228 20877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.000s	sys 0.012s
I20260709 12:24:50.651831 21430 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:50.652886 20878 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:24:50.656412 20735 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:50.656651 20729 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
W20260709 12:24:50.661471 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106839us
I20260709 12:24:50.663758 28690 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:50.620779 28602 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:24:50.674235 28671 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.694409 28877 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.698063 28845 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:50.699442 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3278) took 1403 ms. Trace:
I20260709 12:24:50.700340 28597 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:50.700574 28690 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:50.692981 28559 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.713529 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:50.624660 20883 rpcz_store.cc:276] 0709 12:24:49.272767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.272893 (+   126us) service_pool.cc:224] Handling call
0709 12:24:49.294675 (+ 21782us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:49.296637 (+  1962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.296675 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:49.309781 (+ 13106us) log.cc:844] Serialized 52 byte log entry
0709 12:24:49.588253 (+278472us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b18003939a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:24:50.182587 (+594334us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:50.182632 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.182718 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.539361 (+356643us) raft_consensus.cc:1685] finished
0709 12:24:50.539446 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.540075 (+   629us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1481,"raft.queue_time_us":115161,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":23102,"spinlock_wait_cycles":21600128,"thread_start_us":594122,"threads_started":1,"wal-append.queue_time_us":1019624}
I20260709 12:24:50.695350 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.703495 28876 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.704092 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:50.704970 28747 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:24:50.705366 20850 leader_election.cc:336] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.917s (SENT)
I20260709 12:24:50.731433 28877 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.733939 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:50.734591 28670 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:50.706162 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3131) took 1178 ms. Trace:
I20260709 12:24:50.735580 28656 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:50.707481 28796 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.683311 28773 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:50.662582 28313 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:24:50.710644 20733 rpcz_store.cc:276] 0709 12:24:49.295667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.296065 (+   398us) service_pool.cc:224] Handling call
0709 12:24:49.327048 (+ 30983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.755252 (+428204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.755442 (+   190us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.756076 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.698819 (+942743us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:50.698897 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.699028 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.699094 (+    66us) raft_consensus.cc:1685] finished
0709 12:24:50.699108 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.699343 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":255906,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":28782,"thread_start_us":1195075,"threads_started":2,"wal-append.queue_time_us":639525}
I20260709 12:24:50.741729 28602 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:50.745282 28690 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.662321 28875 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:50.710287 20877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.000s	sys 0.012s
I20260709 12:24:50.736038 28656 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:50.702335 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.727763 20802 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:24:50.729113 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:50.727771 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:50.730140 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:50.754742 20802 pending_rounds.cc:85] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 12
W20260709 12:24:50.759294 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52608us
I20260709 12:24:50.735980 20882 rpcz_store.cc:276] 0709 12:24:49.528047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.808683 (+280636us) service_pool.cc:224] Handling call
0709 12:24:49.843307 (+ 34624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.843857 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.843886 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.844368 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.705612 (+861244us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:50.705693 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.705826 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.705884 (+    58us) raft_consensus.cc:1685] finished
0709 12:24:50.705898 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.706081 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":23793,"thread_start_us":686821,"threads_started":1,"wal-append.queue_time_us":554021}
I20260709 12:24:50.768934 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3125) took 1627 ms. Trace:
I20260709 12:24:50.769788 20883 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:50.771348 20875 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:50.771925 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.773255 20880 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 15.
W20260709 12:24:50.774077 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 252004us
I20260709 12:24:50.779524 28867 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.784173 28656 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.786005 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.769876 20877 rpcz_store.cc:276] 0709 12:24:49.141457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.141631 (+   174us) service_pool.cc:224] Handling call
0709 12:24:49.212304 (+ 70673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.212563 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:50.084404 (+871841us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.768041 (+683637us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:50.768202 (+   161us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:50.768306 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.768503 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.768832 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":315103,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":245,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":246,"thread_start_us":1526148,"threads_started":2,"wal-append.queue_time_us":591531}
I20260709 12:24:50.793620 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.741667 28773 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.758536 20802 pending_rounds.cc:107] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:24:50.796041 28559 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.774748 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:50.800457 20802 raft_consensus.cc:2889] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:50.798247 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:50.771401 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67454.4us
I20260709 12:24:50.786075 28656 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:50.816182 20733 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:50.819936 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:50.824139 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.801302 28796 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:50.829208 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 167003us
W20260709 12:24:50.830224 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 100381us
I20260709 12:24:50.830504 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:50.807672 28747 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:50.816195 28625 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:50.825981 28602 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:50.841084 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:50.835028 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:50.829226 28608 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.859807 28849 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:50.861311 28671 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:50.868285 20880 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:50.867506 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:50.858668 20882 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:50.875836 28824 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.850214 28625 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.901458 28625 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:50.876646 28824 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.906651 28876 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.907133 28824 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:50.883522 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149870us
I20260709 12:24:50.907663 28876 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:50.908893 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:50.909749 28845 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:50.910599 28876 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:50.910449 28845 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:50.911269 28876 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:50.912211 28877 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:50.912724 28845 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:50.913651 28896 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.228s	user 0.002s	sys 0.000s
W20260709 12:24:50.949681 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74261us
W20260709 12:24:50.898854 28318 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.087s	user 0.011s	sys 0.013s
I20260709 12:24:50.911940 28876 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:50.914111 20850 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.231s (SENT)
I20260709 12:24:50.885334 28773 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:50.918979 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4254624100003639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
I20260709 12:24:50.919081 28674 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.919142 28870 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:50.919847 20808 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:50.922732 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.924650 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:50.925928 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:50.927054 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:51.039929 20880 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:50.927492 28893 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.927814 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:50.927942 28873 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.932097 28608 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.933422 28865 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.074915 28865 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.076031 28608 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:50.936465 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:51.079809 20733 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
W20260709 12:24:51.079947 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 226230us
I20260709 12:24:51.081194 28852 raft_consensus.cc:2764] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
W20260709 12:24:51.082779 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 247980us
I20260709 12:24:51.083343 28674 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.937188 28726 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:50.942152 20801 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:51.084398 28726 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:50.942189 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.085743 28674 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:51.086217 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56410.5us
I20260709 12:24:50.943847 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:51.087765 28879 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:51.089118 28879 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:51.089530 28879 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:50.892196 20877 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:50.912935 28877 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:50.946827 20883 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
W20260709 12:24:50.949764 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48517.7us
I20260709 12:24:51.095140 28877 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:50.892993 28367 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:51.100016 28367 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:24:51.101260 28367 ts_tablet_manager.cc:1403] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 7.946s	user 0.094s	sys 0.078s
I20260709 12:24:51.101543 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:50.950543 28597 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:51.040158 20875 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:24:51.045663 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:24:51.045928 20700 connection.cc:256] RPC call timeout handler was delayed by -1.1149624100003166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 12:24:51.050871 28899 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:24:51.050678 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:51.051322 28496 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.134s	user 0.002s	sys 0.025s
I20260709 12:24:51.051431 28881 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:51.112298 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.056465 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.057363 28747 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.064299 20880 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 22 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:24:51.065094 21429 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:51.067384 21428 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:51.067924 28873 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.068856 28902 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:51.083943 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:50.981650 28318 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:24:51.115314 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.122629 20882 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:51.109051 28849 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:51.127889 28625 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.106264 28865 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.133515 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.120708 28877 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.090817 28879 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.180771 28903 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
W20260709 12:24:51.182289 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.182562 28866 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.183312 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 274547us
W20260709 12:24:51.184501 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 154807us
W20260709 12:24:51.187759 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 273659us
I20260709 12:24:51.129670 28893 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.189996 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.190769 28608 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:51.192502 28367 raft_consensus.cc:359] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:51.194057 28881 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:51.203496 28913 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:51.205401 28559 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:51.206616 28672 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:51.170197 28773 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:24:51.230015 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 128568us
I20260709 12:24:51.235060 28672 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.238144 28672 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.239543 28873 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.121923 28870 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:51.240603 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.225169 28875 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:51.242156 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:51.243762 20884 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:24:51.223897 28367 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:24:51.245136 21428 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:24:51.245016 28367 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:51.246191 28771 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:51.122310 28496 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.220000 28608 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:51.225721 28881 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.247846 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:51.256893 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.233476 28845 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:51.264479 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:51.251392 28913 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:51.263105 21428 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:24:51.254058 28608 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:51.266871 28318 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.267648 28866 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.269961 28866 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:51.271263 28614 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.565s	user 0.018s	sys 0.001s
I20260709 12:24:51.256129 28881 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:51.278956 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.279666 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.283744 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:51.275626 20776 consensus_peers.cc:597] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:51.265430 28690 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:51.287086 21435 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:51.286993 28879 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.288055 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.287093 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.289671 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.291033 21432 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:24:51.292266 28849 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:51.272540 28773 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.299644 28872 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.844s	user 0.004s	sys 0.011s
I20260709 12:24:51.300172 28773 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.301592 28913 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.302632 28773 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.266885 28876 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.299541 21431 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:51.303414 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 136528us
I20260709 12:24:51.295769 28367 ts_tablet_manager.cc:1434] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.192s	user 0.008s	sys 0.003s
I20260709 12:24:51.304863 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.306072 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:51.311592 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 227827us
W20260709 12:24:51.313110 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.315263 28771 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.316001 28771 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.286461 28866 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:51.317215 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:51.296885 28687 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:51.320791 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.294502 28690 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.274533 28875 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:24:51.322147 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.306362 28873 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:51.315023 28726 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.273855 28614 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.326294 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:51.326843 28870 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:51.329434 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.337617 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.338093 20875 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:24:51.307368 28849 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:51.324276 28559 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:51.333377 20878 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:51.342176 28559 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.339640 20879 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:51.342594 28559 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.341081 28865 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:51.343442 28865 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.344810 28614 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:51.347539 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:51.348085 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94062.1us
I20260709 12:24:51.349997 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.350878 21435 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:24:51.351830 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.353765 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.354161 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.350818 28687 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 12:24:51.356709 28496 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.353554 28608 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.360800 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:51.362499 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:51.285851 28877 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:24:51.365015 21431 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:51.369573 21435 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:51.402462 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.406029 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.333525 28845 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:51.418358 28845 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.418799 28845 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.419752 20880 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:51.422165 21431 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:24:51.422549 21431 pending_rounds.cc:85] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:24:51.422765 21431 pending_rounds.cc:107] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:24:51.422983 21431 raft_consensus.cc:2889] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:51.423425 28870 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:51.423849 28870 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:51.333706 28690 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.416915 28608 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.417263 28814 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:51.417925 28602 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.003s
I20260709 12:24:51.416451 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.424703 21434 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 12:24:51.338341 28625 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.445709 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.447201 21428 raft_consensus.cc:2410] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 18.
I20260709 12:24:51.448810 21427 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:51.426370 28870 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:51.463372 21429 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:24:51.452699 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:51.466320 20880 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:51.398449 28876 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:24:51.439802 28602 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:51.471825 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.446192 28873 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:24:51.473886 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 160406us
I20260709 12:24:51.429519 28318 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:24:51.486680 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.428632 28851 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
W20260709 12:24:51.481756 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 155563us
I20260709 12:24:51.484129 28625 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:51.492450 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 128857us
I20260709 12:24:51.493470 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:51.499070 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93176.7us
I20260709 12:24:51.484743 28496 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:51.502588 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.507261 28915 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:24:51.507493 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.508591 21428 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:24:51.509120 28845 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.511229 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:51.511552 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68291.3us
I20260709 12:24:51.512303 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:51.512434 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:51.513296 20893 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-1047135f71834dbab6b869eb3c081b81 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:24:51.513921 20893 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-1047135f71834dbab6b869eb3c081b81 on tablet 1047135f71834dbab6b869eb3c081b81 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.514137 28747 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:51.514619 28747 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.515796 28872 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:51.516760 28872 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:51.517715 28656 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:51.518091 28656 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:51.519209 28656 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.520538 28656 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:51.515388 28687 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:51.522056 28687 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.522398 28687 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:51.517104 28918 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:24:51.515000 28747 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:51.497164 28625 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.544776 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.546859 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.547446 20883 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:24:51.547626 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:51.546283 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.549834 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.552703 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.555486 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.556226 20878 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:51.561012 20877 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:51.562196 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:51.562987 28674 raft_consensus.cc:2955] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Committing config change with OpId 22.17: config changed from index 15 to 17, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:24:51.567591 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.562924 20875 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.568694 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:51.551098 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.567826 28496 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:51.573571 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.539951 28625 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:51.580786 20882 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:51.583947 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:51.585673 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96780.2us
I20260709 12:24:51.588595 20876 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:51.574628 28496 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.590904 21428 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:51.591096 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.578375 28915 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:51.569701 28845 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.605569 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:51.586982 28602 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:51.587816 28814 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:51.613852 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.613729 28602 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.622738 28602 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:51.606070 28902 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:24:51.622000 28917 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:51.616619 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.643462 28879 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:51.643844 28917 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:51.644896 28917 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.645831 28670 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:51.646250 28670 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:51.647092 28559 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.647146 28670 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.642087 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:51.626924 28771 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:51.700949 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.726531 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:51.727474 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.728067 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:51.732678 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:51.728664 21428 raft_consensus.cc:2376] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27. Re-sending same reply.
I20260709 12:24:51.658871 28876 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.734777 20850 leader_election.cc:336] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.584s (SENT)
W20260709 12:24:51.736367 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 199091us
I20260709 12:24:51.708261 28845 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.742858 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:51.750664 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:51.743151 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.751354 20876 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.751981 21432 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:24:51.731739 28851 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.744432 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:51.758576 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 171261us
I20260709 12:24:51.760401 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.709884 28881 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.762706 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:51.768586 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:51.754335 28879 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:51.720391 28849 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.765853 28909 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:51.773217 28821 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:51.774111 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:51.778334 28747 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.784500 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.743920 28672 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:24:51.784888 28851 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.771178 28877 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:51.754113 28845 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.798056 28865 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:51.804347 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:51.804459 28936 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:24:51.818327 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.810323 20876 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:51.811426 28877 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.783162 28909 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:51.817399 28845 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:51.820238 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.824720 21429 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.823463 21433 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:24:51.786172 28821 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:51.766203 28876 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.823096 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:51.841220 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 181556us
I20260709 12:24:51.842154 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:51.843801 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.844374 28614 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:51.845012 28614 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:51.845135 28879 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:51.845654 28614 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:51.844156 28870 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:51.832561 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.823863 28877 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:51.848706 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.826638 28814 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:51.851548 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 89460.5us
I20260709 12:24:51.851686 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:51.851872 28879 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:51.852877 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:51.854735 28876 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.827016 28909 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.830981 28821 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.856082 21435 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:51.847083 28687 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:51.884755 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.890964 20876 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:51.892331 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:51.890000 28913 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:51.896919 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.898222 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.899384 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.899689 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:51.882990 28936 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:51.839088 28845 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.829846 28602 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.920158 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:51.932906 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 80115us
I20260709 12:24:51.933960 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.937860 21435 raft_consensus.cc:2410] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 18.
I20260709 12:24:51.919710 21428 raft_consensus.cc:2368] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22.
I20260709 12:24:51.873924 28814 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.883152 28879 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.899770 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:51.956930 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:51.959841 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 188090us
I20260709 12:24:51.955798 20809 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
W20260709 12:24:51.893641 28935 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:24:51.899881 28939 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:51.960676 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:51.967721 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.969456 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.970597 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:51.894759 28559 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.920526 28881 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.917989 28687 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.987422 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:51.961282 28814 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.921723 28851 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.923142 28870 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:51.953991 28915 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:51.914722 28849 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:51.977964 21435 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:51.960485 28879 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.993067 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:51.993767 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.919618 28690 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.979903 28771 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:52.004144 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:51.994078 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:52.022900 21431 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:24:52.012236 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.020071 21427 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:24:52.020646 21432 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 19: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:24:52.020443 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54182.6us
W20260709 12:24:52.013342 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 208319us
I20260709 12:24:51.953243 28625 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:51.953541 28845 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.036538 20809 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:52.038446 21429 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:52.005939 28851 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:52.048089 20802 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:24:52.058851 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:52.067565 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:52.069882 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 177647us
I20260709 12:24:52.085652 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:52.086447 21429 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:52.068686 20879 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:52.086892 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.093830 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:52.094570 21431 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:52.069038 28747 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:52.102245 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.105033 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:52.105720 20879 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:52.108246 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.115338 28849 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:52.115814 28916 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.009s	sys 0.009s
W20260709 12:24:52.116217 28916 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.009s	sys 0.010s
I20260709 12:24:52.117499 28670 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.107049 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:52.123157 21434 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:52.132342 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.106298 28914 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:24:52.157915 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:52.116644 28893 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:52.159170 20875 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:52.159809 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.092372 28917 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 12:24:52.179571 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.183738 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.134132 28913 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:52.189815 20850 leader_election.cc:336] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.197s (SENT)
I20260709 12:24:52.190598 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:52.191329 20875 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:52.192406 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:52.195437 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 93337.3us
I20260709 12:24:52.196549 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.196749 20884 raft_consensus.cc:2955] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 15 to 17, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:52.198719 28852 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.224320 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.224509 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.225129 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.225494 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.226004 20808 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:52.223443 28852 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:52.230127 20875 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:52.230942 20803 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:52.251703 20803 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:52.252661 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:52.253793 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92898.8us
I20260709 12:24:52.255821 28960 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:52.259706 28602 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.143307 28771 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 12:24:52.264562 28690 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.279642 28496 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:52.281551 28849 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:52.282065 28849 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:52.283164 28849 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.284343 28845 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.284919 28796 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.285665 28796 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.288694 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.289783 20801 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:24:52.290303 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 91618.7us
W20260709 12:24:52.243060 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.291077 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.291946 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.292769 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.293365 20806 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:52.293694 20804 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:52.294416 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.294601 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.295730 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.296394 20807 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:52.147877 28914 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:52.296919 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:52.291337 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.289731 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.297600 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:24:52.291468 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.305455 20878 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:24:52.304435 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.307268 20878 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:24:52.295377 20804 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:52.308787 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.309130 20807 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:52.304122 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.306666 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:52.233263 28625 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.312513 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:52.309608 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.311589 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.319049 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.324542 20804 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:52.313265 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:52.331161 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.331216 20804 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:52.312114 28854 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:52.337015 28602 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.341320 28625 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:52.342031 28625 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:52.343051 28625 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.344426 28944 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:52.344906 28944 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.345283 28944 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:52.345973 28726 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:52.325129 28687 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:24:52.331945 20807 raft_consensus.cc:2376] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18. Re-sending same reply.
I20260709 12:24:52.340149 28602 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.322928 28872 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:24:52.364447 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73545.8us
I20260709 12:24:52.373042 20882 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:24:52.375952 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:52.382376 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.363320 20808 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:52.375169 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:52.389225 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:52.386087 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48546.5us
I20260709 12:24:52.353585 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:52.393514 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:52.407176 20878 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:52.391072 20809 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
W20260709 12:24:52.407953 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.335031 28845 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:52.409091 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.409780 20879 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:52.324064 28877 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.412977 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.411491 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:52.417757 21435 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:52.421393 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.349889 28854 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.424666 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.427409 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.339097 28821 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.435110 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:52.434300 21431 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:52.447139 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:52.447455 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.448045 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.468106 20804 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:52.469013 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.418673 28690 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:52.470301 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.448096 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:52.493289 20735 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23.
I20260709 12:24:52.416275 28872 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:52.417016 28771 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:52.429611 20808 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:52.429700 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76225.4us
W20260709 12:24:52.470659 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 215547us
I20260709 12:24:52.489923 21434 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:24:52.412730 28851 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:52.497658 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.503038 21435 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:52.510125 20807 raft_consensus.cc:2368] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26.
I20260709 12:24:52.487954 21431 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:52.427762 28747 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:52.428949 28876 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:24:52.428210 28854 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:52.434722 21432 raft_consensus.cc:2955] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 15 to 17, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:52.491622 28690 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.516928 28690 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:52.517081 28876 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:52.495105 28796 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:52.519371 28747 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:52.519392 28876 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:52.520465 28747 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.521421 28877 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:52.521651 28876 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.523126 28602 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.428555 28845 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:52.524667 28948 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:24:52.526641 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:52.527889 28946 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:24:52.528129 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.528864 20806 raft_consensus.cc:2368] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
W20260709 12:24:52.529157 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.529295 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:52.530562 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.530557 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.522924 28876 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:52.535602 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.538880 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:52.542582 20804 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:52.546212 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.552664 20808 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
W20260709 12:24:52.549067 28614 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.001s
W20260709 12:24:52.532812 28954 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:24:52.535233 28955 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:24:52.511834 28915 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:52.555330 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.390854 28914 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.13, Last appended by leader: 13, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.563632 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.565706 21434 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:52.546763 21431 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.565860 20804 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:52.550377 28877 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:52.562202 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:52.578758 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.581616 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:52.582264 20879 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:24:52.582947 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.583961 28964 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:52.586063 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.586834 28851 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.587869 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:24:52.588348 28909 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.002s
I20260709 12:24:52.588783 28909 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.589061 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:52.589933 28960 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:52.590332 28960 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.590642 28960 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:52.559360 28614 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.591364 28870 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:52.591780 28870 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:52.592175 28870 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:52.592844 28614 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.598050 21430 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:52.598773 28313 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.563553 28496 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:52.599313 28313 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.600960 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.601622 20731 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:52.601720 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.602010 28901 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.602449 20732 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:52.602598 28901 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.603582 20734 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:52.604836 28901 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:52.605697 28877 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:52.579378 28915 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:52.606851 28849 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:52.606709 28877 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.607486 28915 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:52.608657 28879 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.000s
I20260709 12:24:52.609086 28879 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.610236 20801 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:52.607965 28965 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.610796 28965 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:52.611483 28962 consensus_queue.cc:1175] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 25 status { last_received { term: 22 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: [24.16-24.16]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:52.604660 28960 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:52.576252 28881 consensus_queue.cc:1175] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 28 status { last_received { term: 28 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 28 index: 13 } }
I20260709 12:24:52.613195 28965 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:52.613125 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.614559 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:52.619827 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 89337.9us
I20260709 12:24:52.627408 20879 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:52.631994 20802 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:52.623790 21426 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:52.636780 21431 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.638548 20878 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:52.590950 28909 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:52.614274 28814 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.594s	user 0.007s	sys 0.003s
W20260709 12:24:52.686055 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.691905 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:52.695228 20804 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:52.695915 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.696333 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.698834 20734 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:52.706384 20731 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 16.
I20260709 12:24:52.640413 28964 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:52.675559 28879 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.713274 28969 raft_consensus.cc:2764] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:24:52.717567 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.719712 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:52.720435 21427 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:52.722491 21427 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEARNER]: Advancing to term 18
I20260709 12:24:52.723956 28821 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:52.724839 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:52.725615 20734 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:52.731617 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.732430 28964 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:52.736177 20810 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:52.736696 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.743053 21434 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:52.744592 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:52.693946 28597 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.755424 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 263484us
W20260709 12:24:52.762591 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 210787us
I20260709 12:24:52.761221 21431 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:24:52.636727 28690 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.765394 21431 pending_rounds.cc:85] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 14
I20260709 12:24:52.694805 28814 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:52.760337 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.776087 21431 pending_rounds.cc:107] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:24:52.776613 21431 raft_consensus.cc:2889] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.777143 21431 pending_rounds.cc:107] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:24:52.777524 21431 raft_consensus.cc:2889] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.791065 20733 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:52.791621 20730 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:24:52.800029 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:52.619612 28872 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.761751 28879 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:24:52.816840 21400 connection.cc:256] RPC call timeout handler was delayed by -3.0169174900001963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
W20260709 12:24:52.817709 21400 connection.cc:256] RPC call timeout handler was delayed by -2.4535174900001948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 12:24:52.820219 20850 leader_election.cc:336] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.595s (SENT)
I20260709 12:24:52.821014 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.712384 28944 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.824263 20731 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: [19.15-21.16]
I20260709 12:24:52.825826 20881 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:52.810225 28960 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:52.852903 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.854492 20805 raft_consensus.cc:2410] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:24:52.823210 28964 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:52.867666 20727 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:52.796610 28597 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:52.851743 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 337859us
I20260709 12:24:52.841014 28496 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.870436 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.791810 28854 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.827939 28879 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:52.845752 28821 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.875722 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 265719us
I20260709 12:24:52.883888 20733 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:52.840175 28881 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:24:52.890105 21433 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:52.857297 28870 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.839967 28747 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:24:52.866492 28876 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:52.893100 21433 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:52.905267 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 94792.9us
I20260709 12:24:52.905907 20731 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:24:52.910568 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:52.912154 20733 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.908628 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.884550 28875 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:24:52.878924 28496 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.926713 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.932197 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.933594 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:52.938993 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.939725 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.899343 28821 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.948737 20734 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:52.956629 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.942867 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:52.957080 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3158) took 1245 ms. Trace:
I20260709 12:24:52.943131 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.958326 20876 rpcz_store.cc:276] 0709 12:24:51.712125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.819673 (+107548us) service_pool.cc:224] Handling call
0709 12:24:51.822373 (+  2700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.891443 (+ 69070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.956842 (+1065399us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:52.957018 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157300480}
I20260709 12:24:52.908522 28625 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:24:52.960515 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.911602 28876 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.961318 20734 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:52.963442 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:52.944468 28976 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:52.943028 28849 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:52.943470 28873 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.977361 28845 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.979882 21433 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.980952 21433 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:52.981509 28496 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.982847 28870 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:52.937683 28559 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:24:52.984874 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.985508 28967 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:52.985960 28967 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:52.986814 28901 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:52.987636 28614 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.988148 28614 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:52.989032 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:52.990336 28614 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.986858 28967 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:52.992081 28873 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:52.992817 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.992313 28967 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:52.968160 28960 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:52.990733 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:52.997480 28876 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:52.997498 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.998466 20728 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:52.999435 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.999765 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3159) took 1287 ms. Trace:
I20260709 12:24:52.934731 28943 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:24:53.000200 20881 rpcz_store.cc:276] 0709 12:24:51.712478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.840596 (+128118us) service_pool.cc:224] Handling call
0709 12:24:51.841003 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:52.841702 (+1000699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.999448 (+157746us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b5400200b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:52.999695 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239223040}
I20260709 12:24:53.001559 28943 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:52.942921 28966 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:52.970168 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:52.947433 28663 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:24:53.006968 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50462.2us
I20260709 12:24:52.973721 28989 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:52.974098 28597 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:52.937748 28909 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:24:52.965924 28964 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:52.975286 28893 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:52.975625 28986 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:24:52.939657 28965 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:53.002532 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.000475 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 57282us
I20260709 12:24:52.950858 28913 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.027894 28913 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.028414 28909 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:53.028867 28909 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.029027 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1515) took 1131 ms. Trace:
I20260709 12:24:53.029450 21426 rpcz_store.cc:276] 0709 12:24:51.897142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.899101 (+  1959us) service_pool.cc:224] Handling call
0709 12:24:53.028813 (+1129712us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:53.028934 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1664407936}
I20260709 12:24:53.000607 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.030527 20728 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:53.033594 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.019549 28893 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.034421 28893 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.014707 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.036402 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:53.038447 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.009181 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:52.966835 28985 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.018852 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48755.6us
I20260709 12:24:53.002038 28943 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.021999 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.022289 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.038447 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.022588 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.023679 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.046252 20805 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.003772 28814 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:53.037788 28875 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.050002 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:53.050307 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.049960 28814 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:53.051000 21431 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:24:53.033703 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.052234 28814 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.052489 28771 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:53.053015 28771 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:53.053061 20728 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:53.054082 28771 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.021384 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:52.966579 28962 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:53.055539 28962 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:53.055585 20804 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:53.056066 28962 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.058569 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:53.048033 28796 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:53.058872 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.059520 28796 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:53.059840 28876 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:53.059999 28873 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:53.060442 28876 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.060647 28873 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.060508 28796 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.061493 28876 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.061770 28873 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.047322 28985 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.006892 28663 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:53.048528 28602 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:53.039544 20731 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:53.051533 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.048573 28943 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.075001 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.049422 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:53.076035 20731 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:53.076399 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:24:53.077090 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.077711 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:53.078227 28964 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.029918 28909 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.082506 28909 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.083536 28602 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:53.083995 28602 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:53.085213 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.084988 28602 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:53.086516 28602 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.087492 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.088586 20878 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:24:53.064538 20810 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:53.064935 20876 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:24:53.047924 28917 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:53.011133 28690 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.097460 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.039628 28944 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.063969 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:53.067973 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.102504 20728 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:53.015470 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:52.959751 28879 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:53.071136 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45342.2us
I20260709 12:24:53.070473 20881 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:53.087397 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.039153 20877 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:53.095623 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:53.100739 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49274.4us
I20260709 12:24:53.104137 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.072413 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.105046 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.098075 28852 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:53.076030 28943 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:53.116214 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.117110 20735 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:24:53.091650 28875 consensus_queue.cc:1175] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 21 status { last_received { term: 21 index: 14 } last_committed_idx: 10 last_received_current_leader { term: 21 index: 14 } }
I20260709 12:24:53.070329 28960 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:24:53.101994 28854 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:53.118491 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.090276 28965 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
W20260709 12:24:53.129132 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.131542 28992 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:24:53.132772 28313 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:53.134092 28970 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:53.134690 28970 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.135597 28663 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.136538 20732 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:53.137100 28663 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:53.137883 20732 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: [28.13-28.13]
I20260709 12:24:53.121426 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:53.120226 28985 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.138481 21434 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
W20260709 12:24:53.119104 28993 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:24:53.140080 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:53.135600 28970 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.141811 28970 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:53.114794 28771 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:53.110543 28845 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:53.116755 28852 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.143748 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.143832 28845 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.144593 28854 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:53.144692 28845 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.145802 20700 leader_election.cc:400] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:53.146147 20700 leader_election.cc:403] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.146409 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.147768 28901 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.148289 28670 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:53.148923 20732 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:53.149123 20700 leader_election.cc:400] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26.
I20260709 12:24:53.149999 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.150465 20732 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:53.143999 28852 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:53.148491 28901 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:53.152420 20733 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:53.154201 28796 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:53.156474 28917 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:53.158835 28849 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.159610 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.161892 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.149523 28670 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:53.197408 20804 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:53.163333 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.163991 20805 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:53.171891 20735 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:24:53.173504 21433 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:24:53.199908 21433 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 16
I20260709 12:24:53.200181 21433 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:24:53.200425 21433 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:53.175246 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159835us
I20260709 12:24:53.201532 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:53.204586 20882 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:53.205667 20728 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 17.
I20260709 12:24:53.205938 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.206750 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.213955 20729 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:24:53.214184 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.217321 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.172245 28796 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.172142 28917 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.217011 28614 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:53.212812 28872 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:24:53.220135 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71142.9us
I20260709 12:24:53.204245 28670 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:53.171430 28687 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:24:53.236243 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:53.196653 28960 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.206681 28854 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 16. Current term is 17.
I20260709 12:24:53.224704 28917 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.254597 28913 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:53.256543 28625 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:53.257009 28625 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:53.257927 28625 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:53.227624 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 65791.9us
I20260709 12:24:53.238678 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.262643 20882 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.221482 28796 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.172977 28893 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:53.244369 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 94452.6us
I20260709 12:24:53.246070 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.241209 28916 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:53.243628 28909 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.249838 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:53.276127 28909 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.278373 28909 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:53.278996 28614 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:53.274196 28916 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.279583 28614 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:53.279675 28916 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:53.280037 28614 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:53.274185 28854 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:53.250123 28914 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:53.284744 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.287371 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.294200 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:53.297387 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.300441 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.304373 20878 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:53.304565 28989 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:53.314440 20881 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:53.314596 20877 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:53.316944 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.318917 20879 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:53.273375 28625 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:53.327576 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:53.332196 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.332427 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.223956 28849 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.332885 20878 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:53.317842 20882 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:53.235872 28985 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:24:53.337952 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.340162 20809 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:53.306592 28814 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:53.245031 28496 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.088s
W20260709 12:24:53.346745 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.347590 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72555.9us
I20260709 12:24:53.337591 28980 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:53.348431 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:53.266274 28962 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.356233 28877 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:53.356976 28877 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:53.358409 29000 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:24:53.358273 28877 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:53.361120 29001 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:24:53.361739 28845 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:53.360237 28999 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:24:53.306248 28870 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:24:53.337003 28914 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:53.367605 28914 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:53.368026 28914 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.368779 28964 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:53.372856 28962 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.372502 28852 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.374157 20810 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:53.375931 28852 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.378736 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.379055 28849 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.380657 28893 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:53.336057 28876 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:53.336018 28881 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:53.349852 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.393159 20849 leader_election.cc:400] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19.
I20260709 12:24:53.393397 20804 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:53.393536 20849 leader_election.cc:403] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.389011 20882 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:24:53.344415 28814 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:53.398304 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:53.398837 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:53.403059 21426 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:53.403915 20731 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:24:53.411739 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.414916 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.417707 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.345453 28989 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:53.393055 28881 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.425838 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.426671 20805 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.425453 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.430063 20801 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 29 because replica has last-logged OpId of term: 28 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 12.
I20260709 12:24:53.346947 28687 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:24:53.435770 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.402172 28960 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 18. Current term is 19.
I20260709 12:24:53.413499 28814 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.335053 28670 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.448982 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.449841 20809 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:53.429900 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.444962 21428 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:24:53.425472 28881 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.419652 28876 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.452832 20806 raft_consensus.cc:2376] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20. Re-sending same reply.
W20260709 12:24:53.441006 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54591.7us
W20260709 12:24:53.456292 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 125094us
I20260709 12:24:53.460772 28917 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.463600 28917 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.464543 28989 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.464905 28876 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.470814 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.473397 20808 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 17. Current term is 19. Ops: []
I20260709 12:24:53.472941 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:53.448098 28663 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.478380 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.485244 29011 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:24:53.495474 28845 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.496109 28879 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:53.497419 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.500540 21426 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:24:53.511605 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.513312 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:53.440208 28916 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.519913 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.522668 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.476524 28960 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:53.535063 28877 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:53.535601 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.543334 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 65077us
W20260709 12:24:53.542418 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 146565us
I20260709 12:24:53.517066 28845 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:53.547287 28914 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.544178 28881 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.551635 28881 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:53.551934 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.552330 28881 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:53.553346 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.554636 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.554970 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:53.555979 29006 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:53.523588 28893 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:53.559057 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:24:53.561612 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.524405 28964 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:53.565371 20804 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:53.533787 28909 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.531627 28943 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:53.566373 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.508777 28663 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:53.571645 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.592777 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.601572 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:53.602300 21426 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.567116 28602 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:24:53.609198 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85330.3us
W20260709 12:24:53.611212 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.562647 28893 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:53.606864 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.569310 28970 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:53.607316 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.616317 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.565308 28944 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:53.622682 21400 leader_election.cc:400] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23.
I20260709 12:24:53.623489 21400 leader_election.cc:403] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.594633 28967 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:24:53.599608 28877 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:53.628787 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.629755 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.598981 28852 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.635080 20879 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.648056 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:53.647079 21426 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:53.608593 28821 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.621433 28964 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.624861 28893 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.636859 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:53.625777 28876 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:53.681926 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73652us
I20260709 12:24:53.634701 29018 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.634016 28877 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.637938 28726 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:53.692027 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.686247 28663 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:53.683276 28821 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:53.686230 28870 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:24:53.693813 28877 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:53.711301 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.662701 28852 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.711048 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:53.725549 28916 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.690341 28876 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.719684 29015 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:53.724256 20880 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:24:53.720249 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68034.8us
I20260709 12:24:53.748237 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.701891 28559 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.748812 20875 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.749130 28559 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.751395 28559 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:53.759964 28614 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.762777 28614 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:53.788090 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45548.2us
I20260709 12:24:53.688522 28901 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23.
I20260709 12:24:53.722520 20879 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:53.726692 28747 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:53.748049 28876 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.700399 28726 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.693542 29007 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.155s
W20260709 12:24:53.763765 29019 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:24:53.695108 29018 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.767516 28663 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.728180 28821 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:53.785898 29012 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:53.796509 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:53.806191 20875 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:53.807211 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:53.814692 28964 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.816082 28960 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:53.826344 28978 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.926s	user 0.003s	sys 0.000s
I20260709 12:24:53.822623 28893 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.822856 29007 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:53.873733 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:53.851931 21430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1411) took 2748 ms (client timeout 3008 ms). Trace:
I20260709 12:24:53.874394 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.855490 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:53.875473 21432 raft_consensus.cc:2368] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:53.874408 21426 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:53.871958 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1318) took 1240 ms. Trace:
I20260709 12:24:53.819026 28909 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:53.877936 20808 raft_consensus.cc:2368] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:53.879606 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.884716 20804 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:53.874550 21430 rpcz_store.cc:269] 0709 12:24:51.103929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.104125 (+   196us) service_pool.cc:224] Handling call
0709 12:24:53.851709 (+2747584us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:53.851838 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3401961344}
I20260709 12:24:53.876997 20727 rpcz_store.cc:276] 0709 12:24:52.601277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.696031 (+ 94754us) service_pool.cc:224] Handling call
0709 12:24:53.841214 (+1145183us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54001d7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:53.841385 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290396544}
I20260709 12:24:53.896921 20804 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:53.907927 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:53.889185 20878 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 12:24:53.916587 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.924031 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:53.925244 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:53.926029 21433 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:53.929538 20735 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:53.935120 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.938679 20733 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.935683 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:53.942396 20728 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:53.936129 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.951661 20734 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:53.804030 28796 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:53.964308 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:53.964962 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85672.6us
I20260709 12:24:53.966364 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.965003 20730 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:53.971020 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:53.972047 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:53.971020 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.006616 21427 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:54.007159 20735 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:54.006222 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:54.013592 21426 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:24:54.016619 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 268470us
I20260709 12:24:54.019435 21434 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:24:54.022969 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:54.023962 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:54.028744 20802 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:54.045585 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3067) took 1507 ms. Trace:
I20260709 12:24:54.045843 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:54.046464 21432 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:54.046912 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:54.048451 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:54.049990 20701 leader_election.cc:400] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:54.050511 20701 leader_election.cc:403] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:54.046058 20803 rpcz_store.cc:276] 0709 12:24:52.513817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.526391 (+ 12574us) service_pool.cc:224] Handling call
0709 12:24:54.020535 (+1494144us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:54.020656 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867493760}
I20260709 12:24:54.054800 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.057978 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:54.097882 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.101663 20805 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:54.068802 21433 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:24:54.100394 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1935) took 1643 ms. Trace:
I20260709 12:24:54.100236 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.011814 28747 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.103667 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:54.105739 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.057096 20734 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:54.102813 21429 rpcz_store.cc:276] 0709 12:24:52.410995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.411113 (+   118us) service_pool.cc:224] Handling call
0709 12:24:52.427772 (+ 16659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:52.428005 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:52.428027 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:52.433878 (+  5851us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.791391 (+357513us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:52.791429 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.791493 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.825704 (+1034211us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:54.021383 (+195679us) raft_consensus.cc:1685] finished
0709 12:24:54.021406 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.053852 (+ 32446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12103424,"thread_start_us":357189,"threads_started":1,"wal-append.queue_time_us":434835}
W20260709 12:24:54.129936 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81620.5us
I20260709 12:24:54.130820 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:54.131665 20732 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 22. Ops: []
I20260709 12:24:54.132493 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:53.887844 29007 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:53.836725 28663 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.128153 21431 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:24:54.065065 28879 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:54.017787 29024 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:54.133635 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.133744 20733 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:54.134783 20735 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:24:54.053592 28614 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:54.138269 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.100881 29015 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:54.054186 28916 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:24:54.140172 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 343751us
I20260709 12:24:54.152832 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.056653 29033 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:54.130894 28881 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:24:54.173902 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 124154us
I20260709 12:24:54.046706 28821 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.142167 28672 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 17. Current term is 18.
I20260709 12:24:54.184868 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.185905 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:54.176333 28625 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.843s	user 0.008s	sys 0.004s
I20260709 12:24:54.164594 29012 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:54.187152 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.173740 28614 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:54.161024 28881 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:24:54.182243 28916 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:54.197868 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.178061 29029 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:54.203344 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:54.200937 20803 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:24:54.182178 28852 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:54.206864 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.207249 20881 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:54.207540 20876 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:54.199170 28881 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:54.165508 29033 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.227059 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:54.228005 29035 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.228580 29035 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.205610 28893 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.208781 29018 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:54.213230 29012 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:24:54.237203 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.244181 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:54.245189 20803 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:24:54.245239 29037 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:24:54.249917 28814 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.793s	user 0.006s	sys 0.003s
I20260709 12:24:54.250366 28814 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.253824 28663 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:54.244378 29028 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:24:54.206075 28879 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:54.243723 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:54.269577 28893 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:54.270181 28893 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:54.270685 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:54.271742 21431 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:54.271052 28893 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.273589 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:54.274272 21429 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:54.274613 28747 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:54.279721 20875 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:54.225410 28796 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.281186 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3035) took 1298 ms. Trace:
I20260709 12:24:54.281924 21429 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
W20260709 12:24:54.280133 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 147698us
I20260709 12:24:54.281555 20729 rpcz_store.cc:276] 0709 12:24:52.982353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.063899 (+ 81546us) service_pool.cc:224] Handling call
0709 12:24:54.279923 (+1216024us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001d7c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:54.281123 (+  1200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245613312}
I20260709 12:24:54.285192 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:54.266912 20730 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:54.301561 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.302431 20735 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:24:54.282943 28877 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.546s	user 0.006s	sys 0.003s
I20260709 12:24:54.304558 20726 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:54.305469 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.279577 29026 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.279165 29030 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.315303 29035 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.316774 29030 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.317754 29007 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:54.319520 29025 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:24:54.321789 28814 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:54.321661 28879 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.325091 29026 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.327425 29026 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:54.327783 29045 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:24:54.328886 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.331384 28747 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:54.332098 28747 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:54.332789 28747 consensus_queue.cc:1243] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:24:54.333415 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.334185 20807 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:54.334344 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:54.335071 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.314740 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:54.206990 29029 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.335176 29033 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:54.332826 20809 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:54.338275 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.312551 21432 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
W20260709 12:24:54.311079 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55486us
I20260709 12:24:54.339181 20808 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:24:54.335773 20807 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:24:54.318955 29030 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:54.346860 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.348558 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.349236 20730 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:54.349270 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:54.349943 21427 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:24:54.351531 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:54.352908 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.354811 29036 raft_consensus.cc:2764] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:24:54.355782 29027 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:24:54.357568 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:54.358290 21433 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:54.315431 29012 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:54.360280 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:54.353470 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:54.371517 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:54.371865 20808 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:24:54.342900 29026 raft_consensus.cc:2764] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:24:54.375625 20804 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:24:54.374907 29015 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:54.377277 29015 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.377614 29015 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:54.377679 20809 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:54.378153 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3024) took 1403 ms. Trace:
I20260709 12:24:54.378835 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.379527 20803 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:54.383692 29008 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:54.386837 28771 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.224s	user 0.003s	sys 0.011s
I20260709 12:24:54.387939 20883 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:24:54.388314 20883 pending_rounds.cc:85] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 12
I20260709 12:24:54.388535 20883 pending_rounds.cc:107] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:24:54.388764 20883 raft_consensus.cc:2889] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:54.369522 29030 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:24:54.393918 29010 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.826s	user 0.004s	sys 0.006s
I20260709 12:24:54.396706 29029 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:54.352105 29033 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:54.398676 29042 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.399296 29042 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.399926 20730 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:54.400221 28849 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:54.400646 28849 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:54.400995 20807 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:54.401651 29042 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:54.401475 28849 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.343415 28796 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:24:54.402820 28849 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.403571 28796 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:54.404254 28796 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:54.405949 28901 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:54.406384 28901 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:54.406988 28881 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.407315 28901 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.408720 28901 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:54.409965 20806 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:54.310284 28877 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.412711 20810 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:54.413348 28877 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.422029 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:54.425549 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.426496 20802 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:54.428191 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.429301 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.429661 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.430462 29026 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:54.430310 28771 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:54.430876 29026 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.431000 29035 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:54.431291 29026 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:54.431224 28771 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:54.431790 29012 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:54.432204 28771 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:54.433277 29012 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:54.433523 28771 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.434389 29012 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:54.434829 29012 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.435374 28663 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:54.435794 28663 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:54.404270 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:54.378619 20731 rpcz_store.cc:276] 0709 12:24:52.975033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.097203 (+122170us) service_pool.cc:224] Handling call
0709 12:24:54.377868 (+1280665us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:54.378087 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702418176}
I20260709 12:24:54.440997 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.443460 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.442288 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.446409 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.452361 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.453826 20700 leader_election.cc:400] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:54.454193 20700 leader_election.cc:403] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:24:54.460847 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:54.462311 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:54.462571 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.464046 20804 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:24:54.465857 20801 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:54.466656 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.469921 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.471549 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:54.472378 20804 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:54.471565 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.474337 20807 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:54.441175 29012 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:54.485054 20882 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:54.481542 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.460479 29027 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:54.473600 29051 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:54.468178 29011 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:54.464996 28893 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:54.468369 28854 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20.
I20260709 12:24:54.436834 28663 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.460705 29024 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.492467 29041 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:54.487988 28875 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.502264 29048 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:54.487396 28877 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:54.516839 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:54.474485 28965 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:54.499681 28881 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.504513 29011 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:54.526129 20804 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:54.513655 29027 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:54.535778 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.536772 29039 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:54.537384 29039 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:54.538277 29039 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.493355 28821 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:24:54.547224 20806 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 21. Ops: []
I20260709 12:24:54.527102 29037 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.524057 28893 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:54.520851 29024 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.555668 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:54.562911 20882 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:54.565162 29020 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:54.565620 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:54.566643 29057 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:54.565778 29020 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:54.522011 29030 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:24:54.581794 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:54.567283 29021 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:54.571616 29027 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.588068 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.521783 28875 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.599996 20879 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:24:54.630698 20809 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:24:54.635362 29011 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.636436 28965 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:54.596127 28854 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.636541 29011 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:54.583250 28663 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:54.581245 29041 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.639350 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.585153 29036 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:54.599130 28913 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:54.637208 28854 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.586858 28771 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:24:54.633054 29021 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:54.637158 28965 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:54.679251 28854 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:54.656073 29036 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.680892 28965 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.579174 29037 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:54.673306 29054 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:24:54.675962 28875 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:54.639659 28496 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.649686 28913 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:54.687368 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:54.688437 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.691332 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:54.682912 29036 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.689814 20879 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:24:54.688517 20882 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:54.693173 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.697155 20877 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:24:54.697582 28872 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:54.698030 28872 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:54.698127 29021 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.705417 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.706879 20882 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:54.692523 29056 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:54.664752 28870 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:24:54.654650 29041 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.702111 28496 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:54.695693 28893 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.780954 29012 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.699034 28872 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.724202 28913 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.728183 29024 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.728897 29066 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.729879 29048 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:54.733176 29026 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:54.735993 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3169) took 1668 ms. Trace:
I20260709 12:24:54.795877 20883 rpcz_store.cc:276] 0709 12:24:53.068100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.068227 (+   127us) service_pool.cc:224] Handling call
0709 12:24:53.093853 (+ 25626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.389267 (+1295414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.735747 (+346480us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400171e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:54.735931 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2975079168}
I20260709 12:24:54.730312 29020 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:54.742022 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.749352 29051 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:54.746904 20728 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:54.747967 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.749060 29029 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:54.749097 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.742718 29056 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:54.760857 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.787742 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.790903 20803 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:54.792917 20726 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:54.797667 28913 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.803174 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.852676 20730 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:54.805140 29058 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:54.805547 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.854912 20808 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:24:54.855898 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.745476 29037 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.806803 20727 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:54.809830 20731 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:24:54.844372 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.846588 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:54.808732 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.845877 20729 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:54.853858 20884 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:54.857409 20882 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:54.858198 20733 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:24:54.858789 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.866056 20808 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:24:54.858673 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.866950 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.867779 20731 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:54.859256 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:54.868131 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.858271 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:54.861461 20807 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:24:54.859297 20802 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:54.865373 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.864619 20804 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:24:54.869853 20734 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:54.864660 20728 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:54.871061 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.865433 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.883080 20730 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:24:54.886683 20883 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:24:54.883659 20803 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:54.885317 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.886125 20881 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:24:54.872885 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.888208 20731 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:54.863740 20806 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:54.873078 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.849239 29012 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:54.927945 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.928354 29030 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:54.928810 28771 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:54.925630 29018 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.929872 29018 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:54.854199 28602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.202s	user 0.008s	sys 0.035s
I20260709 12:24:54.896178 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.896849 20728 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:54.935082 20879 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:54.900487 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.865746 20877 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:54.925602 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.845463 29029 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:24:54.948742 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.854589 29058 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:54.944996 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:54.945976 20734 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:54.866506 29048 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:54.950518 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:54.963735 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.967633 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.974571 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:54.975418 20730 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:54.967145 20876 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
W20260709 12:24:54.937253 28602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.011s	sys 0.037s
I20260709 12:24:54.978464 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:54.984969 21426 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:54.985530 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:24:54.895681 28916 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:54.934890 28873 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.982064 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:54.810113 29066 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.012181 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:54.988247 21433 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:24:54.944476 28917 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:54.990688 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:54.925710 28967 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:54.993739 29030 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:54.994539 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:54.997177 29061 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 12:24:54.998497 20726 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: [23.13-25.14]
I20260709 12:24:54.998984 29031 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.126s
I20260709 12:24:54.861904 29015 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.000860 29026 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:54.926301 28876 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 12:24:54.934698 28854 raft_consensus.cc:2764] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:24:55.005354 20701 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:54.866688 28313 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.109s
I20260709 12:24:54.979676 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:55.061057 20875 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:55.020051 28870 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:54.974303 29048 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.047427 20727 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:24:55.069458 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:54.926153 28851 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 12:24:54.934544 29033 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:24:54.984704 28602 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:55.096870 28602 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:55.099234 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.099938 29037 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:55.100601 29037 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.100981 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1329) took 1075 ms. Trace:
I20260709 12:24:55.101424 20732 rpcz_store.cc:276] 0709 12:24:54.025138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.053695 (+ 28557us) service_pool.cc:224] Handling call
0709 12:24:54.131512 (+ 77817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.132039 (+   527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.100758 (+968719us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b5400249fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:55.100916 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41720576}
I20260709 12:24:55.106001 29045 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:55.106632 29045 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.108913 29045 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:55.109264 20876 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:55.110023 28663 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:55.019783 28873 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.113348 28873 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:55.059119 28917 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:55.116850 29015 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:55.065631 29031 consensus_queue.cc:1243] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 26.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:24:55.071329 29048 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:55.119129 29066 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:55.122601 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1951) took 1734 ms. Trace:
I20260709 12:24:55.109690 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.136358 20884 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:24:55.135599 21435 rpcz_store.cc:276] 0709 12:24:53.388050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.388174 (+   124us) service_pool.cc:224] Handling call
0709 12:24:53.388610 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.391475 (+  2865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.391520 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.392786 (+  1266us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.457042 (+ 64256us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:53.457085 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.457153 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.072164 (+1615011us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b18003145c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:55.122241 (+ 50077us) raft_consensus.cc:1685] finished
0709 12:24:55.122270 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.122528 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2251,"raft.queue_time_us":46282,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":12617,"spinlock_wait_cycles":12920576,"thread_start_us":31421,"threads_started":1,"wal-append.queue_time_us":43104}
I20260709 12:24:55.100977 29037 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:55.137706 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:55.138073 29023 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.138652 29023 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.138929 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.022760 28967 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.139705 28967 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:55.142037 29023 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:55.142745 28914 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:55.022955 28964 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:55.021524 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:55.144141 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 122731us
I20260709 12:24:55.144906 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:55.130875 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.066706 20726 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:24:55.147416 20881 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:55.147815 28726 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:55.148545 29011 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 12:24:55.047823 29030 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:55.073355 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68132.2us
I20260709 12:24:55.149276 29030 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:55.073478 28893 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:24:55.153344 28893 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.153847 28893 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.072566 21428 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:55.155432 28851 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:55.155866 28851 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:55.156776 28851 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.157997 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:55.109850 20876 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:55.160334 28964 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:24:55.160440 28771 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:55.160935 20878 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3182) took 1549 ms (client timeout 1921 ms). Trace:
I20260709 12:24:55.160604 20876 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.161203 28964 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:24:55.161337 20878 rpcz_store.cc:269] 0709 12:24:53.611433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.623695 (+ 12262us) service_pool.cc:224] Handling call
0709 12:24:55.160722 (+1537027us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:55.160852 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535954816}
I20260709 12:24:55.162983 20731 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:55.138859 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:55.082670 28496 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:55.085166 29041 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:55.176688 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:24:55.069962 29058 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.094220 28670 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:55.192029 20884 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:24:55.119621 29018 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:55.114853 28917 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.082558 29033 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:55.165655 21435 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:55.166008 20875 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:55.166152 20732 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:55.170406 20876 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:55.057662 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68566.8us
I20260709 12:24:55.174307 28893 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:55.062351 29029 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.176854 21433 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:55.194737 20882 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:55.195621 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.213253 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:55.213688 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.190863 28771 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.240626 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.244108 20878 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:24:55.243643 21428 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:24:55.200942 28964 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:55.246361 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:55.247227 20875 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:55.247637 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:55.251389 28969 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:55.251799 28969 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:55.253010 20730 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:55.252745 28969 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.254146 28969 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:55.254878 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:55.255816 21435 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:24:55.257854 21435 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:24:55.260281 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:55.259027 21435 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:55.262559 21428 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:24:55.260856 29060 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.265226 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.265718 29060 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.266721 20732 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:55.267916 29060 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:55.269053 29018 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:55.270483 29059 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:55.270999 29059 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.167040 28663 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.273981 20882 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 18 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:24:55.276603 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.277562 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:55.214221 29033 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:55.278981 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:55.286228 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:55.289264 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:55.290915 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:55.153335 29056 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:55.276041 29063 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:55.280169 28496 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:55.281276 29029 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:55.262326 29026 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:24:55.287287 21428 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 17.
I20260709 12:24:55.249931 28771 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.311256 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:55.316543 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.589s (SENT) [suppressed 25 similar messages]
I20260709 12:24:55.328236 20805 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:55.303717 28821 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.337249 28663 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:55.354169 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:24:55.343369 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.346169 20878 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:55.355629 20876 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:55.333137 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:55.312613 29031 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:24:55.354192 29060 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:55.366943 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1528) took 1091 ms. Trace:
I20260709 12:24:55.466478 21429 rpcz_store.cc:276] 0709 12:24:54.275359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.276834 (+  1475us) service_pool.cc:224] Handling call
0709 12:24:54.277147 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.282521 (+  5374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.366522 (+1084001us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:55.366853 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10468608}
I20260709 12:24:55.370836 20727 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 21. Ops: []
I20260709 12:24:55.374393 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:55.376854 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3301) took 1217 ms. Trace:
I20260709 12:24:55.378278 29026 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:55.381377 28814 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:55.468838 20805 rpcz_store.cc:276] 0709 12:24:54.159769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.159914 (+   145us) service_pool.cc:224] Handling call
0709 12:24:54.160244 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.330982 (+1170738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.376444 (+ 45462us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540027a954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:55.376727 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2685478144}
I20260709 12:24:55.385067 29058 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:55.322167 29011 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:55.313097 28496 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.478600 28496 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:55.478680 29059 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:55.416888 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62814.9us
I20260709 12:24:55.421295 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.483054 20804 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:24:55.423035 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:55.485862 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62957.8us
I20260709 12:24:55.442921 20806 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:55.446815 28964 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:55.441999 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.498927 20810 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:55.353257 29018 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:55.481760 28670 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:55.476866 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:55.477708 29030 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.354713 28821 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.435293 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.489848 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:55.490846 28876 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.436939 28854 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:24:55.495338 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 162373us
W20260709 12:24:55.511997 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.502630 20735 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:24:55.494102 28913 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:24:55.508852 20801 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:24:55.506096 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:55.513352 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 196840us
I20260709 12:24:55.494382 29011 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:24:55.522112 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.480297 28814 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.510358 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:55.528319 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:55.528875 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.474908 29060 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.530534 20734 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:24:55.481001 29026 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.497663 28967 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.554087 20727 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:24:55.553745 28964 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.555563 28967 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:55.501313 29018 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.556353 28967 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:55.557483 28771 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:55.536710 20876 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:24:55.537950 28914 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:55.538410 28845 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:55.526484 28602 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:24:55.509640 29023 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:24:55.539755 29011 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.534946 29045 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.502702 29048 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.546435 28814 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:55.550066 28909 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:55.527307 29082 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:24:55.511221 29067 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:55.552767 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1444) took 1275 ms. Trace:
I20260709 12:24:55.553495 29026 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:55.559248 20731 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:55.561365 20734 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:55.591674 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:55.568338 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:55.601374 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:55.609781 20801 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:55.539795 28872 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:55.556347 29018 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:55.607151 21431 rpcz_store.cc:276] 0709 12:24:54.277832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.277986 (+   154us) service_pool.cc:224] Handling call
0709 12:24:54.278273 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.278615 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.278641 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.107384 (+828743us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:24:55.107935 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.292183 (+184248us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:55.292251 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.292340 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.552448 (+260108us) raft_consensus.cc:1685] finished
0709 12:24:55.552466 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.552701 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":804,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":5416192,"thread_start_us":183552,"threads_started":1,"wal-append.queue_time_us":401602}
I20260709 12:24:55.620817 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:55.537204 28876 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:55.532588 28913 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.590289 28845 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.620472 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:55.603669 28909 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.686947 20734 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:24:55.687987 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.689571 20731 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 12:24:55.596688 29072 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.084s
W20260709 12:24:55.632542 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.699462 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:55.700765 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:55.697005 20731 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:24:55.734580 20735 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:55.620127 28967 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.068s
I20260709 12:24:55.683159 28913 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:55.601356 29029 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.087s
I20260709 12:24:55.606108 29045 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.743991 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:24:55.732201 29013 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:55.698679 28670 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.688027 28909 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:55.686158 28845 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:55.731937 29072 consensus_queue.cc:1243] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:55.623096 29079 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.127s
I20260709 12:24:55.738000 29026 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:55.793614 29026 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.794073 29026 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:55.791090 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 222869us
W20260709 12:24:55.795760 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.786000 28663 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.798692 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:55.801373 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.802253 20804 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:55.804622 20804 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:55.805830 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.809428 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.810184 20804 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:55.811137 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.812923 20804 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:55.753453 28989 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:55.779124 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:55.620458 28559 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.130s
I20260709 12:24:55.749577 28814 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:55.756237 28914 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:55.744916 29090 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:24:55.826393 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:55.754594 28870 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:24:55.830300 20804 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 21. Ops: []
W20260709 12:24:55.816289 29037 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.678s	user 0.004s	sys 0.008s
I20260709 12:24:55.831576 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:55.864336 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:55.864406 20807 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:55.820811 29059 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:55.784796 28670 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.816239 29015 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.540863 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:56.544480 29090 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:56.545106 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:56.402666 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.019s	sys 0.023s
W20260709 12:24:56.555950 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.022s	sys 0.031s
I20260709 12:24:56.568996 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:56.546458 28496 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.067s	user 0.012s	sys 0.011s
W20260709 12:24:56.008136 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 191222us
I20260709 12:24:55.832293 28821 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.546931 29023 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:55.892189 21426 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:24:55.965037 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 373560us
I20260709 12:24:55.821718 28914 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:56.405608 20806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.012s	sys 0.070s
W20260709 12:24:56.575943 20806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.012s	sys 0.080s
W20260709 12:24:56.549278 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.576467 20807 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.711s	user 0.008s	sys 0.004s
I20260709 12:24:55.818256 28989 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.539844 29020 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:24:56.345027 20850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.022s
W20260709 12:24:56.280532 20876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.016s	sys 0.011s
I20260709 12:24:55.877000 29037 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:56.409904 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.006s	sys 0.027s
W20260709 12:24:56.586970 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.007s	sys 0.032s
W20260709 12:24:56.587302 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 787239us
I20260709 12:24:56.539399 28876 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:56.542011 28872 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.921s	user 0.016s	sys 0.009s
I20260709 12:24:55.890622 28814 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.151s
I20260709 12:24:55.820541 28663 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:55.913646 29059 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:24:56.539721 29058 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.055s	user 0.017s	sys 0.004s
W20260709 12:24:56.554736 20801 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.944s	user 0.009s	sys 0.010s
I20260709 12:24:56.555307 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:56.576879 20807 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:24:56.576680 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.582065 20850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.013s	sys 0.026s
I20260709 12:24:56.581777 28989 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:56.586020 28914 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:56.575525 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:24:56.565339 29018 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.012s
I20260709 12:24:56.589586 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:56.564342 29090 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.564195 29060 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.564677 28870 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.662539 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:56.663285 29100 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:24:56.664592 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3105) took 1245 ms. Trace:
I20260709 12:24:56.664947 20808 rpcz_store.cc:276] 0709 12:24:55.418957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.419093 (+   136us) service_pool.cc:224] Handling call
0709 12:24:55.419500 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.419934 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.419962 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.420457 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.792651 (+372194us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:55.792696 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.792771 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.664366 (+871595us) raft_consensus.cc:1685] finished
0709 12:24:56.664386 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.664517 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":837,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":9844,"thread_start_us":318168,"threads_started":1,"wal-append.queue_time_us":1112264}
I20260709 12:24:56.665836 29106 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.666411 29106 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.628996 28872 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.667485 28872 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.669727 28872 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:56.670849 28872 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.671357 28870 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.672654 28916 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:56.673645 28870 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.674746 29045 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.674716 28872 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.677404 28872 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.677650 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3314) took 2233 ms. Trace:
I20260709 12:24:56.677968 28872 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.678534 28872 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:56.579739 28969 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.325s	user 0.010s	sys 0.015s
I20260709 12:24:56.627667 21430 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:24:56.678026 20802 rpcz_store.cc:276] 0709 12:24:54.443937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.447227 (+  3290us) service_pool.cc:224] Handling call
0709 12:24:54.466315 (+ 19088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.869426 (+403111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.677414 (+1807988us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54003d68d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:56.677589 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903294848}
W20260709 12:24:56.684399 28909 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.898s	user 0.016s	sys 0.010s
I20260709 12:24:56.684772 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1445) took 1748 ms. Trace:
I20260709 12:24:56.647251 29018 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.685285 21430 rpcz_store.cc:276] 0709 12:24:54.936376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.936549 (+   173us) service_pool.cc:224] Handling call
0709 12:24:55.372996 (+436447us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:24:55.373306 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.683301 (+1309995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.684544 (+  1243us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:56.684720 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859703040}
I20260709 12:24:56.630126 20801 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:24:56.594036 28913 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.842s	user 0.010s	sys 0.017s
W20260709 12:24:56.631790 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 852693us
W20260709 12:24:56.582273 20876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.017s	sys 0.023s
I20260709 12:24:56.564579 29015 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:56.630685 29105 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.066s	user 0.001s	sys 0.001s
I20260709 12:24:56.652823 29090 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:56.686967 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3060) took 1792 ms. Trace:
I20260709 12:24:56.691473 20875 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:56.696996 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3113) took 1134 ms. Trace:
I20260709 12:24:56.611871 29117 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.581760 28496 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:56.583705 29094 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.846s	user 0.006s	sys 0.008s
I20260709 12:24:56.684821 28909 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:56.712384 28845 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.927s	user 0.014s	sys 0.002s
I20260709 12:24:56.712845 28872 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.714798 29015 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.707424 28913 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.687500 29106 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:56.725353 29087 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.199s	user 0.000s	sys 0.003s
I20260709 12:24:56.718097 20801 rpcz_store.cc:276] 0709 12:24:55.563052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.563282 (+   230us) service_pool.cc:224] Handling call
0709 12:24:56.696926 (+1133644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.716992 20735 rpcz_store.cc:276] 0709 12:24:54.894981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.551329 (+656348us) service_pool.cc:224] Handling call
0709 12:24:55.551696 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.735170 (+183474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.686749 (+951579us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54001d8b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:56.686911 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420019456}
I20260709 12:24:56.689195 29037 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.687070 29018 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:56.736307 29048 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:56.782088 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:56.752524 20809 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:56.727555 29021 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:56.770860 20875 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:56.629357 29059 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:56.682180 28914 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.775116 29106 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.786188 21430 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:24:56.786465 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:56.800318 21435 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:56.802193 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:56.802930 21434 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:56.788046 29021 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:56.788587 28964 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:56.789256 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3107) took 1369 ms. Trace:
I20260709 12:24:56.789779 21432 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:24:56.777294 29015 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.792351 29060 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.793298 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3314) took 1435 ms. Trace:
I20260709 12:24:56.793394 29118 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.794098 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:56.806226 20803 rpcz_store.cc:276] 0709 12:24:55.419608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.419853 (+   245us) service_pool.cc:224] Handling call
0709 12:24:55.470505 (+ 50652us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:55.475135 (+  4630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.475168 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:55.496440 (+ 21272us) log.cc:844] Serialized 52 byte log entry
0709 12:24:55.740197 (+243757us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:55.786444 (+ 46247us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.786857 (+   413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.786960 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.676653 (+889693us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b1800404aa0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:24:56.788934 (+112281us) raft_consensus.cc:1685] finished
0709 12:24:56.789054 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.789203 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4123,"prepare.queue_time_us":44623,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":4904,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":15580,"spinlock_wait_cycles":2623744,"thread_start_us":237421,"threads_started":1,"wal-append.queue_time_us":1249160}
I20260709 12:24:56.818048 21427 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:24:56.774786 28872 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:56.793215 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 94288.8us
I20260709 12:24:56.791661 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1460) took 1516 ms. Trace:
I20260709 12:24:56.750458 28845 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.742931 29117 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.817408 21432 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:56.782783 21428 raft_consensus.cc:2376] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20. Re-sending same reply.
I20260709 12:24:56.799912 29018 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.829032 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:56.832891 20882 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:56.806036 28964 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:56.814764 20727 rpcz_store.cc:276] 0709 12:24:55.357936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.554894 (+196958us) service_pool.cc:224] Handling call
0709 12:24:55.564899 (+ 10005us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.603448 (+ 38549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.603478 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.604103 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.789545 (+185442us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:55.789588 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.789687 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.793102 (+1003415us) raft_consensus.cc:1685] finished
0709 12:24:56.793120 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.793241 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37687,"raft.queue_time_us":40933,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":61117,"thread_start_us":185042,"threads_started":1,"wal-append.queue_time_us":1133355}
I20260709 12:24:56.822942 20883 rpcz_store.cc:276] 0709 12:24:55.275420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.543597 (+268177us) service_pool.cc:224] Handling call
0709 12:24:55.543958 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.544179 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:55.749096 (+204917us) log.cc:844] Serialized 14 byte log entry
0709 12:24:56.791170 (+1042074us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b1800121d80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:56.791348 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:56.791401 (+    53us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:56.791443 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.791503 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.791609 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8439,"prepare.queue_time_us":284985,"prepare.run_cpu_time_us":12789,"prepare.run_wall_time_us":685026,"raft.queue_time_us":235942,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"spinlock_wait_cycles":34030592,"thread_start_us":1011939,"threads_started":2,"wal-append.queue_time_us":1032156}
I20260709 12:24:56.829605 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:56.838379 20880 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:56.821774 29120 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.839249 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:56.836640 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1461) took 1265 ms. Trace:
I20260709 12:24:56.843083 20878 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:56.835148 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:56.843343 20876 rpcz_store.cc:276] 0709 12:24:55.571182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.571360 (+   178us) service_pool.cc:224] Handling call
0709 12:24:55.571778 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.572138 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.572168 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.595096 (+ 22928us) log.cc:844] Serialized 26 byte log entry
0709 12:24:56.699436 (+1104340us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:56.699493 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.699563 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.836433 (+136870us) raft_consensus.cc:1685] finished
0709 12:24:56.836454 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.836578 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6680,"raft.queue_time_us":449,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":8168,"thread_start_us":1019779,"threads_started":1,"wal-append.queue_time_us":1161018}
I20260709 12:24:56.826141 28670 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.840214 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:56.806047 29106 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.868819 29120 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.874006 20880 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:56.875561 29120 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.880407 28914 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.881125 29117 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.881341 29107 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.881910 29107 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.872063 29113 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:56.884172 29107 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.833429 29112 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.887765 28496 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.887506 29112 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.889477 28845 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.889664 29112 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.824569 29060 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.890193 29112 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.884320 29113 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:56.897265 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 321796us
I20260709 12:24:56.836042 28909 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:56.825623 29015 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.823573 29118 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.838838 29018 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.837131 28872 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:56.891247 29026 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.097s	user 0.014s	sys 0.008s
I20260709 12:24:56.815546 29033 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.919831 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:56.909017 29119 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.914565 28496 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:24:56.927461 29018 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:56.929459 28872 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:56.923297 29026 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.932648 21433 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:56.890663 29112 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.912711 29120 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:56.929800 21426 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:24:56.915763 29048 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:56.969319 29048 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:56.969913 29048 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:56.935504 29106 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.932668 29113 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.972774 29113 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.973266 29113 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.975447 28663 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.978719 29112 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:56.990855 29096 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.114s	user 0.010s	sys 0.003s
I20260709 12:24:56.991315 29112 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.992071 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3054) took 2098 ms. Trace:
I20260709 12:24:56.992055 29112 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.992566 20730 rpcz_store.cc:276] 0709 12:24:54.893609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.253907 (+360298us) service_pool.cc:224] Handling call
0709 12:24:55.254271 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.299748 (+ 45477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.299795 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.314905 (+ 15110us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.631283 (+316378us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:55.631347 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.631454 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.990251 (+1358797us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:56.991764 (+  1513us) raft_consensus.cc:1685] finished
0709 12:24:56.991783 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.992004 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31912,"raft.queue_time_us":45482,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4443,"spinlock_wait_cycles":338168576,"thread_start_us":283323,"threads_started":1,"wal-append.queue_time_us":433569}
I20260709 12:24:56.994774 29112 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:56.999728 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79976.6us
I20260709 12:24:57.000980 28602 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:57.001555 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:57.001958 29119 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.002451 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:57.002929 28852 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:57.003280 29117 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:57.003664 29117 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.003944 29106 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:57.004284 29119 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:57.004642 28873 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:24:56.938591 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103505us
I20260709 12:24:56.848783 29021 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.005071 28873 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.958256 28913 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.005476 28873 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:57.006269 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3249) took 1715 ms. Trace:
I20260709 12:24:56.959734 29026 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.017311 20801 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:56.959826 29118 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:56.960551 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3114) took 1397 ms. Trace:
I20260709 12:24:56.961504 29015 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:56.962287 29130 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.963320 29090 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.968235 29059 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:56.930763 29116 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:24:56.980816 29060 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.918488 28964 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:56.981622 29056 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:56.986212 29113 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.989264 29030 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:56.991547 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1520) took 1193 ms. Trace:
I20260709 12:24:57.006620 20875 rpcz_store.cc:276] 0709 12:24:55.291057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.291437 (+   380us) service_pool.cc:224] Handling call
0709 12:24:55.291930 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.785069 (+1493139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.006037 (+220968us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540026d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:57.006204 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3218261888}
I20260709 12:24:56.906948 28670 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:57.019335 29120 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:56.949057 20878 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:57.011211 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:57.060286 20807 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:57.012552 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.061878 20808 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:57.013865 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.015148 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.023944 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.026172 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:57.026535 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:57.028410 28989 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.004009 29117 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:57.031674 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:57.075125 20803 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:57.019851 20810 rpcz_store.cc:276] 0709 12:24:55.563485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.563617 (+   132us) service_pool.cc:224] Handling call
0709 12:24:55.601377 (+ 37760us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:56.547908 (+946531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.547938 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:56.565137 (+ 17199us) log.cc:844] Serialized 52 byte log entry
0709 12:24:56.807438 (+242301us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:56.807479 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.807553 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.960308 (+152755us) raft_consensus.cc:1685] finished
0709 12:24:56.960328 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.960480 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1650,"raft.queue_time_us":147987,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":27136,"thread_start_us":591558,"threads_started":2,"wal-append.queue_time_us":333101}
I20260709 12:24:57.038151 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:57.040235 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:57.063030 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.019104 28814 consensus_queue.cc:1243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 28.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:24:57.008273 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:57.022840 28872 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:57.023801 29072 consensus_queue.cc:1175] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 22 status { last_received { term: 21 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:24:57.081604 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.024056 28663 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:57.035488 29018 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.024590 29041 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 12:24:57.036453 29058 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.031934 29118 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:57.087648 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.063673 20805 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:55.335387 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:57.089381 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.064888 20804 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:24:57.072538 20807 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:24:57.076520 20808 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:57.076459 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:57.077700 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 237571us
I20260709 12:24:57.058280 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.055979 20806 rpcz_store.cc:276] 0709 12:24:55.797900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.819618 (+ 21718us) service_pool.cc:224] Handling call
0709 12:24:55.820081 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.671167 (+851086us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b180039d240. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1ec97bfb 00007f9e1eca264b
0709 12:24:56.684730 (+ 13563us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:56.685041 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 12:24:56.897730 (+212689us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:56.898376 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 12:24:56.898496 (+   120us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:56.905430 (+  6934us) log.cc:844] Serialized 14 byte log entry
0709 12:24:56.905656 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.905742 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.991330 (+ 85588us) raft_consensus.cc:1685] finished
0709 12:24:56.991351 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.991491 (+   140us) inbound_call.cc:177] Queueing success response
0709 12:24:56.993677 (+  2186us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b180039d240. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e22b7358f 00007f9e1bbb2571 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd
Metrics: {"mutex_wait_us":73105,"prepare.queue_time_us":7414,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"raft.queue_time_us":805747,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":15292,"spinlock_wait_cycles":24691584,"thread_start_us":895840,"threads_started":2,"wal-append.queue_time_us":239935}
W20260709 12:24:57.080575 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 72378.7us
I20260709 12:24:57.035511 29130 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.091667 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:57.092113 20801 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 12:24:57.082602 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.46954e+06us
I20260709 12:24:57.056257 29124 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.038010 29112 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:57.064445 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:57.067788 28964 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:57.083278 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3051) took 2190 ms. Trace:
I20260709 12:24:57.056655 29131 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.085583 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:57.103451 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.054090 29011 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:24:57.083355 28814 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.096930 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.101631 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.101948 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.104101 20884 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:57.085433 29128 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.104821 20881 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 12:24:57.079977 29031 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.018s
I20260709 12:24:57.147986 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:57.152592 20880 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:57.061545 28670 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.106047 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.068264 29120 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.097869 29132 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.106415 20624 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 20 to 21, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 21 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:24:57.118578 21428 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:57.100032 29107 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:57.085522 28663 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.154075 20877 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:57.201820 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.086550 29018 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.103361 20728 rpcz_store.cc:276] 0709 12:24:54.892867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.155398 (+262531us) service_pool.cc:224] Handling call
0709 12:24:55.155716 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.155922 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:55.312667 (+156745us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.600594 (+287927us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.003330 (+1402736us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b18001c2820. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:24:57.079590 (+ 76260us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:57.079651 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:57.079710 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.083059 (+  3349us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.083182 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1357237,"prepare.queue_time_us":3975,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":7232,"raft.queue_time_us":1430052,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":64144,"spinlock_wait_cycles":663241472,"thread_start_us":512101,"threads_started":3,"wal-append.queue_time_us":436427}
I20260709 12:24:57.159868 20875 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:57.102877 29114 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:57.172278 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.172564 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.098585 29116 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.079s
I20260709 12:24:57.204051 29125 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.107353 29112 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.206929 29125 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.207499 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:24:57.173740 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.208271 20880 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:57.208652 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:57.209221 21428 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:57.209477 29112 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.173031 29131 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.211906 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:57.212558 21429 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:57.177022 29137 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:57.213992 29131 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.211920 29090 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.163805 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:57.099337 29119 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.215862 29119 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.216799 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.218129 29119 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.128129 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.219776 20879 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:57.144795 29011 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:57.106514 29015 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:57.098431 29030 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:57.083796 28872 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.154934 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:24:57.106758 28771 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:57.128495 29122 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.229511 20880 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:57.129225 29130 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:57.202669 20877 raft_consensus.cc:2376] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24. Re-sending same reply.
I20260709 12:24:57.235926 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:57.236663 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.134636 28873 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.206393 20884 raft_consensus.cc:2376] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25. Re-sending same reply.
I20260709 12:24:57.238045 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.087355 29113 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:57.100215 28916 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:24:57.099892 29023 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:24:57.243943 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:57.192387 28913 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.244609 20878 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:57.093818 29058 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:57.093730 29118 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.227022 21426 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:24:57.228924 29091 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.436s	user 0.000s	sys 0.015s
I20260709 12:24:57.097872 29129 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:57.229059 28989 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.231936 20875 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:24:57.251259 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.133569 29048 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.254779 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:57.193442 29132 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.257048 20884 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:57.234364 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 148847us
I20260709 12:24:57.177146 20803 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:57.209332 29125 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.240309 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:24:57.261471 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.106212 29124 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.173719 29128 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.158028 28814 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.248086 20877 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:57.226504 29015 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:57.226959 29011 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.252344 20880 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:57.231844 29030 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.242841 21429 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:57.247684 28914 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:57.266244 20878 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:24:57.235368 29122 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.251947 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:57.270918 29099 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.358s	user 0.006s	sys 0.009s
W20260709 12:24:57.271924 29127 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 12:24:57.275403 29015 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:57.277129 28872 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.278542 28670 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.323760 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.280553 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:57.281723 28663 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.283313 29120 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.284368 28873 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.285112 29118 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.286204 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.342957 20877 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:57.346649 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:57.287909 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.292023 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.292793 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.232050 28771 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.263927 29045 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:57.298035 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:57.259802 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:24:57.355667 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57741.8us
W20260709 12:24:57.355659 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95922.5us
I20260709 12:24:57.344149 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.357604 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:57.305274 20884 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:57.357857 20877 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:57.359351 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.266752 29059 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:57.309324 29132 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:57.309463 29011 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.309849 29122 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.356477 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:57.309978 28914 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:57.249164 29113 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.312211 29058 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:57.314888 29103 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.750s	user 0.000s	sys 0.002s
I20260709 12:24:57.315670 29030 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.340183 28670 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:57.315734 28814 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.253746 29107 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:24:57.316097 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:57.319988 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.378273 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:57.280184 29018 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:57.324867 20879 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:57.327008 28851 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:57.327852 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:57.328130 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47650.6us
I20260709 12:24:57.254716 28913 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.310946 29124 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:57.337656 21428 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:24:57.351394 20808 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:24:57.384574 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.385445 20808 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:57.300895 20878 pending_rounds.cc:85] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 12
I20260709 12:24:57.307679 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:57.354370 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:57.323980 29129 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:57.366871 20881 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:57.366103 28985 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:57.357215 29118 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.373958 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.374766 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.375566 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.380532 20876 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:57.387813 20878 pending_rounds.cc:107] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:24:57.386795 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.360168 20877 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:57.367719 29119 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:57.796082 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:58.099255 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.026s	sys 0.078s
W20260709 12:24:57.476965 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.163111 20698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.008s	sys 0.007s
W20260709 12:24:58.161679 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.030s	sys 0.085s
W20260709 12:24:57.762056 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.391896 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:57.370040 28670 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.164904 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:58.165055 20802 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:24:57.560278 20881 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:24:58.166096 20878 raft_consensus.cc:2889] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:57.557051 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.160625 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1458) took 1370 ms. Trace:
I20260709 12:24:58.151259 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:58.142628 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 533341us
W20260709 12:24:58.163514 20698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.008s	sys 0.007s
W20260709 12:24:57.401322 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47013.7us
I20260709 12:24:57.830557 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:57.396083 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:24:58.146245 21427 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.005s	sys 0.005s
I20260709 12:24:57.363823 29059 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:57.398008 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:58.142884 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.371552 29060 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:58.105176 21431 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.005s	sys 0.000s
I20260709 12:24:58.160204 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3297) took 3343 ms. Trace:
I20260709 12:24:57.396829 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:57.850996 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.171100 20806 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:24:58.172849 21428 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:58.170637 21433 rpcz_store.cc:276] 0709 12:24:56.790479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.791324 (+   845us) service_pool.cc:224] Handling call
0709 12:24:56.791613 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.939624 (+148011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.160344 (+1220720us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:58.160559 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323370880}
I20260709 12:24:57.366333 28989 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:57.367450 29045 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:58.174816 20877 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:58.228041 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3058) took 3333 ms. Trace:
I20260709 12:24:58.170444 20805 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
W20260709 12:24:57.477897 29097 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.651s	user 0.001s	sys 0.014s
I20260709 12:24:57.385851 29058 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.185443 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:58.265126 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3211) took 1156 ms. Trace:
W20260709 12:24:58.220974 21427 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.006s	sys 0.006s
W20260709 12:24:57.395965 29085 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.953s	user 0.001s	sys 0.002s
W20260709 12:24:58.323248 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 137834us
I20260709 12:24:58.321308 20876 rpcz_store.cc:276] 0709 12:24:57.108422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.166371 (+1057949us) service_pool.cc:224] Handling call
0709 12:24:58.166714 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.184201 (+ 17487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.184238 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:58.184283 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:58.184318 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.264778 (+ 80460us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54003f6e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:58.264846 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.265040 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39700736}
W20260709 12:24:58.160737 29011 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.004s	sys 0.009s
I20260709 12:24:57.419256 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3104) took 2001 ms. Trace:
I20260709 12:24:58.226375 20802 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:24:58.290761 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:24:58.109567 20880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.007s	sys 0.013s
I20260709 12:24:57.504588 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:58.376694 20880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.010s	sys 0.016s
W20260709 12:24:58.232043 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60985.4us
I20260709 12:24:58.376394 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:58.231047 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.385300 29030 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.288602 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:57.501989 28870 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:58.186101 28873 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.201s
I20260709 12:24:58.185839 28943 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.314530 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:57.611403 29037 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.324779 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:58.261152 28970 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.606621 28559 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:58.328109 28944 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.378324 20735 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:24:58.375855 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.379451 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:58.310868 20726 rpcz_store.cc:276] 0709 12:24:54.894532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.303970 (+409438us) service_pool.cc:224] Handling call
0709 12:24:55.307621 (+  3651us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:55.317321 (+  9700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.317377 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:55.415046 (+ 97669us) log.cc:844] Serialized 52 byte log entry
0709 12:24:55.615389 (+200343us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:55.681338 (+ 65949us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.270241 (+1588903us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:58.227416 (+957175us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b54002b9e20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1a733 00007f9e1ec9fdbf 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c
0709 12:24:58.227495 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.227625 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.227704 (+    79us) raft_consensus.cc:1685] finished
0709 12:24:58.227725 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.227952 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2533564,"prepare.queue_time_us":63552,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":49,"raft.queue_time_us":1926452,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":14362,"spinlock_wait_cycles":1614864384,"thread_start_us":130584,"threads_started":1,"wal-append.queue_time_us":321866}
I20260709 12:24:57.401748 29018 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:58.168741 28670 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:58.381260 20728 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:58.317008 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3215) took 1207 ms. Trace:
I20260709 12:24:58.381599 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.397985 20877 rpcz_store.cc:276] 0709 12:24:57.109310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.174294 (+1064984us) service_pool.cc:224] Handling call
0709 12:24:58.174627 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.316723 (+142096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.316950 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.291033 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.327585 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1976) took 1198 ms. Trace:
I20260709 12:24:57.475656 29121 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.380982 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.376616 20809 rpcz_store.cc:276] 0709 12:24:55.418455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.419758 (+  1303us) service_pool.cc:224] Handling call
0709 12:24:55.466022 (+ 46264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.785112 (+1319090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.418991 (+633879us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540019a0d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:57.419162 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2958213376}
I20260709 12:24:58.185750 29041 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.185643 29067 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.384565 20804 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:58.161576 29021 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:57.605147 29079 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.739406 29072 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.382429 28821 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:24:58.320469 29060 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:57.406715 29113 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:57.469312 29128 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:58.310763 20733 rpcz_store.cc:276] 0709 12:24:54.798943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.889704 (+ 90761us) service_pool.cc:224] Handling call
0709 12:24:54.936877 (+ 47173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.963162 (+ 26285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.965925 (+  2763us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.541125 (+575200us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:24:55.541589 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.761407 (+219818us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:55.761468 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.761574 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.314263 (+1552689us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:58.141830 (+827567us) raft_consensus.cc:1685] finished
0709 12:24:58.141852 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.142002 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60484736,"thread_start_us":219358,"threads_started":1,"wal-append.queue_time_us":277441}
W20260709 12:24:58.376649 29011 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.008s	sys 0.013s
I20260709 12:24:58.236977 28875 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.185541 29039 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.245528 28876 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.279932 29107 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:58.390925 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76485.1us
I20260709 12:24:57.378180 29122 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:24:58.245358 29136 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.185279 29135 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:58.231916 28845 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:58.312572 21431 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.008s	sys 0.001s
I20260709 12:24:57.469857 29048 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.244678 28771 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.227326 29059 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.387382 28851 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.611735 28962 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.773s
I20260709 12:24:58.376677 28909 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:58.322978 28877 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.471314 29015 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.245121 28913 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:58.400688 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 255456us
W20260709 12:24:58.459671 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.401028 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:58.461055 20726 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:24:58.380599 28989 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:24:58.399048 21428 rpcz_store.cc:276] 0709 12:24:57.119310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.384134 (+264824us) service_pool.cc:224] Handling call
0709 12:24:58.317242 (+933108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.474797 29117 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:58.407764 20728 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:57.407771 28914 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.416072 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.470827 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:57.406539 29129 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:57.796782 28602 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:58.182852 29118 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.386981 29030 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.477309 28602 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.478674 29118 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.478889 29030 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:58.479651 20879 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:24:58.388481 28870 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:24:58.379485 29058 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.387511 28893 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:24:58.401986 29018 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.433562 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.396118 28559 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:58.435010 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.436256 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.470008 29048 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:58.439363 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:58.498169 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58871us
I20260709 12:24:58.447253 29136 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.500489 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.501641 20733 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:58.408349 29121 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.502525 29136 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.446954 20884 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:24:58.399299 29037 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.485386 29113 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.508244 29037 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.440773 28873 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:58.424033 29024 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:58.512084 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1985) took 1254 ms. Trace:
I20260709 12:24:58.403729 28944 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.396648 28943 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:24:58.509168 29113 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.525838 28943 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.409740 29067 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.526333 20731 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:58.516392 29058 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:58.528267 29067 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.460494 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:58.529183 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68745.1us
I20260709 12:24:58.529948 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:58.532059 20734 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:58.525882 28944 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.466727 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.467823 28913 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:58.451457 28851 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.440933 20624 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 23 to 25, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 25 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.456058 29059 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.443584 29107 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.500411 28602 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.431509 29079 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.470129 21432 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:58.472005 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.541636 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:58.415567 29041 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.531371 28873 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.544414 28873 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.544556 29041 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.432667 29072 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.545156 29079 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.547034 29072 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.408872 29045 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.547497 20728 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:58.474682 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.455510 28909 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:58.462102 29148 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:24:58.479651 20882 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:24:58.441464 28875 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:58.604018 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63098us
I20260709 12:24:58.486223 20735 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:58.470638 28854 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.487351 28962 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:58.488687 29033 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:58.489387 28870 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.458715 28877 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.491919 29060 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:58.492288 29129 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.471076 28989 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.635219 28851 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.604444 28909 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:58.635425 28989 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.443764 29039 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.636003 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:58.637703 29039 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.404477 29150 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:58.638433 29150 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.447899 29135 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.551397 28602 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.640311 28602 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.640878 29135 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.454614 28845 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.642513 28602 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.400907 28970 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.516778 21435 rpcz_store.cc:276] 0709 12:24:57.258152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.620302 (+362150us) service_pool.cc:224] Handling call
0709 12:24:57.620691 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.621155 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.621195 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.153076 (+531881us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.383163 (+230087us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:58.383230 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.383297 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.511758 (+128461us) raft_consensus.cc:1685] finished
0709 12:24:58.511797 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.512008 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29282,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":62057,"thread_start_us":154757,"threads_started":1,"wal-append.queue_time_us":262447}
I20260709 12:24:58.533658 28913 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.537202 29024 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:58.513222 29147 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 12:24:58.504114 29048 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.496026 29018 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:58.469444 29015 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:58.537523 29058 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:58.527024 29113 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.542119 20726 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:24:58.603690 20729 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:58.443938 28876 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.605223 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:24:58.606487 29118 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:58.643042 28845 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.656010 29058 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.656243 29059 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.656531 29058 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:58.656610 29048 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:58.658424 28970 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.656370 28913 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:58.659790 28877 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.659936 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.660710 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1539) took 1384 ms. Trace:
I20260709 12:24:58.661151 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.661850 20726 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:24:58.661067 20807 rpcz_store.cc:276] 0709 12:24:57.276292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.296051 (+ 19759us) service_pool.cc:224] Handling call
0709 12:24:57.296387 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:57.296739 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.296768 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:57.297438 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 12:24:57.535159 (+237721us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:57.722474 (+187315us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.723100 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.723235 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.660500 (+937265us) raft_consensus.cc:1685] finished
0709 12:24:58.660516 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.660657 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31282,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":1214,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":53914,"thread_start_us":189637,"threads_started":1,"wal-append.queue_time_us":895336}
I20260709 12:24:58.662703 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.644809 29150 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.661183 28962 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:58.629082 28854 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.664536 28962 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.665014 29121 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:58.665771 29121 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:58.666164 29121 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:58.665977 28909 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.666577 29121 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:58.667209 20727 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:24:58.667522 20735 raft_consensus.cc:2376] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23. Re-sending same reply.
I20260709 12:24:58.668061 29165 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:58.668869 28876 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.670212 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.670811 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3209) took 1562 ms. Trace:
I20260709 12:24:58.671064 20729 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:58.646116 28875 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:58.672437 20729 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:58.538527 29030 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:58.550971 28559 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:58.502841 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.643118 28602 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:24:58.651749 28870 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:24:58.630560 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 465687us
I20260709 12:24:58.675787 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3252) took 1573 ms. Trace:
W20260709 12:24:58.690510 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 365803us
I20260709 12:24:58.689348 20883 rpcz_store.cc:276] 0709 12:24:57.102493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.102607 (+   114us) service_pool.cc:224] Handling call
0709 12:24:57.147489 (+ 44882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.301290 (+153801us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:57.353697 (+ 52407us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.164056 (+810359us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.164592 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.164738 (+   146us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:58.164769 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.416144 (+251375us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b18002e8940. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:24:58.556291 (+140147us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400366d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:58.556379 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.675496 (+119117us) raft_consensus.cc:1685] finished
0709 12:24:58.675526 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.675714 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":61826,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":18790,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":19800,"spinlock_wait_cycles":357740672,"thread_start_us":820813,"threads_started":2,"wal-append.queue_time_us":924293}
I20260709 12:24:58.691509 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:58.605103 29072 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:58.629801 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 251086us
W20260709 12:24:58.678000 21401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.022s	sys 0.059s
W20260709 12:24:58.683885 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78750.7us
W20260709 12:24:58.724182 21401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.022s	sys 0.059s
I20260709 12:24:58.685489 20726 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:24:58.673995 20878 rpcz_store.cc:276] 0709 12:24:57.107971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.292279 (+184308us) service_pool.cc:224] Handling call
0709 12:24:57.292627 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:58.222392 (+929765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.222436 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:58.223218 (+   782us) log.cc:844] Serialized 52 byte log entry
0709 12:24:58.397718 (+174500us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:58.411942 (+ 14224us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.439148 (+ 27206us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.439390 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.669326 (+229936us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b180032f980. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:24:58.670500 (+  1174us) spinlock_profiling.cc:243] Waited 8.03 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:58.670549 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.670596 (+    47us) raft_consensus.cc:1685] finished
0709 12:24:58.670611 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.670753 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25036,"prepare.queue_time_us":82046,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":9838,"raft.queue_time_us":53682,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":11105,"spinlock_wait_cycles":26570240,"thread_start_us":172601,"threads_started":1,"wal-append.queue_time_us":240692}
I20260709 12:24:58.705966 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:58.731839 20805 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:58.734306 20849 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:24:58.734725 20849 leader_election.cc:403] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:24:58.663383 29117 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.684610 29030 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.722657 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:58.661077 29107 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:24:58.721294 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.670019 29165 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.022s
W20260709 12:24:58.724980 29119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.008s	sys 0.011s
I20260709 12:24:58.741894 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:58.755851 20802 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:58.742826 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:58.756870 20809 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:58.687069 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.005s
I20260709 12:24:58.745677 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:58.758023 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.010s	sys 0.005s
I20260709 12:24:58.758384 20726 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:24:58.748185 20807 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:58.731429 28559 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:58.688921 29011 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:58.803365 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:58.804872 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.806350 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:58.758491 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 607285us
I20260709 12:24:58.724010 29015 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:58.749125 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57563.6us
I20260709 12:24:58.812544 20808 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:58.760116 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:24:58.747906 20806 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:58.740356 29060 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:24:58.807932 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:58.784288 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:58.823140 20883 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:58.825450 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1479) took 1481 ms. Trace:
I20260709 12:24:58.826074 20879 rpcz_store.cc:276] 0709 12:24:57.343881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.396566 (+ 52685us) service_pool.cc:224] Handling call
0709 12:24:58.825177 (+1428611us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:58.825362 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381906688}
I20260709 12:24:58.771513 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:58.812569 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:58.828171 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:58.829012 20879 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:24:58.830392 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:58.831984 20879 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:24:58.816655 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 385713us
I20260709 12:24:58.833287 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:58.834359 20879 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:58.816412 20805 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:24:58.835332 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:58.836351 20808 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:24:58.752168 29030 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:58.846202 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:58.846346 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:58.847726 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 87739.8us
I20260709 12:24:58.855482 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:58.753587 29113 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:24:58.858621 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50806.7us
I20260709 12:24:58.864943 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:58.846098 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:24:58.860961 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89571.6us
I20260709 12:24:58.760574 29119 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:58.737607 29072 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:58.884402 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3258) took 1777 ms (client timeout 1857 ms). Trace:
I20260709 12:24:58.868487 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:24:58.884917 20881 rpcz_store.cc:269] 0709 12:24:57.107273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.216527 (+109254us) service_pool.cc:224] Handling call
0709 12:24:58.884192 (+1667665us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540019d7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:58.884343 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343513856}
I20260709 12:24:58.886643 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:58.888046 20881 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 17.
I20260709 12:24:58.888860 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:58.892248 20883 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:58.892570 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1974) took 1773 ms. Trace:
I20260709 12:24:58.893373 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:58.894349 20883 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:58.836756 29059 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:58.812539 29011 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:58.911744 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:58.912971 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:24:58.890820 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 138029us
I20260709 12:24:58.893059 21431 rpcz_store.cc:276] 0709 12:24:57.118759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.118886 (+   127us) service_pool.cc:224] Handling call
0709 12:24:57.119291 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:57.231994 (+112703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.232024 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:57.248956 (+ 16932us) log.cc:844] Serialized 52 byte log entry
0709 12:24:57.383230 (+134274us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:58.453882 (+1070652us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.892040 (+438158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.892295 (+   255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.892341 (+    46us) raft_consensus.cc:1685] finished
0709 12:24:58.892354 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.892514 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":844393,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":246028,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":5469,"thread_start_us":1579812,"threads_started":4,"wal-append.queue_time_us":226536}
I20260709 12:24:58.900575 20805 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:58.904294 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:58.845912 28909 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:58.899081 20809 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:58.920182 20806 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:24:58.920570 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:58.916734 20728 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:58.917569 21428 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:24:58.914389 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:58.846437 29122 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:58.812924 28821 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 12:24:58.838435 29015 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.876729 29113 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:58.929685 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:59.004410 29169 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.920303 28876 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:58.921029 20726 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:58.886565 29060 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:58.862737 29107 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.876077 29021 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.138s
I20260709 12:24:59.004689 20733 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:59.008752 20879 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:58.947980 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:59.010504 20883 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:24:59.009292 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95029.6us
W20260709 12:24:58.990155 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47915.7us
I20260709 12:24:58.991050 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.012770 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3057) took 4118 ms. Trace:
I20260709 12:24:58.974859 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:58.986567 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.015839 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.014585 20731 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:58.863636 28875 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:58.939450 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.006204 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.023131 20802 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:59.042825 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.045384 20805 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:24:59.038380 20733 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:24:59.006567 29113 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:24:59.053337 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 188510us
W20260709 12:24:59.054247 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 167646us
I20260709 12:24:58.889731 28970 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:59.055595 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.056757 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.065445 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.054435 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:59.013351 20732 rpcz_store.cc:276] 0709 12:24:54.894304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.267540 (+373236us) service_pool.cc:224] Handling call
0709 12:24:55.267856 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.284225 (+ 16369us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:55.303605 (+ 19380us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.548949 (+245344us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b18000d6120. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:24:57.598306 (+2049357us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.174142 (+575836us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.627119 (+452977us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:58.900453 (+273334us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.913251 (+ 12798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.913387 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.012546 (+ 99159us) raft_consensus.cc:1685] finished
0709 12:24:59.012568 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.012704 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1864433,"prepare.queue_time_us":22393,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":1881726,"raft.run_cpu_time_us":501,"raft.run_wall_time_us":83237,"spinlock_wait_cycles":7226752,"thread_start_us":938452,"threads_started":3,"wal-append.queue_time_us":988595,"wal-append.run_cpu_time_us":54589,"wal-append.run_wall_time_us":1861995}
I20260709 12:24:59.062044 20879 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:24:59.063990 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.067687 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:59.082219 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:24:59.084165 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:58.904986 29134 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:59.110004 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:59.112819 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54545.3us
I20260709 12:24:59.016510 29117 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:59.006922 28876 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.117712 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:58.949415 29079 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.180s
I20260709 12:24:59.112026 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.148888 20731 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:24:59.111598 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.161700 20809 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
W20260709 12:24:59.162674 20882 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1476) took 1837 ms (client timeout 2194 ms). Trace:
W20260709 12:24:59.163305 20882 rpcz_store.cc:269] 0709 12:24:57.324890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.397753 (+ 72863us) service_pool.cc:224] Handling call
0709 12:24:59.162512 (+1764759us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54001fe854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:59.162617 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391474432}
I20260709 12:24:59.112335 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.164389 20806 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:59.164479 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:59.162576 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.165576 20805 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:24:58.941148 28845 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.199s
I20260709 12:24:59.171324 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:59.172205 20733 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:59.173019 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.025875 28875 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.028515 29165 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:59.153297 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.185045 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:59.008893 29011 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:58.922950 29082 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:59.154420 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:24:59.185789 21428 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:59.160560 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:24:58.942380 28913 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:59.149427 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 281042us
I20260709 12:24:59.194453 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:24:59.118201 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.158706 28876 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:59.169986 29186 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:59.171083 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.007597 29060 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.174214 20731 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:59.233594 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:59.234750 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.178846 21426 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:24:59.179589 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.235877 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.236229 20805 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:59.061121 29162 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:59.237201 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:59.055166 29161 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:24:59.189951 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:24:59.193310 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:59.243571 20882 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:59.007871 29107 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:59.150372 29117 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:24:59.005008 29169 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:59.196576 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:59.209064 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:59.209942 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.281739 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.211401 21433 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:59.214078 21428 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:24:59.174239 29149 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.732s	user 0.006s	sys 0.005s
I20260709 12:24:59.110819 29159 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:59.007212 28602 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.152597 28970 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:24:59.219107 21400 connection.cc:256] RPC call timeout handler was delayed by -1.8664013490001707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
I20260709 12:24:59.214910 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:59.292671 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.221552 20804 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:59.221884 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.199795 29165 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:59.222719 28913 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:59.208113 29122 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.13, Last appended by leader: 13, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:59.230620 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:59.231989 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.241519 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:59.249272 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.198578 29160 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:59.190418 28875 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:59.224818 28893 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:59.251993 29113 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.195385 29158 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:59.255251 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.380215 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.380411 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:24:59.380508 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.381703 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.187618 29024 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:59.259349 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.253000 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:59.269068 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.246954 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:24:59.386000 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 139147us
I20260709 12:24:59.276674 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:59.158016 29134 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.389945 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.283890 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3218) took 2173 ms. Trace:
W20260709 12:24:59.288666 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 131779us
I20260709 12:24:59.387305 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:24:59.240483 29162 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.216188 29082 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:24:59.293099 21400 connection.cc:256] RPC call timeout handler was delayed by -1.7944013490000543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 12:24:59.112468 29119 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.252331 29174 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:24:59.401700 21400 connection.cc:256] RPC call timeout handler was delayed by -1.720601348999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 12:24:59.403817 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.303834 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:24:59.305449 20803 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:59.244484 29188 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:59.312606 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:24:59.411273 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 255362us
I20260709 12:24:59.316336 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:24:59.311102 29160 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:59.319821 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:59.414726 20731 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:59.320571 29121 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.001s
W20260709 12:24:59.322489 29018 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.000s
I20260709 12:24:59.325055 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.326581 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:59.327826 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:59.329246 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3322) took 1040 ms. Trace:
I20260709 12:24:59.330982 29059 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:59.331854 29015 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:59.420166 20876 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:24:59.420321 20727 rpcz_store.cc:276] 0709 12:24:58.289346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.289498 (+   152us) service_pool.cc:224] Handling call
0709 12:24:59.328992 (+1039494us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54001d8b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:59.329172 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621014912}
I20260709 12:24:59.427424 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:24:59.428058 20727 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:59.332803 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:59.332928 29072 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:24:59.335619 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:59.340652 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.433476 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.434762 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.342217 29117 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:24:59.349171 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1977) took 2208 ms (client timeout 1508 ms). Trace:
I20260709 12:24:59.354203 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.437388 20734 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:24:59.355428 28970 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:59.331007 29030 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:59.292099 29159 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:59.441450 29159 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:59.280213 29186 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:24:59.442727 29030 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:24:59.443535 20881 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:24:59.443260 29186 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.357929 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3207) took 2250 ms. Trace:
I20260709 12:24:59.361954 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:24:59.445755 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.446936 29166 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.447176 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.447556 29166 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.448874 20731 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:24:59.354203 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:59.449863 29011 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.450057 28876 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.381677 20732 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:24:59.393193 29048 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.736s	user 0.008s	sys 0.003s
I20260709 12:24:59.392865 29158 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.301823 29165 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:59.399029 20805 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:24:59.396791 20877 rpcz_store.cc:276] 0709 12:24:57.109958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.403874 (+1293916us) service_pool.cc:224] Handling call
0709 12:24:58.404161 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.404309 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:58.851218 (+446909us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.283498 (+432280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.283559 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:59.283605 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.283686 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.283817 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15899,"prepare.queue_time_us":47947,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2211,"raft.queue_time_us":3636,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":18543,"thread_start_us":846399,"threads_started":2}
I20260709 12:24:59.400538 20701 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:24:59.408885 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:59.410540 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106796us
W20260709 12:24:59.395210 29033 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.000s
I20260709 12:24:59.412146 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:24:59.408392 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:24:59.391314 28893 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:24:59.319644 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:59.419566 20883 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:24:59.341629 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3210) took 2233 ms. Trace:
I20260709 12:24:59.417968 20726 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:59.439599 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:24:59.436478 21432 rpcz_store.cc:269] 0709 12:24:57.141252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.152975 (+ 11723us) service_pool.cc:224] Handling call
0709 12:24:59.348898 (+2195923us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:24:59.349076 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2884593024}
I20260709 12:24:59.405529 28985 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:59.301090 28602 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.406129 29079 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 12:24:59.398138 29113 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:24:59.447942 20730 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:24:59.412212 29191 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:24:59.450969 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 440984us
I20260709 12:24:59.484097 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 1980) took 2341 ms. Trace:
W20260709 12:24:59.634093 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 437563us
I20260709 12:24:59.979799 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3129) took 1081 ms. Trace:
I20260709 12:25:00.033655 20805 rpcz_store.cc:276] 0709 12:24:58.898611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.268637 (+370026us) service_pool.cc:224] Handling call
0709 12:24:59.979738 (+711101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:59.718453 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 503571us
I20260709 12:24:59.980360 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.451035 29015 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:59.981212 28870 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.256s	user 0.004s	sys 0.009s
I20260709 12:24:59.562182 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:24:59.970889 29058 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.314s	user 0.014s	sys 0.000s
W20260709 12:24:59.813651 21400 connection.cc:256] RPC call timeout handler was delayed by -1.043907653000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
I20260709 12:24:59.425457 29018 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:24:59.979786 21433 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:24:59.419227 29160 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:59.986037 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:00.058168 20877 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:24:59.999539 29122 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:59.753517 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:00.060487 20728 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:00.060680 20878 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 12:24:59.921633 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.731304 29030 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:59.988338 29117 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:59.988837 21426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.007s	sys 0.007s
I20260709 12:24:59.988510 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:24:59.425414 29059 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:24:59.991232 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3270) took 1129 ms. Trace:
I20260709 12:24:59.412351 29136 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:59.979280 20884 rpcz_store.cc:276] 0709 12:24:57.108201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.574585 (+466384us) service_pool.cc:224] Handling call
0709 12:24:57.574936 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.503672 (+928736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.341342 (+837670us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:24:59.341557 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2004599552}
I20260709 12:24:59.445389 20880 rpcz_store.cc:276] 0709 12:24:57.107520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.287082 (+179562us) service_pool.cc:224] Handling call
0709 12:24:57.320852 (+ 33770us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:57.338605 (+ 17753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.338658 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:57.343456 (+  4798us) log.cc:844] Serialized 52 byte log entry
0709 12:24:58.824056 (+1480600us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b1800438f40. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1
0709 12:24:58.824307 (+   251us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:58.824365 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.824494 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.905785 (+ 81291us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:24:59.357619 (+451834us) raft_consensus.cc:1685] finished
0709 12:24:59.357649 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.357842 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3108,"raft.queue_time_us":3278,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12933,"spinlock_wait_cycles":24626944,"thread_start_us":1005383,"threads_started":1,"wal-append.queue_time_us":1013793}
I20260709 12:24:59.450806 21435 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:59.419819 29021 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:59.995555 20802 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:24:59.995913 29171 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:59.425380 29121 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.075414 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:24:59.877050 29188 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:24:59.999109 29158 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:24:59.999451 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 591136us
I20260709 12:24:59.999684 29194 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:24:59.999111 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 587066us
I20260709 12:24:59.418542 29173 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.652s
I20260709 12:24:59.979219 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:24:59.398516 29162 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:00.035244 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:24:59.449874 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.035164 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:00.045961 20875 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
W20260709 12:25:00.046936 20727 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.003s
I20260709 12:25:00.036399 21427 rpcz_store.cc:276] 0709 12:24:57.142299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.412816 (+270517us) service_pool.cc:224] Handling call
0709 12:24:57.413162 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.413573 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:58.326684 (+913111us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.813632 (+486948us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.818985 (+  5353us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.819165 (+   180us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:58.819223 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.819292 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.483801 (+664509us) raft_consensus.cc:1685] finished
0709 12:24:59.483830 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.484018 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":225156,"prepare.queue_time_us":825631,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":70680,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":52221,"thread_start_us":1176417,"threads_started":2,"wal-append.queue_time_us":1052600}
W20260709 12:24:59.481146 28670 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.071s	user 0.012s	sys 0.000s
I20260709 12:24:59.984675 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.054615 29058 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.057075 29018 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:24:59.986879 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:00.058873 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3268) took 1197 ms. Trace:
I20260709 12:24:59.468734 29060 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:00.061774 21426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.007s	sys 0.007s
I20260709 12:25:00.045332 29191 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.980985 29048 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.065013 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:00.071686 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.076035 29021 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.063318 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3274) took 1190 ms. Trace:
I20260709 12:25:00.049710 29015 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:00.157533 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 122504us
I20260709 12:25:00.157847 20878 rpcz_store.cc:276] 0709 12:24:58.874196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.611648 (+737452us) service_pool.cc:224] Handling call
0709 12:25:00.063256 (+451608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.084868 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:00.159538 20880 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:25:00.159973 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3276) took 1192 ms. Trace:
I20260709 12:25:00.160275 20880 rpcz_store.cc:276] 0709 12:24:58.968036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.064739 (+1096703us) service_pool.cc:224] Handling call
0709 12:25:00.159916 (+ 95177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.086853 20732 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:25:00.088912 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3220) took 2978 ms. Trace:
I20260709 12:25:00.092386 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:00.064587 29030 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.062388 29204 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:00.148780 20727 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:00.149390 20730 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:00.171607 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 79325.9us
W20260709 12:25:00.172789 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3225) took 2944 ms (client timeout 3366 ms). Trace:
W20260709 12:25:00.173516 20875 rpcz_store.cc:269] 0709 12:24:57.216261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.251704 (+ 35443us) service_pool.cc:224] Handling call
0709 12:25:00.160071 (+2908367us) spinlock_profiling.cc:243] Waited 2.75 s on lock 0x7b540019c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:00.160213 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6321799552}
I20260709 12:25:00.175501 20875 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:25:00.148406 28670 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.150985 21431 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:25:00.151724 20731 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:00.152470 20877 rpcz_store.cc:276] 0709 12:24:58.861287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.985726 (+1124439us) service_pool.cc:224] Handling call
0709 12:25:00.058794 (+ 73068us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.148727 20808 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:25:00.155310 21432 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:00.157322 20733 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:00.185252 21435 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:25:00.063206 20883 rpcz_store.cc:276] 0709 12:24:58.861661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.326236 (+464575us) service_pool.cc:224] Handling call
0709 12:24:59.991132 (+664896us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.162024 20884 rpcz_store.cc:276] 0709 12:24:57.110403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.087744 (+2977341us) service_pool.cc:224] Handling call
0709 12:25:00.088148 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.088380 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.088474 (+    94us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.088533 (+    59us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:00.088597 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.088677 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.088832 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.195628 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1470) took 2970 ms. Trace:
I20260709 12:25:00.196139 20884 rpcz_store.cc:276] 0709 12:24:57.225605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.194508 (+2968903us) service_pool.cc:224] Handling call
0709 12:25:00.194920 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.195172 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.195234 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.195273 (+    39us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:00.195309 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.195385 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.195530 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:00.157335 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 122185us
I20260709 12:25:00.198297 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3072) took 1021 ms. Trace:
I20260709 12:25:00.064174 29136 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:00.226367 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.227339 20733 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:00.199710 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3221) took 3089 ms. Trace:
I20260709 12:25:00.152055 28969 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.064539 29059 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:00.167476 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 102526us
I20260709 12:25:00.181061 20727 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:25:00.198734 20730 rpcz_store.cc:276] 0709 12:24:59.177146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.984271 (+807125us) service_pool.cc:224] Handling call
0709 12:25:00.198224 (+213953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.231051 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:00.200448 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 799912us
I20260709 12:25:00.091912 29171 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:00.382108 20727 pending_rounds.cc:85] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 12
I20260709 12:25:00.382802 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.176635 29203 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:00.233976 20729 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:00.199093 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3222) took 3088 ms. Trace:
I20260709 12:25:00.391173 20880 rpcz_store.cc:276] 0709 12:24:57.110841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.161422 (+3050581us) service_pool.cc:224] Handling call
0709 12:25:00.198548 (+ 37126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.198752 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.198784 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.198828 (+    44us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:00.198865 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.198919 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.199036 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.230345 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3073) took 1053 ms. Trace:
I20260709 12:25:00.176397 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3223) took 3065 ms. Trace:
I20260709 12:25:00.392848 20733 rpcz_store.cc:276] 0709 12:24:59.177429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.193725 (+1016296us) service_pool.cc:224] Handling call
0709 12:25:00.227156 (+ 33431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.230121 (+  2965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.230293 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.315871 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.292850 21429 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:00.316622 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:00.394732 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1356) took 1265 ms. Trace:
I20260709 12:25:00.168164 29021 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.395619 20877 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 29 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:00.398126 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1473) took 3102 ms. Trace:
I20260709 12:25:00.291596 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1473) took 1126 ms. Trace:
I20260709 12:25:00.318980 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:00.399376 21431 rpcz_store.cc:276] 0709 12:24:59.165010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.424916 (+259906us) service_pool.cc:224] Handling call
0709 12:25:00.284044 (+859128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:00.330101 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 294550us
I20260709 12:25:00.179152 29058 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.238418 29136 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:00.338922 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.562s (SENT) [suppressed 24 similar messages]
W20260709 12:25:00.409300 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70429.9us
W20260709 12:25:00.410135 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.410235 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1349) took 1378 ms. Trace:
I20260709 12:25:00.227154 28870 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.179490 29158 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:00.422557 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:00.423565 20880 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:00.230288 20878 rpcz_store.cc:276] 0709 12:24:57.110616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.158811 (+3048195us) service_pool.cc:224] Handling call
0709 12:25:00.199130 (+ 40319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.199339 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.199371 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.199417 (+    46us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:00.199492 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.199553 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.199654 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.385219 20727 pending_rounds.cc:107] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:25:00.171293 29060 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:00.383477 20730 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:00.441094 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3271) took 1579 ms. Trace:
I20260709 12:25:00.392932 20875 rpcz_store.cc:276] 0709 12:24:57.111071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.174679 (+3063608us) service_pool.cc:224] Handling call
0709 12:25:00.175144 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.176030 (+   886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.176326 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.442009 20881 rpcz_store.cc:276] 0709 12:24:58.861804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.912730 (+ 50926us) service_pool.cc:224] Handling call
0709 12:25:00.440781 (+1528051us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:00.441005 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218285440}
I20260709 12:25:00.156602 29191 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:00.093048 29188 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.398715 20877 rpcz_store.cc:276] 0709 12:24:57.296132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.396844 (+3100712us) service_pool.cc:224] Handling call
0709 12:25:00.397286 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.397532 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.397688 (+   156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.397765 (+    77us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:00.397818 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.397902 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.398048 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.395265 20734 rpcz_store.cc:276] 0709 12:24:59.128847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.047260 (+918413us) service_pool.cc:224] Handling call
0709 12:25:00.394643 (+347383us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:00.405391 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 86493.3us
I20260709 12:25:00.454772 20879 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:00.455675 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1482) took 1710 ms. Trace:
I20260709 12:25:00.456051 20879 rpcz_store.cc:276] 0709 12:24:58.745196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.454319 (+1709123us) service_pool.cc:224] Handling call
0709 12:25:00.454607 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.455342 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.455579 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.252259 29199 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.387750 28985 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:00.407500 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 234935us
W20260709 12:25:00.461800 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 230834us
I20260709 12:25:00.410609 20729 rpcz_store.cc:276] 0709 12:24:59.031769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.453669 (+421900us) service_pool.cc:224] Handling call
0709 12:25:00.410176 (+956507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.293387 29122 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:00.415289 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:00.462286 28985 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:00.464241 29122 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.464886 29122 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:00.432849 20727 raft_consensus.cc:2889] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:00.442019 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:00.413470 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1468) took 1250 ms. Trace:
I20260709 12:25:00.291742 29033 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.122s
I20260709 12:25:00.072451 29198 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.217s
I20260709 12:25:00.452251 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:00.316186 28969 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.450238 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1480) took 1705 ms. Trace:
I20260709 12:25:00.469112 20881 rpcz_store.cc:276] 0709 12:24:58.744649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.449106 (+1704457us) service_pool.cc:224] Handling call
0709 12:25:00.449530 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.449786 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.449824 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.449887 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:00.449935 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.450009 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.450156 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.469982 21426 rpcz_store.cc:276] 0709 12:24:59.163371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.431924 (+268553us) service_pool.cc:224] Handling call
0709 12:24:59.432306 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.432679 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.432710 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.433207 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.150225 (+717018us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:00.412322 (+262097us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.413157 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.413237 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.413281 (+    44us) raft_consensus.cc:1685] finished
0709 12:25:00.413294 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.413412 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22301,"prepare.queue_time_us":81865,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":593,"raft.queue_time_us":38021,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":30516,"thread_start_us":903820,"threads_started":2,"wal-append.queue_time_us":554739}
I20260709 12:25:00.469872 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3069) took 1293 ms. Trace:
I20260709 12:25:00.417573 29119 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:00.474386 20727 rpcz_store.cc:276] 0709 12:24:59.176294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.180385 (+1004091us) service_pool.cc:224] Handling call
0709 12:25:00.180707 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.469550 (+288843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.469798 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.477015 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.463091 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:00.199931 29204 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.445741 29060 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:00.497601 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45418.6us
I20260709 12:25:00.500126 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:00.496124 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:00.505427 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:00.507496 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:00.508404 20730 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:00.510775 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:00.511680 20805 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:00.515874 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.518321 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:00.519255 20729 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:00.525962 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3344) took 1640 ms. Trace:
I20260709 12:25:00.526547 20803 rpcz_store.cc:276] 0709 12:24:58.886191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.886382 (+   191us) service_pool.cc:224] Handling call
0709 12:24:58.948513 (+ 62131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.407050 (+458537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.525648 (+1118598us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54003d5ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:00.525889 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808903424}
I20260709 12:25:00.533074 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:00.443609 28870 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.548524 20808 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:00.548715 20849 connection.cc:256] RPC call timeout handler was delayed by -1.2672388840001076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
I20260709 12:25:00.550465 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:00.557209 20801 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:00.475656 29158 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:00.558549 20880 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
W20260709 12:25:00.560050 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98034.4us
I20260709 12:25:00.558303 29158 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.551556 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:00.564229 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:00.560242 20849 connection.cc:256] RPC call timeout handler was delayed by -1.1835388839999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 12:25:00.564859 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:00.531634 29208 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.129s	user 0.001s	sys 0.001s
I20260709 12:25:00.482080 29198 consensus_queue.cc:1243] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:00.553491 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.577239 20727 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:00.565248 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:00.578668 20802 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:25:00.581283 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:00.583302 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:00.584240 20803 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:00.497720 28876 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:00.521034 29058 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.521060 29018 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:00.570920 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:00.595990 20733 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:00.560624 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:00.597993 20876 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:00.598806 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3260) took 1739 ms. Trace:
I20260709 12:25:00.502779 29060 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:00.502931 29033 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:00.564373 20806 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:25:00.481731 29011 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.503562 29113 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:00.569815 20776 connection.cc:256] RPC call timeout handler was delayed by -1.6125863479996951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 12:25:00.588387 21429 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 1979) took 3446 ms (client timeout 2240 ms). Trace:
I20260709 12:25:00.479048 29213 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
W20260709 12:25:00.616763 21429 rpcz_store.cc:269] 0709 12:24:57.142019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.307255 (+165236us) service_pool.cc:224] Handling call
0709 12:25:00.588142 (+3280887us) spinlock_profiling.cc:243] Waited 2.9 s on lock 0x7b5400645ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:00.588311 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6677416704}
W20260709 12:25:00.620487 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 205267us
I20260709 12:25:00.625696 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:00.478248 29199 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.570269 29048 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:00.628494 20700 connection.cc:256] RPC call timeout handler was delayed by -1.5184935569996014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
I20260709 12:25:00.603040 29121 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.616820 29174 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:00.615366 20876 rpcz_store.cc:276] 0709 12:24:58.859050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.597293 (+1738243us) service_pool.cc:224] Handling call
0709 12:25:00.597732 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.598499 (+   767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.598734 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.640900 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3261) took 1781 ms. Trace:
I20260709 12:25:00.611449 29212 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:00.641428 20876 rpcz_store.cc:276] 0709 12:24:58.859366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.639714 (+1780348us) service_pool.cc:224] Handling call
0709 12:25:00.640170 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.640419 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.640457 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.640519 (+    62us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:00.640575 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.640645 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.640822 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.618346 20808 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
W20260709 12:25:00.635461 20700 connection.cc:256] RPC call timeout handler was delayed by -1.4455935569999383s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 12:25:00.643810 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.612419 29033 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:00.644455 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3262) took 1784 ms. Trace:
I20260709 12:25:00.623960 29222 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:00.648661 20876 rpcz_store.cc:276] 0709 12:24:58.859663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.642778 (+1783115us) service_pool.cc:224] Handling call
0709 12:25:00.643171 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.644158 (+   987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.644184 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.644225 (+    41us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:00.644256 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.644307 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.644408 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:00.631791 28875 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.231s	user 0.013s	sys 0.000s
I20260709 12:25:00.634186 28670 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.702782 29219 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:00.703142 28851 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:00.703758 28851 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:00.703482 29219 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.704624 28851 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.706143 29219 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.555594 29171 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.633687 29213 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:00.607717 29058 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.627910 29113 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:00.691994 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 127177us
I20260709 12:25:00.622902 29015 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.693634 20730 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:00.694690 20875 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 20. (term mismatch)
I20260709 12:25:00.651654 29033 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.714594 29219 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.719103 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:00.722993 20875 pending_rounds.cc:85] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 19
I20260709 12:25:00.729110 20875 pending_rounds.cc:107] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 27.20
W20260709 12:25:00.696841 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 196798us
I20260709 12:25:00.652222 29222 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.731472 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:00.717679 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3228) took 3347 ms. Trace:
I20260709 12:25:00.729424 20875 raft_consensus.cc:2889] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:00.732411 20878 rpcz_store.cc:276] 0709 12:24:57.370196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.429929 (+3059733us) service_pool.cc:224] Handling call
0709 12:25:00.461416 (+ 31487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.488204 (+ 26788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.488251 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.488825 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.618936 (+130111us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:00.618988 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.619078 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.717202 (+ 98124us) raft_consensus.cc:1685] finished
0709 12:25:00.717255 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.717544 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26328,"raft.queue_time_us":108870,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":24167,"thread_start_us":129759,"threads_started":1,"wal-append.queue_time_us":218002}
I20260709 12:25:00.738386 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3068) took 1562 ms. Trace:
W20260709 12:25:00.746251 20728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1348) took 1728 ms (client timeout 1861 ms). Trace:
I20260709 12:25:00.738945 20732 rpcz_store.cc:276] 0709 12:24:59.175987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.178320 (+1002333us) service_pool.cc:224] Handling call
0709 12:25:00.178751 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.179130 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.179168 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.179734 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.480335 (+300601us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:00.480377 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.480474 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.738108 (+257634us) raft_consensus.cc:1685] finished
0709 12:25:00.738131 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.738319 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38197,"raft.queue_time_us":2473,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":33119,"thread_start_us":300171,"threads_started":1,"wal-append.queue_time_us":462371}
W20260709 12:25:00.746959 20728 rpcz_store.cc:269] 0709 12:24:59.017768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.065196 (+ 47428us) service_pool.cc:224] Handling call
0709 12:25:00.745964 (+1680768us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b540028c154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:00.746161 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2281336064}
I20260709 12:25:00.751156 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1475) took 1383 ms. Trace:
I20260709 12:25:00.767369 21432 rpcz_store.cc:276] 0709 12:24:59.367953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.981550 (+613597us) service_pool.cc:224] Handling call
0709 12:24:59.981889 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.192929 (+211040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.750761 (+557832us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:00.751057 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398131584}
I20260709 12:25:00.769495 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:00.723320 29209 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 12:25:00.774778 29215 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:25:00.775734 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:00.776268 28602 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:00.776480 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1476) took 1399 ms. Trace:
I20260709 12:25:00.776772 20878 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:25:00.777949 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:00.776994 21433 rpcz_store.cc:276] 0709 12:24:59.376619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.053637 (+677018us) service_pool.cc:224] Handling call
0709 12:25:00.054114 (+   477us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.064013 (+  9899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.064047 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.075715 (+ 11668us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.406941 (+331226us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:00.407002 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.407103 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.776249 (+369146us) raft_consensus.cc:1685] finished
0709 12:25:00.776271 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.776416 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5825,"raft.queue_time_us":9020,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":11190,"thread_start_us":239280,"threads_started":1,"wal-append.queue_time_us":467738}
I20260709 12:25:00.778630 20878 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:00.779963 29188 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.783044 21431 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:25:00.785977 21427 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
W20260709 12:25:00.787846 20810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3338) took 1983 ms (client timeout 1522 ms). Trace:
W20260709 12:25:00.788290 20810 rpcz_store.cc:269] 0709 12:24:58.804573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.804699 (+   126us) service_pool.cc:224] Handling call
0709 12:25:00.759896 (+1955197us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:00.787764 (+ 27868us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395130240}
I20260709 12:25:00.794618 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3264) took 1934 ms. Trace:
I20260709 12:25:00.795082 20879 rpcz_store.cc:276] 0709 12:24:58.860175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.793649 (+1933474us) service_pool.cc:224] Handling call
0709 12:25:00.794102 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.794290 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.794315 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.794359 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:00.794390 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.794446 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.794557 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.796893 20879 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:25:00.797578 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3266) took 1936 ms. Trace:
I20260709 12:25:00.797719 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:00.798058 20879 rpcz_store.cc:276] 0709 12:24:58.860741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.796259 (+1935518us) service_pool.cc:224] Handling call
0709 12:25:00.796617 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.797304 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.797515 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.799167 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:00.794262 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:00.801223 20879 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:00.801963 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1483) took 1934 ms. Trace:
I20260709 12:25:00.802305 20879 rpcz_store.cc:276] 0709 12:24:58.867249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.799609 (+1932360us) service_pool.cc:224] Handling call
0709 12:25:00.799853 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.801648 (+  1795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.801903 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.653390 29212 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.727763 29113 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:00.765197 29213 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:00.812888 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:00.739302 29222 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.711918 28875 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.787241 29015 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:00.738345 29021 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.825977 21426 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:00.848794 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:00.850692 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:00.851488 21432 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:00.806118 29204 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.814433 29011 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:00.800848 29209 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.050s
I20260709 12:25:00.790544 28851 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:25:00.826556 29213 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.824931 29113 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:00.811441 29158 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:00.870898 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:00.809176 28969 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.766623 29219 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.872658 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:00.874382 21432 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:00.808574 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b31661191c14b44ac964555853900f1"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:00.877640 20881 raft_consensus.cc:606] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:25:00.878055 20881 raft_consensus.cc:614] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:00.813661 29225 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:00.852079 29220 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:00.880620 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:00.881498 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:00.887161 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:00.889037 21432 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:00.868654 29011 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.892161 21426 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:25:00.873770 29213 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:00.874282 29158 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:00.909502 29222 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.884682 29188 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:00.900175 29122 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:00.901327 29214 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.405s	user 0.001s	sys 0.000s
W20260709 12:25:00.901634 29059 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.002s
I20260709 12:25:00.906832 29158 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.888867 29174 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:00.889962 29220 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.916081 29212 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.918063 29219 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:00.919991 29136 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.952379 29059 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.954166 29136 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.932498 29058 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:00.929215 29117 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.862s	user 0.011s	sys 0.000s
I20260709 12:25:00.955168 29059 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.936313 29021 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:00.934877 29162 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.743s	user 0.007s	sys 0.000s
I20260709 12:25:00.960026 29162 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.965278 29162 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.910522 29230 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:00.967439 29230 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:00.968730 29230 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.970513 29230 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:00.949010 29188 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:00.941356 29232 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:00.938381 29222 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.962937 29220 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:00.972152 29188 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.974624 29122 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:00.966975 29160 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.903s	user 0.008s	sys 0.001s
I20260709 12:25:01.083988 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.085201 29160 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:01.024276 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.087970 29136 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:01.024689 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 230379us
I20260709 12:25:01.044348 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3085) took 1677 ms. Trace:
I20260709 12:25:01.044641 20735 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23.
I20260709 12:25:01.068630 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3224) took 3956 ms. Trace:
I20260709 12:25:01.092072 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.093915 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:01.092439 20883 rpcz_store.cc:276] 0709 12:24:57.111592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.194156 (+3082564us) service_pool.cc:224] Handling call
0709 12:25:00.319083 (+124927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.483614 (+164531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.483672 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.484302 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.702581 (+218279us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:00.702625 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.702716 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.067930 (+365214us) raft_consensus.cc:1685] finished
0709 12:25:01.067981 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.068345 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123798,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10850,"thread_start_us":291314,"threads_started":2,"wal-append.queue_time_us":284330}
I20260709 12:25:01.094277 20882 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:01.070080 20878 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:25:01.095216 21434 raft_consensus.cc:2376] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21. Re-sending same reply.
I20260709 12:25:00.892251 28870 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:00.986009 29165 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.992s	user 0.013s	sys 0.000s
I20260709 12:25:01.097141 29165 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:00.893932 29011 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.099687 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.091370 20731 rpcz_store.cc:276] 0709 12:24:59.366366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.251404 (+885038us) service_pool.cc:224] Handling call
0709 12:25:00.291084 (+ 39680us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.374616 (+ 83532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.374668 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.388756 (+ 14088us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.653541 (+264785us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:00.653613 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.653726 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.043506 (+389780us) raft_consensus.cc:1685] finished
0709 12:25:01.043592 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.044058 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96872,"raft.queue_time_us":89807,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":29116,"thread_start_us":197658,"threads_started":1,"wal-append.queue_time_us":472671}
I20260709 12:25:01.101513 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:01.102452 21434 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:01.083793 29159 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.642s	user 0.000s	sys 0.014s
I20260709 12:25:01.086815 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.103875 21432 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:00.971266 29158 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.103749 29159 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.105134 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.104385 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3226) took 3838 ms. Trace:
I20260709 12:25:01.105963 21432 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:01.088124 21430 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:01.076110 29191 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
I20260709 12:25:01.076519 28854 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:00.952025 29199 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:00.989271 29213 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:00.870450 28670 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.023s
W20260709 12:25:01.195582 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 104381us
W20260709 12:25:01.197824 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.206125 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:01.272737 20729 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.748s	user 0.013s	sys 0.014s
I20260709 12:25:01.273172 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:01.204159 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.217799 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.219452 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.277988 20882 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:01.220779 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1477) took 1843 ms. Trace:
I20260709 12:25:01.208561 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:01.216887 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.214469 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:01.124315 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:01.106743 29158 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:01.236253 29160 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:01.242161 29107 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:01.213399 29165 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:01.233968 28875 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:01.238010 29030 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:01.279287 21435 rpcz_store.cc:276] 0709 12:24:59.376897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.065557 (+688660us) service_pool.cc:224] Handling call
0709 12:25:00.165944 (+100387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.220904 (+ 54960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.220061 (+999157us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:01.220527 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43473280}
I20260709 12:25:01.289008 29135 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:01.286370 20729 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:01.143985 29159 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:01.293080 29135 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:01.296958 29082 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:01.297137 29193 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:01.298017 29193 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:01.299091 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:01.304412 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.306727 21431 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:01.313699 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.314254 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.229563 29171 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 12:25:01.356226 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.357424 21435 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:01.360456 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.361828 21433 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:01.106148 20884 rpcz_store.cc:276] 0709 12:24:57.266041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.197322 (+2931281us) service_pool.cc:224] Handling call
0709 12:25:00.328594 (+131272us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.498888 (+170294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.498941 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.500096 (+  1155us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.724634 (+224538us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:00.724689 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.724787 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.103717 (+378930us) raft_consensus.cc:1685] finished
0709 12:25:01.103765 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.104188 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145229,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5728,"thread_start_us":321163,"threads_started":2,"wal-append.queue_time_us":340007}
I20260709 12:25:01.365379 20884 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 12:25:01.359118 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.230113 28876 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 12:25:01.370154 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:01.372357 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:01.373103 20726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1343) took 2444 ms (client timeout 1913 ms). Trace:
I20260709 12:25:01.373231 20881 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:01.340728 28893 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:01.376962 20726 rpcz_store.cc:269] 0709 12:24:58.928317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.009267 (+ 80950us) service_pool.cc:224] Handling call
0709 12:25:01.370994 (+2361727us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54002990d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:01.372847 (+  1853us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917716736}
W20260709 12:25:01.378841 20776 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.006s	sys 0.018s
I20260709 12:25:01.378978 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:01.379441 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3282) took 2075 ms. Trace:
I20260709 12:25:01.380196 20877 raft_consensus.cc:2410] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 26 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:01.379976 20881 rpcz_store.cc:276] 0709 12:24:59.304126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.369152 (+2065026us) service_pool.cc:224] Handling call
0709 12:25:01.372604 (+  3452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.377807 (+  5203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.379358 (+  1551us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:01.379916 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.381958 20881 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:01.382575 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.383492 21427 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:01.379428 20776 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.006s	sys 0.018s
I20260709 12:25:01.387687 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:01.389135 29221 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:01.389629 29221 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:01.390197 29060 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.766s	user 0.016s	sys 0.006s
I20260709 12:25:01.391327 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.390863 29221 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.392901 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.392351 29221 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:01.392693 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:01.396831 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:01.396864 28851 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:01.406281 28851 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:01.402567 29166 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:01.402819 29015 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.006s
W20260709 12:25:01.388006 29033 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.006s
I20260709 12:25:01.405058 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.407178 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.378170 20883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
W20260709 12:25:01.414299 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.414471 21434 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:25:01.392494 29011 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.414958 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.406306 29169 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:01.390784 29060 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:01.422642 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 632742us
W20260709 12:25:01.423840 29113 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.010s
I20260709 12:25:01.425432 29011 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:01.425782 20882 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3280) took 2179 ms (client timeout 1617 ms). Trace:
I20260709 12:25:01.409296 28851 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:01.433331 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:01.433393 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49456.4us
I20260709 12:25:01.376485 29191 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.421283 20883 raft_consensus.cc:606] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:01.438359 28854 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:01.436725 20882 rpcz_store.cc:269] 0709 12:24:59.245952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.252728 (+  6776us) service_pool.cc:224] Handling call
0709 12:25:01.425501 (+2172773us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400268254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:01.425684 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3930469248}
I20260709 12:25:01.438949 28854 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:01.441277 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44689.4us
I20260709 12:25:01.443894 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:01.443799 29173 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:01.448961 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.443238 20883 raft_consensus.cc:614] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:01.449630 20876 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:01.451695 21432 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:01.456372 21428 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:01.453101 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.449985 29173 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:01.458225 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:01.458407 29173 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.453972 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:01.459857 21435 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 29 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:25:01.440834 28854 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:01.452838 29191 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:01.450268 20726 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:25:01.436559 28876 consensus_queue.cc:1243] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:01.466760 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:01.466900 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:01.468320 20850 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.572s (SENT)
I20260709 12:25:01.431740 29033 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.449084 29015 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:01.473690 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.485100 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.486481 21433 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:25:01.443033 29113 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.488072 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:01.483963 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:01.491210 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.492484 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:01.495548 20877 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:01.496168 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.495836 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:01.501057 21435 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:01.505875 21434 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:01.509127 20881 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:01.510234 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:01.532629 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.540628 21428 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:01.546849 20729 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.551615 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:01.558776 21427 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:01.550758 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 887884us
I20260709 12:25:01.577401 28851 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.579145 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:01.579452 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:01.599035 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3281) took 2295 ms. Trace:
I20260709 12:25:01.609731 29113 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:01.609812 21435 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:01.548820 20883 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:25:01.614874 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:01.620914 21434 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:25:01.623142 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:01.640780 20731 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:01.624080 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:01.641492 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3230) took 2923 ms. Trace:
I20260709 12:25:01.641788 28970 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:01.625918 29033 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:01.642248 28970 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:01.641957 20875 rpcz_store.cc:276] 0709 12:24:58.717673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.451383 (+1733710us) service_pool.cc:224] Handling call
0709 12:25:00.451677 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.732700 (+281023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.640997 (+908297us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400059e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:01.641276 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557953920}
I20260709 12:25:01.643692 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:01.645228 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:01.646763 20810 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:01.632510 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:01.647567 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3259) took 2788 ms. Trace:
I20260709 12:25:01.646642 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:01.643684 28970 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.634881 21433 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:25:01.647996 20880 rpcz_store.cc:276] 0709 12:24:58.858767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.562546 (+1703779us) service_pool.cc:224] Handling call
0709 12:25:00.562975 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.801949 (+238974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.801997 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.802635 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 12:25:01.359429 (+556794us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:01.359527 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.638357 (+278830us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:01.638410 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.638453 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:01.638469 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.647369 (+  8900us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19698,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":15553,"spinlock_wait_cycles":512103808,"thread_start_us":433950,"threads_started":1,"wal-append.queue_time_us":549839}
I20260709 12:25:01.649503 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:01.650066 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:01.650731 20880 raft_consensus.cc:606] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:25:01.651044 20883 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:25:01.635464 29166 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.652073 20875 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:01.652232 29166 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.653592 20875 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:01.636687 20808 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.651115 20880 raft_consensus.cc:614] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:01.633876 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:01.655715 21428 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:01.638123 28851 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:01.661537 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49148.9us
I20260709 12:25:01.679633 21427 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:01.679756 20729 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:01.654523 29213 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:01.629801 20879 rpcz_store.cc:276] 0709 12:24:59.303833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.811489 (+1507656us) service_pool.cc:224] Handling call
0709 12:25:00.811907 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.812183 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:01.371325 (+559142us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.552753 (+181428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:01.559808 (+  7055us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:01.598412 (+ 38604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.598558 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.598830 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":481026,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":1519,"raft.queue_time_us":36230,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":236,"thread_start_us":594650,"threads_started":2,"wal-append.queue_time_us":227364}
I20260709 12:25:01.691847 20883 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:25:01.681529 29033 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:01.703630 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:01.707677 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:01.708288 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:01.709317 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:01.710381 20807 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:01.710168 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:01.699654 29033 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.718026 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:01.718940 20884 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.721418 29166 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:01.705674 29060 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:01.724885 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:01.725592 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3263) took 2865 ms. Trace:
I20260709 12:25:01.725735 20880 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:01.726044 20876 rpcz_store.cc:276] 0709 12:24:58.859923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.650352 (+1790429us) service_pool.cc:224] Handling call
0709 12:25:00.650932 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.457873 (+806941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.725272 (+267399us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b540019cdd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:01.725531 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1825541504}
I20260709 12:25:01.728807 21426 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 LEARNER]: Advancing to term 20
I20260709 12:25:01.733273 29193 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.736006 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3265) took 2875 ms. Trace:
I20260709 12:25:01.733321 28854 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:01.743474 20774 connection.cc:256] RPC call timeout handler was delayed by -1.083830960999876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3366s for a response
I20260709 12:25:01.682930 29209 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:25:01.745745 29033 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:01.747004 29230 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.729s	user 0.019s	sys 0.002s
I20260709 12:25:01.736455 20878 rpcz_store.cc:276] 0709 12:24:58.860429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.795741 (+1935312us) service_pool.cc:224] Handling call
0709 12:25:00.797152 (+  1411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.095466 (+298314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.735708 (+640242us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540026d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:01.735939 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623967488}
I20260709 12:25:01.723059 20810 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.750406 21429 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:01.750607 29213 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:01.752403 21435 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:01.752532 28854 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:01.699047 29165 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:01.766038 28876 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:01.767855 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:01.767692 28875 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:01.773032 20808 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:01.773141 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3359) took 1232 ms. Trace:
I20260709 12:25:01.774144 20884 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:01.780303 29135 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:01.785112 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:01.806777 20878 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:01.790499 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:01.800298 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:01.791849 20876 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:01.792135 29072 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:01.734978 29193 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:01.815272 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:01.798328 29171 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:01.796012 29213 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:01.819795 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 186145us
W20260709 12:25:01.824869 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.820855 21426 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:25:01.821718 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53997.4us
W20260709 12:25:01.825325 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.826233 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.804913 20809 rpcz_store.cc:276] 0709 12:25:00.540457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.540640 (+   183us) service_pool.cc:224] Handling call
0709 12:25:00.541083 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.428096 (+887013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.428144 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:01.428934 (+   790us) log.cc:844] Serialized 26 byte log entry
0709 12:25:01.487391 (+ 58457us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:01.487449 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.772574 (+285125us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54002b6bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:01.772619 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.772660 (+    41us) raft_consensus.cc:1685] finished
0709 12:25:01.772692 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.772954 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1953,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6502,"spinlock_wait_cycles":2038030080,"thread_start_us":57827,"threads_started":1,"wal-append.queue_time_us":58373}
I20260709 12:25:01.830737 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:01.833803 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67791.7us
I20260709 12:25:01.829860 29194 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:01.800849 29015 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:25:01.835310 29194 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.839655 20731 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:01.840199 29165 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:01.841346 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:01.848085 29135 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:01.856573 28876 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:01.824138 29213 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:01.871743 29201 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.580s	user 0.005s	sys 0.000s
I20260709 12:25:01.875630 29160 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:01.887537 20728 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:25:01.869421 29194 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:01.888501 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:01.892913 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:01.912693 29122 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:01.889043 28876 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:01.920133 28876 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:01.922063 28893 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:01.924518 29072 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:01.925096 29072 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:01.927747 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1492) took 1062 ms. Trace:
I20260709 12:25:01.928288 20881 rpcz_store.cc:276] 0709 12:25:00.865270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.382747 (+517477us) service_pool.cc:224] Handling call
0709 12:25:01.383090 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.512466 (+129376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.927416 (+414950us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:01.927666 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284155136}
I20260709 12:25:01.938021 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:01.944312 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46972.9us
I20260709 12:25:01.926268 29072 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:01.951711 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 67566.3us
W20260709 12:25:01.960426 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 77445.9us
I20260709 12:25:01.963882 20731 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
W20260709 12:25:01.935379 29259 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:25:01.960515 20850 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.691s (SENT)
I20260709 12:25:01.935108 29134 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:01.963207 29201 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:25:01.962080 29221 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.000s
I20260709 12:25:01.966432 29060 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:01.992614 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:01.995517 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.010088 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.995939 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.007792 21431 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
W20260709 12:25:02.005843 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 129086us
W20260709 12:25:02.007151 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.021543 20850 leader_election.cc:336] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.748s (SENT)
I20260709 12:25:02.023026 20734 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 12:25:02.019423 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44033.8us
I20260709 12:25:02.031764 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:02.035398 29165 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:02.036115 29165 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:02.034722 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.036521 29165 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:02.036960 29165 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:02.047251 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.048266 29171 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:02.051081 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.052994 29166 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:02.054742 29166 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:02.056355 20850 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.802s (SENT)
W20260709 12:25:02.057183 20850 leader_election.cc:336] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.877s (SENT)
I20260709 12:25:02.057582 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:02.058341 29173 consensus_meta.cc:287] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.007s
I20260709 12:25:02.058826 29173 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.059401 29169 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:02.059955 29169 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:02.060869 29169 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.038731 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.064522 20726 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:02.070817 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 179482us
I20260709 12:25:02.071837 20880 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
W20260709 12:25:02.073083 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 199725us
I20260709 12:25:02.020578 28875 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.061226 29173 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:02.085539 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1546) took 1011 ms. Trace:
I20260709 12:25:02.075924 29060 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:02.093267 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.088577 21431 rpcz_store.cc:276] 0709 12:25:01.074367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.382273 (+307906us) service_pool.cc:224] Handling call
0709 12:25:02.085304 (+703031us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400088c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:02.085452 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436292352}
W20260709 12:25:02.095809 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.096454 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2000) took 1018 ms. Trace:
I20260709 12:25:02.097782 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:02.101388 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50429.9us
I20260709 12:25:02.103636 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:02.076119 29166 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:02.104568 20880 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:25:02.108894 21427 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:02.109161 20880 raft_consensus.cc:614] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:02.105664 21430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1465) took 2929 ms (client timeout 1764 ms). Trace:
W20260709 12:25:02.097328 20735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1338) took 3183 ms (client timeout 1995 ms). Trace:
I20260709 12:25:02.097941 21428 rpcz_store.cc:276] 0709 12:25:01.078456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.671945 (+593489us) service_pool.cc:224] Handling call
0709 12:25:01.696004 (+ 24059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.768749 (+ 72745us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:01.769071 (+   322us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.095330 (+326259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.095385 (+    55us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:02.095428 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.095490 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.096393 (+   903us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1410,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":6572,"thread_start_us":319305,"threads_started":1}
W20260709 12:25:02.109972 21430 rpcz_store.cc:269] 0709 12:24:59.160913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.161043 (+   130us) service_pool.cc:224] Handling call
0709 12:25:02.080271 (+2919228us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54001e5d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:02.090491 (+ 10220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4351489408}
I20260709 12:25:02.066244 29221 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 12:25:02.119031 29218 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.119855 29221 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:02.120447 29221 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:02.119938 29218 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:02.121554 29221 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.122026 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.122537 29218 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:02.122748 20728 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:02.123201 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.124095 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.124548 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.125064 20728 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:02.125795 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.125995 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.126255 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.127424 20728 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:02.128068 29119 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:02.128175 29266 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:02.128882 29266 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.129947 29241 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:02.122946 29221 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:02.130602 29241 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:02.131970 20726 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:02.131922 29241 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.133803 20726 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:25:02.134140 29241 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:02.134775 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96212.3us
I20260709 12:25:02.133420 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.135679 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.135984 29119 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:02.137233 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:02.081573 28875 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.139868 28875 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:02.140435 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.141593 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.141798 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.143726 29166 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:02.144191 29166 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.144559 29166 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:02.147976 29079 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:02.147737 20624 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 19 to 20, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 20 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:02.151050 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.153432 28970 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:02.150657 20624 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 21 to 22, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 22 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:02.157238 29212 consensus_queue.cc:1175] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 26 status { last_received { term: 26 index: 18 } last_committed_idx: 18 last_received_current_leader { term: 26 index: 18 } }
I20260709 12:25:02.157820 20624 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:02.142491 20730 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:02.144459 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:02.111129 20735 rpcz_store.cc:269] 0709 12:24:58.914300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.940144 (+ 25844us) service_pool.cc:224] Handling call
0709 12:25:02.081883 (+3141739us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54001d7c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:02.097220 (+ 15337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4670800256}
I20260709 12:25:02.142967 29030 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:02.163347 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73239.4us
I20260709 12:25:02.159423 20624 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 17 to 18, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 18 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:02.171731 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:02.172577 20879 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:02.174384 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:02.179909 20883 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:25:02.175714 20881 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:02.176400 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:02.177356 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:02.106571 29113 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:02.146955 29072 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:02.188954 20882 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:02.097247 29033 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.185146 29079 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:02.185402 29266 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:02.210631 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 143359us
W20260709 12:25:02.224884 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 167711us
I20260709 12:25:02.244853 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26ced453445642b09bda798a2963748b"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
W20260709 12:25:02.246368 20775 connection.cc:256] RPC call timeout handler was delayed by -2.07809691500006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 12:25:02.199990 29241 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:25:02.250571 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 106224us
I20260709 12:25:02.252045 20730 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:02.245728 20880 raft_consensus.cc:606] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:25:02.252813 20880 raft_consensus.cc:614] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:02.252866 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:02.256511 20775 connection.cc:256] RPC call timeout handler was delayed by -2.0375969149999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 12:25:02.260694 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 123604us
I20260709 12:25:02.261011 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.262652 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:02.262953 29033 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:02.263737 29033 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:02.264321 29033 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:02.265372 29033 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.267346 20849 leader_election.cc:400] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
W20260709 12:25:02.267619 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 126237us
I20260709 12:25:02.269042 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.270337 29193 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:02.270725 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:02.271725 20850 leader_election.cc:336] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.966s (SENT)
W20260709 12:25:02.279357 20850 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.069s (SENT)
I20260709 12:25:02.289215 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:02.232651 29113 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:02.296730 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52955.6us
I20260709 12:25:02.300441 28893 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.286998 29030 raft_consensus.cc:2764] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:25:02.300782 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:02.301050 28893 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.301594 20880 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:02.301672 29030 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:02.303296 29030 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:02.303278 20879 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:02.304713 29030 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.311348 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.320068 21426 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:02.324848 20875 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:02.305779 28893 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:02.324999 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:02.331288 20882 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:25:02.339303 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:02.338575 20624 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 14 to 15, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 15 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:02.340492 21427 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:02.343524 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:02.344192 29196 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.032s
I20260709 12:25:02.327756 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:02.329099 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:02.307152 29171 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:02.353390 29268 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:25:02.354877 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:02.363317 29166 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.367354 29212 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:02.368443 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:02.372375 21431 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 12:25:02.378415 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67411.9us
I20260709 12:25:02.298908 29241 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:25:02.388713 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:02.403391 21426 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:25:02.401645 20735 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:02.342370 20624 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 16 to 18, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 18 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.333148 29169 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 12:25:02.351028 28875 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.389020 29173 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:25:02.411769 28875 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.386547 29193 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:02.413090 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:02.413448 29135 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.001s
I20260709 12:25:02.414376 20729 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:25:02.389218 28893 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:25:02.413448 29193 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:02.415637 20775 connection.cc:256] RPC call timeout handler was delayed by -1.8872969150002064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
I20260709 12:25:02.416682 29122 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:02.410851 29166 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:02.417723 29175 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:02.418224 29175 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:02.416421 21428 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:02.416824 20775 connection.cc:256] RPC call timeout handler was delayed by -1.7623855850001746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3366s for a response
W20260709 12:25:02.416146 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 103671us
I20260709 12:25:02.419478 29169 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:02.418661 29175 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:02.386085 20878 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.001s
I20260709 12:25:02.419910 29169 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:02.420995 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:02.427772 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:02.414254 28875 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:02.443161 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:02.444170 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:02.450291 29165 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.451285 29136 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:02.451859 29136 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.452234 29136 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:02.452380 29166 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:02.453626 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.454475 20730 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:02.455899 28970 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:02.456398 28970 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.456734 28970 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:02.459729 29169 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.460417 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:02.461136 21427 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:02.461299 29193 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.461128 29169 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:02.410436 29113 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:02.465272 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44353.9us
I20260709 12:25:02.474849 20883 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
W20260709 12:25:02.469689 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 180568us
I20260709 12:25:02.481585 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:02.482237 20875 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 18.
I20260709 12:25:02.482510 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:25:02.484273 20808 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:02.487298 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:02.488303 20880 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:02.498565 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.501837 20728 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:02.497632 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.390612 29194 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.515746 20730 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23.
I20260709 12:25:02.517341 20880 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:25:02.517714 20700 leader_election.cc:336] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.841s (SENT)
I20260709 12:25:02.514930 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:02.518051 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 100853us
I20260709 12:25:02.481046 28851 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:25:02.542934 20729 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:25:02.547591 29072 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:02.548740 29165 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:02.556754 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.550622 29122 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:02.559623 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:02.560504 21435 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:02.562961 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:02.586591 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:02.590375 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:02.590427 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:02.559547 29213 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
W20260709 12:25:02.597846 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.585626 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:02.600472 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.601999 21433 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:02.597332 20730 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:02.603389 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:02.603699 20849 connection.cc:256] RPC call timeout handler was delayed by -1.0907954360000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3247s for a response
I20260709 12:25:02.604022 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:02.598268 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:02.605332 28893 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:02.607756 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.600703 29241 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:02.623908 20775 connection.cc:256] RPC call timeout handler was delayed by -1.7387969149999662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
I20260709 12:25:02.626613 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:02.626725 20775 connection.cc:256] RPC call timeout handler was delayed by -1.4444969149999451s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22890000000000002s for a response
I20260709 12:25:02.627736 20726 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:25:02.628774 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.624404 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.607362 29030 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:02.609891 29171 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:02.642589 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:02.615365 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 178681us
I20260709 12:25:02.604246 29213 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:02.683854 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 124816us
W20260709 12:25:02.688022 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 408665us
I20260709 12:25:02.645330 29171 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:02.675395 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 137720us
I20260709 12:25:02.692535 29171 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:02.713012 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 88738.3us
I20260709 12:25:02.738520 29033 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:02.749540 28875 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:02.741348 29175 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.751667 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:02.752588 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.753243 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:02.753106 29175 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:02.755414 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:02.776790 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:02.777100 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.768747 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.747444 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.757058 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.770193 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.778867 21434 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:02.748389 20728 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:02.756453 29282 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:02.765704 29213 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:02.789152 20882 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:25:02.790164 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:02.791339 20880 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:02.765383 29136 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.794128 29188 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:02.795950 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.793689 29136 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:02.796092 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:02.797400 21434 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:02.795991 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.814952 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.755944 29194 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:02.748950 29030 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:02.827644 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:02.798753 29134 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:02.797449 29193 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:02.841768 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47504.9us
I20260709 12:25:02.833109 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:02.830073 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:02.868031 20880 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:02.809317 29241 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:02.816855 29282 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:02.823331 21434 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:02.843906 20734 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:25:02.844136 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:02.845373 29194 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.845510 29119 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:02.845731 29169 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.845866 29288 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:25:02.846539 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:02.846817 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89835.7us
I20260709 12:25:02.848758 21430 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:02.849411 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:02.768612 29122 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.065s
W20260709 12:25:02.853251 29269 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.568s	user 0.006s	sys 0.001s
I20260709 12:25:02.853775 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.855666 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.855958 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:02.756642 29033 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:25:02.856652 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:02.858281 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:02.841686 29193 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:02.855669 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:02.875349 20727 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
W20260709 12:25:02.896778 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 155120us
I20260709 12:25:02.896868 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:02.896714 21429 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:02.762831 29171 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.905869 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:02.804157 28970 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.910830 20882 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:02.913683 21427 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:02.915794 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:02.904682 29136 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.919162 20729 raft_consensus.cc:2376] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23. Re-sending same reply.
I20260709 12:25:02.921653 20728 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 18: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:02.925530 20809 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:02.927932 20810 raft_consensus.cc:2376] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21. Re-sending same reply.
I20260709 12:25:02.930650 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.932819 20849 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:25:02.933787 29188 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:02.934468 29188 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:02.935472 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 91430.8us
I20260709 12:25:02.935585 29188 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.936363 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:02.937759 29134 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:02.920583 29193 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:02.939649 20807 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:02.934830 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.941133 21429 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:02.817341 29294 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.117s
I20260709 12:25:02.948335 20730 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:02.923799 29221 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:25:02.959758 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.963546 20880 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:02.964897 20810 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:02.975080 29119 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:02.976799 29119 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.941854 29136 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:02.960404 29294 consensus_queue.cc:1243] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:02.986824 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:02.989081 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:02.941946 29266 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:25:02.939277 29169 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:02.963632 29292 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:02.987440 29290 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.991434 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:02.968739 29221 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:02.954826 29279 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:25:02.945796 29194 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.005959 20808 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:03.006781 29221 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.007776 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.008458 20801 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:03.009390 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.009853 29194 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.014868 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.023952 20878 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:03.016180 20806 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: [21.14-21.14]
W20260709 12:25:03.031167 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44388.6us
I20260709 12:25:03.032263 21400 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23.
I20260709 12:25:03.032572 21400 leader_election.cc:403] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:03.034914 29171 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.000051 29290 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.037987 29290 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.038020 29122 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:03.040257 29122 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:02.999951 29292 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.041167 29122 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.042538 29122 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:03.043571 29292 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:03.044045 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.044837 20808 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:02.983820 29030 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.049868 29136 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.051764 29169 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:03.053110 28851 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.053720 28851 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.054862 29241 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.055573 29188 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:03.063656 28851 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.017045 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.072882 20879 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 13. Preceding OpId from leader: term: 30 index: 14. (index mismatch)
I20260709 12:25:03.079707 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.016652 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.079838 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.089781 20802 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:03.093341 20726 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:25:03.096180 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.014019 29311 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.100188 20728 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:03.101837 20878 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:03.100131 20808 raft_consensus.cc:2376] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24. Re-sending same reply.
I20260709 12:25:03.112128 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.113291 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.114128 20879 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:03.109685 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.099789 29011 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:03.115159 20802 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:03.132905 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.134588 20803 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:03.135731 20808 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:03.147517 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.134809 29308 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 12:25:03.108974 29134 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:25:03.166047 29311 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:03.165879 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:03.167097 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:03.170147 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:03.170776 20883 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:03.171926 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:03.175537 29241 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23.
I20260709 12:25:03.177525 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.178857 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.180536 29173 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:03.181250 29173 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:03.182328 29173 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.185000 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.187006 29135 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:03.188530 29288 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:03.152009 29011 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:03.195745 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.199591 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:03.198678 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85871.5us
I20260709 12:25:03.168676 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.203747 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.204442 20810 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.210875 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:03.212126 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.174744 29193 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.197094 29135 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:03.216748 29135 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.217828 29135 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:03.219722 28970 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:03.197701 29288 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:03.223631 29288 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.224189 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.224890 20807 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:03.225047 29288 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:03.225855 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.226501 20807 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:03.227365 29294 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:03.227772 29294 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:03.228649 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.228947 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.229393 20806 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:25:03.230171 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.230257 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.205613 29011 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.234341 29011 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:03.213884 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.228709 29294 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.240135 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.244154 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:03.244840 20733 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:25:03.246121 29282 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:25:03.246368 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 315806us
I20260709 12:25:03.247386 29033 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:03.247675 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:03.249220 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:03.249361 29113 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:03.247406 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 312632us
I20260709 12:25:03.007540 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:03.251181 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:03.226034 29299 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.412s	user 0.005s	sys 0.000s
I20260709 12:25:03.214623 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.242955 20879 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:03.254696 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:03.281889 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.282845 21428 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:03.285087 20810 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:03.287029 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:03.287881 21432 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:03.290154 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.300035 20808 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:03.302053 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.302481 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.304709 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.306721 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.307554 20807 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:03.301913 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.246399 28970 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:03.309482 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.310902 20806 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:03.250154 20624 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:03.308283 21430 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:03.312290 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.313596 20883 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:25:03.314098 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.314098 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.319602 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.297019 29011 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:03.320915 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.320359 20809 raft_consensus.cc:2376] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23. Re-sending same reply.
W20260709 12:25:03.319687 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68623.9us
I20260709 12:25:03.323971 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.329697 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.331318 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.333078 20809 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:03.321411 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:03.334493 20801 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:03.333606 20807 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:25:03.334937 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.309641 29275 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:03.333069 20879 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:03.336041 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.337599 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.338078 20802 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.338774 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.339437 20810 raft_consensus.cc:2410] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:25:03.341068 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.341658 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.318722 29033 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:03.343250 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:03.351454 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102346us
I20260709 12:25:03.351493 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.356479 20803 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.353283 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.357432 20883 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:03.352145 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.350927 20801 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:03.358079 20807 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:03.354279 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:03.371114 21432 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:03.375350 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:03.324692 29011 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:03.378264 20808 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:25:03.381759 20808 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:03.383826 20879 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:03.379359 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.386009 20805 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:03.387259 20810 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:03.355849 29033 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.411675 29033 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:03.414660 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70166.6us
I20260709 12:25:03.383708 29011 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:03.415558 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:03.416865 29113 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:03.417452 29113 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:03.395609 20720 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ccae903e8484f8d8c3af847e2c4c3d4 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 8)) from {username='slave'} at 127.0.0.1:39562
I20260709 12:25:03.418115 29113 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:03.420264 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.399396 20624 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 13 to 16, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 16 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:03.400770 29136 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:03.400837 20700 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.617s (SENT)
I20260709 12:25:03.401778 20808 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:03.403323 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:03.406014 29312 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
I20260709 12:25:03.409382 20810 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:03.334442 28970 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.435616 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:03.437778 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:03.443732 20801 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:25:03.439988 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:03.439752 29136 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.447492 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:03.446480 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.453923 20726 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:03.465291 20732 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:03.489043 20806 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:03.446563 29166 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:03.446620 29213 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:25:03.502054 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.466905 20629 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 273) took 72409 ms (client timeout 15000 ms). Trace:
I20260709 12:25:03.440789 29030 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:03.508065 29039 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:03.450057 29221 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:03.526108 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 122894us
I20260709 12:25:03.453986 29282 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:03.466564 29304 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.299s	user 0.000s	sys 0.002s
I20260709 12:25:03.538048 20803 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
W20260709 12:25:03.546160 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 145351us
I20260709 12:25:03.546680 20807 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:03.547012 20700 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.670s (SENT)
I20260709 12:25:03.522087 29213 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.555812 28970 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.547523 29072 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.532927 29221 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.555802 29280 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:03.552215 29039 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:03.559630 29193 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:03.534840 29282 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:03.588829 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:03.589596 20879 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:03.566685 29213 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:03.572057 29221 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:03.595372 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 148974us
I20260709 12:25:03.596134 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:03.599050 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.599057 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:03.597797 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 285758us
I20260709 12:25:03.577194 29282 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:03.603657 29136 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:03.604568 29039 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.611913 29334 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.612808 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:03.612825 29323 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:25:03.614641 20880 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:03.619604 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:03.621107 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:03.625212 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:03.628505 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 81526.7us
I20260709 12:25:03.629215 20802 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:25:03.629541 20700 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.751s (SENT)
I20260709 12:25:03.561332 29030 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.624758 20880 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:03.571786 29072 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.637638 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:03.641772 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.644762 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:03.631738 29280 raft_consensus.cc:2764] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:25:03.636744 29275 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:03.646693 29166 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:03.654191 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.643615 29136 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:03.658329 29317 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:03.658558 29136 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:03.658887 29113 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.659009 29317 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:03.660460 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 492157us
I20260709 12:25:03.661448 29113 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:03.661609 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:03.662166 29113 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:03.662873 29113 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:03.646392 29334 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:03.659621 29136 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.450911 20624 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 17 to 18, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 18 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:03.673501 20624 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 20 to 22, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 22 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:03.674438 29294 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:03.648483 29171 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:03.675467 29171 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:03.675596 29332 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:03.675925 29171 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:03.676812 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47300us
W20260709 12:25:03.701974 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102986us
I20260709 12:25:03.628228 29011 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.705144 20879 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:25:03.710052 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.710886 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.708503 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:03.715992 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 264345us
I20260709 12:25:03.721774 20619 catalog_manager.cc:5028] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): tablet 7ccae903e8484f8d8c3af847e2c4c3d4 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
W20260709 12:25:03.723302 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60463us
I20260709 12:25:03.703580 29332 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.703660 29288 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:03.730458 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 156798us
W20260709 12:25:03.731700 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96902.8us
I20260709 12:25:03.711827 28854 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.732391 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.729774 21434 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:25:03.720786 29169 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:25:03.521678 20629 rpcz_store.cc:269] 0709 12:23:50.985942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.986366 (+   424us) service_pool.cc:224] Handling call
0709 12:25:03.394894 (+72408528us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:50.280267 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:50.437949 (+157682us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:50.438019 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:24:50.438049 (+    30us) tablet.cc:662] Decoding operations
0709 12:24:50.438680 (+   631us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:50.438755 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:50.438797 (+    42us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:24:50.439273 (+   476us) tablet.cc:701] Row locks acquired
0709 12:24:50.439296 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:24:50.439537 (+   241us) write_op.cc:270] Start()
0709 12:24:50.439690 (+   153us) write_op.cc:276] Timestamp: P: 1783599890439440 usec, L: 0
0709 12:24:50.439721 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:24:50.684557 (+244836us) log.cc:844] Serialized 920 byte log entry
0709 12:24:51.226207 (+541650us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b540007dd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9ae28 00007f9e226d5581 00007f9e226d4d58 00007f9e226d99f1 00007f9e226d99a9 00007f9e226d9939 00007f9e226d9901 00007f9e226d8b2d 0000000000388634 0000000000388479
0709 12:24:51.407022 (+180815us) op_driver.cc:464] REPLICATION: finished
0709 12:24:51.593067 (+186045us) write_op.cc:301] APPLY: starting
0709 12:24:51.593214 (+   147us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:51.625882 (+ 32668us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:51.625934 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:51.744168 (+118234us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:51.829604 (+ 85436us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 12:24:51.829681 (+    77us) write_op.cc:312] APPLY: finished
0709 12:24:51.838346 (+  8665us) log.cc:844] Serialized 33 byte log entry
0709 12:24:51.840243 (+  1897us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:51.840380 (+   137us) write_op.cc:454] Released schema lock
0709 12:24:51.840447 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:51.841008 (+   561us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:51.841034 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1396265,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":175733,"apply.run_cpu_time_us":7459,"apply.run_wall_time_us":247935,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12487,"num_ops":2,"prepare.queue_time_us":1480623,"prepare.run_cpu_time_us":29501,"prepare.run_wall_time_us":965597,"raft.queue_time_us":127238,"raft.run_cpu_time_us":17627,"raft.run_wall_time_us":78287,"replication_time_us":967079,"spinlock_wait_cycles":440037760,"thread_start_us":1557502,"threads_started":4,"wal-append.queue_time_us":191090,"wal-append.run_cpu_time_us":44996,"wal-append.run_wall_time_us":1968828}]]}
W20260709 12:25:03.733153 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 187094us
W20260709 12:25:03.734535 20700 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.866s (SENT)
I20260709 12:25:03.746635 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:03.771018 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 67898us
I20260709 12:25:03.746935 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:03.771911 20733 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:03.772760 20726 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:25:03.773367 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49287.1us
W20260709 12:25:03.773553 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.717751 29196 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:25:03.731805 29188 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:25:03.745401 20624 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 17 to 20, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 20 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:03.784756 29175 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:03.785837 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.788714 29317 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:03.791808 29072 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.731900 29119 raft_consensus.cc:2764] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:25:03.777508 20700 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.876s (SENT)
I20260709 12:25:03.778229 29011 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:03.801028 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.778930 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.795543 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.726262 29292 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:25:03.728116 29331 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:03.803617 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.805140 21431 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:03.808339 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.808790 20700 leader_election.cc:336] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.880s (SENT)
I20260709 12:25:03.793654 29213 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.811712 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.809381 20883 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:03.732005 29308 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:25:03.816130 20804 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:03.816325 20733 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:03.819893 29342 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:03.820185 20700 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.932s (SENT)
I20260709 12:25:03.731774 29332 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.833636 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.741829 29241 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 12:25:03.741938 29266 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.065s
W20260709 12:25:03.821206 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49302.9us
I20260709 12:25:03.823366 29339 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:03.824582 29171 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:03.821346 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 169548us
I20260709 12:25:03.745281 28854 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:03.809201 29320 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.269s	user 0.000s	sys 0.004s
I20260709 12:25:03.806602 29193 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:03.832430 20809 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:03.836966 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:03.840029 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:03.793524 29033 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.843228 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:03.844228 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.832746 29030 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.834075 29011 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.848088 20731 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:03.851819 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.854202 20732 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:03.848363 29275 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:03.855983 29241 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:03.856417 29275 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:03.857913 29337 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:03.857673 29275 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.867341 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:03.846634 28851 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:03.869138 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49012.6us
W20260709 12:25:03.871346 20700 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.963s (SENT)
I20260709 12:25:03.871799 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.847322 29280 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:03.873335 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.883275 29280 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.883699 29280 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:03.885653 29332 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.887027 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:03.887703 20877 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:03.891403 29011 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.892432 29033 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.893836 29011 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:03.873792 29171 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:03.875981 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.894815 20883 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.894945 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.895712 20809 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:03.842813 29193 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.879082 29334 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:03.900432 29334 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:03.900902 29334 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:03.901737 29334 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.880472 28854 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:03.903378 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 159531us
I20260709 12:25:03.903713 29221 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:03.881768 29113 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:03.903033 29334 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:03.905506 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.907065 29221 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:03.911391 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90375.4us
I20260709 12:25:03.911738 29292 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:03.912837 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:03.914645 29292 raft_consensus.cc:2764] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:25:03.882622 29339 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:03.908187 29279 raft_consensus.cc:1064] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
W20260709 12:25:03.908612 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 133345us
I20260709 12:25:03.909574 29072 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:03.901675 29193 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:03.910920 29169 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:03.881938 29282 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:03.916874 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:03.920841 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:03.936851 20883 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:03.915560 29292 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:25:03.947894 20700 leader_election.cc:336] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.170s (SENT)
I20260709 12:25:03.948668 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:03.954532 20877 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.954348 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:03.981199 20883 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:03.937639 29072 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:03.941002 29279 raft_consensus.cc:1068] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Unable to promote non-voter 50c1ab13351d4427b836fbb449c6d7bb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:03.987690 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.982453 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:03.938833 29221 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:03.989310 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:03.991693 20876 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:25:03.948767 28854 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:03.984637 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48280.2us
I20260709 12:25:03.975818 29193 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:03.985080 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:04.011453 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.939575 29169 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:03.998320 29345 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:25:04.002053 29342 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:04.003931 29072 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.007098 29282 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:03.985595 29196 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:25:04.006565 29221 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.008658 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:04.024132 21430 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:03.953094 29339 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:03.978585 29287 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:25:03.990489 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:04.013713 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:04.014098 29193 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.014509 20882 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 18: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:04.028744 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149601us
I20260709 12:25:04.015620 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:04.030746 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:04.014406 21428 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:25:04.006436 29280 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.029208 20881 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:25:04.033933 20807 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:04.015554 29292 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:25:04.041339 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:04.046078 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:04.046440 20804 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:04.031795 29193 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:04.044083 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 96214.1us
I20260709 12:25:04.046540 20883 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:04.043535 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:04.047916 29072 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:04.048069 29339 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.052951 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:04.072351 20809 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:04.063403 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.073671 20878 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:04.035338 29122 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:04.069715 20876 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:04.070214 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56583.1us
W20260709 12:25:04.116933 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52349us
I20260709 12:25:04.118120 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:04.131527 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 100875us
I20260709 12:25:04.046190 29279 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:04.045719 29282 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:04.134652 20881 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:04.137048 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.138834 20878 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:04.139205 20849 leader_election.cc:400] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
I20260709 12:25:04.139528 20849 leader_election.cc:403] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:25:04.135506 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:04.069437 29175 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.070614 29212 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:04.135560 29337 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:04.148948 29350 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24.
W20260709 12:25:04.155850 29347 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:25:04.165740 20809 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
W20260709 12:25:04.155687 20700 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.363s (SENT)
I20260709 12:25:04.166484 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:04.036885 29169 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:04.239567 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57940.3us
W20260709 12:25:04.262856 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 87663.8us
I20260709 12:25:04.265918 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:04.275980 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.171708 29175 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.279351 20880 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:04.260242 29287 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:04.249780 29122 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:04.289048 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:04.290417 20878 raft_consensus.cc:2376] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26. Re-sending same reply.
I20260709 12:25:04.264918 29342 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:04.236503 29279 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.087s
W20260709 12:25:04.289243 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47558.1us
I20260709 12:25:04.261819 28893 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:25:04.291479 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:04.297742 20875 raft_consensus.cc:2376] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23. Re-sending same reply.
I20260709 12:25:04.289368 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.279801 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:04.296490 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:04.292335 21433 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:04.299268 29334 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:04.299571 29267 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.299891 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:04.298388 20876 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:25:04.292738 29268 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.290019 29188 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:04.315241 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 150722us
I20260709 12:25:04.323022 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:04.331651 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:04.337163 21428 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:04.349467 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 193791us
W20260709 12:25:04.351214 20700 leader_election.cc:336] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.426s (SENT)
I20260709 12:25:04.334923 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:04.374370 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:04.376879 20880 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:04.378763 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:04.379482 21431 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:04.355129 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:04.382311 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:04.329229 29292 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:04.385599 29361 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.385514 29292 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.332672 29287 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:25:04.386446 29292 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:04.387861 29287 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:04.386443 29361 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.385798 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:04.388279 29287 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:04.300237 29267 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.392668 20877 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:04.393072 20729 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
W20260709 12:25:04.391333 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56522.7us
I20260709 12:25:04.394119 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:04.397583 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:04.398057 20730 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:04.330025 29166 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:04.385488 29332 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:04.413529 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:04.414456 21435 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:04.421650 20877 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:04.421916 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70770.1us
W20260709 12:25:04.425282 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 159448us
I20260709 12:25:04.426163 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:04.379405 29280 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:04.330281 29268 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.354856 29350 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:25:04.396647 29136 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:04.333104 29188 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.402348 29119 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.396226 29344 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.378763 29033 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:25:04.439271 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:04.426548 29361 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:04.431336 29166 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:04.437644 29136 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.460595 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:04.434844 29355 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:25:04.453775 29342 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:04.435729 29011 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:25:04.495420 21433 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:25:04.432356 29122 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.461608 29136 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:04.497507 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 94656.3us
I20260709 12:25:04.485432 29359 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:04.455128 29344 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.452632 29119 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.463097 29366 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:04.486801 29356 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:04.494316 29355 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:04.498440 29332 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:04.518186 29357 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:25:04.535410 20802 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:04.512264 29359 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:04.519682 29175 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:04.543496 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104380us
I20260709 12:25:04.546308 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:04.523241 29355 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:04.464541 29287 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.544871 29359 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:04.566502 29369 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:04.570115 29188 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:04.571360 20730 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
W20260709 12:25:04.586410 20700 leader_election.cc:336] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.625s (SENT)
I20260709 12:25:04.589716 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:04.609465 29368 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:25:04.611019 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.591009 29196 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.612761 29196 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:04.614622 29072 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.001s
W20260709 12:25:04.615135 29193 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.003s
I20260709 12:25:04.607896 29268 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:04.620559 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.544243 29166 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.591382 29356 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:04.599741 29267 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:04.623808 29372 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.074s
W20260709 12:25:04.625981 29364 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:25:04.617761 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.616498 29287 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:04.615181 29196 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:04.593444 29369 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.627148 29332 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.616758 21431 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:04.612816 29376 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:04.629345 20881 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:04.637020 21435 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
W20260709 12:25:04.643693 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 139622us
W20260709 12:25:04.645033 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55422us
W20260709 12:25:04.645565 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 246880us
I20260709 12:25:04.635851 29119 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:04.635047 29369 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:04.615610 29193 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.654474 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:04.628160 29344 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:04.627254 29136 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.678574 29136 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:04.681761 29375 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.682538 29375 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.683873 29380 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:25:04.653568 29011 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:25:04.685307 29011 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:04.685698 29011 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:04.640413 29376 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:04.691370 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:04.697098 29369 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:25:04.656306 29119 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:25:04.628108 29373 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.109s
W20260709 12:25:04.704866 20700 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.761s (SENT)
I20260709 12:25:04.631405 29355 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.708360 21434 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:04.708084 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55245.4us
I20260709 12:25:04.709945 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:04.635842 29292 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.653512 29196 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.693053 29344 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:04.712095 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:04.695088 29376 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:04.686710 29011 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:04.725699 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.721375 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:04.732252 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.733583 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.733630 21435 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:04.656841 29366 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:25:04.718115 29122 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:04.719138 29294 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:04.718364 29342 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:04.745555 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:04.723393 29221 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:04.743762 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:04.760387 20883 raft_consensus.cc:2376] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29. Re-sending same reply.
I20260709 12:25:04.748962 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:04.727496 29287 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:04.727635 29175 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.762568 29175 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.765947 29175 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:04.757048 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:04.719550 29384 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:04.767436 29384 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:04.768683 29384 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.741119 29369 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.727809 29267 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:25:04.738693 29119 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:04.774545 20875 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:04.723341 29266 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 12:25:04.776539 20700 leader_election.cc:336] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.509s (SENT)
I20260709 12:25:04.908625 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:04.894260 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78533.7us
I20260709 12:25:04.750952 29294 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:04.723778 29196 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.774008 29011 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:04.916321 29377 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:25:04.918972 29375 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:04.922304 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:04.745308 29268 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.148s
I20260709 12:25:04.919811 29342 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:04.892869 29188 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:04.929234 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 156689us
I20260709 12:25:04.910539 29356 raft_consensus.cc:2764] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:25:04.893128 29369 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:04.968891 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:04.969411 29385 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.196s	user 0.004s	sys 0.000s
W20260709 12:25:04.989929 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 213408us
W20260709 12:25:04.992050 20700 leader_election.cc:336] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.528s (SENT)
I20260709 12:25:04.992714 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:04.990331 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:04.996276 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:04.924784 29122 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.060s
W20260709 12:25:04.994745 20700 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.811s (SENT)
I20260709 12:25:05.004061 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:04.998111 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:04.999329 20731 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:04.994136 20732 raft_consensus.cc:2376] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20. Re-sending same reply.
I20260709 12:25:05.005923 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.009050 20728 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:05.009629 29382 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:05.010285 29382 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.012612 29382 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.012780 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:04.988219 29375 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.042474 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.019641 21435 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:05.016993 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:05.017377 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:05.064908 20808 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:05.018432 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.065243 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.017751 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:05.066519 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:05.046463 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51751.9us
W20260709 12:25:05.067879 20700 leader_election.cc:336] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.940s (SENT)
I20260709 12:25:05.077739 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:04.989636 29135 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:04.991484 29193 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.083391 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:04.995486 29266 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:05.085266 20805 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:05.089725 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:04.990098 29119 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:05.092448 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:05.016420 29389 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:25:05.033867 29392 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:25:05.035058 29390 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:25:05.036011 29308 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:05.108388 29308 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:05.027945 29290 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:05.093375 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.093705 21435 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:25:05.099799 29292 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:04.947409 29188 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.017686 29356 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.102658 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:05.117132 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.023372 29396 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:05.003527 29030 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.105556 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 63184.6us
I20260709 12:25:05.093513 28893 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:05.135215 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:05.105672 29266 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.106245 29378 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:05.083287 29275 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:05.032361 29196 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.095304 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.101233 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.108228 29193 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.028688 29369 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.127089 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:05.025321 29359 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.127759 20730 raft_consensus.cc:2376] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30. Re-sending same reply.
I20260709 12:25:05.015355 29391 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:25:05.098169 29354 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.803s	user 0.005s	sys 0.005s
I20260709 12:25:05.130434 20728 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:05.103802 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:05.131247 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 123368us
I20260709 12:25:05.158890 20806 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:05.159150 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:05.161473 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.025564 29365 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.186s
I20260709 12:25:05.141909 29331 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:05.082885 29282 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.146495 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:05.147676 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.144847 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.147645 20808 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:05.145860 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:05.145308 20805 raft_consensus.cc:2376] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27. Re-sending same reply.
I20260709 12:25:05.014830 29386 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.204s
I20260709 12:25:05.085180 29375 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.153180 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.007890 29294 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.113829 29290 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:05.139308 29396 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:05.229561 29396 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.230800 29188 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.014518 29382 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.102s
I20260709 12:25:05.139052 29213 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.233585 29404 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:05.109436 29308 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.238240 29308 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:05.238865 29339 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:05.221313 20731 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:05.223064 29275 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:05.239871 29339 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.135620 29356 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.245653 29356 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.247714 29339 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:05.248809 29376 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 12:25:05.249347 29376 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.249876 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.250588 20807 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:05.250751 20731 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:05.251632 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.251888 20727 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:25:05.251994 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:05.164422 29391 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:05.225391 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60956.4us
I20260709 12:25:05.254473 20734 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:25:05.254277 29391 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.255373 29369 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:05.256889 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:05.224725 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60042us
I20260709 12:25:05.228457 29375 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.144821 29030 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.252331 20735 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:05.253194 20731 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:05.141480 29380 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:25:05.269672 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.219290 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:05.259302 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:05.222067 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75704.2us
I20260709 12:25:05.247849 29359 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.270887 20733 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:05.271415 20726 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:05.155792 29193 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.285459 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.289124 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:05.294242 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1578) took 1430 ms. Trace:
I20260709 12:25:05.268246 29342 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:05.270812 29396 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:05.306734 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.307423 20810 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:05.222436 29282 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.300065 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.300230 29344 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:05.293255 29113 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:05.315728 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:05.316751 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.317624 20809 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:05.294814 20801 rpcz_store.cc:276] 0709 12:25:03.864007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.864202 (+   195us) service_pool.cc:224] Handling call
0709 12:25:03.881316 (+ 17114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.915213 (+ 33897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.915244 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.915674 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:25:04.095444 (+179770us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:04.095504 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.095599 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.026820 (+931221us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:05.288319 (+261499us) raft_consensus.cc:1685] finished
0709 12:25:05.288344 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.294156 (+  5812us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33419,"raft.queue_time_us":33936,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":58,"spinlock_wait_cycles":3734784,"thread_start_us":179375,"threads_started":1,"wal-append.queue_time_us":259800}
I20260709 12:25:05.298928 29193 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:05.309707 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52916.6us
W20260709 12:25:05.310405 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52058.5us
I20260709 12:25:05.312438 20733 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:05.303404 20726 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:05.265949 29404 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:05.331571 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 55611.7us
I20260709 12:25:05.311669 29375 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:05.287478 29188 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:25:05.267549 29119 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.321022 29294 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:05.329811 29332 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:05.303428 29402 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 12:25:05.332072 29344 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:05.353315 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:05.355815 20878 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:05.357455 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:05.355890 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63045.5us
I20260709 12:25:05.361140 20884 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:05.360987 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:05.340149 29376 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.366236 29282 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.339833 29275 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:05.370644 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.371986 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:05.336357 29135 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:05.363468 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.369618 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.320818 29290 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.341925 29375 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.367358 29369 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:05.389494 29188 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:05.351253 29213 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:25:05.400378 20735 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:05.389755 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.354120 29171 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:25:05.379547 29398 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:05.382062 29266 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.361778 29332 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.360235 29330 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:25:05.423504 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.423991 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:05.425340 21430 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:25:05.426357 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.427306 20809 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:05.401437 29188 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.445495 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:05.445814 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69975.2us
I20260709 12:25:05.376183 29275 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:05.447924 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84569.7us
I20260709 12:25:05.454187 20731 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:05.454644 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.455390 20809 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:05.455137 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.449259 20810 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:05.459147 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.461475 20801 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 25. Ops: []
I20260709 12:25:05.417917 29369 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:05.427672 29011 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:05.426549 29290 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:05.467017 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.468009 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.469532 29011 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.470005 29011 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:05.452329 29188 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:05.474987 20728 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:05.468650 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:05.453876 29391 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:25:05.488718 29407 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:25:05.489728 29375 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.492288 29378 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:05.492823 29378 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:05.494035 29378 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.498544 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:05.502094 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:05.504026 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.504802 20731 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:05.506319 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.510558 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:05.513027 29135 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:05.513845 29135 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:05.515601 29400 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:05.516199 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:05.515901 29369 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.517601 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:05.503723 29378 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:05.508075 29212 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:05.447398 29344 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.534469 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:05.543948 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.545346 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:05.426906 29404 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.528721 29171 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:05.548102 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.524101 29332 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.522795 29375 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:05.522871 29135 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:05.549669 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.542095 29400 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:05.556399 29356 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:05.556749 29356 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:05.557019 29356 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:05.557739 29266 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:05.557911 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.559103 29376 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:05.559345 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:05.547673 29380 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:05.561650 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:05.552098 29401 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:05.543669 29212 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.577966 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1525) took 1113 ms. Trace:
I20260709 12:25:05.559605 29376 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:05.583354 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.583971 20805 raft_consensus.cc:2376] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24. Re-sending same reply.
I20260709 12:25:05.578560 21434 rpcz_store.cc:276] 0709 12:25:04.460167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.460307 (+   140us) service_pool.cc:224] Handling call
0709 12:25:05.572852 (+1112545us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:05.573061 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567401216}
I20260709 12:25:05.581137 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:05.585390 20809 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:05.562695 29266 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:05.568845 29332 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.591948 20880 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:25:05.600378 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:05.600764 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.601104 20883 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:05.601379 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:05.601840 20731 raft_consensus.cc:2368] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26.
I20260709 12:25:05.602236 20878 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:25:05.598798 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:05.575064 29400 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:05.552595 29421 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:05.594631 20805 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: [19.11-21.12]
I20260709 12:25:05.570343 29401 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:05.607340 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.603024 20881 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:05.591710 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:05.612557 20809 raft_consensus.cc:2376] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27. Re-sending same reply.
W20260709 12:25:05.596607 20849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.565s (SENT) [suppressed 52 similar messages]
I20260709 12:25:05.605664 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:05.605649 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:05.577039 29330 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:05.561609 29135 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.575169 29171 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:05.560777 29375 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.587743 29376 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:05.586817 29275 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.600436 29380 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:05.587772 29266 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:05.676649 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 80078.3us
I20260709 12:25:05.646250 20878 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:05.683173 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:05.617849 28851 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.618182 29401 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:05.692391 20882 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:05.701032 29403 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.701725 29403 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.688144 29266 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:05.704450 29403 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.652045 28854 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:05.694628 20878 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:05.705530 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:05.607331 29332 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.721673 20734 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:05.722419 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.731061 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:05.735807 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:05.721311 29417 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:05.752053 29428 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:05.752261 29417 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:05.716092 28854 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.752760 29417 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:05.752920 28854 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:05.755713 29212 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.762526 29332 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.766215 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.767551 29171 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.768203 29433 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:05.771713 29332 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:05.773066 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:05.774502 29332 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:05.695621 29380 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.776006 20731 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:05.777844 29380 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.779594 29135 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.779923 28854 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.781304 29375 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.782086 28854 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.782827 20881 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:05.785856 20731 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:05.787545 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:05.741044 29412 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.174s
I20260709 12:25:05.743175 28970 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:05.745877 29409 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:05.749369 29011 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.730494 29113 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:25:05.751554 29427 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:25:05.797060 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.798025 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.815479 21435 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:05.822921 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.798266 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.730119 29282 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.750504 29429 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:05.811755 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.705531 28851 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.832296 21432 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:05.814459 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 281470us
I20260709 12:25:05.828745 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.829345 29308 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:05.843433 29422 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:05.844008 29422 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:05.844494 29422 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:05.844787 29417 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.847469 29443 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:05.849046 29443 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:05.849517 29443 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:05.851873 29113 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:05.852680 29113 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:05.829614 29429 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.854637 29429 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:05.858425 29405 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.633s	user 0.002s	sys 0.000s
I20260709 12:25:05.838169 29308 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.861655 29308 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:05.865219 29266 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.866034 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:05.866665 20876 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:05.866786 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:05.867503 20878 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:05.867703 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:05.869755 20882 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
W20260709 12:25:05.872035 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.871980 29290 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:05.872972 29290 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.830817 29282 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.875615 29409 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.876497 29169 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:05.876360 29409 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.878614 29355 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:05.871508 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:05.876976 29282 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.880803 29290 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.881536 29266 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.883677 29113 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:05.884140 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:05.884693 29409 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.850646 29443 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.885588 20730 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:05.886157 29443 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:05.887754 20732 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 12. Preceding OpId from leader: term: 30 index: 14. (index mismatch)
I20260709 12:25:05.842651 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:05.889209 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60541.8us
I20260709 12:25:05.890367 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:05.891510 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:05.892823 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 161858us
I20260709 12:25:05.894819 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1516) took 1157 ms. Trace:
I20260709 12:25:05.894330 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:05.893913 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:05.899509 20734 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
W20260709 12:25:05.900691 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:05.900691 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58131.7us
I20260709 12:25:05.816052 29376 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.895237 20877 rpcz_store.cc:276] 0709 12:25:04.737443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.737609 (+   166us) service_pool.cc:224] Handling call
0709 12:25:04.737947 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.738417 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:04.751408 (+ 12991us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.031074 (+279666us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18003954a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:05.097503 (+ 66429us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.098650 (+  1147us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:05.110952 (+ 12302us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.111236 (+   284us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:05.111494 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.129058 (+ 17564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.135825 (+  6767us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.857474 (+721649us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:05.894583 (+ 37109us) raft_consensus.cc:1685] finished
0709 12:25:05.894610 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.894751 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20833,"prepare.queue_time_us":16004,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":23885,"raft.run_cpu_time_us":465,"raft.run_wall_time_us":40687,"spinlock_wait_cycles":58959232,"thread_start_us":164980,"threads_started":1,"wal-append.queue_time_us":298970}
W20260709 12:25:05.912357 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:05.914052 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.915016 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.896028 29409 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:05.900321 29361 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:05.923240 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:05.907459 29282 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:05.924724 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:05.916919 29173 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:25:05.932298 21429 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:25:05.920218 29396 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.000s
I20260709 12:25:05.887130 29401 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.930951 29282 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.970310 29282 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:05.920876 29365 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.971998 29365 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.962960 29404 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:05.909041 28970 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:05.974925 29404 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.974768 29365 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:05.975100 28970 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:05.977002 29396 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.977718 28970 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:05.895350 29400 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:25:05.977715 29396 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:05.975600 29404 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:05.978807 29400 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:05.919296 29290 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.921754 29330 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:25:05.918298 29439 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:25:05.985916 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62798.8us
I20260709 12:25:05.982352 29400 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:05.995473 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 91027.1us
W20260709 12:25:05.997006 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.983171 29417 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:05.998624 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74292.7us
W20260709 12:25:05.997751 29213 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.007s	sys 0.007s
I20260709 12:25:05.991225 29378 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:05.995041 29439 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:06.010726 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 119034us
I20260709 12:25:06.004104 29292 raft_consensus.cc:1064] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:25:06.007166 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:05.994619 29356 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:06.007946 29213 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.007s	sys 0.007s
I20260709 12:25:05.992095 29290 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.023618 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:06.009133 29173 consensus_queue.cc:1243] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:06.028267 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:06.031523 28851 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.026587 29400 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.036598 29400 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:06.010551 29378 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:06.041375 29458 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:06.041898 29458 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.041572 29378 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.042397 29458 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:06.044130 29376 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.044782 29376 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:06.045125 29376 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.045449 29376 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:06.007737 29344 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:25:06.047343 29344 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.047606 29424 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.352s	user 0.002s	sys 0.000s
I20260709 12:25:06.047911 29344 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:06.048950 29188 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.546s	user 0.004s	sys 0.008s
I20260709 12:25:06.049515 29188 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.051141 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3389) took 1524 ms. Trace:
I20260709 12:25:06.051554 20803 rpcz_store.cc:276] 0709 12:25:04.526621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.526819 (+   198us) service_pool.cc:224] Handling call
0709 12:25:04.539483 (+ 12664us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.550940 (+ 11457us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:04.600075 (+ 49135us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.489373 (+889298us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b18002c0a40. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:25:05.543027 (+ 53654us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:05.546552 (+  3525us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.546717 (+   165us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:05.546753 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.546826 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.046819 (+499993us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:06.050929 (+  4110us) raft_consensus.cc:1685] finished
0709 12:25:06.050952 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.051082 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8006,"prepare.queue_time_us":47415,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":18198,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":9236,"spinlock_wait_cycles":20152448,"thread_start_us":43746,"threads_started":1,"wal-append.queue_time_us":989875}
I20260709 12:25:06.054189 29030 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.087047 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.060798 29374 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:25:06.102953 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 77380.8us
I20260709 12:25:06.084681 29072 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:06.070417 29344 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.120939 20804 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:25:06.089600 29422 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.145044 29422 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.136420 29072 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.146080 29072 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:06.125523 29433 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.126124 29188 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:06.127617 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 127349us
I20260709 12:25:06.038975 29419 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:06.150518 29308 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.170521 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.171109 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:06.176740 21431 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:25:06.169858 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.171737 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.177556 20877 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:06.171711 29376 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.172392 29457 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:06.173720 20884 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:06.174533 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:06.180269 21434 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:06.176891 20879 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 23 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:25:06.182303 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.183049 21435 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:06.184160 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.184904 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.185158 20728 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:25:06.188635 20776 leader_election.cc:400] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:25:06.188979 20776 leader_election.cc:403] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:25:06.189248 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.191958 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.138197 29294 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.135962 29374 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:06.194716 21427 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:06.136647 29030 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.201740 29030 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.195168 21431 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:06.175010 29396 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.182972 29433 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.212249 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:06.213127 20882 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:06.177722 29419 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:06.214874 29266 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:06.224006 29453 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:25:06.223634 29419 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.190238 29457 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.225312 29419 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:06.175388 29344 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:06.227466 29344 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.205060 29429 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.229456 29344 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.229934 29072 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.214196 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.213569 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:06.182176 29169 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:25:06.219419 29376 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.221061 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.185906 29439 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.217655 21434 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:06.239074 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:06.230888 29429 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.241794 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.203428 29294 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.245448 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.246294 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.255113 20882 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:06.252095 20884 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:06.233723 29457 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.248739 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.258625 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:06.250234 20877 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
W20260709 12:25:06.265254 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.265167 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.267359 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:06.267728 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.259851 29113 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:06.264690 29350 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:06.271390 29072 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.276130 29397 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:06.276669 29397 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:06.276712 29398 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:06.278373 20810 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:06.274978 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.280246 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.280309 29193 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:06.280171 29464 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.050s
I20260709 12:25:06.282598 29375 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:06.285116 20810 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:06.288432 20884 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:06.300855 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:06.307940 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1389) took 1958 ms. Trace:
I20260709 12:25:06.247586 29463 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:25:06.283519 29429 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.285637 29193 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:06.311797 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:06.312125 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:06.308544 20729 rpcz_store.cc:276] 0709 12:25:04.340020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.340156 (+   136us) service_pool.cc:224] Handling call
0709 12:25:06.294823 (+1954667us) spinlock_profiling.cc:243] Waited 9.96 ms on lock 0x7b54002495d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:06.298613 (+  3790us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22903680}
I20260709 12:25:06.290066 29375 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:06.323426 20875 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:06.308619 29350 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:06.309255 29332 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:06.309700 29290 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.325951 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.261297 29374 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.329051 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.329362 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.329743 20808 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:25:06.322271 29429 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.295169 29282 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.334853 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.344040 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3123) took 1094 ms. Trace:
I20260709 12:25:06.367038 20802 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:06.345894 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.369937 20733 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:25:06.323227 29113 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:06.334180 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.372892 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:06.366969 20727 rpcz_store.cc:276] 0709 12:25:05.249622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.252745 (+  3123us) service_pool.cc:224] Handling call
0709 12:25:05.253074 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.253315 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:05.321861 (+ 68546us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.377905 (+ 56044us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.343438 (+965533us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b18003d3420. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:25:06.343551 (+   113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:06.343620 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:06.343683 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.343785 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.343937 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27561,"prepare.queue_time_us":96549,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":4906,"raft.queue_time_us":32061,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":8719,"spinlock_wait_cycles":16227968,"thread_start_us":136330,"threads_started":3,"wal-append.queue_time_us":157361}
I20260709 12:25:06.371511 29421 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:06.375487 20801 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:06.371330 20730 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:06.327216 29166 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.383631 29374 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:06.416679 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.348165 29308 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.294513 29292 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 13, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.408360 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:06.376909 29113 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.414242 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:06.335810 29433 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.383528 29332 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:06.422153 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.445446 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:06.422926 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.423573 29113 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:06.447265 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.424167 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:06.447935 20801 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:06.448269 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.450132 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.451118 29372 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:06.451776 29372 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.452275 29372 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:06.429934 29355 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.000s
I20260709 12:25:06.432363 29266 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:25:06.383528 29193 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.433324 29458 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.456643 29193 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:06.435240 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3194) took 1389 ms (client timeout 1590 ms). Trace:
W20260709 12:25:06.459415 20807 rpcz_store.cc:269] 0709 12:25:05.045489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.064960 (+ 19471us) service_pool.cc:224] Handling call
0709 12:25:06.435033 (+1370073us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540027ae54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:06.435128 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424234240}
W20260709 12:25:06.460372 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.437177 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.438092 29356 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.465808 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3125) took 1215 ms. Trace:
I20260709 12:25:06.443634 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:06.410288 29462 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.099s	user 0.006s	sys 0.005s
I20260709 12:25:06.467854 20727 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:06.468772 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.469228 29458 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.469369 20727 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:06.469992 20729 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:06.471146 20727 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:06.466367 20735 rpcz_store.cc:276] 0709 12:25:05.250257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.272623 (+ 22366us) service_pool.cc:224] Handling call
0709 12:25:05.290788 (+ 18165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.404825 (+114037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.465440 (+1060615us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002274d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:06.465716 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251378432}
I20260709 12:25:06.480389 20735 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:06.480541 29421 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.481436 29421 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:06.481849 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.448231 21435 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:06.487092 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.491806 29030 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.500742 20810 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:06.470790 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:06.447348 29308 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:06.440891 29350 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:06.402940 29404 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.512944 29429 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.513746 29404 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:06.514312 29471 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:25:06.514756 21428 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:06.515470 29473 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:25:06.441202 29375 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.517043 29375 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:06.419894 29166 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.519541 29030 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.520159 29030 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.520840 29166 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.521422 29166 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.522424 29030 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.523524 29332 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.524093 29332 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:06.525537 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.526441 29332 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.527225 29332 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.527602 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.527822 29332 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.529480 20801 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:06.529932 29332 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.530181 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3113) took 1414 ms. Trace:
I20260709 12:25:06.530865 20801 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:06.531517 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.521901 29166 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.536629 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.530552 20732 rpcz_store.cc:276] 0709 12:25:05.115804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.115970 (+   166us) service_pool.cc:224] Handling call
0709 12:25:05.887435 (+771465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.888116 (+   681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.530000 (+641884us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b54000e6ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:06.530125 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1721421696}
W20260709 12:25:06.534816 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88524.1us
I20260709 12:25:06.526298 20726 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:06.536803 20881 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:06.532228 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.547458 20884 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:06.539340 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.550104 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.523044 29030 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.548586 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 104072us
I20260709 12:25:06.567129 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.568069 20884 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:25:06.446453 29282 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.565256 20808 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:06.573642 20808 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:06.573421 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.579797 29166 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.583302 20729 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:06.576540 29422 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:06.584153 20801 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:06.538173 29445 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 12:25:06.584633 29422 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:06.586014 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.573482 29400 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.536s	user 0.003s	sys 0.008s
I20260709 12:25:06.597000 29422 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.560271 29332 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:25:06.597735 29422 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.598299 29332 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:06.598891 29332 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:06.573534 29030 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.599690 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.600575 20883 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:25:06.599936 29332 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.601526 29030 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.602563 21427 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:06.606336 29011 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.609810 21430 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:06.612834 29401 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.616122 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3379) took 1017 ms. Trace:
I20260709 12:25:06.616444 20879 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:06.616506 20734 rpcz_store.cc:276] 0709 12:25:05.598176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.799592 (+201416us) service_pool.cc:224] Handling call
0709 12:25:05.804968 (+  5376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.900047 (+ 95079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.615854 (+715807us) spinlock_profiling.cc:243] Waited 90.1 ms on lock 0x7b54000e7ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:06.616005 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207314816}
W20260709 12:25:06.619588 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3186) took 1609 ms (client timeout 1528 ms). Trace:
I20260709 12:25:06.620055 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:06.620332 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.623421 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.624426 20884 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:06.601462 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.625030 20883 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:06.598366 29422 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.628274 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:06.620003 20804 rpcz_store.cc:269] 0709 12:25:05.010430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.012427 (+  1997us) service_pool.cc:224] Handling call
0709 12:25:06.619354 (+1606927us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:06.619515 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540276096}
I20260709 12:25:06.629068 20878 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:06.665537 20734 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:06.637823 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2024) took 1904 ms. Trace:
I20260709 12:25:06.635571 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.640808 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:06.667886 20731 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:06.667920 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:06.669921 20727 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:06.671576 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.673430 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.623036 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:06.677680 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54736.6us
I20260709 12:25:06.678591 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:06.679708 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:06.692503 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.692654 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.695314 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.695899 20884 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:06.702559 20730 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:25:06.700135 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:06.709388 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.710373 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.715101 29422 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.716454 20731 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:06.718161 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:06.719691 29402 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:06.720147 29402 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.720486 29402 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:06.723138 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 509624us
I20260709 12:25:06.664577 29457 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.726111 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1517) took 1721 ms. Trace:
I20260709 12:25:06.726521 20875 rpcz_store.cc:276] 0709 12:25:05.004937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.005131 (+   194us) service_pool.cc:224] Handling call
0709 12:25:05.008903 (+  3772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.323869 (+1314966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.725878 (+402009us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400367254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:06.726052 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3022631296}
I20260709 12:25:06.728256 20878 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:06.735019 20883 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:06.736856 20884 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:06.737792 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.739720 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:06.703655 29011 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:06.742751 29011 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:06.743681 29011 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.744172 29455 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.744845 29455 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.746102 29466 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:06.746641 29466 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:06.741628 20884 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:06.696636 29193 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:06.704402 29433 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:06.747088 29466 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:06.754009 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:06.744921 29113 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.756455 21428 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:06.752270 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.764380 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.761560 28893 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 12:25:06.748636 29376 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.748814 29412 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:06.749377 29193 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:06.772123 29418 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:25:06.787125 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:06.780311 29193 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:06.758987 29363 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.757138 28970 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:25:06.756780 29422 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:06.791335 29359 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.791852 29458 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:06.792830 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1591) took 1021 ms. Trace:
I20260709 12:25:06.792982 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3205) took 1187 ms. Trace:
I20260709 12:25:06.795392 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.780081 29376 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.796095 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.796909 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.797756 29375 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:06.798447 29476 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:06.799364 29030 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:06.801707 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.799111 29350 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:25:06.802878 29421 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.855605 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.803967 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.741845 29457 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.759927 29433 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:06.805151 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:06.809125 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:06.818411 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1399) took 1030 ms. Trace:
I20260709 12:25:06.819826 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:06.820436 21427 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:06.821115 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:06.760305 29455 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:06.883052 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73998.7us
I20260709 12:25:06.880086 20726 rpcz_store.cc:276] 0709 12:25:05.788049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.888694 (+100645us) service_pool.cc:224] Handling call
0709 12:25:06.526112 (+637418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.538213 (+ 12101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.818181 (+279968us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54002279d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:06.818346 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425743104}
I20260709 12:25:06.880474 21427 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:06.846855 29374 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:06.833503 29363 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.897245 29113 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.851238 29458 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.844713 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.899112 29376 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.899494 29363 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.899981 29458 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.900233 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:06.848831 29359 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.901547 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.902823 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:06.851508 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.907894 20729 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:06.908001 29342 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.908694 29342 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.755230 29372 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:06.912333 29454 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.883s	user 0.004s	sys 0.000s
I20260709 12:25:06.913493 29372 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:06.914316 29372 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:06.838620 29466 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.917871 29466 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:06.872953 20728 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:06.842517 29463 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:06.919592 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.920328 20728 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:06.921290 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.876070 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.923431 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:06.924652 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.843372 20807 rpcz_store.cc:276] 0709 12:25:05.771432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.460828 (+689396us) service_pool.cc:224] Handling call
0709 12:25:06.461217 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.461481 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:06.461881 (+   400us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.792517 (+330636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:06.792563 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:06.792601 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.792673 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.792780 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62927,"prepare.queue_time_us":1936,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":64056,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":45,"thread_start_us":267264,"threads_started":1}
I20260709 12:25:06.926033 21430 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:06.874581 29166 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:06.926702 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.926919 29166 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:06.878746 20808 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:06.831543 20734 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
W20260709 12:25:06.890414 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85373.7us
I20260709 12:25:06.925395 20810 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 18.
I20260709 12:25:06.931391 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:06.891362 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:06.892125 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:06.931984 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:06.894393 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:06.932582 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:06.887392 21427 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.896983 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.903293 29375 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:06.924551 29433 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.942057 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3337) took 1341 ms. Trace:
I20260709 12:25:06.942579 20880 rpcz_store.cc:276] 0709 12:25:05.600529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.604602 (+  4073us) service_pool.cc:224] Handling call
0709 12:25:05.605042 (+   440us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:05.618619 (+ 13577us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:05.618651 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:06.432800 (+814149us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:06.433659 (+   859us) log.cc:844] Serialized 78 byte log entry
0709 12:25:06.737806 (+304147us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:06.737865 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.737966 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.941683 (+203717us) raft_consensus.cc:1685] finished
0709 12:25:06.941721 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.941986 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1454,"raft.queue_time_us":1620,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10590,"spinlock_wait_cycles":60622208,"thread_start_us":303712,"threads_started":1,"wal-append.queue_time_us":461906}
I20260709 12:25:06.880060 29282 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:06.904512 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:06.953637 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:06.904636 29421 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:06.905109 29409 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:06.893487 29418 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:06.905727 29455 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:25:06.906032 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 86302.9us
I20260709 12:25:06.843428 20803 rpcz_store.cc:276] 0709 12:25:05.605582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.053497 (+447915us) service_pool.cc:224] Handling call
0709 12:25:06.053938 (+   441us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:06.054414 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.054454 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.055111 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:25:06.328157 (+273046us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:06.664523 (+336366us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.792543 (+128020us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.792721 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.792780 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.792817 (+    37us) raft_consensus.cc:1685] finished
0709 12:25:06.792828 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.792934 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127515,"prepare.queue_time_us":54716,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":133101,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":204,"thread_start_us":478199,"threads_started":2,"wal-append.queue_time_us":481987}
I20260709 12:25:06.873070 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:06.992636 20805 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:06.939492 29375 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:06.930258 29342 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:06.961851 29384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.012s	sys 0.028s
I20260709 12:25:06.933977 20730 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:06.933399 21427 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:06.935909 29381 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:06.935791 29467 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:25:06.936075 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:07.010847 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:07.011065 20728 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:06.936946 29113 raft_consensus.cc:2764] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:25:06.938184 29193 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.903677 29266 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:06.946256 20882 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:06.946831 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:07.022573 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75841.1us
I20260709 12:25:06.948940 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.949597 20807 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:07.024740 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:06.980566 29409 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:06.893774 29334 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:06.954684 29397 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.903085 29359 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.037675 21429 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
W20260709 12:25:06.959051 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66989.1us
I20260709 12:25:06.930486 29374 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:06.919049 29463 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:06.989115 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:06.986922 20880 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:06.922070 20876 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:06.994766 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.929450 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:07.008380 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:07.047842 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.933084 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:07.048893 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 115882us
I20260709 12:25:07.011737 20732 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:06.952746 29282 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:07.014503 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:07.029285 29387 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:06.990038 29458 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:07.028312 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:07.059065 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:07.013880 29384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.012s	sys 0.028s
I20260709 12:25:07.031299 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:07.036679 29334 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:06.951591 20729 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:07.009579 29375 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:07.029028 29030 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.039595 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:07.052608 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:07.055724 20801 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:07.018998 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.071564 20803 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
W20260709 12:25:07.057060 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 127680us
I20260709 12:25:07.057752 29443 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:07.032241 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:07.059283 20807 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:07.074011 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:06.991384 29294 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.059945 20728 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:07.026432 29381 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:07.031103 29350 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:07.062506 20731 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:07.063220 29397 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:07.058488 29418 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:07.055042 29280 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.056219 29113 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:07.030947 29476 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:25:07.012614 29337 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
W20260709 12:25:07.095019 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62869.8us
I20260709 12:25:07.080439 29458 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:07.090235 29266 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:07.097944 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:07.081629 29363 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:25:07.110388 20730 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:25:07.110641 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.124110 20803 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
W20260709 12:25:07.098275 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.101248 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:07.096431 29350 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.124434 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:07.103384 29287 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.129923 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:07.107272 29403 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:25:07.147303 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.122464 29280 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:07.222424 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 92622us
I20260709 12:25:07.148002 29381 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.135293 29169 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:07.148801 29113 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:07.135866 29337 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:07.112970 29355 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:07.226694 21427 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 18. (term mismatch)
W20260709 12:25:07.229003 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 104675us
I20260709 12:25:07.349004 21427 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 17
I20260709 12:25:07.128413 29467 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:07.233006 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:07.236116 20805 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:07.354498 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.081291 29391 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:25:07.355114 20805 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
W20260709 12:25:07.356099 29482 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.279s	user 0.000s	sys 0.000s
I20260709 12:25:07.356204 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.357615 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:07.358093 29487 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.291s	user 0.000s	sys 0.000s
I20260709 12:25:07.358738 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.361231 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:07.362017 29193 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.337378 29294 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.363894 29294 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.366694 29294 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.368520 29391 consensus_queue.cc:1243] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:07.369107 29391 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:07.343216 29334 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.369431 29391 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:07.081954 29030 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:07.313010 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.128147 29400 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.332640 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.236474 20809 raft_consensus.cc:2955] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 15 to 17, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:07.135576 29350 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:07.132958 29384 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:07.349442 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:07.254704 28893 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:07.279593 20801 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:07.237413 29266 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:07.384516 20732 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:07.349320 21427 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:25:07.339735 29455 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:07.390116 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:07.394318 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:07.394528 29030 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.343995 29113 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:07.397933 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:07.401556 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:07.416771 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:07.343678 29342 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.405496 21427 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:07.338128 29442 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.418139 20807 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:07.109570 29282 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.132211 29463 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.423281 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.395972 29175 raft_consensus.cc:2764] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 12:25:07.352449 29402 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.012s
I20260709 12:25:07.423480 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:07.402552 28893 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.406524 29337 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:07.230762 29287 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.350845 29169 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.370335 29391 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.468093 20807 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:07.394366 29489 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:07.409654 29455 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.410725 29421 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.388892 29400 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:06.672641 21433 rpcz_store.cc:276] 0709 12:25:04.733583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.734247 (+   664us) service_pool.cc:224] Handling call
0709 12:25:04.745457 (+ 11210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.745667 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:06.347843 (+1602176us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b18003a5460. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf
0709 12:25:06.348046 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.637376 (+289330us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:06.637450 (+    74us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:06.637513 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.637607 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.637751 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":63358,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":44029,"raft.queue_time_us":144,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3124,"spinlock_wait_cycles":21681408,"thread_start_us":455097,"threads_started":2}
I20260709 12:25:07.422010 29355 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:07.517660 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.429610 29011 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:07.434198 29361 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:07.430080 29344 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
W20260709 12:25:07.527052 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.490885 28893 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:07.529083 29030 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:07.427366 29476 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:07.521588 29433 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:07.534832 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:07.537569 29492 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:25:07.537427 29355 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.538530 29011 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:07.538235 29113 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.542150 29457 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.484203 29294 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:07.418476 29458 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.549147 29400 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.551936 29266 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.430125 29442 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.561882 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:07.533485 29374 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.535677 29391 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:07.569537 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.022s	sys 0.035s
I20260709 12:25:07.572165 20804 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:07.544343 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:07.494118 29402 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.496713 29337 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.567778 29455 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:07.588153 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 471772us
I20260709 12:25:07.594553 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:07.596359 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.022s	sys 0.036s
W20260709 12:25:07.615399 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 584716us
I20260709 12:25:07.429852 29342 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.575135 29433 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:07.598910 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:07.608445 20801 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:07.571301 29421 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.613061 29442 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:07.599640 29113 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:07.567940 29476 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.568174 29030 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.655375 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:07.599243 29280 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.660820 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:07.663049 21434 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:07.618706 29294 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:07.666564 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:07.617293 29376 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.655117 29011 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.661365 29496 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:25:07.685081 29496 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:07.685882 29496 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.686424 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3290) took 1152 ms. Trace:
W20260709 12:25:07.688297 29494 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:25:07.689850 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3390) took 1531 ms. Trace:
I20260709 12:25:07.690325 20802 rpcz_store.cc:276] 0709 12:25:06.158800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.474558 (+315758us) service_pool.cc:224] Handling call
0709 12:25:06.474970 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.794661 (+319691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.794712 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.795409 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.135090 (+339681us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:07.135132 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.135249 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.689622 (+554373us) raft_consensus.cc:1685] finished
0709 12:25:07.689642 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.689793 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":301133,"raft.queue_time_us":301486,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":339209,"threads_started":1,"wal-append.queue_time_us":730969}
I20260709 12:25:07.691690 29342 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:07.692442 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.692687 20802 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 12:25:07.694540 20802 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:25:07.687057 20876 rpcz_store.cc:276] 0709 12:25:06.534324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.534452 (+   128us) service_pool.cc:224] Handling call
0709 12:25:06.534767 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.054970 (+520203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.686056 (+631086us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400305a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:07.686330 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890203392}
I20260709 12:25:07.708596 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:07.709482 20876 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:07.670882 29442 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:07.712357 21433 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
W20260709 12:25:07.707525 29375 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.000s
I20260709 12:25:07.716895 20807 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:07.715082 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:07.719852 20807 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:07.704138 29490 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.668946 29280 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:25:07.672966 29294 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.675257 29376 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.730391 20807 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 12:25:07.725950 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 302634us
I20260709 12:25:07.720942 29400 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:07.725644 29506 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:07.766988 20808 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:25:07.735497 29287 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.721139 29375 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.719498 29442 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:07.772267 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.762040 29502 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:07.725472 29490 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.783438 29502 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:07.766398 29400 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:07.770219 29506 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.760637 29344 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.024s
I20260709 12:25:07.763036 29457 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:25:07.719276 29433 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.789700 29502 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:07.806914 20807 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:07.814857 29496 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.815713 29030 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.768754 29403 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:25:07.816998 29350 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:07.798826 29337 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:07.793509 29498 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:25:07.819157 29350 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:07.777611 29188 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.024s
W20260709 12:25:07.820067 29501 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
W20260709 12:25:07.820937 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 160234us
I20260709 12:25:07.797778 29506 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:07.824040 29499 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:25:07.831478 29442 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:07.831379 29400 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.832792 29375 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:07.833786 29376 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.821790 29280 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:07.844661 29280 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:07.848189 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70201.7us
I20260709 12:25:07.840915 29188 consensus_queue.cc:1243] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:07.848410 20731 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:07.849417 20876 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:07.849961 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:07.841460 29509 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:07.825307 29402 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:07.866192 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:07.867280 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:07.867551 20876 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 20: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:07.842648 28893 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.868279 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:07.867504 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.872829 20883 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:07.847114 29280 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.873754 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.874630 20876 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:07.877642 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:07.875495 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.864641 29509 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:07.878904 20876 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 18.
I20260709 12:25:07.879910 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:07.880715 20876 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:07.881006 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:07.881453 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:07.882529 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.883287 29280 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:07.883605 29294 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.883766 29509 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:07.885063 29509 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:07.891505 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:07.893915 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.894829 20729 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:07.901540 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:07.902848 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:07.897691 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:07.898029 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.904631 20876 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:07.907956 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:07.908582 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:07.908932 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:07.909314 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.910251 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.911099 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:07.908013 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1607) took 1030 ms. Trace:
I20260709 12:25:07.911616 20876 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:07.913021 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.913565 20876 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:07.915612 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:07.918859 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:07.886651 29376 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.919718 20729 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:07.922431 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:07.925014 20728 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:07.914189 29280 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.922459 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:07.922601 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:07.933525 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:07.934412 20876 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:07.906834 29113 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:25:07.937234 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:07.920581 29375 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:07.925951 29513 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.945016 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1552) took 1123 ms. Trace:
I20260709 12:25:07.966007 21431 rpcz_store.cc:276] 0709 12:25:06.821491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.622634 (+801143us) service_pool.cc:224] Handling call
0709 12:25:07.623032 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.625791 (+  2759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.625840 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.626937 (+  1097us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.805163 (+178226us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:07.805207 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.805285 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.944736 (+139451us) raft_consensus.cc:1685] finished
0709 12:25:07.944759 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.944940 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":907,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":132875,"threads_started":1,"wal-append.queue_time_us":211209}
W20260709 12:25:07.946017 29512 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:07.957996 29490 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:07.923237 29376 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:07.970068 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.069312 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:08.069731 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.078254 20728 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:08.094223 21431 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:08.099012 21431 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:08.179966 29514 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
W20260709 12:25:08.216670 29521 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 12:25:08.204717 29523 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:25:08.274341 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.090981 29363 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.088521 29361 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:08.096658 29113 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:07.939968 29496 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.167s
I20260709 12:25:08.072197 29513 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.068549 29403 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:07.934648 29280 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.396706 20876 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:25:08.397420 21431 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:25:08.435549 28893 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.381917 29433 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.396478 29509 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:08.455931 29509 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.446947 29403 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:08.423751 29455 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.798s	user 0.013s	sys 0.004s
I20260709 12:25:08.459658 29455 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.460172 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1553) took 1614 ms. Trace:
I20260709 12:25:08.460595 21433 rpcz_store.cc:276] 0709 12:25:06.846223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.721446 (+875223us) service_pool.cc:224] Handling call
0709 12:25:07.738301 (+ 16855us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.738821 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.738860 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.763285 (+ 24425us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.886889 (+123604us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:07.886934 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.887022 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.459927 (+572905us) raft_consensus.cc:1685] finished
0709 12:25:08.459949 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.460110 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2783,"raft.queue_time_us":122,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":19215,"thread_start_us":89282,"threads_started":1,"wal-append.queue_time_us":187625}
I20260709 12:25:08.462152 29455 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.464200 20805 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22.
W20260709 12:25:08.465804 29375 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 12:25:08.466351 29375 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:08.470693 29391 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.007s
I20260709 12:25:08.477111 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.481745 29344 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:08.482364 29344 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:08.488528 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1600) took 1645 ms. Trace:
I20260709 12:25:08.489156 20808 rpcz_store.cc:276] 0709 12:25:06.842660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.690110 (+847450us) service_pool.cc:224] Handling call
0709 12:25:07.690486 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.767422 (+ 76936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.488189 (+720767us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b540025d5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:08.488446 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175053056}
I20260709 12:25:08.491029 28893 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.491840 28893 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.494701 28893 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.451756 29433 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.450673 29113 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:08.514249 29113 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.515838 29433 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:08.454185 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2037) took 1604 ms (client timeout 2004 ms). Trace:
W20260709 12:25:08.516547 21434 rpcz_store.cc:269] 0709 12:25:06.849808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.953378 (+103570us) service_pool.cc:224] Handling call
0709 12:25:08.453913 (+1500535us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54005de154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:08.454106 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626391296}
I20260709 12:25:08.517205 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3300) took 1449 ms. Trace:
I20260709 12:25:08.455204 29376 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.497169 29280 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:08.499526 21426 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2025) took 3765 ms (client timeout 1974 ms). Trace:
I20260709 12:25:08.501041 29332 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:08.519150 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 135929us
I20260709 12:25:08.501375 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1538) took 1495 ms. Trace:
I20260709 12:25:08.427996 29361 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.523779 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:08.509586 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.524405 20808 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:08.447170 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.422230 29363 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.511355 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:08.511812 20735 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:25:08.462100 29502 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.005s
I20260709 12:25:08.517639 20877 rpcz_store.cc:276] 0709 12:25:07.068010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.516357 (+1448347us) service_pool.cc:224] Handling call
0709 12:25:08.516674 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.516865 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.516894 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.516933 (+    39us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:08.516973 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.517029 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.517137 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:08.519985 21426 rpcz_store.cc:269] 0709 12:25:04.733870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.734835 (+   965us) service_pool.cc:224] Handling call
0709 12:25:08.499220 (+3764385us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:08.499413 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5291669376}
I20260709 12:25:08.521341 20880 rpcz_store.cc:276] 0709 12:25:07.006514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.053922 (+ 47408us) service_pool.cc:224] Handling call
0709 12:25:07.054377 (+   455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.396538 (+342161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.396575 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.419727 (+ 23152us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.740368 (+320641us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:07.740426 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.500856 (+760430us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54001fe854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:08.500923 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.500991 (+    68us) raft_consensus.cc:1685] finished
0709 12:25:08.501022 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.501299 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":313,"raft.queue_time_us":132560,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"spinlock_wait_cycles":785149184,"thread_start_us":216821,"threads_started":1,"wal-append.queue_time_us":278749}
I20260709 12:25:08.528971 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:08.532809 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1554) took 1580 ms. Trace:
W20260709 12:25:08.536638 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.543771 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:08.544318 20880 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:08.550282 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1539) took 1064 ms. Trace:
I20260709 12:25:08.554658 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3306) took 1110 ms. Trace:
I20260709 12:25:08.555305 20880 rpcz_store.cc:276] 0709 12:25:07.444099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.542098 (+1097999us) service_pool.cc:224] Handling call
0709 12:25:08.542467 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.554356 (+ 11889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.554585 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.523375 29375 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:08.550923 20878 rpcz_store.cc:276] 0709 12:25:07.485762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.549172 (+1063410us) service_pool.cc:224] Handling call
0709 12:25:08.549591 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.549808 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.549836 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.549895 (+    59us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:08.549943 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.550031 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.550201 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.585793 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:08.572108 20880 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 12:25:08.588105 20732 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:08.531903 29369 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:08.588852 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1540) took 1102 ms. Trace:
I20260709 12:25:08.592501 20880 rpcz_store.cc:276] 0709 12:25:07.486843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.556379 (+1069536us) service_pool.cc:224] Handling call
0709 12:25:08.556809 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.588348 (+ 31539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.588693 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.594966 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3367) took 1072 ms. Trace:
I20260709 12:25:08.596334 20880 rpcz_store.cc:276] 0709 12:25:07.522119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.593778 (+1071659us) service_pool.cc:224] Handling call
0709 12:25:08.594213 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.594475 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.594507 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.594573 (+    66us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:08.594630 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.594704 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.594890 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.542783 29459 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:08.603574 29290 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:08.604044 29513 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.531226 29502 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:08.600400 29506 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.752s	user 0.011s	sys 0.006s
I20260709 12:25:08.544080 29507 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:08.560770 29030 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:08.585275 29380 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:08.527554 29344 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.627097 29502 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:08.631524 20879 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:08.609750 29290 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:08.533288 21427 rpcz_store.cc:276] 0709 12:25:06.952376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.916142 (+963766us) service_pool.cc:224] Handling call
0709 12:25:07.917422 (+  1280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.917656 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:08.381251 (+463595us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.526596 (+145345us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.532436 (+  5840us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.532506 (+    70us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:08.532568 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.532638 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.532758 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":535491,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":1955,"spinlock_wait_cycles":9259264,"thread_start_us":414433,"threads_started":2}
I20260709 12:25:08.636153 29290 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.637533 29290 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:08.639567 20881 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:08.640056 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1550) took 1819 ms. Trace:
I20260709 12:25:08.640224 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1542) took 1074 ms. Trace:
I20260709 12:25:08.640551 20881 rpcz_store.cc:276] 0709 12:25:07.565654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.639008 (+1073354us) service_pool.cc:224] Handling call
0709 12:25:08.639370 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.639984 (+   614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.640171 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.640451 21429 rpcz_store.cc:276] 0709 12:25:06.820896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.821668 (+   772us) service_pool.cc:224] Handling call
0709 12:25:06.821957 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.038581 (+216624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.639820 (+1601239us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540050c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:08.639998 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495390976}
I20260709 12:25:08.642462 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.643128 20803 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:08.644146 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.644837 20803 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:08.645587 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:08.646237 20803 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:08.646332 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1416) took 1725 ms. Trace:
I20260709 12:25:08.646699 20731 rpcz_store.cc:276] 0709 12:25:06.920730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.121823 (+201093us) service_pool.cc:224] Handling call
0709 12:25:07.127880 (+  6057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.849413 (+721533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.646085 (+796672us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54000e74d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:08.646273 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1656321792}
I20260709 12:25:08.648721 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:08.610682 29459 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.651073 29459 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.654457 29169 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.620339 29506 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.655750 20875 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: [25.12-25.12]
I20260709 12:25:08.656800 29506 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:08.619035 20808 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:08.652667 29363 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.661079 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1543) took 1092 ms. Trace:
I20260709 12:25:08.661491 20881 rpcz_store.cc:276] 0709 12:25:07.568474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.641492 (+1073018us) service_pool.cc:224] Handling call
0709 12:25:08.641796 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.660700 (+ 18904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.660728 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.660772 (+    44us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:08.660816 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.660879 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.660995 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.662093 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.665236 20808 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:08.672201 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:08.679018 20808 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:08.646978 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.662969 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:08.709719 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46858us
I20260709 12:25:08.601248 29375 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:08.621603 29380 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.682502 29280 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.682777 29360 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:08.717176 29506 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:08.613998 29507 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.658233 29344 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:08.654495 29361 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.694023 29166 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.630982 29509 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.659162 29030 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:08.724155 20876 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:25:08.695374 29459 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:08.696029 20881 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:08.698324 20734 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:08.718376 29360 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.723778 29496 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.726619 29380 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.698326 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:08.704128 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3203) took 3096 ms. Trace:
I20260709 12:25:08.731446 20731 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:08.711856 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:08.714154 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.687633 29363 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.715622 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:08.748291 20730 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:08.713418 29294 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:08.679582 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:08.750620 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.718665 29506 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:08.736646 20806 rpcz_store.cc:276] 0709 12:25:05.591908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.592296 (+   388us) service_pool.cc:224] Handling call
0709 12:25:05.592679 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:05.592919 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.592955 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:05.593553 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.616298 (+ 22745us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:05.681302 (+ 65004us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.683628 (+  2326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.683764 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.118247 (+1434483us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800370de0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1
0709 12:25:08.657420 (+1539173us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b1800370de0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:08.688428 (+ 31008us) raft_consensus.cc:1685] finished
0709 12:25:08.688459 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.688690 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8430,"prepare.queue_time_us":125596,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":102,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4070,"spinlock_wait_cycles":11543808,"thread_start_us":86593,"threads_started":2,"wal-append.queue_time_us":100684}
I20260709 12:25:08.762176 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.762996 20806 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:08.768606 20808 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:25:08.769857 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:08.770833 29522 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.351s	user 0.001s	sys 0.002s
I20260709 12:25:08.719945 29507 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:08.720377 20732 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:08.752350 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.793228 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:08.794276 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:08.796065 29266 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.796571 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.796705 29266 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.797788 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:08.798803 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:08.772154 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:08.799717 29266 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.800024 29513 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:25:08.777233 29495 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.224s	user 0.015s	sys 0.001s
I20260709 12:25:08.800503 29513 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:08.800916 29513 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:08.803586 29490 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:08.804333 29292 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:08.804792 29292 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:08.804899 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:08.804224 29266 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:08.805680 29292 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.806259 29266 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:08.807068 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1547) took 2029 ms. Trace:
I20260709 12:25:08.804255 29490 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.807462 21428 rpcz_store.cc:276] 0709 12:25:06.777441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.777697 (+   256us) service_pool.cc:224] Handling call
0709 12:25:06.778099 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.778531 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.778573 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.782970 (+  4397us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.135638 (+352668us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:07.135677 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.135754 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.776541 (+1640787us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:08.806790 (+ 30249us) raft_consensus.cc:1685] finished
0709 12:25:08.806812 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.807010 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":655,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":30197,"spinlock_wait_cycles":25745280,"thread_start_us":339460,"threads_started":1,"wal-append.queue_time_us":721205}
W20260709 12:25:08.784314 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1595) took 1944 ms (client timeout 1517 ms). Trace:
I20260709 12:25:08.810434 29490 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:08.810801 20804 rpcz_store.cc:269] 0709 12:25:06.840219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.851231 (+ 11012us) service_pool.cc:224] Handling call
0709 12:25:08.784119 (+1932888us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b5400127d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:08.784240 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528130432}
I20260709 12:25:08.739260 20881 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:08.812438 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.768642 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.806679 29266 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:08.813133 29421 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.789249 29403 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.786900 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3365) took 1704 ms. Trace:
I20260709 12:25:08.814065 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.814170 29266 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.814383 20879 rpcz_store.cc:276] 0709 12:25:07.082363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.476753 (+1394390us) service_pool.cc:224] Handling call
0709 12:25:08.786682 (+309929us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400267854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:08.786834 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352658816}
I20260709 12:25:08.804490 29387 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:08.815452 29266 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:08.815654 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.815919 29387 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:08.816480 20806 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:08.772636 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:08.817039 20810 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:08.818694 29282 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:08.806702 29292 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:08.820096 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:08.812448 20803 raft_consensus.cc:2376] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23. Re-sending same reply.
I20260709 12:25:08.821158 29387 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:08.823118 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3370) took 1051 ms. Trace:
I20260709 12:25:08.823640 20876 rpcz_store.cc:276] 0709 12:25:07.771975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.446850 (+674875us) service_pool.cc:224] Handling call
0709 12:25:08.822885 (+376035us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400200b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:08.823044 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456613632}
I20260709 12:25:08.813432 20804 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 12:25:08.824717 29504 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.096s	user 0.006s	sys 0.002s
W20260709 12:25:08.792140 29493 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.241s	user 0.000s	sys 0.002s
I20260709 12:25:08.825644 20804 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:08.826341 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3215) took 1753 ms. Trace:
I20260709 12:25:07.911743 20802 rpcz_store.cc:276] 0709 12:25:06.877193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.695858 (+818665us) service_pool.cc:224] Handling call
0709 12:25:07.696187 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.696395 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:07.821160 (+124765us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.907440 (+ 86280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:07.907520 (+    80us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:07.907599 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.907725 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.907907 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":140710,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":146,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":4991,"thread_start_us":195205,"threads_started":2}
I20260709 12:25:08.818631 29490 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.827150 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3204) took 3232 ms. Trace:
I20260709 12:25:08.836745 20809 rpcz_store.cc:276] 0709 12:25:05.594765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.652406 (+ 57641us) service_pool.cc:224] Handling call
0709 12:25:05.680333 (+ 27927us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:06.766406 (+1086073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.766451 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.785138 (+ 18687us) log.cc:844] Serialized 590 byte log entry
0709 12:25:07.069570 (+284432us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:07.456102 (+386532us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.730652 (+274550us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.730944 (+   292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.764185 (+1033241us) spinlock_profiling.cc:243] Waited 5.99 ms on lock 0x7b18003bfc20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:25:08.765741 (+  1556us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b54002b57d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:08.765800 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.791405 (+ 25605us) spinlock_profiling.cc:243] Waited 4.5 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:08.826907 (+ 35502us) raft_consensus.cc:1685] finished
0709 12:25:08.826934 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.827089 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":392360,"prepare.queue_time_us":331785,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":1514,"raft.queue_time_us":374707,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":44959,"spinlock_wait_cycles":1759095168,"thread_start_us":487044,"threads_started":2,"wal-append.queue_time_us":704670}
I20260709 12:25:08.826683 20804 rpcz_store.cc:276] 0709 12:25:07.073168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.824948 (+1751780us) service_pool.cc:224] Handling call
0709 12:25:08.825303 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.826056 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.826273 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.726981 29360 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.829728 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3296) took 1769 ms. Trace:
I20260709 12:25:08.831002 20877 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:25:08.848233 20802 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:08.849081 20808 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:08.731521 29166 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:08.849470 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3216) took 1776 ms. Trace:
I20260709 12:25:08.850641 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.851046 20802 rpcz_store.cc:276] 0709 12:25:07.073415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.836699 (+1763284us) service_pool.cc:224] Handling call
0709 12:25:08.837125 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.849198 (+ 12073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.849407 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.848304 20875 rpcz_store.cc:276] 0709 12:25:07.060459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.513893 (+1453434us) service_pool.cc:224] Handling call
0709 12:25:08.514288 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.656196 (+141908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.829476 (+173280us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540019c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:08.829668 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324601728}
I20260709 12:25:08.854771 20731 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:08.726928 29496 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.867686 29030 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.761622 29363 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:08.871817 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.873481 29443 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.874004 29443 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.876065 29375 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:08.876467 29443 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.721177 29280 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.753072 29459 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:08.832463 29332 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:08.846818 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 43839.1us
I20260709 12:25:08.886765 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:08.879415 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 199954us
W20260709 12:25:08.887283 21430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1596) took 2265 ms (client timeout 1716 ms). Trace:
I20260709 12:25:08.878288 29363 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:08.879179 20810 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
W20260709 12:25:08.889573 21430 rpcz_store.cc:269] 0709 12:25:06.621592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.621707 (+   115us) service_pool.cc:224] Handling call
0709 12:25:08.887051 (+2265344us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400750154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:08.887178 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535001472}
I20260709 12:25:08.891327 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:08.880692 29403 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.880528 29509 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:08.829264 29290 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:25:08.898458 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:08.898645 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:08.829787 29294 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.883795 29332 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:08.902309 21429 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:08.902069 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:08.903909 21428 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:25:08.900907 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:08.906077 21427 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:08.879056 29490 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.913563 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:08.911695 29455 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:08.915967 29169 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:08.916373 29169 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.916716 29169 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:08.920130 29409 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.920753 29280 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:08.921288 29409 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:08.922722 20805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3218) took 1848 ms (client timeout 1807 ms). Trace:
W20260709 12:25:08.923899 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.925536 29372 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:08.926258 29372 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.902674 29530 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:08.905102 29400 raft_consensus.cc:2764] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:25:08.907426 29332 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:08.928999 20805 rpcz_store.cc:269] 0709 12:25:07.073960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.358475 (+284515us) service_pool.cc:224] Handling call
0709 12:25:08.922472 (+1563997us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002bff54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:08.922643 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2538663168}
I20260709 12:25:08.932408 20809 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:08.908249 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:08.903764 29403 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:08.901144 29506 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:08.947824 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:08.939396 29530 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:08.954794 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.955430 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.963721 20876 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
W20260709 12:25:08.958182 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 422974us
I20260709 12:25:08.851637 20803 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:08.973261 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:08.990921 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.992333 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.993551 20876 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:08.994318 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.994367 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3421) took 1087 ms. Trace:
I20260709 12:25:08.995589 20803 rpcz_store.cc:276] 0709 12:25:07.906801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.821598 (+914797us) service_pool.cc:224] Handling call
0709 12:25:08.994165 (+172567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.996150 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.997503 20876 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:08.998114 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.999406 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:09.000597 20875 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:09.002394 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.003415 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:09.009588 21427 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:09.016451 20803 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:25:09.016634 20879 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:09.009979 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:09.010613 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49065.3us
I20260709 12:25:09.000109 28970 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:09.021994 28970 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.022783 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:08.970773 29509 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.053s
I20260709 12:25:09.025091 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:09.027117 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:09.028025 29496 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.029351 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:09.030126 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3213) took 1957 ms. Trace:
I20260709 12:25:09.030580 20807 rpcz_store.cc:276] 0709 12:25:07.072666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.738324 (+665658us) service_pool.cc:224] Handling call
0709 12:25:07.738620 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.807364 (+ 68744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.029885 (+1222521us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b540019a0d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:09.030061 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156203904}
I20260709 12:25:08.931299 29455 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:25:09.033455 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:09.031783 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:09.034618 20807 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:09.035964 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:09.038393 20807 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:09.042430 21427 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:09.039038 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:08.962754 29375 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.045823 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3419) took 1156 ms. Trace:
I20260709 12:25:09.046707 20807 rpcz_store.cc:276] 0709 12:25:07.888968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.037620 (+1148652us) service_pool.cc:224] Handling call
0709 12:25:09.038166 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.044747 (+  6581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.045524 (+   777us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.047791 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3360) took 2240 ms. Trace:
I20260709 12:25:08.963003 29502 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:08.964169 29369 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:09.001325 29171 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:25:08.990680 29387 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.006983 29418 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:08.991356 29337 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:09.050928 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1618) took 1142 ms. Trace:
I20260709 12:25:09.020066 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:08.812868 29513 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.030880 20876 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:09.001124 29400 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:09.079336 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:08.912744 29459 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.072867 20875 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:09.119531 20729 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:09.053639 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.120707 21429 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:25:09.120862 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:09.051467 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.085484 20807 rpcz_store.cc:276] 0709 12:25:07.908977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.049238 (+1140261us) service_pool.cc:224] Handling call
0709 12:25:09.050200 (+   962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.050488 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.050533 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.050589 (+    56us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:09.050635 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.050719 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.050856 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:09.076363 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55295.8us
I20260709 12:25:09.104553 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.123255 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:09.123488 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.114249 20876 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:09.080615 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3354) took 2295 ms. Trace:
I20260709 12:25:09.124245 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.124452 20884 rpcz_store.cc:276] 0709 12:25:06.783600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.859916 (+ 76316us) service_pool.cc:224] Handling call
0709 12:25:06.860426 (+   510us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:06.873719 (+ 13293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.873773 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:06.874660 (+   887us) log.cc:844] Serialized 52 byte log entry
0709 12:25:08.732852 (+1858192us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b18003079c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:08.972707 (+239855us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:08.972751 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.972818 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.078761 (+105943us) raft_consensus.cc:1685] finished
0709 12:25:09.078782 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.079141 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12519,"raft.queue_time_us":16916,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6333,"spinlock_wait_cycles":41395968,"thread_start_us":239653,"threads_started":1,"wal-append.queue_time_us":2180042}
I20260709 12:25:09.126060 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.127202 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:09.131422 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.133483 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.134984 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.135665 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:09.133859 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.136548 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.138006 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.138998 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3366) took 1648 ms. Trace:
I20260709 12:25:09.138985 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.140285 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.141508 21427 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:09.139676 20878 rpcz_store.cc:276] 0709 12:25:07.490260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.590008 (+1099748us) service_pool.cc:224] Handling call
0709 12:25:08.590463 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.590721 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:08.955958 (+365237us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.112575 (+156617us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.119738 (+  7163us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.121882 (+  2144us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.133409 (+ 11527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.133464 (+    55us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:09.138671 (+  5207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.138769 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.138927 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35824,"prepare.queue_time_us":400787,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":25465,"spinlock_wait_cycles":59032192,"thread_start_us":475541,"threads_started":2}
I20260709 12:25:09.031147 29378 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:09.053268 20882 rpcz_store.cc:276] 0709 12:25:06.807458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.034792 (+227334us) service_pool.cc:224] Handling call
0709 12:25:07.040410 (+  5618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.040656 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:07.060064 (+ 19408us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.065556 (+  5492us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.638220 (+1572664us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b180034e340. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:25:08.691953 (+ 53733us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b180034e340. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:08.925265 (+233312us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b180034e340. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2557 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:09.047052 (+121787us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.047122 (+    70us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:09.047405 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.047526 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.047715 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":57819,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":5764,"raft.queue_time_us":24520,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":15263,"spinlock_wait_cycles":53897984,"thread_start_us":408691,"threads_started":1,"wal-append.queue_time_us":2376}
W20260709 12:25:09.066699 29535 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:09.144160 20875 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:25:09.147806 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.145416 20879 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:09.146411 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.147966 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.148653 21433 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:09.150308 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.151434 21431 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:25:09.004577 29443 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.111s
I20260709 12:25:09.158473 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:09.047605 29529 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:09.079365 29543 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.149004 20726 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:09.157019 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.051307 29534 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:09.078809 29363 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.179488 20733 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:25:09.174726 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.056628 29372 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.160099 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.139914 20883 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:09.183116 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.161188 20735 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:09.077850 29361 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.190308 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67159.7us
I20260709 12:25:09.191473 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.183874 20728 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:09.063316 29418 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:09.065151 28970 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.184442 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.204931 20729 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:09.204986 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.119555 29282 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.078660 29380 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.113440 29509 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.076713 29545 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:09.207232 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.128788 29193 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:09.158844 29290 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.200254 20802 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:09.088212 29375 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.079728 29421 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.102s
I20260709 12:25:09.192371 21427 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:09.228322 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.243180 20735 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:09.056869 29502 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.205940 20732 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:25:09.060110 29387 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.182833 29534 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:09.256067 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 308284us
I20260709 12:25:09.258984 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:09.343683 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.204134 29418 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:09.157714 29171 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.085s
I20260709 12:25:09.157827 29404 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.074s
W20260709 12:25:09.528723 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 269797us
I20260709 12:25:09.529083 20876 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:09.529893 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:09.511494 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:09.524185 20802 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:25:09.463887 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 224696us
I20260709 12:25:09.080763 29530 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.057816 29369 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:09.511368 29547 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.565327 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53924us
I20260709 12:25:09.177971 29378 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.566934 20735 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: [23.11-25.12]
I20260709 12:25:09.564455 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.262125 29420 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:09.576049 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.514730 29294 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:09.530253 29526 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:25:09.188577 29543 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.188752 29363 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:09.606698 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:09.255447 29193 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.513504 29534 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:09.188911 29529 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:09.614876 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.601893 20802 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:09.257401 29282 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.202401 29361 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:09.581529 29518 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.668s	user 0.004s	sys 0.000s
I20260709 12:25:09.634575 20732 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: [28.13-30.14]
I20260709 12:25:09.461539 29545 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:09.232203 29509 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:09.587985 29266 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.772s	user 0.002s	sys 0.010s
I20260709 12:25:09.572751 29460 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.561177 29541 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:09.257230 29380 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:09.617357 29525 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:25:09.636370 20810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.042s	sys 0.053s
I20260709 12:25:09.636821 20726 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:09.577862 29119 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 12:25:09.629750 20622 maintenance_manager.cc:419] P 9f77a9352772433bb5809bf72fd4f0ee: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033371
I20260709 12:25:09.595093 29559 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:09.617717 29490 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:09.695452 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.656852 29527 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.461704 29290 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:09.643216 29506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.027s	sys 0.089s
W20260709 12:25:09.637596 29538 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.630s	user 0.000s	sys 0.001s
I20260709 12:25:09.672449 20735 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:09.669746 20810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.043s	sys 0.053s
I20260709 12:25:09.582336 29547 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:09.691418 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 483333us
I20260709 12:25:09.715103 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:09.698851 20727 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:09.681455 29507 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.890s	user 0.008s	sys 0.004s
I20260709 12:25:09.706957 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:09.668184 29404 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.671998 29334 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:09.695323 29559 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:09.706723 29506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.027s	sys 0.089s
I20260709 12:25:09.676323 29542 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:09.720187 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:09.715868 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3415) took 2905 ms. Trace:
W20260709 12:25:09.726657 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 534943us
I20260709 12:25:09.713045 29282 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.699342 29403 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:09.726341 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 196574us
I20260709 12:25:09.728142 20801 rpcz_store.cc:276] 0709 12:25:06.810654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.672645 (+861991us) service_pool.cc:224] Handling call
0709 12:25:07.673143 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.673454 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.673496 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.704072 (+ 30576us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.792791 (+ 88719us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:07.792833 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.792905 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.115449 (+1322544us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:09.715562 (+600113us) raft_consensus.cc:1685] finished
0709 12:25:09.715585 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.715806 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7046912,"thread_start_us":68751,"threads_started":1,"wal-append.queue_time_us":124148}
I20260709 12:25:09.736673 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:09.693449 29513 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.668573 29558 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:25:09.672713 29460 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.746800 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.713093 29547 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:09.732622 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:09.748584 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.749830 20731 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:25:09.748376 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:09.750645 20734 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:25:09.754783 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.781530 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:09.785351 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52820.2us
I20260709 12:25:09.728010 29542 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:09.748366 29344 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.025s	user 0.008s	sys 0.004s
I20260709 12:25:09.724377 29559 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:09.801841 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.795053 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:09.714237 29527 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:09.822474 20803 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:09.738052 29443 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:09.723871 29404 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.828352 21432 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:25:09.755051 29403 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:09.837100 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.721992 29507 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:09.830600 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 237021us
I20260709 12:25:09.839653 20731 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:25:09.839321 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 232728us
I20260709 12:25:09.761209 29502 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.856524 20734 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:09.800182 29375 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.870939 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:09.821146 29387 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.859833 29344 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:25:09.905043 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.907305 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.908756 20731 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:09.868667 29355 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:09.923866 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 176414us
I20260709 12:25:09.870780 29541 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:09.880040 29490 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:09.867275 29421 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.874053 29403 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:09.927403 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:09.868870 29458 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.085s
I20260709 12:25:09.868760 29568 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:09.875068 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:09.924243 29193 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:09.873656 29562 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.137s
I20260709 12:25:10.031438 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:10.044809 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 249801us
I20260709 12:25:09.900116 29376 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:09.745280 29545 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:09.999815 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:09.914736 29344 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.848381 29443 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:09.878271 29558 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
W20260709 12:25:09.952977 29553 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:25:09.876469 29378 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:09.929301 29363 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:09.999310 29555 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 12:25:10.002074 29282 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:10.030479 29509 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:09.925595 29546 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.112s
I20260709 12:25:09.911547 29334 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:10.005117 29490 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:10.032876 29290 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.037153 29380 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.050863 29344 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:10.070901 29570 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.011469 29421 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:10.081921 29564 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:25:10.050351 29376 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:10.104487 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.065244 29568 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:25:10.089303 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85624us
I20260709 12:25:10.129797 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3429) took 1587 ms. Trace:
I20260709 12:25:10.130427 20807 rpcz_store.cc:276] 0709 12:25:08.542537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.131564 (+589027us) service_pool.cc:224] Handling call
0709 12:25:09.131879 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.132141 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:09.631476 (+499335us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.128869 (+497393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:10.129233 (+   364us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:10.129460 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.129556 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.129715 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":604042,"prepare.run_cpu_time_us":2988,"prepare.run_wall_time_us":31647,"raft.queue_time_us":17848,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":14149,"thread_start_us":946554,"threads_started":3}
W20260709 12:25:10.079208 29560 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
I20260709 12:25:10.069540 29171 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:25:10.141723 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:10.078585 29579 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:10.169479 29355 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:10.170125 29355 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:10.170321 20807 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:10.077024 29282 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.176716 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1553) took 1139 ms. Trace:
I20260709 12:25:10.185961 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.517680 20807 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
W20260709 12:25:10.170207 29332 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.224s	user 0.009s	sys 0.003s
I20260709 12:25:10.140367 29568 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.521104 29568 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:10.520823 29332 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.524113 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.136475 29460 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.109192 29546 consensus_queue.cc:1243] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 12:25:10.175859 29409 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.254s	user 0.011s	sys 0.001s
W20260709 12:25:10.177631 29566 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 12:25:10.526377 29547 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.526623 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.176805 29584 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.526885 29547 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.528087 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2055) took 1387 ms. Trace:
I20260709 12:25:10.172363 29443 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.157177 29579 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:10.529189 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.528543 21435 rpcz_store.cc:276] 0709 12:25:09.141081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.142279 (+  1198us) service_pool.cc:224] Handling call
0709 12:25:09.142892 (+   613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.143334 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.143358 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.177147 (+ 33789us) log.cc:844] Serialized 564 byte log entry
0709 12:25:09.643640 (+466493us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:09.643696 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.643802 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.527842 (+884040us) raft_consensus.cc:1685] finished
0709 12:25:10.527865 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.528023 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10624,"thread_start_us":466104,"threads_started":1,"wal-append.queue_time_us":693124}
I20260709 12:25:10.530722 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.171047 29527 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:10.180218 29404 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:10.529143 29563 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:10.531854 29563 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:10.216229 29555 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:10.532318 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.194509 29496 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:10.110407 29587 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.104s
I20260709 12:25:10.085655 29570 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.532825 20807 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:10.533154 29496 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.532661 29563 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.534811 29570 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.178169 29334 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:10.273411 29418 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.003s
I20260709 12:25:10.536207 29418 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:10.257678 29530 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:10.209940 29490 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.539081 29530 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.541522 29530 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.310681 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3418) took 2422 ms. Trace:
W20260709 12:25:10.326846 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 76181.2us
I20260709 12:25:10.542325 20806 rpcz_store.cc:276] 0709 12:25:07.888213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.961297 (+1073084us) service_pool.cc:224] Handling call
0709 12:25:08.961657 (+   360us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:09.185013 (+223356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.185044 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:09.193866 (+  8822us) log.cc:844] Serialized 78 byte log entry
0709 12:25:09.845440 (+651574us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:09.885956 (+ 40516us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.886564 (+   608us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.889315 (+  2751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.889444 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.310346 (+420902us) raft_consensus.cc:1685] finished
0709 12:25:10.310378 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.310596 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22496,"prepare.queue_time_us":104703,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":8145,"raft.queue_time_us":112553,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":7722,"spinlock_wait_cycles":30508800,"thread_start_us":511629,"threads_started":2,"wal-append.queue_time_us":738503}
I20260709 12:25:10.544436 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:10.300552 29169 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.382s	user 0.008s	sys 0.011s
W20260709 12:25:10.334472 29166 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.457s	user 0.014s	sys 0.000s
I20260709 12:25:10.545188 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.341584 29355 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.545596 29169 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.546396 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:10.547295 29280 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.547015 20806 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:10.547930 29355 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.547791 29280 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.342957 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1615) took 2490 ms. Trace:
I20260709 12:25:10.534830 29563 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:10.347386 29534 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.687s	user 0.015s	sys 0.002s
I20260709 12:25:10.549137 29570 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.549429 29534 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.549650 29570 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.321338 29559 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.551651 20806 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:10.548641 20810 rpcz_store.cc:276] 0709 12:25:07.852896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.903321 (+1050425us) service_pool.cc:224] Handling call
0709 12:25:08.929681 (+ 26360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.708950 (+779269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.708993 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.716879 (+  7886us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.861438 (+144559us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:09.861499 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.861609 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.342726 (+481117us) raft_consensus.cc:1685] finished
0709 12:25:10.342747 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.342886 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97068,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":865294,"threads_started":2,"wal-append.queue_time_us":543357}
I20260709 12:25:10.552234 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1624) took 1538 ms. Trace:
I20260709 12:25:10.552680 29559 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.352383 29421 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.412847 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:10.403746 29455 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:10.553666 29455 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:10.553858 20726 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:10.554581 29534 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.554833 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:10.554528 29455 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.556289 29455 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:10.556687 20726 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:25:10.420336 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.425521 20775 connection.cc:256] RPC call timeout handler was delayed by -1.5686597429998983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
I20260709 12:25:10.557710 29609 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:25:10.558637 20775 connection.cc:256] RPC call timeout handler was delayed by -1.4615597430001799s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
W20260709 12:25:10.559275 20775 connection.cc:256] RPC call timeout handler was delayed by -1.3404571149999356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
W20260709 12:25:10.560503 20775 connection.cc:256] RPC call timeout handler was delayed by -1.1807597430001806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2388s for a response
I20260709 12:25:10.451478 29578 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:10.561177 29554 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.988s	user 0.011s	sys 0.012s
I20260709 12:25:10.561305 29578 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.563591 29578 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:10.197190 20850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.009s	sys 0.031s
I20260709 12:25:10.509073 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:10.357446 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3368) took 2831 ms. Trace:
I20260709 12:25:10.973212 20880 rpcz_store.cc:276] 0709 12:25:07.526149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.598066 (+1071917us) service_pool.cc:224] Handling call
0709 12:25:08.598562 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.950158 (+351596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.950255 (+    97us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.320572 (+1370317us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:10.321198 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.347683 (+ 26485us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:10.347719 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.347808 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.357082 (+  9274us) raft_consensus.cc:1685] finished
0709 12:25:10.357112 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.357374 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89399,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":5375,"spinlock_wait_cycles":29209856,"thread_start_us":376921,"threads_started":2,"wal-append.queue_time_us":34031}
I20260709 12:25:10.347697 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3378) took 1381 ms. Trace:
I20260709 12:25:10.981237 20884 rpcz_store.cc:276] 0709 12:25:08.966565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.126018 (+159453us) service_pool.cc:224] Handling call
0709 12:25:09.126273 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.527478 (+401205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.527513 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.527556 (+    43us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:09.527594 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.347464 (+819870us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54001fde54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.347516 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.347647 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922173312}
I20260709 12:25:10.181715 20878 rpcz_store.cc:276] 0709 12:25:09.037287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.148549 (+111262us) service_pool.cc:224] Handling call
0709 12:25:09.155392 (+  6843us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.156047 (+   655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.156090 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.156719 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.624306 (+467587us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:09.624369 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.624622 (+   253us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.176463 (+551841us) raft_consensus.cc:1685] finished
0709 12:25:10.176489 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.176649 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1023979,"raft.queue_time_us":120,"thread_start_us":467093,"threads_started":1,"wal-append.queue_time_us":564627}
I20260709 12:25:10.078378 29590 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.127s
I20260709 12:25:10.527773 29584 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.985536 29384 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:10.987608 29584 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.552801 20810 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:10.987782 29547 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:10.988408 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3437) took 1243 ms. Trace:
I20260709 12:25:10.988755 20810 rpcz_store.cc:276] 0709 12:25:09.745139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.552334 (+807195us) service_pool.cc:224] Handling call
0709 12:25:10.552625 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.988154 (+435529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.988354 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.552580 20806 rpcz_store.cc:276] 0709 12:25:09.013876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.551018 (+1537142us) service_pool.cc:224] Handling call
0709 12:25:10.551352 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.552002 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.552177 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.314371 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:11.002177 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 687846us
I20260709 12:25:10.564960 29610 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.567544 20807 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:11.004395 29610 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:10.566298 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:11.007841 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 441597us
I20260709 12:25:11.008635 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:11.010749 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:10.568099 20726 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:11.011698 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3431) took 1047 ms. Trace:
I20260709 12:25:11.012039 20726 rpcz_store.cc:276] 0709 12:25:10.003715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.567538 (+563823us) service_pool.cc:224] Handling call
0709 12:25:10.567803 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.011397 (+443594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.011620 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:11.012543 29294 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:11.012957 29294 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:11.013357 29294 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:11.013401 29455 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:10.583644 20802 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:10.565781 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:11.016772 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 451061us
I20260709 12:25:11.017555 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:10.633199 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1599) took 1509 ms. Trace:
W20260709 12:25:10.657344 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.635390 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1421) took 1520 ms. Trace:
I20260709 12:25:11.019340 20733 rpcz_store.cc:276] 0709 12:25:09.114807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.117869 (+  3062us) service_pool.cc:224] Handling call
0709 12:25:09.178839 (+ 60970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.180615 (+  1776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.635165 (+1454550us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b540028d050. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:10.635329 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41884544}
I20260709 12:25:10.581336 29282 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.589277 29280 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:10.524641 29332 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:10.345937 29344 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.019981 29610 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:10.669564 20774 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.015s	sys 0.009s
W20260709 12:25:11.040154 20774 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.017s	sys 0.010s
W20260709 12:25:11.040443 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 1.16493e+06us
I20260709 12:25:10.654294 29361 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.042184 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:10.553244 29404 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:10.666568 29558 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:11.046917 29558 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:10.564168 29476 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.180198 29507 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.658264 29490 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:11.023680 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:11.051085 29404 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:11.050817 29490 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.053195 29490 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.080086 29541 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 30.14, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:10.617292 29556 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.893s	user 0.002s	sys 0.004s
I20260709 12:25:10.661886 29542 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.067979 29542 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:10.564510 29496 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.538594 29418 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.577386 29568 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:11.080895 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:11.082515 29568 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:10.180408 29567 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.310s	user 0.001s	sys 0.001s
I20260709 12:25:10.485836 29529 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:10.648206 29543 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.530263 29579 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 30.14, Last appended by leader: 14, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:10.181465 29573 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.086927 29573 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.089130 29573 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:10.564494 29570 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:10.697404 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3377) took 1731 ms. Trace:
I20260709 12:25:11.090896 20876 rpcz_store.cc:276] 0709 12:25:08.966282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.124798 (+158516us) service_pool.cc:224] Handling call
0709 12:25:09.125075 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.529751 (+404676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.697010 (+1167259us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.697308 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927889280}
I20260709 12:25:10.714619 20730 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:11.094020 20730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3423) took 2133 ms (client timeout 1847 ms). Trace:
W20260709 12:25:11.094424 20730 rpcz_store.cc:269] 0709 12:25:08.960484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.960626 (+   142us) service_pool.cc:224] Handling call
0709 12:25:11.093789 (+2133163us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54003ae8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:11.093941 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3904928640}
I20260709 12:25:10.732954 20621 maintenance_manager.cc:643] P 9f77a9352772433bb5809bf72fd4f0ee: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.997s	user 0.037s	sys 0.007s Metrics: {"bytes_written":5600,"delete_count":0,"lbm_write_time_us":992,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":16264,"threads_started":1,"update_count":211,"wal-append.queue_time_us":15921}
I20260709 12:25:11.096480 20729 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
W20260709 12:25:10.736577 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.744557 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:42714 (request call id 0) took 1848 ms. Trace:
I20260709 12:25:11.098726 29579 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:11.098874 21430 rpcz_store.cc:276] 0709 12:25:08.896137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.896438 (+   301us) service_pool.cc:224] Handling call
0709 12:25:10.744224 (+1847786us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca5735 00007f9e1ec9d9fa 00007f9e23c654c9 00007f9e1d1e1c54 00007f9e1d1e1be2 00007f9e1d1e1b11 00007f9e1d1e1a8c 00007f9e1d1e0d42 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.744462 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3031626496}
W20260709 12:25:10.755918 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3298) took 3694 ms (client timeout 1858 ms). Trace:
W20260709 12:25:10.779589 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.035s	sys 0.030s
W20260709 12:25:11.100095 20883 rpcz_store.cc:269] 0709 12:25:07.062009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.874227 (+812218us) service_pool.cc:224] Handling call
0709 12:25:10.755590 (+2881363us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:10.755837 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2672553728}
W20260709 12:25:11.100726 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.035s	sys 0.030s
W20260709 12:25:11.101099 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.05413e+06us
I20260709 12:25:11.106004 29529 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.106243 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.106604 29529 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.107625 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.108183 21433 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:11.108664 29529 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:11.108970 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:11.109587 21433 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:25:10.824213 20881 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:10.833837 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:11.110425 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.111083 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 277309us
I20260709 12:25:11.111481 29361 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:11.111608 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3142) took 1903 ms. Trace:
I20260709 12:25:11.112110 29361 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.844008 20801 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: [24.17-26.18]
I20260709 12:25:11.112591 29563 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:11.112702 29361 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.114132 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1622) took 2101 ms. Trace:
I20260709 12:25:11.114581 29580 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:11.115043 29361 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.115304 29580 consensus_queue.cc:1243] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:11.115723 29361 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:11.116277 29546 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:25:10.851655 20850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.013s	sys 0.031s
I20260709 12:25:10.871299 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.122159 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.123696 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.125101 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:11.125356 21426 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:10.885026 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3137) took 1755 ms. Trace:
I20260709 12:25:10.889130 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:10.897397 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:10.898118 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:10.923273 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3422) took 3011 ms. Trace:
I20260709 12:25:10.924559 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3143) took 1463 ms. Trace:
I20260709 12:25:10.927644 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3145) took 1396 ms. Trace:
I20260709 12:25:10.941980 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1611) took 3182 ms. Trace:
I20260709 12:25:10.961283 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3304) took 3890 ms. Trace:
I20260709 12:25:11.002269 29547 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:10.578450 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3217) took 3504 ms. Trace:
I20260709 12:25:11.015775 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1621) took 2003 ms. Trace:
I20260709 12:25:10.548103 29169 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.021199 29355 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.021538 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.020134 29559 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:11.025054 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.028837 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:11.319592 20806 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:11.029506 29534 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:11.030241 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:11.021085 29584 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:11.020869 29119 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:11.018857 21432 rpcz_store.cc:276] 0709 12:25:09.123680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.123996 (+   316us) service_pool.cc:224] Handling call
0709 12:25:10.632960 (+1508964us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:10.633129 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614999296}
I20260709 12:25:11.033600 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.327699 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:11.328428 20808 raft_consensus.cc:2376] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25. Re-sending same reply.
I20260709 12:25:11.329517 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.330255 20808 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:11.035118 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.331739 20810 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:11.039531 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:11.040861 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.621878 29376 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.039549 29332 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.335770 20807 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:25:11.336087 29332 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.318284 29355 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.338235 29332 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:11.043464 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.044334 29344 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.338652 29355 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.046422 29280 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.339001 20726 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:11.339388 29280 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.340106 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.341543 29280 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.341446 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:10.604169 29421 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:11.343954 29421 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.346083 29421 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.346728 20734 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
W20260709 12:25:11.347792 20734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3428) took 1437 ms (client timeout 1582 ms). Trace:
W20260709 12:25:11.348156 20734 rpcz_store.cc:269] 0709 12:25:09.910786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.948635 (+ 37849us) service_pool.cc:224] Handling call
0709 12:25:11.347644 (+1399009us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54001d7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:11.347741 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2990854912}
I20260709 12:25:11.349268 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:11.350073 20734 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:11.047940 29558 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.352555 29558 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:11.354020 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.355270 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:11.356292 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.356817 20810 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:25:11.357568 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.358242 20810 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:11.055994 29476 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:11.057687 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.058871 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.059636 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1617) took 3182 ms. Trace:
W20260709 12:25:11.045869 20775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.542s (SENT) [suppressed 16 similar messages]
I20260709 12:25:11.060417 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.061887 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.072744 29294 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:10.791985 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.005s	sys 0.002s
I20260709 12:25:10.822075 20882 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:11.113971 29563 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:10.875965 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:10.881019 29292 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:11.114496 20801 rpcz_store.cc:276] 0709 12:25:09.012860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.736840 (+723980us) service_pool.cc:224] Handling call
0709 12:25:09.737294 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:11.112809 (+1375515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.113885 (+  1076us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540027b854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:11.114075 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544342400}
I20260709 12:25:11.125991 29541 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:11.128012 29572 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.128402 29548 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:11.128487 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3375) took 2642 ms. Trace:
I20260709 12:25:11.130358 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.131067 21433 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
W20260709 12:25:11.131394 20850 connection.cc:256] RPC call timeout handler was delayed by -1.21473650300004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 12:25:11.112066 20729 rpcz_store.cc:276] 0709 12:25:09.208026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.249829 (+ 41803us) service_pool.cc:224] Handling call
0709 12:25:11.096285 (+1846456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.096921 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.111423 (+ 14502us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b54003ae8d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:11.111552 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4245637504}
I20260709 12:25:11.230957 20802 rpcz_store.cc:276] 0709 12:25:09.012289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.656199 (+643910us) service_pool.cc:224] Handling call
0709 12:25:10.583213 (+927014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.014213 (+431000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.015569 (+  1356us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54002a99d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:11.015722 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087089920}
I20260709 12:25:11.231791 20727 rpcz_store.cc:276] 0709 12:25:09.129467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.143329 (+ 13862us) service_pool.cc:224] Handling call
0709 12:25:09.143583 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.715915 (+572332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.884720 (+1168805us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54002990d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.884953 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276551424}
I20260709 12:25:11.233198 20803 rpcz_store.cc:276] 0709 12:25:07.911462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.135845 (+1224383us) service_pool.cc:224] Handling call
0709 12:25:09.137217 (+  1372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.836686 (+699469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.922916 (+1086230us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.923159 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1575301376}
I20260709 12:25:11.233860 20809 rpcz_store.cc:276] 0709 12:25:07.073687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.839872 (+1766185us) service_pool.cc:224] Handling call
0709 12:25:08.840203 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.937334 (+ 97131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.578047 (+1640713us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b54002a99d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.578297 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211607424}
I20260709 12:25:11.234527 20877 rpcz_store.cc:276] 0709 12:25:07.071129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.530611 (+1459482us) service_pool.cc:224] Handling call
0709 12:25:08.830651 (+300040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.848456 (+ 17805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.960952 (+2112496us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b5400200b50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:10.961146 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689291008}
I20260709 12:25:11.235381 21434 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:11.235719 20732 rpcz_store.cc:276] 0709 12:25:09.461637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.473213 (+ 11576us) service_pool.cc:224] Handling call
0709 12:25:09.534896 (+ 61683us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:09.635229 (+100333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.924232 (+1289003us) spinlock_profiling.cc:243] Waited 99.3 ms on lock 0x7b54000e6ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.924490 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228419456}
I20260709 12:25:11.236392 20804 rpcz_store.cc:276] 0709 12:25:07.759623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.860137 (+1100514us) service_pool.cc:224] Handling call
0709 12:25:08.860526 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.912711 (+ 52185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.912757 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.912818 (+    61us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:08.912873 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.941667 (+2028794us) spinlock_profiling.cc:243] Waited 51.9 ms on lock 0x7b54003a0554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:10.941731 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.941903 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119371648}
I20260709 12:25:11.232667 20728 rpcz_store.cc:276] 0709 12:25:09.531169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.531482 (+   313us) service_pool.cc:224] Handling call
0709 12:25:10.927407 (+1395925us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400298bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:10.927576 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3095381760}
W20260709 12:25:11.244714 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 101447us
I20260709 12:25:11.135651 29562 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.137034 29573 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.129513 29496 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.322781 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.366961 20879 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:25:11.361611 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 303973us
I20260709 12:25:11.395659 29562 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.334590 29559 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.048772 29282 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:11.396809 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.397447 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:11.398475 29332 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:11.399740 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.399850 29559 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.396389 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:11.396615 20850 connection.cc:256] RPC call timeout handler was delayed by -1.1382365029999164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
I20260709 12:25:11.402295 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.402638 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.403081 29529 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:11.403400 20732 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:11.403577 29529 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.404352 29562 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.404814 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.404959 29573 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.405442 20729 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:11.405551 29529 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:11.406330 29529 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:11.406566 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.407605 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:11.408274 29292 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.408942 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:11.409247 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 357652us
I20260709 12:25:11.410004 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:11.410113 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:11.410315 29292 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.410802 29584 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:11.411360 29282 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:11.411612 29584 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:11.412096 29282 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:11.412808 29282 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:11.413115 29344 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:11.413511 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.413199 29496 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.413885 29344 consensus_queue.cc:1243] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:11.414757 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:11.415808 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.413233 29282 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:11.359947 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.416917 20806 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:11.444052 20807 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:11.445091 20806 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:11.446460 20806 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:11.417207 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.418546 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.418628 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:11.361678 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.419816 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.420228 20881 rpcz_store.cc:276] 0709 12:25:08.486353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.812624 (+326271us) service_pool.cc:224] Handling call
0709 12:25:08.812903 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.110628 (+2297725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.128229 (+ 17601us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b5400268254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:11.128425 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4624984320}
I20260709 12:25:11.365109 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:11.421068 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.365983 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:11.364640 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:11.365530 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:11.398524 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.358932 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.350867 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.424125 29294 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.424041 29572 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.424620 29355 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.425091 29541 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.421346 29548 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.426231 29280 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:11.427866 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.445360 29282 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:11.421701 20805 rpcz_store.cc:276] 0709 12:25:07.876985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.937297 (+1060312us) service_pool.cc:224] Handling call
0709 12:25:08.950081 (+ 12784us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.950835 (+   754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.950904 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:08.963715 (+ 12811us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.233479 (+269764us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:09.233537 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.233629 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.616700 (+1383071us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:11.059283 (+442583us) raft_consensus.cc:1685] finished
0709 12:25:11.059315 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.059574 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":43576,"spinlock_wait_cycles":4272512,"thread_start_us":269447,"threads_started":1,"wal-append.queue_time_us":732016}
I20260709 12:25:11.431025 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.401046 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:11.424532 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:11.468263 20877 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:11.438730 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3318) took 1564 ms. Trace:
I20260709 12:25:11.360020 29476 consensus_queue.cc:1243] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:11.447501 20808 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:11.450871 20883 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 12:25:11.456357 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 196411us
I20260709 12:25:11.456560 20884 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:11.451454 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.473246 20881 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:11.452250 20875 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:11.452261 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.453576 20810 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:11.417860 29573 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.452617 20878 raft_consensus.cc:2376] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24. Re-sending same reply.
I20260709 12:25:11.454950 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.476815 20807 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:25:11.455143 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.427259 29547 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:11.463342 20880 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:25:11.465194 29548 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:11.443042 29562 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.483388 29562 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.485756 29562 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.486370 20803 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:11.461810 29119 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:25:11.489434 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.466117 20809 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:25:11.407712 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.446897 29584 raft_consensus.cc:2764] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:25:11.492420 20804 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:11.469305 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.493641 20877 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:11.468937 20882 rpcz_store.cc:276] 0709 12:25:09.874312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.034909 (+160597us) service_pool.cc:224] Handling call
0709 12:25:10.035286 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.423306 (+1388020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.438370 (+ 15064us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:11.438644 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1808492928}
I20260709 12:25:11.461136 29559 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.498041 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73593us
W20260709 12:25:11.475677 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.478724 21429 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:25:11.479087 20880 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 14
I20260709 12:25:11.496351 29476 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:11.499542 20880 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:25:11.499629 29476 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.499949 20880 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:11.500181 29476 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:11.439677 29292 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:11.501524 29292 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.478725 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.480252 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.504230 20884 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:11.480528 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.505321 21433 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:11.465417 29280 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.481961 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:11.505167 29282 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.506299 29280 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:11.479594 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.507038 20883 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:25:11.507889 29282 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.508671 29282 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:11.508806 29541 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.509541 29496 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.509614 29282 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.467932 20879 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:11.510987 29572 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:11.512431 29282 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.513396 29292 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.494402 29573 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:11.496215 29559 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.517236 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:11.518283 29559 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.518469 29632 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:11.518982 29496 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:11.483402 29547 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.519598 29541 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:11.520226 29541 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:11.520702 29541 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:11.525394 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:11.525784 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.526578 21432 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:11.527529 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.529353 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.529739 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.526782 21433 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:11.530308 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.530524 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.531632 21428 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:11.531924 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.532609 20875 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:11.465721 29355 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.538172 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.538542 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:11.538349 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.546118 29332 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:11.534502 29119 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:11.541460 29570 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:11.550908 29570 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.546943 29548 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.556710 29570 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:11.552071 29119 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.557668 29404 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 12:25:11.569720 29476 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.571856 29548 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.098726 20622 maintenance_manager.cc:419] P 9f77a9352772433bb5809bf72fd4f0ee: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.289892
I20260709 12:25:11.315420 20628 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 22, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 22 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:25:11.066068 20624 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 202) took 39098 ms (client timeout 15000 ms). Trace:
I20260709 12:25:11.609368 20628 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 22, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 22 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:11.616370 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.568s	user 0.007s	sys 0.003s
I20260709 12:25:11.631796 20801 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
W20260709 12:25:11.607787 20624 rpcz_store.cc:269] 0709 12:24:31.967659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.968175 (+   516us) service_pool.cc:224] Handling call
0709 12:25:11.065968 (+39097793us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:04.179853 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:04.385931 (+206078us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:04.385982 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:25:04.385994 (+    12us) tablet.cc:662] Decoding operations
0709 12:25:04.387234 (+  1240us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:04.387307 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:04.387332 (+    25us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:25:04.388500 (+  1168us) tablet.cc:701] Row locks acquired
0709 12:25:04.388548 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:25:04.388712 (+   164us) write_op.cc:270] Start()
0709 12:25:04.388859 (+   147us) write_op.cc:276] Timestamp: P: 1783599904388647 usec, L: 0
0709 12:25:04.388877 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:25:04.389577 (+   700us) log.cc:844] Serialized 5259 byte log entry
0709 12:25:05.335397 (+945820us) op_driver.cc:464] REPLICATION: finished
0709 12:25:05.683690 (+348293us) write_op.cc:301] APPLY: starting
0709 12:25:05.691646 (+  7956us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:05.851588 (+159942us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:05.851716 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:06.051830 (+200114us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:06.052165 (+   335us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:25:06.052218 (+    53us) write_op.cc:312] APPLY: finished
0709 12:25:06.052604 (+   386us) log.cc:844] Serialized 121 byte log entry
0709 12:25:06.052801 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.052968 (+   167us) write_op.cc:454] Released schema lock
0709 12:25:06.053038 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:06.053509 (+   471us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.053543 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9436451,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":243398,"apply.run_cpu_time_us":22060,"apply.run_wall_time_us":434246,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58467,"num_ops":13,"prepare.queue_time_us":131379,"prepare.run_cpu_time_us":18149,"prepare.run_wall_time_us":517699,"raft.queue_time_us":186161,"raft.run_cpu_time_us":23353,"raft.run_wall_time_us":243549,"replication_time_us":931769,"thread_start_us":493595,"threads_started":4,"wal-append.queue_time_us":329894,"wal-append.run_cpu_time_us":19708,"wal-append.run_wall_time_us":2333681}]]}
I20260709 12:25:11.442258 20729 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:11.636222 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:11.637807 20809 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:25:11.404364 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.433775 20733 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:11.634446 20628 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:11.434325 20735 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:11.447911 20730 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:25:11.618284 20700 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.807s (SENT)
I20260709 12:25:11.620600 20726 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:11.649595 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.453541 20727 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
W20260709 12:25:11.455859 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 410000us
W20260709 12:25:11.453920 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.656160 20728 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:25:11.644047 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 228133us
I20260709 12:25:11.478796 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.663103 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.663506 20802 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:11.664356 29355 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.665118 29547 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:11.666203 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:11.667339 29476 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:11.667512 29547 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:11.668969 29529 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:11.671933 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:11.673557 29638 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:11.674371 29580 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.625829 20805 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:25:11.627019 20806 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:11.634214 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:11.417088 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:11.333349 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:11.686939 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 269786us
I20260709 12:25:11.687072 20731 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:11.640465 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:11.688498 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48589.8us
I20260709 12:25:11.688660 29361 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:11.689512 29361 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:11.680769 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08001a7760 after lost signal to thread 19606
I20260709 12:25:11.680662 29562 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:11.683966 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:11.685348 29563 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:11.698222 29361 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.700805 29496 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:11.701305 29496 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:11.707613 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.708796 21434 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:11.709851 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:11.710808 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50027.6us
I20260709 12:25:11.711062 21435 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:25:11.702684 29496 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.712393 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:11.713692 29458 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:11.719480 29332 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:11.721552 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.722506 20727 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:11.723755 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:11.724973 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.725706 29559 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:11.726163 29559 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.726570 29559 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:11.731827 29119 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:11.732936 29355 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:11.675784 29580 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.737133 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:11.738296 29580 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:11.744261 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 110398us
I20260709 12:25:11.745615 29548 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:11.746316 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:11.746748 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:11.747414 29638 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:11.748065 29638 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:11.747895 21434 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 31 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 17.
I20260709 12:25:11.749240 29638 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.755441 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.760421 20805 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 20 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:25:11.760955 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08002d5020 after lost signal to thread 20667
I20260709 12:25:11.760308 20628 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 22 to 24, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:11.766127 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.767133 20735 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
W20260709 12:25:11.766446 20831 debug-util.cc:398] Leaking SignalData structure 0x7b080021de20 after lost signal to thread 20759
I20260709 12:25:11.768332 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.756563 29496 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:11.765103 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:11.781791 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4ac0 after lost signal to thread 20834
W20260709 12:25:11.782783 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4aa0 after lost signal to thread 20835
I20260709 12:25:11.781616 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.784399 20809 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
W20260709 12:25:11.785655 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.775097 29580 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.789506 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.781108 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.794780 29546 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:11.795375 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.801890 20805 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
W20260709 12:25:11.802935 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 119281us
I20260709 12:25:11.803345 29562 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:25:11.804548 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48735.6us
I20260709 12:25:11.800115 20803 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:11.807780 29458 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:11.808276 20731 pending_rounds.cc:85] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 12
I20260709 12:25:11.808708 20731 pending_rounds.cc:107] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:25:11.808992 20731 raft_consensus.cc:2889] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:11.810480 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98320.4us
I20260709 12:25:11.808234 29458 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.811520 29458 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:11.811861 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:11.813668 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67575.7us
I20260709 12:25:11.815440 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.818480 20801 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
W20260709 12:25:11.823488 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.823745 29507 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:11.824319 29507 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:11.825543 29507 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:11.828017 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 156457us
I20260709 12:25:11.828747 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.829030 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:11.829043 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.829830 20876 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:11.830372 20875 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:11.831372 29332 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:11.831830 29332 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.831806 29580 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.832345 29332 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:11.833042 20727 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:11.834426 29580 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.836836 29280 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.839705 29280 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.842989 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.845623 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.847486 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:11.802663 29541 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.853657 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.853688 20728 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:25:11.858811 20733 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:11.860780 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.774390 20628 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 21, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 21 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:11.831015 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.875950 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.877147 20803 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:11.878512 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:11.879446 20808 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:11.883718 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.884889 20805 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:11.901521 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.905715 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:11.906651 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83456.9us
W20260709 12:25:11.901489 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57071.8us
I20260709 12:25:11.911804 20776 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:25:11.912200 20776 leader_election.cc:403] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:11.917198 20808 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:11.918743 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:11.915684 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.920606 20733 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:11.924674 20728 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:11.924767 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 312763us
I20260709 12:25:11.932297 29641 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
W20260709 12:25:11.924104 29642 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:25:11.944900 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 137376us
I20260709 12:25:11.930626 20726 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:25:11.945344 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:11.953830 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08000a87a0 after lost signal to thread 20908
I20260709 12:25:11.953846 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:11.923339 29638 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:11.927489 29562 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:25:11.960417 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 148696us
I20260709 12:25:11.950178 29355 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:25:11.954746 20733 raft_consensus.cc:2376] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27. Re-sending same reply.
I20260709 12:25:11.958351 20801 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
W20260709 12:25:11.965034 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134135us
I20260709 12:25:11.967352 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.968384 20875 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:25:11.967389 29292 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.969305 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:11.969436 29292 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:11.969900 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:11.970763 20879 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:11.971262 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.973997 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:11.979290 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:11.981817 29559 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.950270 20628 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 16 to 21, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 21 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:11.988988 20628 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 24 to 28, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 28 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:25:11.991389 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62153.2us
W20260709 12:25:11.992987 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57663.3us
I20260709 12:25:11.993182 29380 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:11.994333 29380 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:11.996186 29541 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:11.942837 29563 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:25:11.996855 29559 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:11.997467 29563 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:11.997875 29563 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:11.989015 29332 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:11.994513 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:11.990769 20628 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 24 to 26, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 26 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:11.958403 29562 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:11.930179 29570 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.027585 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:12.020049 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.029310 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:12.031968 20883 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
W20260709 12:25:12.030031 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70290.5us
W20260709 12:25:12.037763 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08003e4920 after lost signal to thread 21459
W20260709 12:25:12.038341 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08003de8e0 after lost signal to thread 21460
W20260709 12:25:12.040361 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800138c60 after lost signal to thread 29525
I20260709 12:25:12.043882 29119 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:12.044430 29119 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.044827 29119 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:12.045137 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:12.051295 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47502.4us
I20260709 12:25:12.056010 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:12.062834 29541 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:12.063612 29541 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.064093 29541 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:12.067826 29292 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:12.068679 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.016970 29562 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:12.015854 29332 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:12.080890 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:11.998844 29563 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.092288 20728 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:12.097199 21435 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:12.098254 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79454.5us
I20260709 12:25:12.017838 29458 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.100420 20628 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 15 to 20, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 20 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:12.115867 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.119213 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:12.110423 29292 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:12.091486 29361 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:12.132499 20881 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:12.132993 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:12.134387 20727 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:12.132592 20882 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: [21.14-23.15]
I20260709 12:25:12.147061 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:12.148056 20877 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:12.147363 29280 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:12.153343 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.144774 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:12.138768 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:12.156180 29280 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:12.165088 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48407.2us
I20260709 12:25:12.166950 29280 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:12.171360 29292 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.172021 29292 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.174258 29292 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:12.174834 29292 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:12.175307 29558 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:25:12.137930 20628 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:12.186837 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.187417 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.187790 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.189667 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:12.191769 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.192485 20880 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:12.193400 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.193610 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.194335 20879 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:12.196792 29541 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:12.198431 29645 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:12.199788 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:12.200796 20804 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:12.175144 29292 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.201359 29292 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:12.202183 20802 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:12.203711 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:12.205842 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.206421 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.207312 20878 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:12.207406 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.207940 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:12.208236 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.208580 20881 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:12.208911 29418 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:12.209331 29418 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.209475 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.209712 29418 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:12.210947 20878 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:12.213372 29361 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:12.211963 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:12.156651 29563 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:12.230734 20727 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:12.214165 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.229763 20883 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:12.199175 29541 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:12.246835 20877 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:12.254292 20876 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:25:12.236191 29570 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:12.256359 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.268985 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.269841 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
W20260709 12:25:12.272269 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.271533 20628 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 16 to 23. New cstate: current_term: 23 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:12.277565 20809 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: [21.14-23.15]
I20260709 12:25:12.292120 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.293510 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.291479 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:12.275709 29332 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:12.299130 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.300769 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.298249 20879 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:12.304208 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.305222 20877 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:25:12.300869 20810 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 15
I20260709 12:25:12.306960 20810 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:25:12.307260 20810 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:12.308569 20804 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:12.309217 20808 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:12.243153 29507 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:25:12.285485 29458 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:12.320744 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.298122 20628 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 17, term changed from 18 to 22, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193), VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 22 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:12.327591 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:12.331256 29570 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:25:12.332011 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 200214us
I20260709 12:25:12.338506 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.339570 20808 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:12.341010 20628 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:12.343500 20628 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:12.347762 21434 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:12.307878 29496 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.005s
I20260709 12:25:12.351465 20879 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:12.349433 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 137581us
W20260709 12:25:12.348711 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 142537us
I20260709 12:25:12.338495 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.362109 20884 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:12.365168 20881 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:12.364006 20882 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:12.372083 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.372116 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.345371 29570 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.373312 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:12.374253 20727 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:25:12.377010 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.391624 20876 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:12.386055 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:12.397140 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:12.386953 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:12.394968 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 271406us
I20260709 12:25:12.398659 20731 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:12.380270 20628 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 12, term changed from 15 to 19, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to <none>, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 19 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:25:12.377842 29570 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:12.396731 20808 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:12.402751 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:12.417594 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:12.422767 20804 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:12.400586 29529 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.420437 29541 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.434211 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:12.440223 29663 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:12.442394 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:12.397729 29280 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.434777 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.448532 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:12.453557 21431 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:12.434600 29529 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.451010 29541 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.461504 29529 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:12.450222 29663 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.462229 29541 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:12.462455 29663 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:12.462618 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:12.402889 29418 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.466280 29404 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:12.470415 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:12.471586 29641 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:12.474674 20806 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:12.472584 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:12.481899 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:12.472759 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 23 to 27. New cstate: current_term: 27 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:12.482656 20876 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
W20260709 12:25:12.490139 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54369.7us
I20260709 12:25:12.476775 29119 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.497483 29119 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:12.477396 29404 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.521217 20628 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 22, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 22 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:25:12.522634 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 75281us
I20260709 12:25:12.443583 29665 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:25:12.520925 29404 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:12.506390 29660 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:12.530026 20628 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 20 to 23, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 23 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.527001 29418 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:12.564302 20628 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:12.538116 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:12.571703 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 206186us
W20260709 12:25:12.569061 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 96871.8us
W20260709 12:25:12.584534 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 114218us
I20260709 12:25:12.588042 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:12.535010 29292 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.613302 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:12.616695 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 78112.3us
I20260709 12:25:12.621363 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:12.593720 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60904.7us
I20260709 12:25:12.623914 29665 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:12.624562 29665 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:12.624465 29680 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:12.626109 29665 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.627271 29671 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:12.627756 29671 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.628135 29671 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:12.631646 20628 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 19, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 19 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:12.624228 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:12.637590 29660 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:12.643754 20628 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 14 to 23, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 23 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.645613 20628 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.651351 21428 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:12.652365 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:12.633322 29662 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:12.653962 29662 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:12.655035 29662 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.656721 29292 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:12.656791 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:12.657656 29292 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:12.658294 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:12.590320 29361 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:12.631690 29665 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:12.682122 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:12.668601 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:12.690452 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54083.8us
I20260709 12:25:12.690604 29496 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:12.691358 29496 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:12.691920 29496 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:12.692505 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:12.696976 21431 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
W20260709 12:25:12.672549 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 250614us
I20260709 12:25:12.665328 29355 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:12.707504 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49315.1us
I20260709 12:25:12.707988 29507 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:12.708233 29671 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.708861 29507 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:12.711014 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:12.711840 29669 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:25:12.712006 29292 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:12.712623 29679 raft_consensus.cc:2764] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:25:12.713083 29580 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:12.713726 29580 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:12.713860 29292 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:12.672290 29418 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:25:12.710189 29507 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:12.717738 20628 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:12.716265 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:12.717149 29676 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:25:12.739017 29280 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:12.845383 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 269884us
I20260709 12:25:12.728288 29361 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.715014 29559 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:12.712029 29673 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:12.734673 29638 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.776s	user 0.004s	sys 0.009s
W20260709 12:25:12.741880 29562 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.001s
I20260709 12:25:12.863229 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:12.725145 29641 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:12.864509 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 163280us
W20260709 12:25:12.716187 29674 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:25:12.865475 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:12.731359 29529 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:12.887748 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:12.887920 29529 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.888350 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:12.888600 29529 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:12.889258 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.889629 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:12.891301 20807 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:12.894142 20803 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:25:12.895576 20733 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 12. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:25:12.895603 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:12.896962 29458 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:12.897244 20730 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 12:25:12.698544 29558 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:12.897558 29458 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:12.898479 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:12.880271 29677 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:12.899181 29558 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:12.899529 29677 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:12.899751 29558 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:12.899938 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:12.900243 29558 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:12.900723 29677 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.746119 29404 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.902161 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:12.902940 20729 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:12.903530 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:12.903731 29404 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:12.903999 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:12.879719 29562 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:12.906610 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:12.715032 29292 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:12.876494 29638 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:12.879618 29685 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:12.879727 29684 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:12.873515 29559 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.897130 20735 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:12.714953 29580 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.901270 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:12.887981 20804 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 12. (term mismatch)
I20260709 12:25:12.725301 29332 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.912132 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:12.710994 29671 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:12.902675 29677 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:12.907464 29660 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:12.872839 29673 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.923122 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 235056us
I20260709 12:25:12.846622 29663 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:12.967444 20880 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
W20260709 12:25:12.963231 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 116585us
I20260709 12:25:12.943295 20805 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:25:12.920683 20628 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 21, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 21 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.880268 29546 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:12.944383 20806 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:12.942559 20732 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:12.913403 29292 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:12.920848 20804 pending_rounds.cc:85] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 11
I20260709 12:25:12.989024 20804 pending_rounds.cc:107] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 25.12
I20260709 12:25:12.989440 20804 raft_consensus.cc:2889] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:12.878047 29570 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:12.987389 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 319236us
I20260709 12:25:12.922547 29332 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:12.995417 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 382262us
W20260709 12:25:12.972617 29674 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:25:12.924592 29559 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:12.948896 29660 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:12.985457 29563 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.749s	user 0.013s	sys 0.000s
I20260709 12:25:12.967329 29673 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:12.971581 29546 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:13.048864 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 332695us
I20260709 12:25:13.058763 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:13.047462 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63464.5us
I20260709 12:25:13.022378 29660 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:13.077313 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:13.077093 20628 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:12.898852 29458 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.084880 20628 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 20, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 20 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:13.091097 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:12.991286 29684 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.080801 29119 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:13.081564 29562 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:13.098786 20628 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 17 to 21, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 21 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:13.111642 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 197838us
I20260709 12:25:13.095883 29680 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:13.123843 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:13.106120 29119 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:13.129853 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97307.8us
W20260709 12:25:13.114825 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.131135 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.118754 29680 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:13.135133 20628 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 25, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 25 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:13.141790 20628 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 28, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 28 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:13.143085 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 111296us
I20260709 12:25:13.134534 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:13.145289 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.121827 29418 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:25:13.102319 29662 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:25:13.131645 29687 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:13.144419 29680 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:13.115892 29292 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.165162 29638 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:13.165335 29701 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:25:13.165995 29699 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.076s
I20260709 12:25:13.166664 29119 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:13.166793 29686 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:25:13.167106 29684 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.155257 29458 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:13.154775 29361 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.180502 20804 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:13.181056 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:13.181774 29697 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:13.187481 29570 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.192489 20805 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:13.146741 29663 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:13.215579 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63337.7us
I20260709 12:25:13.230403 20803 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:13.230130 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:13.235406 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54470.2us
I20260709 12:25:13.237972 20735 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:13.238379 20807 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:13.240558 29119 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:13.241170 29119 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:13.233595 29562 raft_consensus.cc:2764] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:25:13.129489 29546 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.242185 29119 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.256799 29680 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.257817 29529 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.179181 29558 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.247805 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:13.250486 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:13.236105 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:13.259884 20803 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:13.261597 21429 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:13.265408 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:13.267760 20807 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:13.268460 20804 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:13.268273 21400 leader_election.cc:336] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.517s (SENT)
I20260709 12:25:13.268631 20731 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 20. Current term is 21. Ops: [19.13-20.14]
I20260709 12:25:13.269182 20726 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:13.269949 20728 pending_rounds.cc:85] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 15
I20260709 12:25:13.272776 20728 pending_rounds.cc:107] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:25:13.273057 20728 raft_consensus.cc:2889] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:13.259337 29698 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
W20260709 12:25:13.263542 29496 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.003s
I20260709 12:25:13.236569 29292 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:13.306229 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70195.6us
W20260709 12:25:13.306875 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.261108 29558 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.259153 29680 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.308604 29638 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:13.340432 29711 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:13.343711 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72274.3us
W20260709 12:25:13.360819 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 92568.1us
I20260709 12:25:13.345028 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:13.328924 29570 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:25:13.371618 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44377.4us
I20260709 12:25:13.343273 29638 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.364171 29711 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.400243 29711 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:13.363080 29558 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.404639 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3405) took 1207 ms. Trace:
I20260709 12:25:13.324359 29496 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.390509 29529 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:13.395973 29638 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:13.396090 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3262) took 1011 ms. Trace:
I20260709 12:25:13.399266 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:13.399560 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54648.7us
I20260709 12:25:13.369732 29570 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:13.375512 20806 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:13.430403 20802 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:25:13.410471 20732 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:13.405063 20884 rpcz_store.cc:276] 0709 12:25:12.197655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.377015 (+179360us) service_pool.cc:224] Handling call
0709 12:25:12.393750 (+ 16735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.530846 (+137096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.530881 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.564679 (+ 33798us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.672507 (+107828us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:12.711246 (+ 38739us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.711482 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.711569 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.404407 (+692838us) raft_consensus.cc:1685] finished
0709 12:25:13.404429 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.404575 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":298,"prepare.queue_time_us":38522,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":76405,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":223,"thread_start_us":197825,"threads_started":2,"wal-append.queue_time_us":294174}
I20260709 12:25:13.420737 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:13.418826 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 298821us
I20260709 12:25:13.436307 20807 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:13.419611 20809 rpcz_store.cc:276] 0709 12:25:12.385088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.385257 (+   169us) service_pool.cc:224] Handling call
0709 12:25:12.396828 (+ 11571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.430635 (+ 33807us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:12.586222 (+155587us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.020232 (+434010us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.071843 (+ 51611us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.071988 (+   145us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:13.072200 (+   212us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.072356 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.072428 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.395807 (+323379us) raft_consensus.cc:1685] finished
0709 12:25:13.395829 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.395994 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171395,"prepare.queue_time_us":168897,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":90,"raft.queue_time_us":88218,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":37856,"thread_start_us":357133,"threads_started":2,"wal-append.queue_time_us":457663}
I20260709 12:25:13.448066 20734 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:13.447340 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:13.452013 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 320857us
I20260709 12:25:13.457516 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:13.457758 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:13.458026 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 96387.5us
I20260709 12:25:13.431152 29570 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:13.437091 29563 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:13.446579 29355 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.464776 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44128.3us
W20260709 12:25:13.462874 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 336005us
I20260709 12:25:13.466842 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:13.468586 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:13.455876 29529 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:13.430500 29697 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:25:13.419085 29558 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:13.463079 29665 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.003s
I20260709 12:25:13.472725 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:13.475306 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 389199us
I20260709 12:25:13.427362 29559 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.485178 29707 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:13.485939 29529 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:13.499555 29548 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:13.490451 29292 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:13.511478 29548 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:13.514097 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.491677 29705 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:13.497931 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:13.438019 29660 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.465204 29680 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:13.453661 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:13.513727 29548 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.511682 29687 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:13.515787 29292 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.519532 29715 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.478080 29701 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:13.533401 29507 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:13.539774 29507 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.540789 29687 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:13.541910 29687 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.542407 29507 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.544472 29673 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.520429 29570 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.530993 29680 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.550994 29673 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:13.551415 29570 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.552225 21431 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:13.470687 29355 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.553244 29496 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:13.553529 29713 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:13.552654 29715 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:13.490824 29677 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.000s
I20260709 12:25:13.529858 20808 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:25:13.530058 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:13.501677 29707 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:13.531023 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 79340.3us
I20260709 12:25:13.535450 29292 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:13.538030 29671 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:13.575459 29671 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.576031 29671 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:13.576361 29715 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.577116 29715 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:13.578769 29709 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:13.579214 29709 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:13.538097 29548 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:13.545830 29701 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:13.546058 29668 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.966s	user 0.005s	sys 0.012s
I20260709 12:25:13.546672 29710 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:13.548022 29660 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:13.536232 29665 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:25:13.551708 29529 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:13.554931 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.522138 29558 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:13.517254 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 388958us
W20260709 12:25:13.557179 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90467us
I20260709 12:25:13.501550 29562 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.569639 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:13.531564 29705 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.570441 29707 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:13.583165 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 85292.9us
W20260709 12:25:13.597971 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 140358us
I20260709 12:25:13.604851 29709 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.606966 29559 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:13.620471 20775 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22.
I20260709 12:25:13.626080 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:13.608093 29713 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:13.604557 29680 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:13.624389 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166955us
I20260709 12:25:13.630956 29705 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:13.622634 29332 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.002s
I20260709 12:25:13.617383 29710 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:13.638896 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:13.663029 20881 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:13.663234 29710 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.654914 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:13.664470 20882 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:13.640931 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:13.647604 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:13.648418 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:13.667433 29559 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:13.668331 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:13.658775 29702 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:25:13.660445 29355 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:13.662178 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3253) took 1401 ms. Trace:
I20260709 12:25:13.656157 29701 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.670948 29559 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:13.672046 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102560us
I20260709 12:25:13.671726 29700 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:13.625226 29496 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:13.622191 29562 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.673421 29665 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.673636 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47625.6us
I20260709 12:25:13.669183 20801 rpcz_store.cc:276] 0709 12:25:12.260868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.261194 (+   326us) service_pool.cc:224] Handling call
0709 12:25:12.278097 (+ 16903us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:12.314515 (+ 36418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.314554 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:12.315210 (+   656us) log.cc:844] Serialized 52 byte log entry
0709 12:25:12.399432 (+ 84222us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:12.399489 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.399647 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.545285 (+1145638us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:13.661836 (+116551us) raft_consensus.cc:1685] finished
0709 12:25:13.661859 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.662081 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5827,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":9042688,"thread_start_us":83813,"threads_started":1,"wal-append.queue_time_us":179426}
I20260709 12:25:13.674497 29709 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:13.668462 29332 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.704878 29332 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:13.677547 29580 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:13.662355 29292 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.679937 29355 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:13.707605 29355 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:13.708673 29355 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:13.685416 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.691064 20700 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.748s (SENT)
I20260709 12:25:13.712464 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:13.713564 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43637.5us
W20260709 12:25:13.714301 20700 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.756s (SENT)
I20260709 12:25:13.714715 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:13.698266 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63358.2us
W20260709 12:25:13.722311 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.690260 29559 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:13.717149 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 261024us
W20260709 12:25:13.759374 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.761817 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66421.2us
I20260709 12:25:13.755823 29418 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:13.781102 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:13.782881 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90700.8us
I20260709 12:25:13.716629 29292 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:13.787674 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:13.683889 29705 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:25:13.769164 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:13.795876 21433 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:13.799599 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:13.800539 21432 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:25:13.801496 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:13.803730 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b31661191c14b44ac964555853900f1"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:13.804612 20878 raft_consensus.cc:606] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:25:13.804961 20878 raft_consensus.cc:614] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:13.807446 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.807663 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.808390 20731 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:25:13.808527 20734 raft_consensus.cc:2376] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22. Re-sending same reply.
I20260709 12:25:13.809202 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.809751 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.809954 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.810313 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.810705 20727 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:13.810576 20734 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:13.811498 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:13.871798 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 55125.9us
I20260709 12:25:13.872380 20735 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:13.811714 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.811633 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.873348 20732 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:13.812748 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.813654 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:13.762742 29665 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.874944 20729 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
W20260709 12:25:13.820447 29708 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.559s	user 0.005s	sys 0.007s
I20260709 12:25:13.821610 29711 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:13.822235 20880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1568) took 1629 ms (client timeout 1649 ms). Trace:
W20260709 12:25:13.822644 29690 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.580s	user 0.001s	sys 0.011s
I20260709 12:25:13.853701 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:13.867952 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 102759us
I20260709 12:25:13.836225 29681 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:13.876286 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69180.2us
I20260709 12:25:13.878299 20726 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 22 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:25:13.877785 29361 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.882673 20880 rpcz_store.cc:269] 0709 12:25:12.192469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.193143 (+   674us) service_pool.cc:224] Handling call
0709 12:25:13.821965 (+1628822us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54001fe854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:13.822091 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778593152}
W20260709 12:25:13.768582 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 129106us
I20260709 12:25:13.888967 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:13.891750 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 196153us
I20260709 12:25:13.881937 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.907994 20730 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:25:13.884337 20731 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:13.890437 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:13.909238 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.910319 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:13.911873 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:13.916438 29683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.009s	sys 0.032s
I20260709 12:25:13.918362 20801 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:13.918763 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:13.919364 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:13.919365 20802 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:13.920143 20809 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:13.922930 20808 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:13.926249 20804 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:13.928561 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:13.929364 20806 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:13.877975 29687 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:13.930438 20808 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
W20260709 12:25:13.930784 20700 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.941s (SENT)
I20260709 12:25:13.933117 20875 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: [21.14-23.15]
I20260709 12:25:13.930459 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:13.929709 29689 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.935940 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:13.910759 29710 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.937594 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:13.935904 29689 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.938014 29710 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:13.938659 29717 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.228s	user 0.001s	sys 0.004s
I20260709 12:25:13.923383 29713 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:13.939750 29713 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:13.939885 29689 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:13.940909 29713 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.936175 20731 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:25:13.943979 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.944304 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.945041 20731 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:13.945991 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.946303 20732 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:25:13.947453 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.954187 29562 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.938601 29702 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:13.955598 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:13.957406 29580 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:13.957862 29580 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:13.958261 29580 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:13.959528 29699 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:13.960155 29699 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:13.962280 29699 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.963055 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.963732 20726 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:13.908447 29496 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:13.965785 29496 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.968667 29711 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:13.918390 29728 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:13.969753 29728 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:13.970700 29728 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.937620 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:13.973312 29658 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.734s	user 0.001s	sys 0.014s
I20260709 12:25:13.877661 29563 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.096s
I20260709 12:25:13.974691 29563 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:13.975096 29563 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:13.945217 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:13.980739 20729 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:13.888037 29332 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:13.995638 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:13.996331 20801 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:13.982108 20803 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:13.982757 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.985122 20726 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
W20260709 12:25:13.992151 29638 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.000s
I20260709 12:25:13.997764 20732 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:14.006145 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.006946 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.007647 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:14.008885 20809 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 12:25:14.010715 20731 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:14.012929 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:14.019685 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64148.5us
W20260709 12:25:14.020401 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.021656 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 113645us
W20260709 12:25:13.981014 29708 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:25:13.881469 29677 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
W20260709 12:25:14.023171 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90947.2us
W20260709 12:25:13.979485 29683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.009s	sys 0.033s
I20260709 12:25:14.025103 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:14.033241 29496 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.020443 29638 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:13.973578 29580 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.036970 29638 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.038376 20880 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:14.039970 20878 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:14.040843 29580 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:14.006327 29690 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:25:14.045238 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.045418 20809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:25:14.046197 20809 raft_consensus.cc:606] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
W20260709 12:25:14.045701 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.914614 29361 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.048401 20809 raft_consensus.cc:614] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:14.054796 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.057288 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:13.976046 29563 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.054324 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:14.061513 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 140458us
W20260709 12:25:14.062587 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 125079us
I20260709 12:25:14.066335 20803 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:14.068602 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.073388 29700 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:14.042518 29332 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.103955 29529 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:14.108055 29529 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.109328 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:14.110339 29529 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.111883 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:14.113612 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:14.076020 29710 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.119652 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.120105 29332 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.120388 20883 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:14.090210 29361 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:14.121440 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71565.9us
I20260709 12:25:14.122851 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:14.077687 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.124622 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:14.024049 29671 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.096351 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.095840 21434 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:14.026051 29687 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.128933 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:14.130671 20700 leader_election.cc:336] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.980s (SENT)
I20260709 12:25:14.135545 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:14.140153 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.053714 29580 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:25:14.144162 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.145633 20732 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:14.146566 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.148160 20877 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:25:14.104851 29563 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:14.157337 20884 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:14.099829 29546 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:14.155951 20883 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:14.069409 29559 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.159439 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.068590 29496 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:25:14.065472 29638 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.081s
W20260709 12:25:14.210264 29707 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 12:25:14.214342 20810 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:14.163116 29458 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:14.211699 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.156253 29698 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:14.143056 29680 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:14.157066 29683 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:14.158492 29332 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:14.213260 20875 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:25:14.219503 20879 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:14.219328 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151757us
I20260709 12:25:14.214466 29707 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:14.220073 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151533us
I20260709 12:25:14.243786 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:14.244455 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.219257 29458 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:14.221752 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 98986.8us
I20260709 12:25:14.247411 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.222193 29711 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:14.223368 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 150841us
I20260709 12:25:14.162451 29723 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:14.234366 29671 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.158737 29529 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:14.251637 29638 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:14.251838 20879 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:25:14.252182 29638 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.252718 29638 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:14.254391 20880 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:14.255662 29707 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:14.235863 29730 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.299s	user 0.003s	sys 0.000s
I20260709 12:25:14.245076 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:14.235863 29732 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:25:14.257622 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:14.257979 29529 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:14.258464 29529 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.258643 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:14.254235 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.258833 29529 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:14.264940 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.265578 20877 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:14.266125 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.266956 20876 raft_consensus.cc:2410] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:25:14.267817 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.268400 20876 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:14.271472 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.272754 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.274058 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.274588 20879 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:14.275082 20730 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:14.276976 29332 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:14.277598 29332 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:14.241788 20882 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:25:14.247766 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.281229 20883 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:14.250730 20802 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:14.284981 20877 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:14.288718 20804 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:25:14.290474 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.290699 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.290980 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:14.291672 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.292232 20875 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:14.287906 29355 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:14.293023 29698 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:14.293111 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.294202 20876 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:14.294145 29355 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:14.295327 29701 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:14.295696 29701 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:14.296559 29701 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.299671 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.300426 20876 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:14.301367 20883 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:14.302126 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:14.302233 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.299286 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.304898 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.307456 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:14.246717 29665 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:14.298434 29701 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:14.313671 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:14.315160 20808 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:14.315084 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.309381 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.309717 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:14.317248 20807 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:14.310865 20801 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:14.299721 29735 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:14.312487 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:14.278530 29332 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.245198 29680 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.308079 29700 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:14.260350 29711 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.280205 29723 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.310894 29671 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:14.316862 29736 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:25:14.355108 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.336673 29680 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:14.337159 29496 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:14.355955 29562 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:14.356235 29496 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:14.356499 29562 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:14.356683 29496 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:14.357460 29562 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.357729 29496 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.337809 29735 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:14.359962 29735 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.339241 29705 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:14.346392 29702 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:14.363310 29119 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:14.305569 29662 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:25:14.363862 29702 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.364428 29662 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:14.364550 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.364838 29662 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:14.366161 29702 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:14.351898 29738 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:25:14.352551 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.353415 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.367805 20882 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:14.319538 29699 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:14.368846 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.367805 20875 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:14.370820 29660 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:14.371424 29660 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:14.371286 29529 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.371789 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.372411 20875 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:14.372462 29660 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.373459 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.373669 29529 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:14.358959 29496 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:14.351095 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 103752us
I20260709 12:25:14.382195 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:14.383347 29699 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:14.383823 29699 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.384209 29699 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:14.387310 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.389809 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:14.353785 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:14.376933 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:14.351496 29458 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.335072 29698 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:14.391597 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:14.365803 29662 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.389966 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:14.404524 20732 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:14.398872 29677 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:14.415874 29734 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.499s	user 0.004s	sys 0.005s
I20260709 12:25:14.377737 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:14.395836 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:14.397174 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 319003us
I20260709 12:25:14.397842 29687 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:14.399845 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 275274us
I20260709 12:25:14.452734 20878 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:14.400008 29418 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:14.401163 29119 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:14.403277 20806 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:14.409752 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.392308 20879 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:14.442847 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:14.443594 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.392383 20730 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:14.442184 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.460489 20877 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:25:14.461333 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:14.462989 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 53326.1us
I20260709 12:25:14.465723 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.467022 20875 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:14.468178 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.468890 21433 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:14.469074 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.469383 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:14.469848 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:14.470139 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.477681 20732 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 12:25:14.478219 20732 pending_rounds.cc:85] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 13
I20260709 12:25:14.478502 20732 pending_rounds.cc:107] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 21.14
W20260709 12:25:14.478572 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100908us
I20260709 12:25:14.478812 20732 raft_consensus.cc:2889] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:14.480801 20732 pending_rounds.cc:107] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:25:14.481448 20732 raft_consensus.cc:2889] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:14.487478 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.489773 21435 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:14.491779 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:14.492710 20879 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:14.494585 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:14.494741 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:14.496037 20883 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:14.486208 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.489233 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:14.501340 21432 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:25:14.495610 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:14.508066 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:14.509848 20700 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.053s (SENT)
I20260709 12:25:14.510574 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.455003 29418 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.403854 29680 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 12:25:14.464638 29119 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:14.519157 20875 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:14.520146 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 130412us
W20260709 12:25:14.539103 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.497184 29458 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:14.497725 29292 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:14.530527 20880 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:14.477028 29698 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:14.485819 29700 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:14.552796 20883 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 12:25:14.551841 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.564072 20876 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:25:14.554044 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.564713 21432 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:14.515908 29677 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:14.511723 29662 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:14.524807 29418 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:14.512081 29580 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:14.604082 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:14.569038 29728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.024s	sys 0.028s
I20260709 12:25:14.522500 29705 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:14.599138 20884 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 12:25:14.548732 29675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.000s	sys 0.027s
I20260709 12:25:14.595104 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.616626 21429 raft_consensus.cc:2368] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:25:14.556133 29683 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:14.637359 20883 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:25:14.555445 29292 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.640403 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.651566 20875 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:25:14.597826 29735 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:14.646373 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51398.7us
I20260709 12:25:14.566709 29698 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:14.656735 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:14.658011 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46448.6us
W20260709 12:25:14.629782 29728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.024s	sys 0.028s
I20260709 12:25:14.657214 29292 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:14.628378 29700 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.672284 29119 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.673836 29119 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:14.675460 21428 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:14.676537 29733 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.539s	user 0.006s	sys 0.006s
I20260709 12:25:14.670902 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:14.677991 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:14.678373 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:14.678648 21426 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:14.679426 29681 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:14.679517 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:14.680167 29681 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:14.680691 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.680264 21427 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:14.681283 29681 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.695456 29700 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.699728 29684 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:14.700372 29684 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:14.629109 29580 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.701620 29684 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.643217 29559 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:14.651367 29355 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:14.704164 29711 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:14.645254 29675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.000s	sys 0.028s
I20260709 12:25:14.649721 29723 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:14.681996 29712 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:14.710903 21432 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:14.688125 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:14.687439 29705 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:14.734833 29563 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:25:14.736076 29563 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:14.708678 29698 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:14.736402 29705 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.655450 29562 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:25:14.772931 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 94623us
W20260709 12:25:14.774197 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 264399us
W20260709 12:25:14.784325 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.646947 29683 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:14.685590 29744 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:25:14.834692 20700 leader_election.cc:336] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.073s (SENT)
I20260709 12:25:14.845446 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:14.785058 29713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.002s	sys 0.030s
I20260709 12:25:14.670096 29707 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:14.830403 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:14.744704 29710 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.854439 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:14.736435 29563 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:14.854552 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.776513 29711 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.800468 29712 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:14.861975 20727 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:14.798480 29355 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:14.833586 29683 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:14.850463 29713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.002s	sys 0.031s
W20260709 12:25:14.865710 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.844125 29663 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.835428 29687 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:14.852298 29684 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:14.869124 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:14.851995 29689 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.828886 29671 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:25:14.857710 29546 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.800073 29728 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 12:25:14.860880 29736 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:14.854323 29665 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:14.909350 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:14.903343 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:14.863981 29712 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:14.908663 20621 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.898s	user 0.020s	sys 0.024s
I20260709 12:25:14.913518 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:14.918668 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 473044us
I20260709 12:25:14.969024 21429 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:14.869647 29580 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:14.922150 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 370399us
I20260709 12:25:14.972474 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:14.871129 29355 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.863761 29710 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:14.926519 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 437379us
W20260709 12:25:14.931391 29748 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:25:14.932010 29638 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.679s	user 0.015s	sys 0.000s
W20260709 12:25:14.945088 29701 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.001s
I20260709 12:25:14.945499 29673 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:25:14.927589 29723 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:14.947665 29699 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.003s
W20260709 12:25:14.949043 29496 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.002s
I20260709 12:25:14.938637 29698 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.950912 20804 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:14.950874 29563 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:14.963218 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59939.3us
I20260709 12:25:14.903640 29663 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:15.021502 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 205240us
I20260709 12:25:14.923957 29689 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:14.978997 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:14.979367 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 339877us
I20260709 12:25:14.924474 29737 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:15.046171 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.870987 29711 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:14.980152 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:14.909077 29687 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.004971 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:15.070802 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 91213.2us
I20260709 12:25:14.977283 29665 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:15.006227 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.015388 29736 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.974083 29707 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:14.973717 29712 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:15.016430 20882 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:25:14.978694 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:15.001960 29404 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:15.002010 29767 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:15.010354 29638 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.070005 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.075897 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.072834 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:15.076813 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.079579 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.094274 20875 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:15.093031 21429 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:14.969290 29546 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.081424 20728 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:15.016040 29699 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.081923 20727 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:15.105621 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.082266 29707 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:15.112977 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.120527 20884 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:15.039921 29292 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.069819 29701 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:25:15.080616 29559 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:25:15.125425 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:15.019729 29723 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.088302 29759 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:15.140329 20810 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.15-31.15]
I20260709 12:25:15.147420 20803 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:25:15.149279 21429 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:15.108466 29641 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:15.095136 29404 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:15.195678 21426 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.007s
I20260709 12:25:15.197397 21426 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
W20260709 12:25:15.199504 21427 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.008s
I20260709 12:25:15.199983 21427 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:15.129153 29707 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:15.090278 29736 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:15.214455 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89132.7us
W20260709 12:25:15.217823 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 117259us
I20260709 12:25:15.105708 29580 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:25:15.108136 29680 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.088s
W20260709 12:25:15.227090 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 121766us
I20260709 12:25:15.141165 29769 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:15.233530 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 134681us
W20260709 12:25:15.236759 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 261484us
I20260709 12:25:15.246239 20850 leader_election.cc:400] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 20 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 20. Current term is 21.
I20260709 12:25:15.248600 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:15.252657 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.252571 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:15.253575 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:15.215174 29559 consensus_queue.cc:1243] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 28.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 12:25:15.242329 29758 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:25:15.235689 29701 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.271615 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:15.273090 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:15.273648 21432 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.000s
I20260709 12:25:15.274216 21432 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 12:25:15.274492 21432 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 14
I20260709 12:25:15.256322 29735 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:25:15.312100 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74686.4us
I20260709 12:25:15.269110 29559 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:15.312073 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:15.265445 29779 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.277052 29700 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:15.277513 29728 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:15.280977 29737 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:15.280736 29496 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:15.331641 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94171.6us
I20260709 12:25:15.335794 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:15.274688 21432 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 20.15
I20260709 12:25:15.340267 21432 raft_consensus.cc:2917] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Aborting config change with OpId 20.15: Aborted: Op aborted by new leader
I20260709 12:25:15.309273 29641 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:15.341415 21432 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:15.346501 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 123951us
I20260709 12:25:15.321136 29355 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.323452 29698 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.280565 29701 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.383644 29699 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.384204 29638 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:15.385511 29292 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.385623 29641 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.352355 29680 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:25:15.388685 29680 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:15.389139 29680 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:15.390106 29680 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:15.350514 29761 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.270s	user 0.001s	sys 0.001s
I20260709 12:25:15.379315 29559 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:15.391722 29559 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:15.392239 29559 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:15.395692 29770 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:25:15.348909 29735 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:15.398964 29735 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:15.399434 29735 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:15.399410 29680 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:15.406183 29771 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 12:25:15.408342 29775 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:25:15.408854 29723 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:15.410416 29763 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:25:15.411974 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3480) took 1125 ms. Trace:
W20260709 12:25:15.413260 29418 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.772s	user 0.008s	sys 0.004s
I20260709 12:25:15.412427 20809 rpcz_store.cc:276] 0709 12:25:14.286349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.286531 (+   182us) service_pool.cc:224] Handling call
0709 12:25:14.292212 (+  5681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.297185 (+  4973us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:14.521003 (+223818us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.998042 (+477039us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.998584 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.998732 (+   148us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:14.998768 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.998840 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.411697 (+412857us) raft_consensus.cc:1685] finished
0709 12:25:15.411731 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.411899 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12742,"prepare.queue_time_us":373167,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":5533,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":8678,"thread_start_us":542096,"threads_started":2,"wal-append.queue_time_us":623009}
I20260709 12:25:15.413663 29418 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.414753 29710 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.415966 29687 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.381310 29689 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.350699 29665 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:15.340220 29779 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.416635 29418 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.417604 29665 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:25:15.418567 29764 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:25:15.348899 29404 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.423815 29404 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:15.424252 29404 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:15.425091 29404 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.426357 29404 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:15.351146 29711 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.345392 29663 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.427980 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:15.429392 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:15.430652 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:15.340451 29772 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.082s
W20260709 12:25:15.407100 29458 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.005s
W20260709 12:25:15.446262 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:15.440088 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 128271us
W20260709 12:25:15.443632 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 109865us
I20260709 12:25:15.458038 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.465646 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:15.454457 29662 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.838s	user 0.014s	sys 0.000s
I20260709 12:25:15.415923 29546 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.477803 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:15.432730 29783 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:15.480695 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.479915 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:15.478652 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 246066us
I20260709 12:25:15.483590 20735 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:15.446928 29119 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.000s
I20260709 12:25:15.482124 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.486454 21433 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:25:15.487527 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.434693 29793 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:15.418606 29779 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.415911 29736 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:15.431128 29712 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.454456 29458 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:15.482167 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 255434us
I20260709 12:25:15.455716 29710 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:15.435073 29789 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:25:15.511134 29675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.000s	sys 0.012s
I20260709 12:25:15.541044 21428 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:15.541677 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.478408 29698 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:25:15.542366 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.555074 20731 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:25:15.555588 20883 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:15.538298 29529 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.554963 29689 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.566249 29689 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.567603 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.545689 29355 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.568658 29689 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.550948 29546 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.478802 29790 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 12:25:15.570439 29767 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:15.570886 29767 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:15.571924 29767 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.572870 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1578) took 1308 ms. Trace:
I20260709 12:25:15.573285 20882 rpcz_store.cc:276] 0709 12:25:14.264820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.368508 (+103688us) service_pool.cc:224] Handling call
0709 12:25:15.572655 (+1204147us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:15.572808 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484297472}
I20260709 12:25:15.574222 29496 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:15.574491 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:15.576196 29683 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.001s
I20260709 12:25:15.576284 20807 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:15.576798 20807 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:15.576725 29683 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.577674 20807 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.463876 29728 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:15.510823 29292 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.579504 29292 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:15.580333 29684 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.003s
I20260709 12:25:15.581898 29292 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:15.550187 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.582262 29712 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.578812 29728 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.538625 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.583527 21427 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:15.584853 21432 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:15.585348 29728 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:15.542258 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.483026 29783 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:15.586944 29783 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.492343 29723 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:15.578495 29707 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.558387 29701 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.558297 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.543244 29418 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:15.563822 20726 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:25:15.563903 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.564698 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.565322 29529 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.606163 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:15.569438 29791 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:15.570107 29546 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.545411 29793 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:15.542795 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.546435 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:15.552197 20876 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:25:15.612311 20878 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:15.578970 20807 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:15.431744 29711 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:25:15.551843 29675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.001s	sys 0.013s
I20260709 12:25:15.597978 20731 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:15.448083 29559 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.600791 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.600051 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.621824 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:15.555471 29663 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.433627 29735 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.614835 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:15.610771 29418 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.617914 29529 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.619570 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.588979 29683 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.568954 29355 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.605759 29783 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:15.628187 20878 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:15.621824 20727 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:15.596127 29789 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:15.632059 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.605121 29723 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:15.551426 29779 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.609324 29791 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:15.653800 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.632236 20732 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:25:15.625185 29785 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.610879 29496 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.051s
I20260709 12:25:15.675498 20876 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:15.679318 20807 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:25:15.692030 20807 pending_rounds.cc:85] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 16
I20260709 12:25:15.692344 20807 pending_rounds.cc:107] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:25:15.692571 20807 raft_consensus.cc:2889] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:15.613340 29736 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 12:25:15.661222 21434 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:25:15.665030 21432 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:15.698021 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.699537 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:15.623580 29563 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:15.657812 29404 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:15.681612 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.681303 29779 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.682660 29791 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.704509 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:15.702847 20735 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:15.659296 29711 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:15.707324 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.704715 29762 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:15.709735 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:15.710410 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.710999 29355 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.711407 29707 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.711603 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.712199 20732 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:15.631579 29786 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.160s
I20260709 12:25:15.713217 29711 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.682631 29789 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.632385 29663 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.717159 29663 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.718842 29793 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.700950 29785 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.718859 29546 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.719784 20734 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:15.720589 29418 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.720885 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.721282 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.721616 20734 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 19.
I20260709 12:25:15.689656 29119 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 23.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:25:15.722168 21426 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:25:15.722661 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.723237 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:15.723338 21435 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:15.727013 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:15.728765 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.729429 21427 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:15.731914 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.732547 20879 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:15.732589 20878 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:15.735177 20735 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:15.735236 20732 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:15.734087 29792 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:15.738238 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.740094 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:15.741003 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.743470 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:15.744479 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.746174 29562 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:15.747017 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.746714 29562 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.747748 20733 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:15.748306 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.752700 20728 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:15.749549 21429 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:15.755208 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:15.756363 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.757392 20732 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:15.769241 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.770140 20807 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:25:15.771975 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.771975 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:15.772652 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:15.773717 20726 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:25:15.780352 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:15.781622 20806 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:25:15.739079 29559 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:15.780459 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 153723us
I20260709 12:25:15.789816 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:15.792392 20807 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:15.792588 20734 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:15.795428 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.797250 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:15.797803 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:15.799304 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:15.791433 20732 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:15.804558 20806 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:15.778613 29684 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.774238 29797 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:15.784106 29705 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:15.813659 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.814267 20883 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:15.784169 29790 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:15.815325 20879 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:15.811071 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.812242 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.817019 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.818248 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.813140 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.782655 29710 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:25:15.826149 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.831022 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:15.832312 20876 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:15.833385 20727 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:15.826752 20733 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:25:15.784345 29701 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:25:15.828970 29705 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:15.783036 29792 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:25:15.786058 29662 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:25:15.783360 29707 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:25:15.831645 20734 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:15.793186 29541 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:25:15.823235 29797 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:15.845737 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:15.821259 29665 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:15.820693 29641 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:15.861006 20806 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:15.850037 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52282.9us
I20260709 12:25:15.861933 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:15.816375 29404 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
W20260709 12:25:15.868254 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:15.813293 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:15.795644 29723 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:15.819757 29684 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.813223 29795 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.056s
I20260709 12:25:15.878430 20806 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:15.880802 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:15.883033 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84046.5us
I20260709 12:25:15.883793 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:15.861126 29665 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:15.826365 29790 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:15.889991 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:15.898880 20734 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:15.943836 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 244385us
I20260709 12:25:15.945889 20727 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:15.947316 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:15.947937 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:15.876447 29735 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:15.890805 29798 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 12:25:15.876101 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 65446.4us
I20260709 12:25:15.952149 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:15.871979 29712 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.075s
I20260709 12:25:15.956638 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:15.952569 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90025.8us
I20260709 12:25:15.891355 29665 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:15.962347 20876 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:15.955638 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:15.964768 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:15.880046 29680 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.073s
I20260709 12:25:15.949775 29792 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:15.970664 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:15.974483 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83333.7us
I20260709 12:25:15.976080 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:15.947464 29677 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:25:15.957310 29662 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:15.981086 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:15.982075 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:15.967829 29404 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:25:15.985589 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:15.986632 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:15.987535 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:15.987674 29756 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 28 current term: 28 is higher.
I20260709 12:25:15.974524 29711 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:15.988713 29711 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:15.990973 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:15.994365 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:15.972295 29789 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:15.975733 29735 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.005159 29404 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:16.006174 29404 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEARNER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:16.007929 29785 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:15.977692 29779 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:16.009650 29805 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:25:16.010345 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:16.010864 29802 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:25:16.015998 29562 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:15.982638 29687 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:15.979384 29355 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:16.028894 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.030155 20808 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:15.984048 29529 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:16.000450 29712 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:16.032464 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:16.001788 29762 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:16.047030 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:16.002274 29789 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:16.046595 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:16.031301 29529 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:16.059018 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.026415 29687 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:16.029939 29355 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.073827 20808 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:16.030499 29792 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:16.057950 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:16.059559 29789 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:15.978714 20621 maintenance_manager.cc:643] P 9f77a9352772433bb5809bf72fd4f0ee: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 4.375s	user 0.146s	sys 0.073s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":27113,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":65853,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":23,"lbm_write_time_us":9940,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":9,"peak_mem_usage":283250,"raft.queue_time_us":501976,"raft.run_cpu_time_us":10731,"raft.run_wall_time_us":105888,"reinsert_count":0,"thread_start_us":637187,"threads_started":4,"update_count":234,"wal-append.queue_time_us":430989}
I20260709 12:25:15.972198 29705 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.032675 29779 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:25:16.072330 29562 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:16.054320 29404 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:16.104176 29355 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:16.020608 29559 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:25:16.032171 29662 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:25:16.088020 29687 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:16.161444 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 322812us
I20260709 12:25:16.161798 20808 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:25:16.026768 29735 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.155875 20806 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
W20260709 12:25:16.173110 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 182239us
I20260709 12:25:16.173807 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:16.174789 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 192798us
I20260709 12:25:16.181157 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.161458 29562 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.163002 29404 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.162040 29792 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:16.186609 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.185200 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:16.166669 29684 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:16.191987 29404 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:16.195601 29816 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
W20260709 12:25:16.198596 29806 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:25:16.200095 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.201103 29792 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.204479 20880 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:25:16.196529 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:16.197642 20622 maintenance_manager.cc:419] P 9f77a9352772433bb5809bf72fd4f0ee: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 12:25:16.188696 29562 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:16.206813 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.053860 29711 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.210026 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.187690 29790 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:15.971963 29797 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.194917 29813 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.188s
W20260709 12:25:16.223868 20804 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3482) took 1922 ms (client timeout 1975 ms). Trace:
I20260709 12:25:16.224393 29816 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:25:16.224272 20804 rpcz_store.cc:269] 0709 12:25:14.300964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.301876 (+   912us) service_pool.cc:224] Handling call
0709 12:25:16.223685 (+1921809us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54002b5cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:16.223792 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248240512}
I20260709 12:25:16.225005 29816 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:16.225822 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.225986 29816 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.226503 20803 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:16.229681 29813 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:16.230116 29813 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:16.230974 29813 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.232252 29813 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:16.232321 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.233289 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:16.234964 29712 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:16.235526 29712 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.235601 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:16.235957 29712 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:16.235635 29826 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:16.237239 29826 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:16.238193 29826 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:16.239125 29826 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:16.239269 29684 consensus_queue.cc:1175] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 21 status { last_received { term: 21 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 21 index: 16 } }
I20260709 12:25:16.240724 29826 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:16.241079 29826 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:16.241886 29826 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:16.212448 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 328049us
W20260709 12:25:16.212437 29807 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 12:25:16.221014 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:16.222563 29817 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:25:16.236052 29816 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:16.217734 20621 maintenance_manager.cc:643] P 9f77a9352772433bb5809bf72fd4f0ee: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 12:25:16.256012 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:16.218760 29783 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.003s
I20260709 12:25:16.260617 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.261449 20806 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:16.217754 29689 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:16.266295 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:16.247385 29779 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:16.243165 29826 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:16.276221 20803 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:16.255013 29546 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:16.253681 29800 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:16.286589 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:16.206072 29529 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.288034 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:16.292284 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.294166 20808 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:16.325546 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3362) took 1242 ms. Trace:
I20260709 12:25:16.326099 20884 rpcz_store.cc:276] 0709 12:25:15.046237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.046805 (+   568us) service_pool.cc:224] Handling call
0709 12:25:15.077657 (+ 30852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.126609 (+ 48952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.288202 (+1161593us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:16.288366 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97968512}
I20260709 12:25:16.329983 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:16.281894 29735 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:16.283007 29800 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.261559 29813 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
W20260709 12:25:16.330875 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149793us
I20260709 12:25:16.285038 29496 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.339579 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.294030 29546 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.325033 29673 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:16.353452 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:16.359792 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.325778 29710 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:16.334079 29810 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:16.328979 29541 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:16.358104 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.368664 20808 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:16.366978 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.356314 20804 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 12:25:16.371443 20804 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 14
I20260709 12:25:16.371811 20804 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 20.15
I20260709 12:25:16.353288 29845 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:16.344427 29800 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:16.374434 20804 raft_consensus.cc:2917] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Aborting config change with OpId 20.15: Aborted: Op aborted by new leader
I20260709 12:25:16.375241 20804 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.354868 29735 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:16.374161 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44257.9us
W20260709 12:25:16.383697 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 150462us
I20260709 12:25:16.383888 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:16.384616 20808 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:16.384620 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:16.387729 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.390216 20808 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 12:25:16.390671 20808 pending_rounds.cc:85] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 12
I20260709 12:25:16.390945 20808 pending_rounds.cc:107] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:25:16.391259 20808 raft_consensus.cc:2889] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.398155 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:16.400476 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 168225us
I20260709 12:25:16.369194 29541 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.401082 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:16.400753 29563 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.693s	user 0.012s	sys 0.000s
I20260709 12:25:16.355823 29496 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.409659 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:16.410549 21432 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:16.403913 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.408334 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:16.406917 20730 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:25:16.411957 21431 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:16.412111 20808 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:16.412951 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:16.414057 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:16.414672 20804 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:25:16.414947 21435 raft_consensus.cc:2376] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24. Re-sending same reply.
I20260709 12:25:16.417912 21434 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 18. (term mismatch)
I20260709 12:25:16.418321 21434 pending_rounds.cc:85] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 17
I20260709 12:25:16.418493 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:16.418529 21434 pending_rounds.cc:107] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 27.18
I20260709 12:25:16.419586 21434 raft_consensus.cc:2889] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.420534 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:16.405684 29827 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:16.420029 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:16.424052 20808 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:16.423779 21426 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:25:16.362495 29546 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.439149 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:16.405994 29541 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:16.438822 21428 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:25:16.450505 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:16.411432 29735 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:16.454113 20734 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:25:16.455011 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:16.419919 29673 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.458091 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:16.460294 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:16.461911 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.466101 29824 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 26.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:25:16.470011 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:16.478088 29355 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.479429 20801 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:25:16.482223 29496 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:16.483559 29687 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.483866 29673 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.485785 29687 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:16.486123 29673 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:16.491351 21431 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:16.491581 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:16.493314 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:16.494228 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:16.494585 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.494781 20876 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:16.496030 20877 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: [26.15-28.16]
I20260709 12:25:16.497054 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.497732 20877 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:16.509027 21430 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:16.510125 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:16.420555 29794 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:16.511698 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.524833 20877 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
W20260709 12:25:16.525236 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 140849us
I20260709 12:25:16.525774 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.515127 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.516566 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.517223 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:16.516912 29404 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.517906 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.385499 29845 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.441650 29812 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.187s
I20260709 12:25:16.523531 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.529178 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:16.534933 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:16.537324 20876 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:16.525966 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:16.541234 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 86305.6us
W20260709 12:25:16.510411 29665 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.006s
I20260709 12:25:16.423579 29803 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.265s
W20260709 12:25:16.526264 29827 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:25:16.551111 20735 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:16.551376 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:16.554390 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.554327 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.555332 20805 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:16.557559 21435 raft_consensus.cc:2368] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30.
I20260709 12:25:16.557520 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:16.539426 29713 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:16.570779 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44883.1us
I20260709 12:25:16.572577 29663 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:16.577551 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:16.577031 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.303s (SENT) [suppressed 36 similar messages]
I20260709 12:25:16.583341 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:16.534222 29712 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.587589 20879 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:25:16.654232 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.574123 29723 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:16.573741 29803 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 26.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:16.552307 29665 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.530503 29794 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:16.735708 29829 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:25:16.736588 29833 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:25:16.737262 29804 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.647s	user 0.003s	sys 0.000s
I20260709 12:25:16.737887 29843 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:16.738446 29835 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:16.738745 29846 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.152s
W20260709 12:25:16.733969 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 156744us
I20260709 12:25:16.751286 20804 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:25:16.752013 20804 raft_consensus.cc:606] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:16.752458 20804 raft_consensus.cc:614] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:16.703769 29663 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:16.764353 29663 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.765067 29355 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:16.767026 29546 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:16.768451 20879 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:16.764429 29712 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:16.586926 29662 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.154s
I20260709 12:25:16.771652 29845 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:16.779038 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 202006us
I20260709 12:25:16.751560 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:16.705996 29723 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.793484 29723 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:16.579780 29735 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.754014 29810 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:16.802623 29665 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:16.810524 29826 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.001s
W20260709 12:25:16.778337 29816 consensus_meta.cc:287] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
W20260709 12:25:16.813908 29789 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.651s	user 0.005s	sys 0.006s
I20260709 12:25:16.811826 20878 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:25:16.812918 29794 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:16.582407 29529 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.160s
I20260709 12:25:16.817523 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:16.822232 29849 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 12:25:16.823426 29712 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:16.823999 29712 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.824424 29712 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:16.827587 29823 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.828440 29823 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.832959 29404 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:16.841037 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:16.848661 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:16.851588 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:16.853354 29562 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.000s
I20260709 12:25:16.853621 20735 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:25:16.854172 29677 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:16.853842 29562 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.854909 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:16.856467 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:16.857148 29562 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:16.858659 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 68926.9us
I20260709 12:25:16.859479 20734 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:25:16.850627 29803 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:16.860440 29843 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:16.862787 20733 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:25:16.854909 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:16.864990 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:16.865592 20734 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:25:16.814563 29789 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:16.870833 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.874369 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:16.879412 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 284292us
I20260709 12:25:16.881733 20735 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:25:16.883744 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.885241 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.885514 20803 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 19.
I20260709 12:25:16.887642 20802 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:16.887917 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.872898 29677 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:16.894815 29847 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:16.874778 29701 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:25:16.897472 29701 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:16.867656 29541 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:16.886592 29355 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:16.905989 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.877568 29562 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:25:16.891390 29662 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:16.919955 20884 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:16.907811 29701 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:16.925000 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.904327 29663 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
W20260709 12:25:16.928527 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.931584 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54914.9us
W20260709 12:25:16.924636 29675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.014s	sys 0.000s
I20260709 12:25:16.913540 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:16.946931 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.947336 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:16.943094 29675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.015s	sys 0.000s
I20260709 12:25:16.966627 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3490) took 2494 ms. Trace:
I20260709 12:25:16.967331 20810 rpcz_store.cc:276] 0709 12:25:14.459946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.460070 (+   124us) service_pool.cc:224] Handling call
0709 12:25:14.460393 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.156282 (+695889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.951372 (+1795090us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:16.954675 (+  3303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532878720}
I20260709 12:25:16.942932 29823 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:16.947991 29701 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:16.975490 29803 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:16.976225 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:16.977228 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:16.977994 20732 raft_consensus.cc:2368] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20.
I20260709 12:25:16.979333 20775 leader_election.cc:400] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20.
I20260709 12:25:16.979679 20775 leader_election.cc:403] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:25:16.979527 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:16.980654 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1639) took 1968 ms (client timeout 2303 ms). Trace:
I20260709 12:25:16.977876 29867 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:16.964596 29662 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:25:16.981082 21433 rpcz_store.cc:269] 0709 12:25:15.004443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.004624 (+   181us) service_pool.cc:224] Handling call
0709 12:25:16.972345 (+1967721us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:16.972490 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954461056}
I20260709 12:25:16.966600 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:16.983498 29867 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:16.985188 29774 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:16.986398 29701 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:16.954293 29847 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:25:16.993320 20734 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:16.999588 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:17.007439 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:17.004783 29541 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.005347 29662 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:17.012425 29857 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:17.023530 29789 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.024137 29857 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:17.024720 29857 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:17.025530 29857 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.989125 29866 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:16.987830 29843 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:25:17.026556 29866 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:16.995786 29867 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:17.029071 29866 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.014834 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1651) took 1200 ms. Trace:
I20260709 12:25:17.029915 20807 rpcz_store.cc:276] 0709 12:25:15.814189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.814353 (+   164us) service_pool.cc:224] Handling call
0709 12:25:15.814726 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.814987 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:16.075330 (+260343us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.014304 (+938974us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b1800262780. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:25:17.014411 (+   107us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:17.014486 (+    75us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:17.014557 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.014629 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.014767 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8888,"prepare.queue_time_us":260545,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":32183,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2749,"spinlock_wait_cycles":11303168,"thread_start_us":376508,"threads_started":3,"wal-append.queue_time_us":152112}
I20260709 12:25:17.000288 29774 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:17.026044 29800 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.642s	user 0.006s	sys 0.003s
I20260709 12:25:17.011003 29660 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:17.026489 29776 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:17.057189 29541 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:17.042176 29800 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.062799 29541 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.064476 29800 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.055603 29774 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.066011 29541 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.066605 29774 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:17.050655 29660 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:17.051528 29712 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.072767 29677 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:17.087847 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:17.089663 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:17.090306 21426 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:17.090647 20882 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:17.091290 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.090164 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.095559 29826 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.096112 29826 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.096932 29797 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:17.097694 29797 consensus_queue.cc:1243] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:17.098191 29826 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:17.100845 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93482.1us
W20260709 12:25:17.102941 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:25:17.086934 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107521us
I20260709 12:25:17.118589 29843 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:17.119927 29877 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:17.120524 29877 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:17.121833 29872 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:25:17.124533 29877 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.126628 29681 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:17.127262 29681 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.129318 29681 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:17.133302 20850 leader_election.cc:336] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.682s (SENT)
I20260709 12:25:17.133813 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:17.135691 29811 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:17.136565 29660 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.20, Last appended by leader: 20, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.142146 29791 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:17.146785 29701 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 19. Current term is 20.
W20260709 12:25:17.146976 29765 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 2.023s	user 0.015s	sys 0.018s
I20260709 12:25:17.148666 29712 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.149737 29867 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:17.155305 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.168548 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50172.9us
W20260709 12:25:17.171787 29759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.955s	user 0.013s	sys 0.016s
W20260709 12:25:17.175446 29759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.959s	user 0.013s	sys 0.016s
I20260709 12:25:17.175779 29759 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:17.177907 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:17.181736 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:17.205075 20726 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:25:17.205767 20726 pending_rounds.cc:85] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 16
I20260709 12:25:17.205972 20726 pending_rounds.cc:107] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:25:17.206224 20726 raft_consensus.cc:2889] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.206702 29867 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.207713 29712 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.208428 29712 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.218245 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:17.218717 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:17.219475 20807 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:17.224037 21432 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:17.225778 21432 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:25:17.227454 20850 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.756s (SENT)
W20260709 12:25:17.228338 20850 leader_election.cc:336] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.773s (SENT)
I20260709 12:25:17.228686 21434 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:17.229063 21434 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:17.228704 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:17.229902 21434 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.230587 29824 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:17.231043 21434 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:17.227103 21432 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:25:17.243731 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.176942 29759 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.252565 20807 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:17.286757 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:17.251415 29824 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:17.288161 20875 raft_consensus.cc:606] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:25:17.288616 20875 raft_consensus.cc:614] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:17.288228 29824 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.289475 29562 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:17.274923 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.290057 29562 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.280673 21434 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:25:17.295941 21434 pending_rounds.cc:85] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:25:17.297307 21434 pending_rounds.cc:107] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:25:17.292269 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.303069 21434 raft_consensus.cc:2889] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.303283 20875 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:17.298198 29829 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.299547 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.300154 29846 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:17.300206 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.314978 20733 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:17.318324 20735 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:25:17.319669 20882 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:17.258924 29774 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:17.329111 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.329383 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.330133 20879 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:17.312706 29846 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:17.337852 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.341248 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:17.341830 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.314952 29824 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.328625 29847 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:17.351285 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49347.4us
I20260709 12:25:17.333294 29774 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.357617 29774 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.314440 29829 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:17.349937 29559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.153s	user 0.012s	sys 0.014s
W20260709 12:25:17.349387 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 104360us
I20260709 12:25:17.364320 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.365124 20879 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:25:17.366072 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.366742 20879 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 28
W20260709 12:25:17.377085 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76082.3us
I20260709 12:25:17.378584 29689 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:17.364084 29559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.012s	sys 0.015s
I20260709 12:25:17.381373 29774 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.397266 29559 consensus_queue.cc:1243] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:25:17.397511 29774 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.398881 29762 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:17.399351 29762 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:17.400210 29762 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.400267 29774 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.402269 29762 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:17.388377 29562 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.394618 20875 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:17.395581 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.405520 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:17.406128 20882 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:17.396245 29846 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:17.409271 29882 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:17.421475 20879 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:17.426396 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.427532 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:17.379060 29878 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.358606 29847 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.467353 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:17.472394 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:17.470920 29878 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.479050 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:17.430028 29784 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:17.487893 29853 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.727s	user 0.003s	sys 0.000s
I20260709 12:25:17.488420 29767 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:17.492141 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 130473us
I20260709 12:25:17.475026 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:17.476445 29834 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.030s	sys 0.037s
W20260709 12:25:17.520181 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.473477 29791 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:17.513198 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:17.475705 29663 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:25:17.519429 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 140986us
I20260709 12:25:17.418774 29689 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.491325 29824 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.478979 29705 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:17.520125 29834 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.030s	sys 0.037s
I20260709 12:25:17.524075 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:17.469771 29894 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 12:25:17.480489 29559 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:25:17.522475 29785 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:17.534017 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.539950 29665 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:17.540273 29816 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.540326 29681 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:17.502413 29784 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.537425 29837 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:17.529843 29737 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:17.524219 29711 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:17.555897 29562 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:17.561365 20728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1451) took 2006 ms (client timeout 1667 ms). Trace:
I20260709 12:25:17.588289 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.542770 29878 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.557195 29665 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:17.572788 29852 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.812s	user 0.001s	sys 0.001s
W20260709 12:25:17.418429 29835 consensus_queue.cc:1175] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 29 status { last_received { term: 25 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: [26.15-28.16]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:17.576534 29894 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:17.576853 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.578059 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.577890 29779 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:17.578599 29355 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:17.578950 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3379) took 1079 ms. Trace:
I20260709 12:25:17.535604 29824 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.538679 29705 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.556035 29899 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:25:17.560415 29681 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.559103 29816 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.554183 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:17.557214 29785 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:17.587550 20728 rpcz_store.cc:269] 0709 12:25:15.553065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.553167 (+   102us) service_pool.cc:224] Handling call
0709 12:25:17.559563 (+2006396us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54003b3b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:17.559687 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274013440}
I20260709 12:25:17.578831 29689 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.597393 29711 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.580184 29837 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.598132 20879 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:17.608906 20884 rpcz_store.cc:276] 0709 12:25:16.499801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.527766 (+ 27965us) service_pool.cc:224] Handling call
0709 12:25:17.578759 (+1050993us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:17.578892 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828275584}
I20260709 12:25:17.616747 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3374) took 1123 ms. Trace:
I20260709 12:25:17.621707 20882 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
W20260709 12:25:17.622205 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68105.1us
I20260709 12:25:17.641121 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:17.605890 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.622478 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:17.622702 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 264276us
I20260709 12:25:17.642251 20876 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:17.634119 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.553634 29898 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 12:25:17.636533 20878 rpcz_store.cc:276] 0709 12:25:16.493759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.494382 (+   623us) service_pool.cc:224] Handling call
0709 12:25:17.616555 (+1122173us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400200b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:17.616692 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727629184}
I20260709 12:25:17.559381 29897 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.066s
I20260709 12:25:17.622673 29681 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:17.655308 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 92165.1us
I20260709 12:25:17.652510 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:17.611347 29847 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.614457 29789 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.656863 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:17.662700 20802 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:17.621191 29829 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.663800 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:17.659024 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:17.616861 29824 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.614529 29784 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.632170 29864 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.651302 29683 raft_consensus.cc:2764] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:25:17.635557 29878 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:17.661655 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:17.667045 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.598673 29737 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:17.668447 21434 pending_rounds.cc:85] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 13
I20260709 12:25:17.674209 21434 pending_rounds.cc:107] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:25:17.674611 21434 raft_consensus.cc:2889] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.660444 29895 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:25:17.680610 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62297us
I20260709 12:25:17.682143 29895 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:17.682891 29895 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.685459 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:17.649291 29826 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:25:17.679152 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:17.689265 29895 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.689314 20884 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:25:17.692143 20875 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 25. Ops: []
I20260709 12:25:17.668685 29789 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.617607 29562 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.696409 29737 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.699368 29898 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:17.699622 29910 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:17.700867 29705 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.701231 29837 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.674654 29864 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.705202 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:17.711498 29665 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.712169 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:17.714874 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.738166 20875 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:17.726451 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47382.9us
I20260709 12:25:17.731030 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.678761 29759 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:17.682976 29904 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.677310 29767 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:25:17.744935 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:17.742715 20808 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:17.660424 29785 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.735289 29847 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:17.755398 20727 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:25:17.730669 29898 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:17.742172 29759 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:17.749663 29665 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.009s
W20260709 12:25:17.766170 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.753643 29847 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:17.776609 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 117652us
I20260709 12:25:17.764699 29898 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:17.765381 29759 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:17.790324 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:17.770527 29897 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:17.796217 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:17.779572 29847 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:17.743995 29904 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.805414 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:17.805900 20727 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:17.808620 20850 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 2.202s (SENT)
I20260709 12:25:17.810573 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:17.811553 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:17.815366 20802 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 25. Ops: []
I20260709 12:25:17.799751 29665 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:17.830610 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.773149 29901 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:25:17.874759 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:17.877029 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:17.853171 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.883133 20734 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:17.884148 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:17.888664 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:17.888561 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:17.889438 20727 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:17.889597 20734 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:17.878355 29882 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:25:17.891537 29891 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:25:17.895411 29677 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:17.896225 29677 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:17.912824 20805 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:25:17.914136 29681 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.915436 29711 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:17.919405 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3497) took 2155 ms. Trace:
I20260709 12:25:17.919838 20729 rpcz_store.cc:276] 0709 12:25:15.755413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.824549 (+ 69136us) service_pool.cc:224] Handling call
0709 12:25:15.841500 (+ 16951us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.869212 (+ 27712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.869246 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.869755 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.580452 (+1710697us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b18003b41c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:17.760479 (+180027us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:17.760521 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.760600 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:17.900511 (+139911us) raft_consensus.cc:1685] finished
0709 12:25:17.900530 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.910761 (+ 10231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27268,"raft.queue_time_us":30992,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":17488,"spinlock_wait_cycles":3435264,"thread_start_us":179721,"threads_started":1,"wal-append.queue_time_us":1883445}
I20260709 12:25:17.821468 29918 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:17.924314 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:17.926618 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.930027 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:17.881151 20802 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:25:17.932446 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:17.803153 29897 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:17.937093 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:17.923480 29918 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:17.939157 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:17.937721 20880 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:17.912079 21400 connection.cc:256] RPC call timeout handler was delayed by -1.1976638070000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
I20260709 12:25:17.936821 29816 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:17.959885 29736 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:17.960565 29736 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:17.962802 29665 raft_consensus.cc:670] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:17.968668 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3207) took 2266 ms. Trace:
I20260709 12:25:17.976204 20726 rpcz_store.cc:276] 0709 12:25:15.702427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.774556 (+ 72129us) service_pool.cc:224] Handling call
0709 12:25:15.774822 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.206766 (+1431944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.968395 (+761629us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002495d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:17.968572 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3288047872}
I20260709 12:25:17.991073 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:17.991904 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.019033 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.021781 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.022902 20726 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:17.957015 29816 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.024901 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:17.955955 29918 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:18.025977 21433 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:25:18.028488 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:18.029789 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:17.974730 29681 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.039084 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.035044 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.035872 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.040663 20729 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:18.041184 21433 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:18.051462 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.052455 20726 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:18.055800 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:18.036677 29762 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.004s
W20260709 12:25:18.058959 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72177.4us
I20260709 12:25:18.062872 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.059966 21433 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:18.064647 20880 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:18.063623 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:18.036278 29915 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:18.071682 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.075891 20801 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:25:17.965579 29898 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.080466 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.081965 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.082396 21430 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:18.083005 21433 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:18.084525 20726 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:18.078399 29119 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:25:18.042131 29683 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:25:18.105975 21433 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:18.038463 29816 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.123888 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.125483 20726 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:18.105490 29910 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:18.147491 29932 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:18.148033 20726 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:25:18.156219 29736 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.122397 29795 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:18.106134 29562 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.164250 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.178623 20810 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:18.178190 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 114795us
I20260709 12:25:18.185050 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2106) took 1694 ms. Trace:
I20260709 12:25:18.168087 29795 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:18.185866 21429 rpcz_store.cc:276] 0709 12:25:16.491088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.133879 (+1642791us) service_pool.cc:224] Handling call
0709 12:25:18.143771 (+  9892us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.174962 (+ 31191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.174995 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:18.175596 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:25:18.183205 (+  7609us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:18.183251 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.183327 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.184842 (+  1515us) raft_consensus.cc:1685] finished
0709 12:25:18.184862 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.184987 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30607,"raft.queue_time_us":33027,"thread_start_us":7160,"threads_started":1,"wal-append.queue_time_us":7567}
I20260709 12:25:18.177155 29683 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.194185 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:18.177606 29332 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.202343 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:18.203267 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 168297us
I20260709 12:25:18.205972 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:18.203780 29683 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.211644 29829 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:18.212230 29829 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:18.212059 29759 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.212862 29829 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:18.214385 29759 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:18.215950 29735 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:18.216427 29735 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.216740 29935 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:18.216948 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:18.218485 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:18.221364 29864 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.223048 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.223570 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:18.224339 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.216814 29735 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:18.225900 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:18.231626 29915 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:18.231851 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.232643 21427 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:18.233626 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.234722 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.235468 29675 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:18.235948 29675 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.236315 29675 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:18.238391 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.239717 29756 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:25:18.240221 29924 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:25:18.242419 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.243012 29361 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.243755 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.205085 29332 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.244854 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.246337 21427 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:18.195288 29736 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:18.251654 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:18.243453 29932 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:18.243577 29361 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.250140 29759 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:18.240217 29756 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:18.249188 29736 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:18.295202 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43609.8us
I20260709 12:25:18.295959 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3236) took 1318 ms. Trace:
I20260709 12:25:18.296072 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:18.278566 29829 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.261283 29361 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.266462 21434 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:18.301833 21434 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:18.268767 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.296409 20732 rpcz_store.cc:276] 0709 12:25:16.977183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.978662 (+  1479us) service_pool.cc:224] Handling call
0709 12:25:16.978936 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.979123 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:16.979399 (+   276us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.979764 (+   365us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.561824 (+582060us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18002e39c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:18.295524 (+733700us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b18002e39c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:25:18.295630 (+   106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.295681 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:18.295725 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.295790 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.295901 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1000,"prepare.queue_time_us":1481,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":53889,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":5196,"spinlock_wait_cycles":82800768,"thread_start_us":31836,"threads_started":1}
I20260709 12:25:18.253793 29932 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.275131 29918 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.275864 29712 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.309041 29918 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:18.276361 29927 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:25:18.278831 29332 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:18.245667 29916 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.033s
I20260709 12:25:18.274504 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:18.311177 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:18.311287 29641 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:18.311499 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.303825 29119 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:18.312332 21426 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:18.312605 29119 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:18.313123 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:18.313551 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.313510 29119 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.314869 29119 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:18.249352 29897 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.249015 29915 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:18.315757 29915 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.316120 29915 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.293994 21427 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:18.305603 21434 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.306780 29932 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.285899 29759 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.335448 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.340620 20728 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:18.348037 29332 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:18.348183 29756 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.348691 29332 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:18.302040 29797 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:25:18.349887 29925 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:25:18.351094 29938 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:18.351784 29938 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:18.352324 29938 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:18.345214 29816 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.329232 29736 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.349718 29332 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.349699 29759 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:18.357039 29736 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:18.323648 29936 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:18.359784 29934 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:18.360809 29934 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:18.365718 20730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3201) took 2908 ms (client timeout 1718 ms). Trace:
W20260709 12:25:18.366185 20730 rpcz_store.cc:269] 0709 12:25:15.457615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.457774 (+   159us) service_pool.cc:224] Handling call
0709 12:25:18.365437 (+2907663us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:18.365608 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180023168}
I20260709 12:25:18.345742 29638 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:18.344633 29898 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.328884 29829 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.370044 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.268800 29677 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.334522 29877 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:18.366566 29759 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.372092 21429 raft_consensus.cc:2368] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23.
I20260709 12:25:18.374877 21434 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:25:18.338640 29847 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.001s
W20260709 12:25:18.385797 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.394497 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 169602us
I20260709 12:25:18.410945 21434 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 20. Preceding OpId from leader: term: 25 index: 21. (index mismatch)
I20260709 12:25:18.424733 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.427847 21434 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:18.429896 20730 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:18.425869 29793 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 25 is higher.
I20260709 12:25:18.436179 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.386663 29847 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.437412 20726 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:25:18.438740 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.439493 21434 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:18.440574 29904 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.429112 29829 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:25:18.443677 29683 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.447429 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:18.431936 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:18.448444 20730 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:18.446030 29735 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.451133 29829 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:18.451327 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2114) took 1233 ms. Trace:
I20260709 12:25:18.451772 29829 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:18.452015 21430 rpcz_store.cc:276] 0709 12:25:17.217706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.217858 (+   152us) service_pool.cc:224] Handling call
0709 12:25:18.451134 (+1233276us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54001e5d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:18.451260 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984979712}
I20260709 12:25:18.452458 20883 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 25. Ops: []
I20260709 12:25:18.452755 29829 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.453351 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.453912 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.454097 29829 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:18.454640 20878 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:18.361689 29934 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.455516 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.465780 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:18.466452 20726 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:25:18.467823 29789 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.467845 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:18.469260 29938 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.478652 20726 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:18.482290 20730 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:25:18.483321 20731 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:18.483572 20727 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:18.488072 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:18.488611 21430 raft_consensus.cc:1217] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Deduplicated request from leader. Original: 23.16->[25.17-25.17]   Dedup: 25.17->[]
I20260709 12:25:18.489135 21430 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:25:18.451357 29735 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.492139 21430 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:18.482075 29930 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.018s
I20260709 12:25:18.500909 29915 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.502377 21426 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26.
I20260709 12:25:18.504179 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.427467 29929 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
W20260709 12:25:18.504544 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.504964 21434 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:18.511117 21430 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:18.509403 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:18.506875 29938 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.487681 29675 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.509949 29877 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:18.520272 29877 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.521005 29932 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.523147 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:18.520221 20879 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:18.524487 20850 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26.
I20260709 12:25:18.526470 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.532827 21434 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:18.521606 29675 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.528326 29736 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.010s
I20260709 12:25:18.528209 29929 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:18.522408 29877 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.534281 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:18.531345 29938 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:18.540794 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:18.541988 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.542169 20733 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.548171 20876 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:18.550251 29665 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:18.551086 29665 consensus_queue.cc:1243] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:18.553381 29915 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.555024 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3449) took 1262 ms. Trace:
I20260709 12:25:18.544445 29675 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:18.556049 29938 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:18.558374 29938 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:18.563140 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3291) took 2101 ms (client timeout 2070 ms). Trace:
W20260709 12:25:18.567597 20808 rpcz_store.cc:269] 0709 12:25:16.461535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.461667 (+   132us) service_pool.cc:224] Handling call
0709 12:25:18.562906 (+2101239us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54002bff54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:18.563038 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2943900032}
I20260709 12:25:18.567996 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:18.568517 20882 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:18.568667 20876 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.544203 29929 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:18.555074 29847 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.570026 29929 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.571620 29932 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:18.574580 20878 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:18.576900 20876 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:18.577396 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:18.565986 21433 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:18.584415 29404 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:25:18.588217 29404 consensus_queue.cc:1243] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:25:18.588820 29404 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:18.588899 29877 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:18.589210 29404 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:18.589534 29877 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:18.590163 29404 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.590513 29877 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.594333 29759 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.587945 29916 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:18.587517 29559 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:18.607227 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:18.616215 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:18.599131 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:18.617224 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 687724us
I20260709 12:25:18.608355 20803 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:18.599982 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:18.606779 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:18.611129 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:18.621387 20882 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:18.555467 20881 rpcz_store.cc:276] 0709 12:25:17.292860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.431146 (+138286us) service_pool.cc:224] Handling call
0709 12:25:17.617533 (+186387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.617756 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:17.646635 (+ 28879us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.738656 (+ 92021us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:17.738712 (+    56us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:17.738758 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.738819 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.554856 (+816037us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400200b50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:18.554966 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2006,"prepare.queue_time_us":31738,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":2364,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":3135,"spinlock_wait_cycles":341382528,"thread_start_us":89225,"threads_started":1,"wal-append.queue_time_us":128869}
I20260709 12:25:18.614152 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.607543 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:18.623540 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:18.599061 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:18.624600 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.625202 20733 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:18.625315 20876 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:18.621405 20806 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:18.602504 29932 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:18.630875 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:18.636137 20810 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.631572 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:18.636859 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.626580 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.637032 20884 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:25:18.627882 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.641409 20808 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:18.639513 20850 leader_election.cc:400] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23.
I20260709 12:25:18.642467 20850 leader_election.cc:403] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:18.640363 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.645810 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.647735 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.648864 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.600318 29824 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:18.625396 29847 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:18.601166 29909 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.651281 29689 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:25:18.651542 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.695799 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:18.696810 20881 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:18.601094 29662 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:18.635519 29680 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:18.630678 29559 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.652176 29932 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.599056 29900 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:18.654855 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.656785 20882 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:18.665253 20806 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:18.668625 21430 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:18.684643 20876 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:18.604800 29404 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:18.687820 20884 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:18.691007 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:18.717176 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.699415 20881 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:25:18.702164 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.721084 20809 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:18.705016 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.708724 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:18.723985 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:18.710513 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:18.731539 20808 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:18.735383 29838 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:18.736055 29838 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.737103 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:18.738296 29838 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.739890 29355 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.740437 29355 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:18.743716 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 144740us
I20260709 12:25:18.745761 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:18.746526 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:18.746971 29701 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.747520 29701 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.748657 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3461) took 1387 ms. Trace:
I20260709 12:25:18.749679 29701 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.749816 29835 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:18.750217 29875 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:18.750742 29875 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:18.751140 29875 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:18.751406 29711 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:18.751847 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:18.753072 29711 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:18.753726 29711 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.750258 29701 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:18.754496 29701 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.755601 29864 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:18.694490 29803 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.740711 29834 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:18.753746 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:18.718809 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.767040 21433 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:18.714262 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:18.768862 21433 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 18. (term mismatch)
I20260709 12:25:18.769328 21433 pending_rounds.cc:85] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 17
I20260709 12:25:18.769621 21433 pending_rounds.cc:107] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 29.18
I20260709 12:25:18.769915 21433 raft_consensus.cc:2889] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:18.770072 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.771533 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.667637 29916 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:25:18.763314 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:18.763451 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:18.753078 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:18.756109 29864 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.766227 20806 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:18.760077 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.654165 29816 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:25:18.774190 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:18.754382 29711 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.705873 29663 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:25:18.766825 29789 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:18.705440 29361 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:18.779897 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.782823 20876 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:18.812829 20876 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 12:25:18.783378 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69238.8us
I20260709 12:25:18.811808 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:18.783833 20882 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 20. Preceding OpId from leader: term: 25 index: 21. (index mismatch)
I20260709 12:25:18.715972 29829 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23.
I20260709 12:25:18.787469 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.786530 29791 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:18.788735 20802 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:18.765168 29680 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:18.797632 29816 consensus_queue.cc:1243] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 12:25:18.798868 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45861us
I20260709 12:25:18.801713 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.797294 29877 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:18.749061 20877 rpcz_store.cc:276] 0709 12:25:17.360812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.606948 (+246136us) service_pool.cc:224] Handling call
0709 12:25:17.607267 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.649439 (+ 42172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.649491 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:17.649560 (+    69us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:17.649608 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.748426 (+1098818us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:18.748472 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.748588 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96324480}
I20260709 12:25:18.705090 29662 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.828558 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.829316 20809 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:18.829533 20802 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:18.811105 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.833511 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.817240 20805 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:18.781713 29641 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:18.789639 29864 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.770750 20801 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:18.789062 29767 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:18.700383 29909 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.743078 29355 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.955173 29701 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.693080 29824 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:18.819387 29785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.000s	sys 0.007s
I20260709 12:25:18.807786 29663 consensus_queue.cc:1243] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 28.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:25:18.966087 29785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.000s	sys 0.008s
I20260709 12:25:18.970440 29824 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.975059 29701 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.975955 29332 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.975860 29701 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.976787 29909 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.978430 29701 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.961933 29950 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:18.980022 29701 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.979672 29875 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.980566 29701 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.982510 29875 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.983065 29701 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.976634 29332 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.987033 29332 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:18.877481 29789 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:25:18.993971 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1656) took 2547 ms (client timeout 1794 ms). Trace:
W20260709 12:25:18.994403 21428 rpcz_store.cc:269] 0709 12:25:16.446733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.509339 (+ 62606us) service_pool.cc:224] Handling call
0709 12:25:18.991459 (+2482120us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:18.993895 (+  2436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2562277632}
I20260709 12:25:18.782917 29834 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.876181 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:18.995580 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.865756 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:18.996244 21428 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
W20260709 12:25:18.997407 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 121302us
I20260709 12:25:18.998378 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.865789 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:18.953683 29735 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:18.949883 29662 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.929289 29938 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
W20260709 12:25:18.938488 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.876344 29932 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.774554 29803 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.000864 29735 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:19.002810 29638 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:19.002650 29404 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:19.005246 29803 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.004943 29735 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.005473 29816 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:18.869243 29791 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:19.005927 29816 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:19.006625 29791 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.007262 29816 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.008800 29816 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:18.873323 29711 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.810094 29844 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:25:18.797735 29675 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:25:18.923135 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3241) took 1236 ms. Trace:
I20260709 12:25:19.011417 20728 rpcz_store.cc:276] 0709 12:25:17.686309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.686478 (+   169us) service_pool.cc:224] Handling call
0709 12:25:18.922922 (+1236444us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b540028d554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:18.923066 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385937152}
I20260709 12:25:18.810384 29361 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.919087 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:19.012722 29361 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.820385 29898 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:19.013048 29834 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:18.834654 29680 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:19.014252 29680 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:18.869931 29904 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:19.012912 20808 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:19.018723 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.019479 20804 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:19.020920 20804 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:19.023167 29361 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.026644 20808 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:19.028106 29361 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:18.956482 20810 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:18.958316 21429 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
W20260709 12:25:18.961396 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.033190 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:19.033617 20806 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:18.961151 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.034435 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:18.824703 20876 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:19.035153 20801 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:25:18.979228 29930 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.036331 29930 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.037616 20801 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.15-31.15]
I20260709 12:25:18.959322 29559 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:18.867533 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.926412 29756 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:18.992301 21435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1651) took 2575 ms (client timeout 1756 ms). Trace:
I20260709 12:25:18.993413 21431 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:18.999322 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:18.996510 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 130840us
I20260709 12:25:19.001902 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:19.000653 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:19.006112 29735 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:18.757624 29900 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:18.960453 29641 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.043627 29677 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:19.043876 29864 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.044898 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:19.077286 20802 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:19.047538 29789 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:19.039029 20801 raft_consensus.cc:1217] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Deduplicated request from leader. Original: 23.13->[25.14-29.17]   Dedup: 26.15->[27.16-29.17]
I20260709 12:25:19.080479 20801 pending_rounds.cc:85] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 15
I20260709 12:25:19.051107 21430 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.080865 20801 pending_rounds.cc:107] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 28.16
I20260709 12:25:19.081383 20801 raft_consensus.cc:2889] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.027827 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:19.113842 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.055757 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:19.070055 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.069370 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:19.069703 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 69129.4us
W20260709 12:25:19.068317 21435 rpcz_store.cc:269] 0709 12:25:16.416776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.417000 (+   224us) service_pool.cc:224] Handling call
0709 12:25:18.992057 (+2575057us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b540050c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:18.992200 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318119168}
I20260709 12:25:19.076298 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:19.078641 20806 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [24.17-28.19]
W20260709 12:25:19.077382 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 78186.3us
I20260709 12:25:19.041065 20810 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:19.070243 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:19.035141 21429 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
W20260709 12:25:19.119467 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49297.4us
I20260709 12:25:19.120117 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:19.121603 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:19.124835 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.125524 20881 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:19.124673 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.126351 20877 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:19.127856 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.073232 29559 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:19.142716 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:19.142881 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.130195 21435 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:19.132748 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.128971 21428 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:19.141492 20806 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:25:19.145402 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.149230 20877 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:25:19.151167 20808 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:19.077021 29677 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:19.155532 29816 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:19.152220 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:19.154642 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:19.158758 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:19.155290 20802 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:19.115688 29675 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:19.143396 29404 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:19.171497 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:19.184415 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.113847 29857 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.059s
I20260709 12:25:19.174849 29816 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:19.148298 29559 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.155126 29638 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:19.116483 29834 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 12:25:19.193939 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2119) took 1511 ms. Trace:
I20260709 12:25:19.172482 29711 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.176388 29701 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:19.169878 29916 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:19.195941 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43814.8us
I20260709 12:25:19.176213 29930 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:19.191318 29119 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:19.203163 21432 rpcz_store.cc:276] 0709 12:25:17.667302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.667445 (+   143us) service_pool.cc:224] Handling call
0709 12:25:17.675669 (+  8224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.676110 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.676153 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.676775 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 12:25:18.921733 (+1244958us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b18000cd0c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1
0709 12:25:18.921900 (+   167us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:18.959134 (+ 37234us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.170229 (+211095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.170354 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.170393 (+    39us) raft_consensus.cc:1685] finished
0709 12:25:19.177376 (+  6983us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.178495 (+  1119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1745,"prepare.queue_time_us":66697,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":39203,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":16239,"spinlock_wait_cycles":8637184,"thread_start_us":284865,"threads_started":3,"wal-append.queue_time_us":97092,"wal-append.run_cpu_time_us":1865,"wal-append.run_wall_time_us":1114182}
I20260709 12:25:19.178172 29675 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:19.215803 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:19.209559 29929 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:19.196465 29404 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:19.224062 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:19.204677 29898 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:19.205600 29559 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:19.210193 29701 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.239596 29701 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:19.214923 29711 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.245743 29711 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.247100 29864 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.247514 29816 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.248216 29900 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.249511 20809 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:25:19.226503 29909 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:25:19.219568 29834 consensus_queue.cc:1243] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:19.237529 29361 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:19.220613 29916 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.220472 29675 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:19.216991 29332 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:19.213997 29844 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:19.224253 29875 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:19.222283 29119 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.262207 21435 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:19.218480 29680 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:19.229521 29930 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:19.231513 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:19.298475 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.346567 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 115120us
I20260709 12:25:19.349083 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:19.275398 29932 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:19.278504 29918 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:19.268187 29662 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.089s
W20260709 12:25:19.368276 29966 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:25:19.269821 29361 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.402657 29736 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:19.406929 29956 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.486s	user 0.001s	sys 0.001s
W20260709 12:25:19.410321 29955 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.500s	user 0.000s	sys 0.001s
I20260709 12:25:19.411301 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1657) took 1456 ms. Trace:
I20260709 12:25:19.411769 29641 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.368773 29638 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 12:25:19.412917 29950 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.356762 29960 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:19.286490 29844 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.278775 29332 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.413373 29950 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:19.345036 29680 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.411715 21434 rpcz_store.cc:276] 0709 12:25:17.955016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.533606 (+578590us) service_pool.cc:224] Handling call
0709 12:25:18.543259 (+  9653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.543614 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.543643 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:18.584774 (+ 41131us) log.cc:844] Serialized 26 byte log entry
0709 12:25:18.954445 (+369671us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:19.073693 (+119248us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.079996 (+  6303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.080079 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.406236 (+326157us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:19.410959 (+  4723us) raft_consensus.cc:1685] finished
0709 12:25:19.411059 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.411240 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12731,"prepare.queue_time_us":29250,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":10711,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":40803,"spinlock_wait_cycles":97665408,"thread_start_us":358875,"threads_started":2,"wal-append.queue_time_us":275337}
I20260709 12:25:19.435730 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.368284 29834 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.058s
W20260709 12:25:19.444691 29969 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:25:19.441470 29894 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:19.378393 29932 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.450877 29894 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.453244 29932 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:19.455168 29844 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.457262 20875 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:19.385021 29918 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.460505 29559 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.461263 29918 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:19.462759 29119 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:19.463449 29894 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.463599 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.464988 29846 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:19.465232 29361 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:19.466455 29559 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:19.376147 29404 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.433310 29960 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:19.488888 20804 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:25:19.491333 20804 pending_rounds.cc:85] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 14
I20260709 12:25:19.491629 20804 pending_rounds.cc:107] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:25:19.491849 20804 raft_consensus.cc:2889] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.493969 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.503899 29675 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:19.510012 20876 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:19.513031 29929 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:19.526401 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3489) took 3865 ms. Trace:
I20260709 12:25:19.526887 20731 rpcz_store.cc:276] 0709 12:25:15.661286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.661399 (+   113us) service_pool.cc:224] Handling call
0709 12:25:18.373354 (+2711955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.483767 (+110413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.039297 (+555530us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:19.526198 (+486901us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b540024a4d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:19.526341 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6449177216}
I20260709 12:25:19.558372 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.564339 29960 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.18, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.565346 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:19.565948 21428 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:19.571538 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:19.572170 20882 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:19.525499 29680 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:19.569496 29954 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.648s	user 0.002s	sys 0.000s
W20260709 12:25:19.581171 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.583544 20882 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:19.585110 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.585829 20882 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
W20260709 12:25:19.587987 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 239003us
I20260709 12:25:19.584985 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:19.633347 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48479.7us
I20260709 12:25:19.589006 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:19.634508 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.637315 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:19.642452 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:19.643071 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.648248 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:19.648918 20882 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:19.653065 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64163.6us
W20260709 12:25:19.653988 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.655501 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.657183 20727 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:19.658393 20731 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:19.575830 29837 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.665863 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:19.666695 20727 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:19.677868 20809 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:19.678097 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.678494 20809 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:19.680006 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:19.586369 29915 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:19.681190 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.681943 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.562777 29795 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.632462 29929 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.688407 20882 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:19.689440 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.692871 20731 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:25:19.693234 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:19.689746 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:19.697386 21428 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:19.659613 29559 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:19.696346 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:19.727382 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.664295 29361 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:19.631167 29916 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:19.679904 29932 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:19.681587 29681 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.743531 20810 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:19.681118 29675 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:19.790166 29795 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:19.791901 29975 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:25:19.741103 29958 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.790s	user 0.002s	sys 0.002s
I20260709 12:25:19.699338 29929 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.795873 29935 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:19.796300 29973 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:25:19.796430 29935 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.798502 20881 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:19.833855 21433 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:25:19.657647 29638 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 12:25:19.856030 29795 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.878836 29795 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.879868 29332 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:19.880913 29949 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.973s	user 0.003s	sys 0.000s
I20260709 12:25:19.881441 29795 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.882439 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2124) took 1946 ms. Trace:
I20260709 12:25:19.882854 21426 rpcz_store.cc:276] 0709 12:25:17.936041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.516068 (+580027us) service_pool.cc:224] Handling call
0709 12:25:18.519541 (+  3473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.529791 (+ 10250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.529826 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:18.532339 (+  2513us) log.cc:844] Serialized 26 byte log entry
0709 12:25:18.655808 (+123469us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:18.655874 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.655976 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.880280 (+1224304us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:19.882130 (+  1850us) raft_consensus.cc:1685] finished
0709 12:25:19.882152 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.882374 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3384,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2992,"spinlock_wait_cycles":129432832,"thread_start_us":110030,"threads_started":1,"wal-append.queue_time_us":282924}
I20260709 12:25:19.886984 21426 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:25:19.855748 29930 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:19.889376 29930 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.890856 29541 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.891462 29541 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:19.892635 20878 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:19.893430 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1655) took 3470 ms. Trace:
I20260709 12:25:19.894484 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3471) took 1295 ms. Trace:
I20260709 12:25:19.895033 20878 rpcz_store.cc:276] 0709 12:25:18.598724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.598845 (+   121us) service_pool.cc:224] Handling call
0709 12:25:19.894263 (+1295418us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540019c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:19.894407 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2933459712}
I20260709 12:25:19.680696 29837 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.893978 21431 rpcz_store.cc:276] 0709 12:25:16.422598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.998831 (+1576233us) service_pool.cc:224] Handling call
0709 12:25:18.993083 (+994252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.068519 (+ 75436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.890712 (+822193us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b540050c8d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:19.893351 (+  2639us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285606272}
W20260709 12:25:19.903070 20879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.039s	sys 0.085s
W20260709 12:25:19.835137 29970 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
W20260709 12:25:19.903689 20879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.039s	sys 0.086s
I20260709 12:25:19.737406 29361 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:19.695160 29915 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:19.905019 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 215344us
W20260709 12:25:19.856230 29974 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:25:19.833168 29907 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:19.839504 29904 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.835589 29916 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.737557 29701 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.857506 29932 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:19.924263 29894 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:19.918007 29907 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:19.921471 29559 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:19.936096 29559 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.939695 21431 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:19.941067 29915 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.941916 29662 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.942150 29915 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:19.942575 29915 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:19.942523 29662 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:19.945606 29964 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.578s	user 0.002s	sys 0.000s
I20260709 12:25:19.946218 29662 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.947573 29978 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.949262 29978 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.952003 29978 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:19.952572 29962 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.706s	user 0.007s	sys 0.003s
I20260709 12:25:19.960234 29404 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:19.961205 29404 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.965054 29774 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:19.968142 29874 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.968771 29874 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.970248 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:19.971287 21431 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:19.972290 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:19.972860 29874 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:19.974385 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:19.976276 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3518) took 1340 ms. Trace:
I20260709 12:25:19.976441 29404 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.977686 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3527) took 1106 ms. Trace:
I20260709 12:25:20.029865 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:19.981647 29837 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.934496 29907 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:19.965858 29774 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:19.989315 21434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.011s	sys 0.042s
I20260709 12:25:19.834930 29681 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.044277 29934 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:20.045143 29681 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:19.994558 21431 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
I20260709 12:25:19.996532 20807 rpcz_store.cc:276] 0709 12:25:18.635860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.636047 (+   187us) service_pool.cc:224] Handling call
0709 12:25:18.636323 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.116356 (+480033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.116398 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.116442 (+    44us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:19.116475 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.975965 (+859490us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54003b6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:19.976032 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.976202 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103487360}
I20260709 12:25:20.001175 20810 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:19.922729 29904 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:20.051558 29662 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.012s
I20260709 12:25:20.034410 29915 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.053504 29894 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:19.909057 29361 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:20.054471 29894 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.029500 20804 rpcz_store.cc:276] 0709 12:25:18.870925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.022840 (+151915us) service_pool.cc:224] Handling call
0709 12:25:19.023111 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.492260 (+469149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.977383 (+485123us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54003ca354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:19.977602 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068733696}
W20260709 12:25:20.040314 21434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.012s	sys 0.042s
I20260709 12:25:20.034493 20806 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.18-31.18]
I20260709 12:25:20.047134 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:20.061113 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:20.063328 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3328) took 1055 ms. Trace:
I20260709 12:25:20.063804 20801 rpcz_store.cc:276] 0709 12:25:19.008176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.038409 (+ 30233us) service_pool.cc:224] Handling call
0709 12:25:19.038780 (+   371us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:25:19.121552 (+ 82772us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:19.121587 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:19.139851 (+ 18264us) log.cc:844] Serialized 52 byte log entry
0709 12:25:19.217199 (+ 77348us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:19.217351 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.217423 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.063098 (+845675us) raft_consensus.cc:1685] finished
0709 12:25:20.063117 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.063266 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7508,"raft.queue_time_us":39969,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":17646,"thread_start_us":76951,"threads_started":1,"wal-append.queue_time_us":383888}
I20260709 12:25:20.030088 29795 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:19.925357 29916 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:20.098225 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 293437us
I20260709 12:25:20.092568 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:20.087082 29681 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:20.040995 29680 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:20.077281 29978 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:20.126690 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.161120 29939 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:20.161564 29939 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:20.103224 29837 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:20.139003 29948 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.331s	user 0.004s	sys 0.000s
I20260709 12:25:20.149305 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3330) took 1122 ms. Trace:
I20260709 12:25:20.175724 20808 rpcz_store.cc:276] 0709 12:25:19.026952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.027566 (+   614us) service_pool.cc:224] Handling call
0709 12:25:20.148996 (+1121430us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b5400126e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:20.149217 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84264832}
W20260709 12:25:20.153455 29951 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.257s	user 0.003s	sys 0.000s
W20260709 12:25:20.155822 29952 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.219s	user 0.004s	sys 0.000s
I20260709 12:25:20.159531 29979 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:20.160146 29935 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:20.161023 29978 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.135356 29932 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.164407 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:20.220444 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.228435 21431 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:20.226823 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2131) took 1528 ms. Trace:
I20260709 12:25:20.110340 29795 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.231364 21429 rpcz_store.cc:276] 0709 12:25:18.698590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.154292 (+455702us) service_pool.cc:224] Handling call
0709 12:25:19.566172 (+411880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.566367 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.566397 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.566444 (+    47us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:19.566481 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.566540 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.226554 (+660014us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400750150. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:20.226733 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892535424}
I20260709 12:25:20.162295 29680 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.121093 29681 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.179314 29837 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.235966 29774 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.225018 29874 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:25:20.245680 29701 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.278771 21429 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:25:20.286376 29983 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:20.286518 29986 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.274910 29844 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:25:20.287106 29986 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.289489 29986 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.290807 29986 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:20.291360 29986 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:20.292714 29681 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.292296 29986 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:20.298336 29904 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:20.304085 29982 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:20.307034 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1660) took 2320 ms. Trace:
I20260709 12:25:20.223526 29979 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.311110 21427 rpcz_store.cc:276] 0709 12:25:17.986542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.552231 (+565689us) service_pool.cc:224] Handling call
0709 12:25:18.552595 (+   364us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:18.553022 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.553051 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:18.553675 (+   624us) log.cc:844] Serialized 52 byte log entry
0709 12:25:18.635360 (+ 81685us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:18.702219 (+ 66859us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.758334 (+ 56115us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.767142 (+  8808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.767276 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.562123 (+794847us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b18001ef280. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:19.974822 (+412699us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b18001ef280. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:20.138198 (+163376us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:20.306737 (+168539us) raft_consensus.cc:1685] finished
0709 12:25:20.306767 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.306944 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38125,"prepare.queue_time_us":109764,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":3337,"raft.queue_time_us":38041,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":43071,"spinlock_wait_cycles":31780352,"thread_start_us":81268,"threads_started":1,"wal-append.queue_time_us":202779}
I20260709 12:25:20.272843 29939 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.327170 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2135) took 1346 ms. Trace:
W20260709 12:25:20.326570 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 391465us
W20260709 12:25:20.334029 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.322355 29950 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.878s	user 0.013s	sys 0.000s
I20260709 12:25:20.341198 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.322700 29701 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.345813 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:20.344295 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 222390us
I20260709 12:25:20.339278 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:20.313701 29361 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.349113 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:20.326483 29878 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.347445 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:20.351864 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:20.354626 29681 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:20.356333 29689 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:20.356674 29681 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.356922 29689 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:20.357162 29681 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:20.357410 29689 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:20.357756 29874 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:20.359216 29874 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:20.360338 29874 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:20.361307 29795 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:20.385242 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.362237 20882 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:20.363473 30004 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:20.364611 29837 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.366451 29681 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.369449 30003 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:20.369947 29904 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.352926 29701 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.370975 20879 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:20.408504 29826 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:20.409161 29826 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.410256 29681 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.410811 29979 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.371817 29874 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:20.411573 29874 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:20.353426 29784 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:20.412506 29784 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.376466 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.413224 30003 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:20.413431 20875 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:20.387753 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.414410 29680 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.355001 29939 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:20.389705 30004 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:20.411116 29701 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.344516 29950 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.412890 29784 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:20.415902 30004 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.417187 29950 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.417265 30004 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:20.414201 30003 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.418377 30003 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:20.419643 29995 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:20.372397 29878 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.420197 29995 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.404209 20808 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:20.421773 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.383589 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.423166 20806 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:20.357242 29978 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.383265 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:20.432443 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.431569 20881 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:20.432930 20701 connection.cc:256] RPC call timeout handler was delayed by -1.3487293019998106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:25:20.433717 20802 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:25:20.435417 20701 connection.cc:256] RPC call timeout handler was delayed by -1.2153536490000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:25:20.436306 20701 connection.cc:256] RPC call timeout handler was delayed by -1.1822859039998548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260709 12:25:20.396914 29689 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:20.437095 20701 connection.cc:256] RPC call timeout handler was delayed by -1.0699469429996498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
I20260709 12:25:20.437152 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.439144 20882 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:20.443320 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.444101 20802 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 21.
I20260709 12:25:20.444207 20881 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:20.445159 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.445822 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
W20260709 12:25:20.435468 29985 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
I20260709 12:25:20.449498 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.410288 29361 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.454571 20732 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:20.455857 29995 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.457420 29995 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:20.457610 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1682) took 1391 ms. Trace:
I20260709 12:25:20.457973 29995 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.457993 21426 rpcz_store.cc:276] 0709 12:25:19.065971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.887925 (+821954us) service_pool.cc:224] Handling call
0709 12:25:19.888262 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.910485 (+ 22223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.910518 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:19.910981 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.175866 (+264885us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:20.175904 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.175974 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.457392 (+281418us) raft_consensus.cc:1685] finished
0709 12:25:20.457417 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.457556 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33774,"raft.queue_time_us":35090,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":5817,"thread_start_us":242027,"threads_started":1,"wal-append.queue_time_us":509756}
I20260709 12:25:20.460076 29541 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:20.467325 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.468082 20732 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:20.471488 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3417) took 1206 ms. Trace:
I20260709 12:25:20.471935 20877 rpcz_store.cc:276] 0709 12:25:19.264762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.793605 (+528843us) service_pool.cc:224] Handling call
0709 12:25:19.794010 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.794424 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.794466 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:19.795015 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.231786 (+436771us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:20.245696 (+ 13910us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.273150 (+ 27454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.273251 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.471287 (+198036us) raft_consensus.cc:1685] finished
0709 12:25:20.471309 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.471431 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27040,"prepare.queue_time_us":13634,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":119746,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":3328,"thread_start_us":434912,"threads_started":1,"wal-append.queue_time_us":574790}
I20260709 12:25:20.470525 29681 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:20.474730 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.475855 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.476667 29681 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.449141 29997 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:25:20.471853 29361 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:20.497999 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 117548us
W20260709 12:25:20.501158 29967 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.254s	user 0.005s	sys 0.011s
I20260709 12:25:20.519408 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.520428 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:20.522171 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:20.522867 20802 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:20.523052 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:20.522200 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.522069 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.524871 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.523761 20733 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:20.501912 29689 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.525075 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3392) took 1918 ms. Trace:
I20260709 12:25:20.526585 20884 rpcz_store.cc:276] 0709 12:25:18.606356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.715206 (+108850us) service_pool.cc:224] Handling call
0709 12:25:18.715621 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.971073 (+255452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.971110 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.971159 (+    49us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:18.971235 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.524849 (+1553614us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54000a2dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:20.524894 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.525018 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586925312}
I20260709 12:25:20.527321 20726 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:20.526602 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.527977 29875 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:20.528445 29875 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:20.507903 29997 consensus_queue.cc:1243] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:20.511688 29681 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.523725 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:20.498121 29995 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.530424 20806 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:20.532651 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.508560 29878 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.527830 30010 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:25:20.536862 29874 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.513387 29784 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.539168 29874 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.541229 29784 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.541523 20732 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:20.543169 29929 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:20.549996 29946 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.901s	user 0.003s	sys 0.000s
I20260709 12:25:20.546886 29907 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:20.506388 29361 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.555861 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.556521 20735 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:20.560895 20802 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:20.561935 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:20.564927 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.567842 20803 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
I20260709 12:25:20.568753 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.565884 30004 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:20.569410 20735 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:20.572223 20730 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 33 because replica has last-logged OpId of term: 31 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 14.
I20260709 12:25:20.568111 29843 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:25:20.573822 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.574646 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:20.591357 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.591540 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.592114 20729 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:20.596117 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.596561 20733 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:20.597435 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.597532 29875 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.598006 20733 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:20.598174 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.598840 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.618395 29728 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.618580 20733 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:20.618904 29728 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.619529 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.619906 20884 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:20.619556 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.621263 29728 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.621766 20882 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:20.563202 29681 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:20.619820 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.595546 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.623071 20883 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:20.624150 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.625385 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.625921 20734 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:20.627893 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.631067 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.632576 20806 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:20.633657 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.633841 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.613708 21427 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
W20260709 12:25:20.636227 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.631742 20801 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:20.639267 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.599315 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.600101 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.600392 29361 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.604142 20729 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:20.569947 30010 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:25:20.608347 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:20.608515 29875 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.610273 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:20.649574 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.650178 20728 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:20.612672 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:20.612947 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:20.650835 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.651173 21426 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:20.651774 20884 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 24: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:20.652199 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.613193 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:20.613683 29857 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:20.617120 29660 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:20.617933 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.643433 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.657490 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:20.644270 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.644745 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.658329 20807 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:20.644897 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.646330 20731 raft_consensus.cc:2410] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:20.647887 20735 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:20.659655 20810 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:20.650606 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.660434 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.661084 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.661590 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.661728 20731 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:20.662564 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.649595 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.653184 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.664383 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:20.653496 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.665107 20733 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:20.665560 20883 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
W20260709 12:25:20.652231 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 43965.7us
I20260709 12:25:20.666375 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.667514 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:20.668291 20733 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 26: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:20.670336 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.674427 20731 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:20.674973 20733 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:20.664037 20882 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:20.664395 29660 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.678439 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:20.678814 30011 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:20.680351 29660 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.688794 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.689671 20728 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:20.673755 29875 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:20.662511 29857 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.656409 29843 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:20.660357 29728 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:20.692474 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 162494us
I20260709 12:25:20.692770 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:20.693325 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.702069 20729 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:20.663044 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:20.705262 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:20.706673 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:20.697082 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:20.692090 29929 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:20.707049 20734 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:20.707234 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:20.701042 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.708410 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:20.726173 20728 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
W20260709 12:25:20.725353 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.741173 20733 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:20.742888 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:20.710474 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:20.750790 20728 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:20.752156 20733 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: [23.16-23.16]
W20260709 12:25:20.755676 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47316.5us
I20260709 12:25:20.758510 21426 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:20.750917 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:20.762106 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 204096us
I20260709 12:25:20.763388 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:20.749750 29541 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:20.771709 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:20.771047 30017 raft_consensus.cc:2764] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:25:20.741503 29934 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:20.747375 29978 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:20.770278 21426 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:20.707966 29361 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:20.674183 29822 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:25:20.748099 29878 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:20.774147 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:20.764374 30003 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:20.792987 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:20.692088 29875 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:20.771061 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 63915.6us
I20260709 12:25:20.777529 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:20.783583 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73233.9us
I20260709 12:25:20.796819 20733 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:20.778296 29934 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:20.767957 29834 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:20.785181 29784 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:20.807794 29826 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.788298 30011 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:25:20.782383 30004 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:20.808519 30004 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:20.808678 29875 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.808980 30004 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:20.809523 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:20.749979 29846 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:25:20.795293 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:20.777894 29736 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:20.816800 20700 leader_election.cc:400] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
I20260709 12:25:20.817307 20700 leader_election.cc:403] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.784554 29978 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.794195 29878 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.786209 29997 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:20.760813 29894 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:25:20.783488 29900 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:20.764164 29675 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:25:20.748564 29728 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.833577 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.845839 20810 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:20.817871 29939 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:20.820704 29756 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:20.835865 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:20.812390 30016 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:20.812705 29979 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:20.813760 29934 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:20.779862 29935 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:25:20.844815 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:20.795861 30003 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.815562 29736 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.848063 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:20.834563 20807 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:20.850637 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:20.879048 20802 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:20.824139 29978 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:20.827210 29878 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:20.820048 29834 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.881134 20803 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:20.846802 29784 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
I20260709 12:25:20.848062 29939 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.880518 29736 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:20.890339 29784 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:20.891922 30011 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:20.892364 30011 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:20.893579 30011 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.895026 30011 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:20.895735 30014 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:25:20.898057 29736 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.864499 29541 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:20.868845 29979 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.877292 29935 consensus_queue.cc:1243] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:20.877584 30003 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:20.876116 29950 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:20.900676 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95101.7us
I20260709 12:25:20.880707 29756 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:20.876784 30016 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:20.899827 30020 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:20.852128 29900 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.887291 30019 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:20.901681 29939 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:20.925514 29736 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:20.909301 29960 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:20.908972 29979 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:20.331939 21435 rpcz_store.cc:276] 0709 12:25:18.980484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.355057 (+374573us) service_pool.cc:224] Handling call
0709 12:25:19.380478 (+ 25421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.381246 (+   768us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:19.381667 (+   421us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.985318 (+603651us) spinlock_profiling.cc:243] Waited 7.51 ms on lock 0x7b180049b940. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:25:20.326814 (+341496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:20.326870 (+    56us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:20.326918 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.326986 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.327111 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":1434,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5829,"spinlock_wait_cycles":17278720,"thread_start_us":341253,"threads_started":1}
I20260709 12:25:20.941753 29756 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.921185 30019 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:20.921272 29857 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:20.955294 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85189us
I20260709 12:25:20.942530 29960 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:20.944450 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:20.916939 29950 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:20.955840 30019 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:20.976091 29900 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.986425 20734 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:20.989679 29934 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:20.996758 29834 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:20.997699 29960 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.004030 29816 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:21.020042 29816 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.022419 29816 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:21.024256 29728 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.032657 29950 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:20.998958 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:20.999384 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2137) took 2018 ms. Trace:
I20260709 12:25:21.003779 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1677) took 2473 ms. Trace:
I20260709 12:25:20.960867 29689 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:20.982846 29916 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:20.947463 29361 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 12:25:20.992815 29986 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:21.050191 29934 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.054441 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:21.055356 20876 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:20.960289 29857 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.037322 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:21.040282 21428 rpcz_store.cc:276] 0709 12:25:18.981035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.901270 (+920235us) service_pool.cc:224] Handling call
0709 12:25:19.901747 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.041296 (+139549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:20.041329 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.041787 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.274016 (+232229us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:20.274071 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.998963 (+724892us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b540042ed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:20.999011 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.999069 (+    58us) raft_consensus.cc:1685] finished
0709 12:25:20.999083 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.999319 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56439,"raft.queue_time_us":60046,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13842,"spinlock_wait_cycles":220501888,"thread_start_us":170848,"threads_started":1,"wal-append.queue_time_us":336482}
W20260709 12:25:21.042065 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 227397us
I20260709 12:25:21.040414 21433 rpcz_store.cc:276] 0709 12:25:18.527301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.782330 (+255029us) service_pool.cc:224] Handling call
0709 12:25:18.783954 (+  1624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.834359 (+1050405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.000021 (+1165662us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:21.000265 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413579008}
I20260709 12:25:21.043311 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:21.071461 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1647) took 1110 ms. Trace:
I20260709 12:25:21.071904 21431 rpcz_store.cc:276] 0709 12:25:19.961170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.229496 (+268326us) service_pool.cc:224] Handling call
0709 12:25:20.229875 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.406971 (+177096us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:20.407018 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:20.407079 (+    61us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:20.407125 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.071223 (+664098us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:21.071276 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.071406 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16085,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5734,"spinlock_wait_cycles":406287744}
I20260709 12:25:21.081812 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:21.083143 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:21.084838 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:21.085412 21431 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:21.088166 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2136) took 2107 ms. Trace:
I20260709 12:25:21.088333 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.088646 21434 rpcz_store.cc:276] 0709 12:25:18.980776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.444718 (+463942us) service_pool.cc:224] Handling call
0709 12:25:19.445036 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.475295 (+ 30259us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:19.475568 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.088803 (+613235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:20.088854 (+    51us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:20.088914 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.087938 (+999024us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b5400645ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:21.087984 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.088102 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31906,"prepare.queue_time_us":4535,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":33761,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":4079,"spinlock_wait_cycles":69002624,"thread_start_us":580937,"threads_started":1,"wal-append.queue_time_us":700887}
I20260709 12:25:21.090531 21433 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:21.094197 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.031002 30003 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.104177 20876 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
W20260709 12:25:21.056931 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134397us
I20260709 12:25:21.105177 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.106959 21428 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:25:21.048439 30027 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.110638 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.111462 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:21.046872 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47968.9us
I20260709 12:25:21.121480 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.075727 29960 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:21.134188 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:21.130244 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.139544 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.140316 20810 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:21.141350 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.142663 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.125375 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.151166 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:21.152460 21433 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:21.154593 21434 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:21.155447 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.156618 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.095781 29774 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:21.095938 29987 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:21.171250 20802 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:21.111620 21428 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:20.968142 30016 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.165661 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:21.096094 29995 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:21.109416 30017 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:21.124229 29361 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.090626 29834 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 12:25:21.192667 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:21.187563 30034 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.193641 30025 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:21.157792 29932 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:25:21.170810 29987 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:21.168211 30011 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:21.196758 29978 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.197803 29929 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:21.145453 29934 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:21.199328 29361 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.159094 30004 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.167963 30032 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:25:21.202152 29979 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.144773 30027 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.319588 30027 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.320508 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.321724 29978 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.290990 29932 consensus_queue.cc:1243] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:21.291320 29904 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:21.326108 29904 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:21.326577 29904 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:21.329258 30019 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.331405 29934 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:21.332437 29934 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.274648 30034 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.339382 29875 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:21.339888 29875 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:21.340696 29875 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.341354 30034 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.341889 29875 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:21.116070 29843 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:25:21.217692 29930 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:21.228240 29935 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:21.232702 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2132) took 2400 ms. Trace:
I20260709 12:25:21.233018 29907 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:21.235041 30030 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:21.235095 29662 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:21.236514 29939 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.239230 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:21.240886 30003 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.242031 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:21.359308 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:21.360145 20884 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:21.242717 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:21.243446 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:21.082010 29857 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.253314 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:21.364847 20883 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:21.254419 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:21.256007 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:21.366371 20878 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:21.258491 20881 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:25:21.267899 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75317.1us
I20260709 12:25:21.368031 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:21.371235 20875 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:21.268460 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.372553 20803 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:18.448287 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:21.075362 29541 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:21.345651 29361 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.167835 29677 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.306151 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.259030 29728 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.309185 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.384780 20802 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:25:21.312543 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.315094 30004 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.291539 29987 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:21.345834 20806 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:21.349438 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.388262 20808 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:21.349001 21430 rpcz_store.cc:276] 0709 12:25:18.832227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.177147 (+344920us) service_pool.cc:224] Handling call
0709 12:25:19.192025 (+ 14878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.343966 (+151941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.344014 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.344074 (+    60us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:19.344133 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.232467 (+1888334us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:21.232512 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.232642 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348822400}
I20260709 12:25:21.356093 20876 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:25:21.348696 29935 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:25:21.297336 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.298524 30011 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.205859 29878 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.381161 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:21.352422 29907 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:21.385382 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:21.407984 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.409201 21400 connection.cc:256] RPC call timeout handler was delayed by -1.3190969190000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
W20260709 12:25:21.274132 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.411296 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:21.412057 20883 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:21.413049 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:21.418434 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:21.418735 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.384398 29900 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:21.420632 20734 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:21.423233 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:21.424662 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.424973 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.425441 20731 raft_consensus.cc:2376] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24. Re-sending same reply.
I20260709 12:25:21.425709 20734 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:21.426875 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:21.427223 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.386415 30019 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.429157 20731 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:21.435072 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:21.435703 21430 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 18
I20260709 12:25:21.436141 21430 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 27.19
I20260709 12:25:21.436456 21430 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.421742 29816 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:21.439311 29816 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.441582 29816 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.441941 29689 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:21.442147 29816 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.443257 30031 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:21.443758 30031 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:21.444162 30031 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:21.421648 30030 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:21.445240 30030 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:21.445638 30030 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:21.447299 30027 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:21.447780 30027 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:21.448171 30027 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:21.453819 29875 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:21.456959 30030 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.442703 29816 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.461404 29816 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.461544 30030 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.465153 29857 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.468032 30027 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.470646 30027 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.474006 29979 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:21.391947 29935 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:21.502691 29935 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.475744 29939 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:21.408651 29793 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:21.476696 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.430943 29900 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.477880 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.507488 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.508191 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.420708 29978 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.479035 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:21.509865 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 375744us
W20260709 12:25:21.506779 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 338807us
I20260709 12:25:21.480620 20727 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:21.511790 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.513226 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.514005 20727 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:21.490262 20729 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:21.516511 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.517360 20729 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:21.408586 29907 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:21.518800 20729 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:21.498919 20730 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:21.398895 29728 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.417111 29759 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.532687 20732 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:21.510200 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.533782 20726 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:21.528909 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.549846 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.484679 30031 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.560776 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:21.562772 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.525864 29900 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:21.562099 30039 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:21.577196 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:21.591840 20804 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:25:21.578071 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.486649 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:21.581374 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:21.593529 20732 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 26
W20260709 12:25:21.593789 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 107259us
I20260709 12:25:21.586834 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:21.595643 20730 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:21.598726 20883 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:21.537147 29677 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:21.525512 29662 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:21.605649 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.575790 20809 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.501044 29904 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.610025 20808 pending_rounds.cc:85] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 16
I20260709 12:25:21.549998 29979 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 12:25:21.608569 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:21.611248 20808 pending_rounds.cc:107] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:25:21.611647 20808 raft_consensus.cc:2889] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.616827 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:21.607813 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.617522 20883 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:21.618037 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:21.621116 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.621606 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:21.549789 29759 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.588393 30034 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:21.554433 30026 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:25:21.632076 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.536865 29978 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.627899 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:21.625934 20727 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:21.603274 29689 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:21.596939 29932 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:21.535956 29793 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.645900 20804 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
W20260709 12:25:21.655498 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54463.9us
I20260709 12:25:21.670168 29793 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.672261 21434 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:21.672664 29728 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.677920 29689 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.678632 29689 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:21.681352 29689 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.654472 29932 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.685467 29784 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:21.685948 29784 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:21.686244 29932 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:21.625418 30045 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.691390 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:21.692122 20804 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:21.614312 29662 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.653158 20883 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:21.702316 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 279164us
I20260709 12:25:21.713667 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:21.719020 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:21.723155 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.716478 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:21.725898 21427 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:25:21.718945 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.726725 21435 raft_consensus.cc:2368] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25.
I20260709 12:25:21.728482 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.729127 21434 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 18.
I20260709 12:25:21.726917 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.714669 29728 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.715715 29878 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.739066 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.732311 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:21.686836 29784 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.736112 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.724762 30057 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:21.744664 20731 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:21.730953 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.718565 30039 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.731760 20804 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:21.742345 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:21.745841 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.752182 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.750119 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.759279 21434 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:21.697597 30045 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.753602 29759 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.768132 21430 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
I20260709 12:25:21.755510 29904 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.737371 29756 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:21.749786 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:21.772264 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.746862 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:21.752533 30031 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.759743 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:21.751137 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:21.730337 30047 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:21.744108 30030 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:21.754606 29978 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.771422 21434 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:21.772948 21430 pending_rounds.cc:85] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 18
I20260709 12:25:21.788748 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3388) took 3184 ms. Trace:
I20260709 12:25:21.758447 30039 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.759342 30025 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.776196 20804 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:21.768247 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:21.780810 21427 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:21.795506 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:21.795864 21430 pending_rounds.cc:107] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:25:21.796222 21430 raft_consensus.cc:2889] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.796357 30047 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:21.765532 29875 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:21.796799 30047 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:21.797216 30047 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:21.761663 30057 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:21.739773 29728 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.798322 30057 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.799376 29728 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.770511 29878 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:21.800114 29878 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.802212 29878 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.803540 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.804260 20731 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:21.805505 30047 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.807655 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:21.808281 20804 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:21.778192 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:21.824388 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57490.4us
I20260709 12:25:21.825282 20804 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:21.825757 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:21.781837 29541 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:21.789237 20880 rpcz_store.cc:276] 0709 12:25:18.603875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.604056 (+   181us) service_pool.cc:224] Handling call
0709 12:25:18.604361 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.604698 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.604728 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:18.630551 (+ 25823us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.553379 (+1922828us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b18001220e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:21.787381 (+1234002us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b5400323320. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec09090 00007f9e1ec166b8 00007f9e1ec9f92b 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c
0709 12:25:21.787457 (+    76us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:21.787498 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.787580 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.787637 (+    57us) raft_consensus.cc:1685] finished
0709 12:25:21.787655 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.787810 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":432,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":25854,"spinlock_wait_cycles":107254016,"thread_start_us":32917,"threads_started":1,"wal-append.queue_time_us":1935387,"wal-append.run_cpu_time_us":2805,"wal-append.run_wall_time_us":1003172}
I20260709 12:25:21.833396 20804 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
W20260709 12:25:21.792874 30050 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:25:21.835951 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:21.839354 20727 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:21.834578 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56496.4us
I20260709 12:25:21.786309 29756 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:21.789127 29824 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:21.789063 29759 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:21.840950 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:21.831590 30049 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:25:21.829973 30041 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:25:21.837198 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:21.844069 29978 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:21.839510 30030 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:21.860921 29662 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:21.852320 29793 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.009s
W20260709 12:25:21.861847 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.851441 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:21.869844 29793 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:21.870592 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:21.871726 29736 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:21.872287 29736 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:21.872982 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:21.873768 29816 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:21.873337 29736 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.875362 29816 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:21.875761 29987 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:21.876873 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.878142 29987 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:21.879791 30056 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 12:25:21.881258 30055 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:25:21.882328 30054 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:21.882817 30039 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:21.841212 29929 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:25:21.884352 29907 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.843524 29541 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:21.886026 20806 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:21.886245 29541 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.850304 30027 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:21.887714 30027 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:21.888129 29907 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:21.888293 30006 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.888579 30027 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.838532 29837 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:21.889072 29756 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.889163 30006 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:21.829424 30025 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:21.829995 29932 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.021s
W20260709 12:25:21.896028 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.900652 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.642s (SENT) [suppressed 15 similar messages]
I20260709 12:25:21.871734 29793 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:21.893787 29736 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:21.917738 30047 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:21.928639 30025 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:21.936307 30048 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.292s	user 0.003s	sys 0.001s
I20260709 12:25:21.983785 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.986244 20731 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:25:21.986567 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.989037 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.989284 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1490) took 1381 ms. Trace:
I20260709 12:25:21.990149 20727 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:21.851500 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:21.996371 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3357) took 1163 ms. Trace:
W20260709 12:25:22.001542 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.003813 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 152183us
I20260709 12:25:22.004760 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:22.006493 20808 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:21.984390 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:21.989980 20733 rpcz_store.cc:276] 0709 12:25:20.607698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.756606 (+148908us) service_pool.cc:224] Handling call
0709 12:25:20.756938 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.797885 (+ 40947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.988550 (+1190665us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b5400226fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:21.989054 (+   504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90991488}
I20260709 12:25:21.907271 29793 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:22.019840 29793 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.022867 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.024376 20803 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:22.025920 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.031600 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:22.036010 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.038540 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:22.039877 20729 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:21.997124 20805 rpcz_store.cc:276] 0709 12:25:20.832939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.833168 (+   229us) service_pool.cc:224] Handling call
0709 12:25:20.833499 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.057964 (+224465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.058012 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.058642 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.259482 (+200840us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:21.260918 (+  1436us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.294155 (+ 33237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.294235 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.870513 (+576278us) spinlock_profiling.cc:243] Waited 9.73 ms on lock 0x7b18003be420. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:21.995246 (+124733us) raft_consensus.cc:1685] finished
0709 12:25:21.995470 (+   224us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.995994 (+   524us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14179,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":67627,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":10372,"spinlock_wait_cycles":22387456,"thread_start_us":424490,"threads_started":2,"wal-append.queue_time_us":362215}
I20260709 12:25:21.934970 30045 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.049197 29904 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:22.049829 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:22.050451 29904 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:22.059350 30025 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:22.064015 30025 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.065856 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.066983 20804 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:22.067293 30025 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:22.054971 30045 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:22.068349 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.071596 30057 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.073668 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:22.074676 29759 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.076748 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.077028 29903 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:22.071053 30039 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:22.079986 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.084811 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.085516 29857 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.088321 29915 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:22.093200 29935 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:22.137647 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.139467 30034 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:22.139273 30057 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.140658 20880 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:22.142189 30057 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:22.145733 29793 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.150321 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:22.155638 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82234.7us
I20260709 12:25:22.156363 29904 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:22.157083 29904 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:22.157848 29904 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:22.157707 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:22.159948 30032 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:22.160529 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.116681 29759 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:22.162382 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:22.124782 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.124344 20876 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:22.114952 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.125478 30039 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.166242 30039 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.127681 29857 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:22.131354 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
W20260709 12:25:22.131888 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 82479.1us
I20260709 12:25:22.167922 20730 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:22.109565 29903 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:22.127146 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.169759 20877 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
I20260709 12:25:22.169353 29903 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.160708 30032 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:22.172623 30032 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.175556 29903 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:22.178004 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.181320 20876 raft_consensus.cc:2368] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:25:22.182691 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:22.183832 21434 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:25:22.185389 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.186054 29793 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.187834 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.196521 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3355) took 1364 ms. Trace:
I20260709 12:25:22.188309 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.197002 20801 rpcz_store.cc:276] 0709 12:25:20.832302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.835612 (+  3310us) service_pool.cc:224] Handling call
0709 12:25:22.196042 (+1360430us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b5400127d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:22.196304 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1731319680}
W20260709 12:25:22.198434 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48527.5us
I20260709 12:25:22.198668 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.187129 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.169432 29857 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.182775 20808 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:22.202425 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:22.202577 20809 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:22.178252 30011 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.187667 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:22.217291 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.217284 30047 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:25:22.220103 20804 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:22.140018 30034 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:22.224435 30034 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.226300 30034 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:22.218750 20807 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:22.165014 20729 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:22.214996 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46714.3us
I20260709 12:25:22.227842 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1684) took 1106 ms. Trace:
I20260709 12:25:22.229956 20807 rpcz_store.cc:276] 0709 12:25:21.121027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.121158 (+   131us) service_pool.cc:224] Handling call
0709 12:25:22.205280 (+1084122us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aeee 000000000036eb39 00007f9e23bee1e0 00007f9e23cb70c4 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:25:22.205566 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.227528 (+ 21962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.227773 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4994560}
I20260709 12:25:22.234097 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:22.235136 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:22.235728 29960 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:22.236374 29960 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.236841 29960 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:22.246505 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.246332 29824 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:22.247640 20883 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:25:22.248792 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:22.249234 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 349140us
I20260709 12:25:22.249714 30047 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:22.250278 30047 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:22.250720 30047 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:22.251688 29932 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:22.252161 29932 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.252570 29932 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:22.254175 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.195103 29784 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:22.131021 30006 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.268489 20801 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:25:22.276854 20803 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:22.279320 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:22.281284 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.278280 29824 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:22.286208 29793 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:22.287062 29987 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:22.283496 20881 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:22.277298 30039 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:22.297114 20850 leader_election.cc:400] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25.
I20260709 12:25:22.297632 20850 leader_election.cc:403] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:22.291890 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54645.7us
I20260709 12:25:22.308449 20809 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:25:22.317402 29892 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:25:22.317505 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.295140 29736 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:22.295058 29987 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.325796 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.339579 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.340996 30011 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:22.341563 30072 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.096s	user 0.000s	sys 0.002s
I20260709 12:25:22.343995 20881 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:25:22.347229 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.348860 29997 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.318570 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3537) took 2678 ms. Trace:
I20260709 12:25:22.365579 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.377182 20880 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:22.367046 20809 rpcz_store.cc:276] 0709 12:25:19.640462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.640614 (+   152us) service_pool.cc:224] Handling call
0709 12:25:19.677741 (+ 37127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.510524 (+832783us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b4c000326c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1cf6ba20 00007f9e1cf7ca29 00007f9e1cf42644 00007f9e1cf69f2d 00007f9e1cf69b93 00007f9e1ec9a18b 00007f9e1ec9acb5 00007f9e1ec95f30 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416
0709 12:25:21.556504 (+1045980us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5400511ba0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1d06f 00007f9e1ec95f60 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62
0709 12:25:22.201830 (+645326us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5800433ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:25:22.313426 (+111596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.318459 (+  5033us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215640064}
W20260709 12:25:22.380699 29900 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.012s
W20260709 12:25:22.380821 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 105665us
I20260709 12:25:22.381389 29900 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.382493 29987 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.383669 20802 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:22.383513 30011 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.383576 29824 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25.
I20260709 12:25:22.324564 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:22.387660 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:22.388398 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97190.9us
I20260709 12:25:22.388620 29793 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:22.392702 29793 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:22.394493 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:22.395454 21434 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:22.394183 29793 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.383591 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.363688 29736 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.407902 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:22.363600 30039 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:22.387033 30011 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:22.444657 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.499259 29756 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.497750 30013 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.429412 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.442736 29960 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.503165 30013 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.505154 29960 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.423086 30047 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.506292 30013 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.507457 29756 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.508306 30047 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.440698 29904 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.453250 29736 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:22.472178 29824 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:25:22.515221 29756 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.433221 20808 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:22.449751 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:22.483096 29541 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:22.480760 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.488494 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:22.410354 29997 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.420202 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:22.503659 20807 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:22.515367 29824 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:22.427635 20809 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:22.485847 21429 raft_consensus.cc:1813] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:25:22.500792 21432 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:22.480059 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 96952.3us
I20260709 12:25:22.428326 20733 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:22.515367 21435 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:22.427742 20806 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:22.526047 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.428853 21433 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:22.452335 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:22.531608 21429 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
W20260709 12:25:22.519129 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 304) took 27445 ms (client timeout 15000 ms). Trace:
I20260709 12:25:22.482901 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:22.510957 29904 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.535794 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:22.523037 29824 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:22.538802 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.535513 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:22.469787 29900 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.536692 20734 raft_consensus.cc:2376] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26. Re-sending same reply.
I20260709 12:25:22.498800 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:22.544975 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.540164 21430 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
W20260709 12:25:22.545859 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 340787us
I20260709 12:25:22.546036 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.453502 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:22.545002 20802 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:22.556136 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.488315 29784 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:22.577190 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.581759 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.578157 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:22.576740 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:22.578130 20731 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:22.541692 30047 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:25:22.587121 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 167028us
I20260709 12:25:22.586699 20633 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 12 to -1, term changed from 19 to 9, leader changed from <none> to 657bc62d20374e539b23641715f22745 (127.19.36.194), NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) evicted. New cstate: current_term: 9 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:22.596637 20880 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:22.596364 29824 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.597836 29824 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:22.599781 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.538462 29987 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:22.425420 29932 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.586541 29784 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.489593 29759 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.615528 20883 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:22.618868 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:22.624225 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.499986 30039 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.627993 21434 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:22.628654 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:22.629685 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.632009 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43845.9us
I20260709 12:25:22.638796 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.641322 20806 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:25:22.643132 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.644591 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:22.645354 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70649.2us
I20260709 12:25:22.646181 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:22.647904 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.648877 20731 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:22.649834 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151177us
I20260709 12:25:22.650835 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:22.651826 29119 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:22.624256 30047 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:22.663915 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.665418 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:22.680215 29930 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:22.680799 29997 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.680998 29930 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.684324 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3546) took 1103 ms. Trace:
I20260709 12:25:22.684762 20726 rpcz_store.cc:276] 0709 12:25:21.580604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.583775 (+  3171us) service_pool.cc:224] Handling call
0709 12:25:21.991377 (+407602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.992146 (+   769us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:22.017977 (+ 25831us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.530610 (+512633us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:22.530755 (+   145us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:22.530814 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.530899 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.683177 (+152278us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b180027efe0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4
0709 12:25:22.684011 (+   834us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b5400226fd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:22.684148 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114418,"prepare.queue_time_us":25654,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":122287,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":12850,"spinlock_wait_cycles":1039722624,"thread_start_us":405105,"threads_started":2,"wal-append.queue_time_us":590339}
I20260709 12:25:22.688242 29784 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.689494 29541 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:22.689939 29541 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.690331 29541 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.691869 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2150) took 2079 ms. Trace:
I20260709 12:25:22.692271 21429 rpcz_store.cc:276] 0709 12:25:20.612240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.612387 (+   147us) service_pool.cc:224] Handling call
0709 12:25:22.691589 (+2079202us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54007b2ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:22.691794 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4219457408}
I20260709 12:25:22.695645 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:22.696839 21426 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:25:22.697217 21426 pending_rounds.cc:85] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 14
I20260709 12:25:22.697456 21426 pending_rounds.cc:107] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 22.15
I20260709 12:25:22.697810 21426 raft_consensus.cc:2917] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Aborting config change with OpId 22.15: Aborted: Op aborted by new leader
I20260709 12:25:22.698066 29904 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:22.698194 21426 raft_consensus.cc:2889] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.698504 29904 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:22.698943 29904 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:22.701078 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:22.703015 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.652542 29987 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.698887 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:42714 (request call id 16) took 4495 ms. Trace:
W20260709 12:25:22.709923 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69111.7us
I20260709 12:25:22.711848 20734 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:22.714419 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.715134 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:22.711388 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:22.715209 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:22.712602 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:22.718387 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:22.720109 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:22.727219 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.728078 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:22.714737 29916 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:22.676638 30034 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:25:22.736482 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:22.738941 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:22.742311 29875 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:22.532514 20628 rpcz_store.cc:269] 0709 12:24:55.081597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.082554 (+   957us) service_pool.cc:224] Handling call
0709 12:25:22.518891 (+27436337us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:13.922057 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:14.040308 (+118251us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b5c001a630c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24e91331 00007f9e24e91221 00007f9e24e8bf6c 00007f9e226e9d69 00007f9e226d50a6 00007f9e226d4d58 00007f9e226d99f1 00007f9e226d99a9 00007f9e226d9939 00007f9e226d9901 00007f9e226d8b2d
0709 12:25:14.040407 (+    99us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:14.040449 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:25:14.040461 (+    12us) tablet.cc:662] Decoding operations
0709 12:25:14.327737 (+287276us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:14.327834 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:14.327867 (+    33us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:25:14.330449 (+  2582us) tablet.cc:701] Row locks acquired
0709 12:25:14.330506 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:25:14.330687 (+   181us) write_op.cc:270] Start()
0709 12:25:14.330929 (+   242us) write_op.cc:276] Timestamp: P: 1783599914330616 usec, L: 0
0709 12:25:14.330969 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:25:14.352773 (+ 21804us) log.cc:844] Serialized 11487 byte log entry
0709 12:25:15.433266 (+1080493us) op_driver.cc:464] REPLICATION: finished
0709 12:25:15.613040 (+179774us) write_op.cc:301] APPLY: starting
0709 12:25:15.628821 (+ 15781us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:16.522803 (+893982us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:16.522839 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:16.905226 (+382387us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:16.956675 (+ 51449us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 12:25:16.956738 (+    63us) write_op.cc:312] APPLY: finished
0709 12:25:16.957403 (+   665us) log.cc:844] Serialized 252 byte log entry
0709 12:25:16.968255 (+ 10852us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:16.968502 (+   247us) write_op.cc:454] Released schema lock
0709 12:25:16.980570 (+ 12068us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:17.012951 (+ 32381us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:17.012983 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2018704,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":163144,"apply.run_cpu_time_us":48132,"apply.run_wall_time_us":1398357,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125347,"cfile_init":1,"lbm_read_time_us":418,"lbm_reads_lt_1ms":4,"num_ops":29,"prepare.queue_time_us":261325,"prepare.run_cpu_time_us":13306,"prepare.run_wall_time_us":432095,"replication_time_us":1102092,"spinlock_wait_cycles":124542848,"thread_start_us":286954,"threads_started":3,"wal-append.queue_time_us":45434,"wal-append.run_cpu_time_us":2308,"wal-append.run_wall_time_us":1115110}]]}
I20260709 12:25:22.757912 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:22.733836 29997 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:22.712018 21426 rpcz_store.cc:276] 0709 12:25:18.203521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.727477 (+523956us) service_pool.cc:224] Handling call
0709 12:25:19.048931 (+321454us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b54007b32a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1c520 00007f9e1eca5b42 00007f9e1eca4800 00007f9e1ec9d9fa 00007f9e23c654c9 00007f9e1d1e1c54 00007f9e1d1e1be2 00007f9e1d1e1b11 00007f9e1d1e1a8c 00007f9e1d1e0d42
0709 12:25:19.121854 (+ 72923us) log.cc:844] Serialized 501 byte log entry
0709 12:25:19.959669 (+837815us) spinlock_profiling.cc:243] Waited 981 us on lock 0x7b54007b32a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec18142 00007f9e1ebf8eaa 00007f9e1ebffd53 00007f9e1ebffcd9 00007f9e1ebffc69 00007f9e1ebffc31 00007f9e1ebfedfd 0000000000388634 0000000000388479 00007f9e1bbb2475
0709 12:25:20.053930 (+ 94261us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b54007b32a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1fb26 00007f9e1ec1949a 00007f9e1ec18714 00007f9e1ebf8eaa 00007f9e1ebffd53 00007f9e1ebffcd9 00007f9e1ebffc69 00007f9e1ebffc31 00007f9e1ebfedfd 0000000000388634
0709 12:25:21.302412 (+1248482us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b54007b32a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1fb26 00007f9e1ec1949a 00007f9e1ec18714 00007f9e1ebf8eaa 00007f9e1ebffd53 00007f9e1ebffcd9 00007f9e1ebffc69 00007f9e1ebffc31 00007f9e1ebfedfd 0000000000388634
0709 12:25:22.698784 (+1396372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23577,"raft.queue_time_us":1084,"raft.run_cpu_time_us":12083,"raft.run_wall_time_us":3078468,"spinlock_wait_cycles":396267392,"thread_start_us":52686,"threads_started":1,"wal-append.queue_time_us":134497,"wal-append.run_cpu_time_us":3875,"wal-append.run_wall_time_us":1626929}
I20260709 12:25:22.746614 30057 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:22.718284 29987 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.746506 29793 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:22.748433 29900 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:22.777935 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.778628 20806 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:22.748057 29960 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.748167 30047 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:22.785175 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 113568us
I20260709 12:25:22.771762 30057 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.802338 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:22.762888 29997 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:22.802446 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82067.5us
I20260709 12:25:22.780279 29932 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:22.798481 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:22.802877 29903 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:22.809872 29997 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.806980 30057 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:22.838074 30027 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:25:22.839672 29784 raft_consensus.cc:2764] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:25:22.833333 29930 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:22.840816 29930 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:22.800590 29960 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:22.854424 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.855331 30091 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:22.855929 20809 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:22.856455 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3552) took 1035 ms. Trace:
I20260709 12:25:22.856683 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.857452 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.858234 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:22.861255 29759 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.862071 30032 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:22.810688 29675 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.150s
W20260709 12:25:22.866693 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 12:25:22.868254 29987 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:22.870666 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1655) took 1768 ms. Trace:
I20260709 12:25:22.944765 21426 rpcz_store.cc:276] 0709 12:25:21.102006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.102143 (+   137us) service_pool.cc:224] Handling call
0709 12:25:22.696258 (+1594115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.843486 (+147228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.870433 (+ 26947us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54007b2ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:22.870595 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3563807488}
I20260709 12:25:22.821354 29916 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.065s
I20260709 12:25:22.885871 29736 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.887140 29932 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:25:22.887470 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:22.956566 20733 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.951656 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:22.929184 20801 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:22.837684 29903 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:22.929975 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 220689us
I20260709 12:25:22.821544 29900 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.923753 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:22.887231 30069 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 12:25:22.963056 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 306293us
W20260709 12:25:22.943248 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.947530 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 81047.5us
I20260709 12:25:22.922713 30091 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:22.969558 30078 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.164s	user 0.002s	sys 0.004s
W20260709 12:25:22.975318 30071 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.350s	user 0.008s	sys 0.000s
I20260709 12:25:22.928046 29793 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:25:22.981168 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:22.982259 20727 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:22.983351 29987 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:22.983939 29987 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.984396 29987 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:22.989637 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3555) took 1417 ms. Trace:
I20260709 12:25:22.992197 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:22.994036 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:22.998411 30091 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:22.950538 30013 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.053s
I20260709 12:25:23.004489 30013 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:23.004957 30013 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:23.005947 30013 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.007365 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:23.008543 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49166.2us
I20260709 12:25:23.009433 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:23.010288 29915 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:23.010757 29915 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:23.011139 29119 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:23.011250 29915 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:23.012212 29119 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:22.923485 20728 rpcz_store.cc:276] 0709 12:25:21.735755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.740233 (+  4478us) service_pool.cc:224] Handling call
0709 12:25:21.752330 (+ 12097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.760161 (+  7831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.760190 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.760690 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.140693 (+380003us) spinlock_profiling.cc:243] Waited 915 us on lock 0x7b18000e4460. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:22.616619 (+475926us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:22.622008 (+  5389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.628451 (+  6443us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.742991 (+114540us) raft_consensus.cc:1685] finished
0709 12:25:22.743030 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.771237 (+ 28207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7439,"raft.queue_time_us":7568,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":5839,"spinlock_wait_cycles":2104704,"thread_start_us":405225,"threads_started":1,"wal-append.queue_time_us":944079}
I20260709 12:25:23.016863 20726 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:23.004621 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:23.021960 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98355.7us
I20260709 12:25:23.023061 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:22.995107 20729 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:22.950860 29675 consensus_queue.cc:1243] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:25:22.961293 29903 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:23.055787 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:22.971541 29736 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:23.060914 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 201905us
I20260709 12:25:23.062916 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:23.068535 20731 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:23.049117 29857 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:22.990252 20810 rpcz_store.cc:276] 0709 12:25:21.572574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.576265 (+  3691us) service_pool.cc:224] Handling call
0709 12:25:21.576584 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.583086 (+  6502us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:21.609556 (+ 26470us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.040402 (+430846us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b1800348dc0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:25:22.125226 (+ 84824us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b1800348dc0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:22.641375 (+516149us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.675895 (+ 34520us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:22.729297 (+ 53402us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.729460 (+   163us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:22.729501 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.729603 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.875872 (+146269us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:22.989083 (+113211us) raft_consensus.cc:1685] finished
0709 12:25:22.989117 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.989405 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12435,"prepare.queue_time_us":44695,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":22862,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":14552,"spinlock_wait_cycles":45631872,"thread_start_us":545658,"threads_started":2,"wal-append.queue_time_us":981687}
I20260709 12:25:23.034807 30013 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:23.085371 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.076961 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:23.030216 29935 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.053s
W20260709 12:25:23.099434 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.104266 29784 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:23.105392 29784 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:23.105719 29784 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:23.090191 29119 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:23.062610 30057 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.126257 20879 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:23.106637 29784 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:23.129067 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111249us
W20260709 12:25:23.210393 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.210579 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.211390 20882 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:25:23.211702 29784 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:23.087378 29904 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:23.114895 29824 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.005s
I20260709 12:25:23.145711 30034 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.257480 30084 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:23.267841 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.291404 29960 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.302712 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:23.304136 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65766.5us
I20260709 12:25:23.305207 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:23.309124 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 174646us
I20260709 12:25:23.313060 29904 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.317765 30057 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.337972 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:23.348356 29759 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:23.349740 29916 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:23.354635 30089 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.113s	user 0.000s	sys 0.004s
I20260709 12:25:23.354976 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.355778 20879 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26.
I20260709 12:25:23.363538 20733 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
W20260709 12:25:23.351421 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.070s	sys 0.174s
W20260709 12:25:23.365054 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.071s	sys 0.175s
W20260709 12:25:23.365470 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 549190us
I20260709 12:25:23.368299 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:23.374152 20732 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:23.369261 20726 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
W20260709 12:25:23.376228 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65165.3us
I20260709 12:25:23.393810 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:23.418655 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 358491us
I20260709 12:25:23.353889 30034 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.383584 29904 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:23.432781 29939 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:25:23.433665 21433 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:25:23.434131 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:23.390321 29857 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:23.435276 29857 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.437754 29857 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.434788 21429 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:23.428236 20879 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:23.447165 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.451588 20882 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
W20260709 12:25:23.452320 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 147267us
I20260709 12:25:23.420686 29892 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:23.454756 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 368632us
I20260709 12:25:23.428623 29904 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.441219 30034 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.461376 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:23.462453 29875 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:23.463696 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:23.473258 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:23.473371 30084 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:25:23.475492 30095 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.231s	user 0.000s	sys 0.002s
I20260709 12:25:23.496258 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:23.479176 29119 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:23.497639 30032 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:23.498117 30032 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:23.498517 30032 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:23.498772 21426 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:23.498915 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:23.500119 20804 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
W20260709 12:25:23.476881 29541 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.786s	user 0.013s	sys 0.000s
I20260709 12:25:23.501209 29541 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.472478 20810 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:25:23.504062 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:23.480433 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 119861us
W20260709 12:25:23.483953 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.483989 21429 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
W20260709 12:25:23.457033 29997 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 12:25:23.485134 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:23.506829 20808 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:23.485594 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:23.486145 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.485081 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:23.490100 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:23.488267 21433 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:23.492007 20728 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:23.492113 20735 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:25:23.492682 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:23.510172 20729 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 33
I20260709 12:25:23.492506 29892 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:23.493263 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:23.449287 29987 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:23.505537 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.508070 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 232620us
W20260709 12:25:23.511981 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.521755 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 577135us
I20260709 12:25:23.524264 20801 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:23.484066 29904 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:23.527611 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:23.525661 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:23.530009 20727 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:23.530735 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:23.531472 20732 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:23.531649 20809 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:25:23.532625 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:23.521504 29915 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 12:25:23.525720 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:23.535862 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.530858 29892 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:23.505368 30094 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:23.544857 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:23.546380 20731 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:23.552942 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:23.550601 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:23.557216 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:23.557978 20727 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:23.551613 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:23.527870 29736 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:23.574235 20879 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:23.576890 20726 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:23.574193 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:23.546279 29784 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.627427 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53350.1us
I20260709 12:25:23.525400 29997 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:23.627839 20881 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:23.630533 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:23.582278 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:23.630659 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.553902 29892 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:23.628358 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:23.549793 29119 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:23.633414 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83694.3us
I20260709 12:25:23.646541 20732 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:23.654498 20808 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:25:23.655727 20877 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:23.655807 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.546049 29915 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.659690 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:23.564924 30094 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:23.662113 20880 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:23.664784 20803 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:23.561451 29541 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.672206 20878 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:25:23.574595 30100 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:23.636519 29932 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:23.659510 29950 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.701212 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52091.5us
I20260709 12:25:23.658775 29756 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:23.703537 21434 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:23.701517 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:23.706949 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.705304 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 162667us
W20260709 12:25:23.701444 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49935.6us
I20260709 12:25:23.684631 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:23.715775 20878 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:25:23.716099 20729 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
W20260709 12:25:23.679937 29900 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.001s
I20260709 12:25:23.672672 29892 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:23.721151 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:23.720968 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 241481us
I20260709 12:25:23.674005 30034 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:23.743161 20880 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:23.713388 29756 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.702203 29930 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:25:23.759337 20881 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:23.673542 29736 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:23.742909 29903 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.003s
I20260709 12:25:23.675397 29784 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.756644 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:23.742388 29900 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:23.774933 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 84054us
W20260709 12:25:23.778548 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63947.8us
I20260709 12:25:23.760315 29756 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:23.781121 20878 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:23.785964 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:23.765527 30106 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:23.807165 20733 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:23.687747 30100 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.716802 29950 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:23.805132 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:23.777578 30027 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:23.780141 29930 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.787590 29987 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.807121 30084 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:23.824893 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3370) took 1003 ms. Trace:
I20260709 12:25:23.825450 29950 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:23.825336 20803 rpcz_store.cc:276] 0709 12:25:22.821801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.821928 (+   127us) service_pool.cc:224] Handling call
0709 12:25:22.831881 (+  9953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.665613 (+833732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.824646 (+159033us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54003d5ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:23.824831 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1914910976}
I20260709 12:25:23.808382 29935 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:25:23.809679 29857 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
W20260709 12:25:23.827728 30104 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:25:23.828363 29735 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:25:23.777156 29903 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.830237 29997 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:23.812206 30099 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:23.831576 30099 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:23.831694 29915 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:23.832031 30099 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:23.820627 29930 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:23.835894 30102 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:25:23.836584 29784 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.837711 29930 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:23.838409 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:23.839238 21430 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:23.843953 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:23.820878 30106 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.814870 29904 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.846408 30106 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.852523 20727 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
W20260709 12:25:23.852918 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73701.2us
I20260709 12:25:23.808151 29939 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 12:25:23.854316 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:23.814406 30103 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.803949 29541 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:25:23.865020 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:23.866024 21434 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:23.847288 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:23.871673 20806 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:23.864475 29701 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:23.851992 29759 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.883038 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66127.8us
I20260709 12:25:23.884249 21399 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26.
I20260709 12:25:23.884685 21399 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:23.881824 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:23.816777 30116 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:25:23.887974 29915 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:23.894170 29915 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:23.894774 29915 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:23.885296 30084 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:23.899693 30084 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.900297 30084 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:23.906494 30103 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:23.907684 30100 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.909256 30103 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:23.808144 29892 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.910550 29119 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:23.912453 29903 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:23.885474 29759 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.918068 29759 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:23.919521 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:23.922876 30106 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:23.923488 30110 raft_consensus.cc:2764] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:25:23.923645 30106 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:23.924250 30106 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:23.926612 30106 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:23.891129 30107 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:23.892109 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:23.928421 30032 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.930354 29904 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:23.931627 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52302us
I20260709 12:25:23.935413 20802 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:23.936053 30097 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:23.936625 30097 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:23.937601 30097 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:23.938408 30108 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.121s	user 0.001s	sys 0.002s
I20260709 12:25:23.939180 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:23.943017 30091 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:23.943909 30047 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:23.944339 30047 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:23.945235 30047 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:23.810635 30094 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:23.946497 30047 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:23.949312 29736 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:23.950486 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95869us
W20260709 12:25:23.935331 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 144996us
I20260709 12:25:23.930917 30032 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:23.962330 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70320.6us
I20260709 12:25:23.965767 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:23.966485 21426 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:23.967603 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:23.971560 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:23.972294 20733 raft_consensus.cc:2376] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23. Re-sending same reply.
W20260709 12:25:23.972869 30113 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:25:23.973796 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:23.977834 29915 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:23.952205 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:23.980149 29915 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:23.983605 29736 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:23.984236 29736 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:23.980178 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:23.993685 20734 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:23.991077 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:23.996816 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:23.998685 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:24.001060 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 117012us
I20260709 12:25:24.001961 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:24.006976 20802 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:25:24.007599 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:24.009248 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:24.010075 20803 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:24.009456 20804 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:24.012020 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:24.008894 29915 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:25:24.014132 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:24.015050 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:24.015391 29759 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:24.016127 29759 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:23.996575 29701 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.017657 20733 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:25:23.998278 29784 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:24.026644 20729 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:23.962456 30119 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:24.023270 30099 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.011127 29903 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:24.038391 29903 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:24.009102 30116 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:24.005985 30100 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:24.046031 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:24.033712 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:24.050277 20727 raft_consensus.cc:2368] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24.
I20260709 12:25:24.034508 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:24.036906 20733 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:24.016589 29759 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:24.068622 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66731.7us
I20260709 12:25:24.070142 20732 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:25:24.032267 30119 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:24.099418 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55945.4us
I20260709 12:25:24.100561 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:24.026485 29119 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:25:24.103063 20726 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:24.029234 29784 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:24.104842 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.105907 20730 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:24.107051 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:24.042794 29904 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:23.990739 29736 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.110312 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:24.114037 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:24.043274 30116 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:24.102865 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:24.046196 30100 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:24.039263 29793 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:25:24.140187 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:24.146054 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:24.108492 29784 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:24.103996 30119 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:24.156862 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:24.141072 29950 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:24.109839 29857 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:24.164743 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 143891us
I20260709 12:25:24.166069 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:24.143273 30100 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:24.140493 30116 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:24.136132 29930 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.181602 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:24.171705 20728 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:24.032527 29701 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:24.173307 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 366669us
I20260709 12:25:24.140816 30103 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.155256 29892 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:24.186208 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:24.188705 20732 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:24.138644 29997 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
W20260709 12:25:24.200377 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43641.5us
W20260709 12:25:24.199940 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 125540us
I20260709 12:25:24.180938 30034 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.194715 29735 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:24.163602 29960 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.024s
I20260709 12:25:24.132936 29903 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.193969 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:24.208144 30094 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:24.216033 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:24.189949 29875 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:24.179756 29857 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.188621 29930 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.221148 30034 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.187796 29541 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:25:24.190122 29950 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:25:24.222817 30094 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:24.225101 29935 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:24.187407 30084 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:25:24.268174 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 96418.2us
I20260709 12:25:24.270267 29903 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:24.278463 29930 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:24.274111 30094 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:24.303206 29950 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:24.303954 30123 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:25:24.304577 30129 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:25:24.304872 30122 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.118s	user 0.002s	sys 0.000s
I20260709 12:25:24.291604 29559 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:24.225368 29759 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.208115 30103 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.240060 30130 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:24.307085 20726 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 12:25:24.315065 29756 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.004s
I20260709 12:25:24.274937 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:24.313140 20728 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
W20260709 12:25:24.314043 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48628us
I20260709 12:25:24.268278 29119 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:24.239931 29939 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:25:24.273687 29935 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.338589 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:24.339581 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:24.340355 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:24.345256 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 680696us
W20260709 12:25:24.349216 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 133252us
I20260709 12:25:24.493808 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:24.466265 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:24.435056 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 217311us
I20260709 12:25:24.350463 29559 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:24.488940 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 137492us
W20260709 12:25:24.497007 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.500159 20735 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:24.503247 21434 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:24.502296 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 227410us
I20260709 12:25:24.505435 21433 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:24.509225 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:24.498348 30124 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.227s	user 0.000s	sys 0.011s
W20260709 12:25:24.522720 30128 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:25:24.523793 30136 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:24.524258 30126 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.258s	user 0.003s	sys 0.005s
I20260709 12:25:24.524448 30136 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.526875 30136 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:24.527915 29857 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:24.499348 29935 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:24.490145 29541 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:24.492280 29736 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.502956 30013 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.383155 29756 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:24.571808 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59816.4us
I20260709 12:25:24.409476 30130 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:24.580271 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 114254us
W20260709 12:25:24.580394 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 52559.9us
I20260709 12:25:24.494330 29119 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.503964 30119 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.602739 20806 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: [23.18-23.18]
W20260709 12:25:24.606331 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.574569 29701 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:24.632249 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:24.632041 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 122913us
I20260709 12:25:24.586126 29903 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:24.596827 29987 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.671504 29987 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.672812 30119 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:24.673444 30119 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.673835 29987 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:24.673942 30119 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.580844 29541 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.676795 20733 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 28 index: 13. (index mismatch)
I20260709 12:25:24.676966 30116 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:24.679837 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:24.680444 30084 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:24.680666 20728 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:24.682152 30100 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.632817 29559 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.685606 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:24.686883 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:24.640591 29756 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:24.640918 29119 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:24.641227 30103 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:24.641639 30047 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.695s	user 0.009s	sys 0.006s
I20260709 12:25:24.593741 30013 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.642491 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:24.647260 30135 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:24.636730 29939 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:25:24.647734 30121 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:24.647827 20807 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 28 index: 13. (index mismatch)
W20260709 12:25:24.648636 29915 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.001s
I20260709 12:25:24.648973 30120 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.649828 30094 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.739152 21428 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:24.605515 29735 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:25:24.654929 30131 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:25:24.658730 30134 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.238s
I20260709 12:25:24.659765 29784 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.661059 30130 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:24.575722 29736 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.629973 30152 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.661867 30151 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.662230 29930 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.663008 30150 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:24.678856 29541 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:24.691125 20735 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:24.711648 20810 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:24.737527 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:24.768172 21429 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:24.739355 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:24.742954 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 249210us
I20260709 12:25:24.637517 29903 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.671563 29875 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:24.747591 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:24.747907 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:24.748260 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:24.745743 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:24.748998 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:24.750213 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:24.753651 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:24.754042 20809 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:24.893980 29935 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.756004 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:24.747126 29915 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26.
I20260709 12:25:24.764801 20734 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:24.744776 29735 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.088531 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:25.088490 29735 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:25.091383 29939 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:25.092003 29939 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.094300 29939 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.094962 30147 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:25.086020 29904 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:25.095504 30147 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.095912 30147 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:25.095839 29904 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.096421 30099 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.098032 30034 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.098508 29904 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:24.772030 20878 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 19: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:25.100260 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:25.100924 20878 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:25.101385 29784 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.062630 30109 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.101840 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:25.102229 30109 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.103154 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:25.117646 20878 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:25.119153 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:25.114976 21428 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:25.110069 21429 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
W20260709 12:25:25.041180 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.105993 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 397895us
I20260709 12:25:25.005798 30159 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:24.935374 30135 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.762440 30152 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.773800 30130 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.097s
I20260709 12:25:24.854384 30146 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.874329 29541 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:24.756102 29559 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:24.874421 30153 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.768883 30151 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:24.873485 30138 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:25:24.877154 30013 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.137920 29904 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:25.035331 20801 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:24.834705 29930 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:24.773144 29903 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:24.773378 29875 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:25.180012 29875 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.181214 29875 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:24.872375 29824 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:25.183504 29824 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.188026 21430 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:25.034513 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:24.856966 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 87487.2us
I20260709 12:25:25.189615 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:25.190156 30152 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.190784 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:24.920251 29736 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:24.995553 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:25.143843 30157 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.846746 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:24.918503 30144 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:25:25.063298 21434 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:25.062935 20876 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:25:25.069346 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 170893us
I20260709 12:25:25.228410 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:25.229453 20876 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:25.060403 30100 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:24.917006 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:24.974117 30119 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:24.918393 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:25.230931 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:24.759589 30116 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:24.754741 21435 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:25.252745 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.156198 30150 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:24.921798 30094 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:25.145016 20879 raft_consensus.cc:2368] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33.
I20260709 12:25:25.111688 30132 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:25.136476 29784 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:24.758409 30057 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:25.104692 30109 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.170387 20878 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:25.172178 29997 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:25.152838 29735 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.173835 29930 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.285827 20878 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:25.162377 29904 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:25.149780 30159 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:24.858397 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 87773.2us
I20260709 12:25:25.192270 20732 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:25.193321 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:25.196689 30155 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:24.917857 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:25.306109 20884 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:25.210515 20877 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:25.150738 29541 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:25.115860 30147 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.231041 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:25.232182 20699 leader_election.cc:400] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24.
I20260709 12:25:24.744760 30120 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.112278 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:25.315392 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:25.206698 30153 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.253494 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.252449 29736 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:25.254501 30152 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:25.303961 29904 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:25.209559 30013 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:25:25.309136 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:25.310282 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 117035us
I20260709 12:25:25.341243 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:25.311264 20735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1523) took 1222 ms (client timeout 1550 ms). Trace:
I20260709 12:25:25.162716 29559 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.352674 20732 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:25.300684 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:25.315603 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 204118us
I20260709 12:25:25.274688 30132 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:25.263459 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:25.265530 20730 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:25.316717 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:25.154745 30135 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:25.319226 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 165023us
I20260709 12:25:25.253039 30157 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.252647 30160 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 12:25:25.156646 30146 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.288497 30094 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.331008 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:25.362643 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:25.362785 21426 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:25.301802 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:25.364041 21428 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:25.289448 30116 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:25.356534 20727 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:25.317267 30145 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:25.281965 30057 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.321812 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:25.327934 30156 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:25.277189 30150 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.356189 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2198) took 1012 ms. Trace:
I20260709 12:25:25.355760 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:25.350770 20735 rpcz_store.cc:269] 0709 12:25:24.071224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.071333 (+   109us) service_pool.cc:224] Handling call
0709 12:25:25.293814 (+1222481us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540028d054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:25.293961 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":834237824}
I20260709 12:25:25.384838 21430 rpcz_store.cc:276] 0709 12:25:24.338559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.339235 (+   676us) service_pool.cc:224] Handling call
0709 12:25:25.336363 (+997128us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:25.351353 (+ 14990us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369510272}
I20260709 12:25:25.350013 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:25.367810 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3581) took 1260 ms. Trace:
W20260709 12:25:25.368564 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68001.5us
I20260709 12:25:25.365823 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:25.372912 29960 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:25.373832 30162 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:25:25.374861 30097 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:25.376685 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.378578 29735 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.353477 29824 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.418700 20727 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:25.335392 29736 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.397195 20735 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:25.335851 30152 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.432713 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.393744 20729 rpcz_store.cc:276] 0709 12:25:24.106458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.265741 (+159283us) service_pool.cc:224] Handling call
0709 12:25:24.276173 (+ 10432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.315885 (+ 39712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.315935 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.345775 (+ 29840us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.566089 (+220314us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:24.566153 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.566261 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.366970 (+800709us) raft_consensus.cc:1685] finished
0709 12:25:25.366997 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.367132 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31051,"raft.queue_time_us":31219,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11572,"thread_start_us":219809,"threads_started":1,"wal-append.queue_time_us":408968}
I20260709 12:25:25.408479 20882 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:25.398259 20728 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:25.411859 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 249946us
I20260709 12:25:25.385972 30109 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:25.418747 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.432811 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:25.381134 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:25.355016 29759 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:25:25.442466 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.444031 20730 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:25.392989 30149 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.441874 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:25.365876 30148 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:25:25.413684 30145 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:25.371003 30094 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.441769 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.444701 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:25.410252 30097 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:25.418339 29935 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.457329 20804 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:25.387686 30116 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.492561 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:25.493420 20805 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:25.446331 30160 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:25.458922 30149 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.429435 30099 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.465607 30176 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:25:25.442447 30032 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:25:25.465747 30097 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.494463 20884 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:25.497170 20699 leader_election.cc:400] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
W20260709 12:25:25.458794 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93036us
I20260709 12:25:25.538147 20699 leader_election.cc:403] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:25:25.515146 29759 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:25:25.517685 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73064.7us
I20260709 12:25:25.525720 20735 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:25.527750 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.529934 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.463833 30109 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:25.398479 29541 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.594875 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.594412 30109 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.596221 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.597347 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.598327 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.528877 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.496894 29939 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:25:25.538877 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 177247us
I20260709 12:25:25.544500 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:25.496563 30160 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.456254 30138 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.071s
I20260709 12:25:25.601025 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.608197 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.609340 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.611955 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.613034 20735 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:25.613799 20730 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:25.620446 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:25.615108 21434 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:25.617157 20727 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:25.617519 20729 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:25.622339 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.622989 20804 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:25.618500 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.623898 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:25.544658 30151 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.544080 29759 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:25.524845 29904 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.600699 30027 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:25.608777 30013 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:25:25.534528 30099 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.616176 30160 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:25.614110 30169 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.099s
I20260709 12:25:25.619083 30186 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 12:25:25.715158 30150 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:25.716197 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 235886us
I20260709 12:25:25.717149 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:25.720037 30180 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:25:25.627308 30178 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:25.720114 30195 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:25.721395 30178 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.665246 30100 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:25.724097 30130 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:25.724020 30100 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.725775 30130 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:25:25.730108 30182 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:25:25.733247 30153 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.735813 30159 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.738335 30157 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.659662 30097 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:25.739992 30177 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:25:25.741364 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3582) took 1634 ms. Trace:
I20260709 12:25:25.742947 30149 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.741851 20731 rpcz_store.cc:276] 0709 12:25:24.106733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.296882 (+190149us) service_pool.cc:224] Handling call
0709 12:25:24.306645 (+  9763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.327687 (+ 21042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.327722 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.344752 (+ 17030us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.113752 (+769000us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b18002b91e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:25.267473 (+153721us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:25.267539 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.267638 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.740947 (+473309us) raft_consensus.cc:1685] finished
0709 12:25:25.740986 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.741283 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":873,"raft.queue_time_us":166,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":11102,"spinlock_wait_cycles":16007040,"thread_start_us":153462,"threads_started":1,"wal-append.queue_time_us":1101703}
I20260709 12:25:25.743741 30120 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:25.744310 30120 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:25.745640 30160 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.635526 21428 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:25.761078 20805 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:25.640375 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.664291 20730 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 15. Preceding OpId from leader: term: 33 index: 16. (index mismatch)
I20260709 12:25:25.770350 20727 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
W20260709 12:25:25.776943 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 155103us
I20260709 12:25:25.785835 20810 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:25.787061 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.792735 21426 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:25.811434 20882 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:25.632910 30135 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:25.741690 30146 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.635509 30192 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.752156 29736 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.883514 29904 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:25.885761 30178 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.788759 30100 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.821079 29559 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:25.877851 29875 consensus_meta.cc:287] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.001s
I20260709 12:25:25.887240 30192 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.931298 30139 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:25.868701 30150 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:25.932266 30150 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:25.932349 30192 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:25.932772 30150 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:25.933717 29903 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.758s	user 0.005s	sys 0.005s
I20260709 12:25:25.875483 30157 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.934229 29903 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:25.934777 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3467) took 1016 ms. Trace:
W20260709 12:25:25.904275 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 359822us
I20260709 12:25:25.955940 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:25.957082 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 137256us
I20260709 12:25:25.958078 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:25.891238 30057 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:25.959019 30094 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:25.959347 30100 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:25.960281 30152 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.961177 30099 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.934902 30157 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.964017 30157 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:25.964855 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:25.965590 20733 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:25.966907 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:25.943436 20875 rpcz_store.cc:276] 0709 12:25:24.918354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.269849 (+351495us) service_pool.cc:224] Handling call
0709 12:25:25.270293 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.270795 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:25.270837 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.280735 (+  9898us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.479901 (+199166us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:25.479943 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.480018 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.934492 (+454474us) raft_consensus.cc:1685] finished
0709 12:25:25.934515 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.934674 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2994,"raft.queue_time_us":4241,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4258,"thread_start_us":148738,"threads_started":1,"wal-append.queue_time_us":472355}
I20260709 12:25:25.982793 29892 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:25.983333 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3586) took 1791 ms. Trace:
I20260709 12:25:25.983700 30134 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:25.984266 30134 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:25.891129 29759 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:25.983839 20734 rpcz_store.cc:276] 0709 12:25:24.192481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.328191 (+135710us) service_pool.cc:224] Handling call
0709 12:25:24.350532 (+ 22341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.088207 (+737675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.982955 (+894748us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54003b5454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:25.983258 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952199552}
I20260709 12:25:25.990648 20806 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:25:25.953887 29900 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:25.991600 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:25.993481 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.954979 30160 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:25.997799 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:25.996714 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:25.979941 29997 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:26.000052 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.001317 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:26.002832 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:25.985388 30099 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:26.003829 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:25.983369 29892 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:25.995504 29900 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:26.003779 30013 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:26.024181 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.001842 29759 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:26.025537 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.002014 30100 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:25:26.031472 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 411074us
I20260709 12:25:26.034904 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:26.035387 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.035593 20807 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:26.023572 30120 raft_consensus.cc:2764] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:25:26.011485 30155 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:26.038861 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3471) took 1072 ms. Trace:
I20260709 12:25:25.985466 30134 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.045569 30134 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:26.045082 29892 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.047537 30192 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:26.047945 29892 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:26.044790 20883 rpcz_store.cc:276] 0709 12:25:24.966515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.262330 (+295815us) service_pool.cc:224] Handling call
0709 12:25:25.283809 (+ 21479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.457994 (+174185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.458025 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:25.458066 (+    41us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:25.458095 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.038549 (+580454us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400267d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:26.038630 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.038791 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400063744}
I20260709 12:25:26.049737 30135 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:26.050704 30192 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:26.051069 30192 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:26.052980 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.053797 20808 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:26.054082 30132 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.054689 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.057920 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.061802 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.062481 20808 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:26.058848 20804 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:26.063709 20733 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:26.065800 21433 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:26.066202 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:26.067430 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.071329 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45912.8us
I20260709 12:25:26.072271 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:26.047446 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:26.075763 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.078972 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:26.079787 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3463) took 1200 ms. Trace:
I20260709 12:25:26.077642 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:26.077956 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.081590 20808 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:26.080185 20882 rpcz_store.cc:276] 0709 12:25:24.865976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.866177 (+   201us) service_pool.cc:224] Handling call
0709 12:25:26.065731 (+1199554us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001d01d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:26.065833 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357983104}
I20260709 12:25:26.086864 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.097576 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3600) took 1403 ms. Trace:
W20260709 12:25:26.060650 30156 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.014s	sys 0.019s
W20260709 12:25:26.061300 30145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.023s	sys 0.040s
W20260709 12:25:26.110129 20700 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.534s (SENT)
I20260709 12:25:26.119382 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3462) took 1229 ms. Trace:
I20260709 12:25:26.124223 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.063582 29915 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:26.077291 30132 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:26.122676 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50443.6us
I20260709 12:25:26.098161 20806 rpcz_store.cc:276] 0709 12:25:24.693824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.697062 (+  3238us) service_pool.cc:224] Handling call
0709 12:25:24.746199 (+ 49137us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.994683 (+1248484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.097233 (+102550us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:26.097490 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2856319360}
W20260709 12:25:26.095150 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47812.7us
I20260709 12:25:26.129525 21399 leader_election.cc:400] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33.
I20260709 12:25:26.129894 21399 leader_election.cc:403] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.130796 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.131465 20726 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:26.132768 30194 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:26.123138 30145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.024s	sys 0.040s
I20260709 12:25:26.139715 30145 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:26.140013 30199 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:26.140748 30145 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.143354 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.144126 30100 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.144625 30100 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.146719 30100 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:26.147317 30155 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.147775 30155 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.149694 30192 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.124660 20880 rpcz_store.cc:276] 0709 12:25:24.855307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.855464 (+   157us) service_pool.cc:224] Handling call
0709 12:25:24.857198 (+  1734us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:25.392930 (+535732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.392980 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:25.417997 (+ 25017us) log.cc:844] Serialized 52 byte log entry
0709 12:25:25.528923 (+110926us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:25.528962 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.076713 (+547751us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54003f6e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:26.076762 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.076806 (+    44us) raft_consensus.cc:1685] finished
0709 12:25:26.076819 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.084870 (+  8051us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1664,"raft.queue_time_us":129,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":16781,"spinlock_wait_cycles":1230787456,"thread_start_us":62740,"threads_started":1,"wal-append.queue_time_us":188720}
I20260709 12:25:26.155478 20807 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:25:26.126829 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.125491 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:26.121795 30156 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.014s	sys 0.020s
I20260709 12:25:26.157598 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:26.163077 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.165102 20803 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 14. Preceding OpId from leader: term: 33 index: 16. (index mismatch)
W20260709 12:25:26.166764 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 176733us
I20260709 12:25:26.167584 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.124257 30203 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:26.171442 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.171296 21433 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 LEARNER]: Advancing to term 22
I20260709 12:25:26.172915 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.159662 30192 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:26.153443 30100 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:26.179677 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.181263 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.182152 20735 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:25:26.181977 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.181267 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.191234 29903 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:26.191933 30013 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:26.192772 20726 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:26.122706 29900 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.192767 30013 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:26.193718 29900 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:26.194036 30013 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:26.194547 21433 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:26.194772 30013 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:26.205647 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:26.211947 21426 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:25:26.212743 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.213377 20808 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:26.216027 30155 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:26.160043 30156 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:26.232681 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.236119 20809 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:26.237437 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134594us
I20260709 12:25:26.238441 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.239610 20735 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:26.243919 21433 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:25:26.268798 20728 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:25:26.272413 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.273541 20808 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:26.279654 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.277135 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:26.280926 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:26.280769 20809 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
W20260709 12:25:26.284685 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.291699 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:26.293247 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.294620 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.279109 30100 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.308717 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.309588 20882 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:26.311093 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.300414 30198 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:26.317914 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:26.319736 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.319825 20881 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:26.320632 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.321285 20728 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:25:26.322440 30179 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.322930 30179 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.325042 30179 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:26.325541 30179 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:26.325959 30179 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.326385 30027 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:26.326922 30027 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:26.299614 30164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.023s	sys 0.024s
W20260709 12:25:26.327705 30164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.023s	sys 0.025s
I20260709 12:25:26.333575 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:26.337263 21433 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:26.330108 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.338618 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
W20260709 12:25:26.336737 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152262us
I20260709 12:25:26.300240 30200 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:26.346159 21433 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
I20260709 12:25:26.346822 29939 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:26.347383 29939 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:26.349658 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.349948 29784 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:26.350358 29784 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:26.350657 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.351516 30152 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:26.351300 29784 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.352097 30152 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.352515 30152 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:26.352757 29784 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:26.354604 30203 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.356056 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:26.357261 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.357817 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.397480 21430 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:26.398536 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:26.403741 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.404772 20734 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:26.350956 30194 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33.
I20260709 12:25:26.400929 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:26.338667 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 155708us
I20260709 12:25:26.408423 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:26.408595 29932 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:26.409487 20731 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:25:26.355129 30203 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.410706 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.411028 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.415300 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.425733 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.420676 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.311735 29756 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.100s
I20260709 12:25:26.358096 30208 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:26.397389 30169 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:26.439131 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:26.461294 20729 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:26.311313 30156 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.451634 21430 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: [20.15-22.16]
I20260709 12:25:26.401750 30155 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:25:26.466171 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 65317.9us
I20260709 12:25:26.336370 30100 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.470146 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.471325 30142 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:26.428795 30057 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:26.427906 30097 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:26.478498 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.480496 20802 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:26.479617 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:25:26.466413 30205 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:26.463529 30169 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.473920 30157 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:25:26.494067 21434 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:25:26.423623 30200 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:25:26.474001 30142 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.480334 30057 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:26.481914 30097 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:26.489189 30205 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.493137 30169 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:26.504870 30221 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:26.513190 30221 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.507120 30205 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:26.516999 30097 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.517688 29541 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:26.518201 30097 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:26.520565 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:26.521382 21432 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:26.521689 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3381) took 1102 ms. Trace:
I20260709 12:25:26.522168 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.522523 29735 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:26.523610 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.523891 21431 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:26.524377 21428 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:26.524986 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.525224 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:26.525827 21428 raft_consensus.cc:2376] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19. Re-sending same reply.
I20260709 12:25:26.526233 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.526643 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.527922 21433 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:26.528524 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.528990 30100 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:26.530714 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:26.513561 30221 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:26.531627 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.532241 21429 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:26.533600 30203 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:26.534677 21435 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:26.538976 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.504166 30179 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:26.502297 30142 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:26.542130 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:26.540870 30152 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.508188 30213 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:26.544690 30213 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.544763 30152 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:26.545086 30213 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:26.546697 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.547153 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.522971 29735 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:26.522986 21432 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:26.527110 21431 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:25:26.552175 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:26.522099 20802 rpcz_store.cc:276] 0709 12:25:25.419621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.422190 (+  2569us) service_pool.cc:224] Handling call
0709 12:25:25.445559 (+ 23369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.489345 (+1043786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.521426 (+ 32081us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003a0f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:26.521625 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2377473408}
W20260709 12:25:26.557112 30150 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.002s
I20260709 12:25:26.557601 30150 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.559250 30057 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.560456 29939 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:26.565716 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.563804 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
W20260709 12:25:26.573707 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.600546 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.564226 20875 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:26.609820 21431 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:25:26.576534 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 138957us
I20260709 12:25:26.570616 20729 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:25:26.602432 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.616971 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.617660 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 209303us
I20260709 12:25:26.619786 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.620415 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:26.621121 21428 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:25:26.621356 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.625342 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.627240 21435 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:26.629222 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.630016 20878 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:26.632639 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:26.633338 20878 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:26.609264 30150 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:26.620312 30109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.021s	user 0.017s	sys 0.019s
I20260709 12:25:26.644483 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.646224 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.651073 20884 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:26.647212 21428 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:26.641042 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.641150 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.601693 30155 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:26.618904 30152 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.611814 30208 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:26.664240 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.668913 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.665155 20876 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
W20260709 12:25:26.656950 30109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.051s	user 0.017s	sys 0.020s
I20260709 12:25:26.654495 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:26.676569 20877 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:26.674173 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:26.676621 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.623063 29735 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:26.678920 29735 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:26.684100 30142 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.686190 20877 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:26.673602 30194 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:26.687681 30194 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.690068 30194 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:26.693521 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.698962 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.699733 21430 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
W20260709 12:25:26.699747 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 238019us
I20260709 12:25:26.645643 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.704769 20878 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:26.665583 29903 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:26.721432 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.666504 30185 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:26.725443 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.728405 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.729285 21430 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:26.718532 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.732107 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.733222 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.734388 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.735086 20884 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:26.726531 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:26.741041 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95486.7us
I20260709 12:25:26.742585 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:26.743237 20733 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:25:26.748489 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.753146 20876 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:26.760286 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.761070 20810 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:26.766390 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.767601 30135 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:26.767768 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.768092 20806 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:26.768182 30135 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.768926 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.761085 29903 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:26.770035 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:26.770635 30135 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:26.770820 20877 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:26.770217 29903 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:26.771095 30223 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:25:26.771724 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.772825 20810 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:26.773147 20877 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:26.774132 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.774904 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.775866 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:26.778692 20879 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:25:26.780076 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.779856 30155 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.782894 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.786057 20876 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:26.783994 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:26.791502 30222 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:26.791838 20884 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:26.789245 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:26.674794 30152 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.791651 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.715173 30142 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:26.806314 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:26.796813 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54980.4us
W20260709 12:25:26.759614 21400 connection.cc:256] RPC call timeout handler was delayed by -1.4160875480001778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
W20260709 12:25:26.826647 21400 connection.cc:256] RPC call timeout handler was delayed by -1.3332875480000439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
W20260709 12:25:26.827023 21400 connection.cc:256] RPC call timeout handler was delayed by -1.1478875480001989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 12:25:26.827348 21400 connection.cc:256] RPC call timeout handler was delayed by -1.1082875480001348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 12:25:26.820245 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.828692 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:26.829216 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.830153 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.820323 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.770164 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:26.825367 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:26.830788 30194 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:26.798058 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:26.747395 30150 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:25:26.838238 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62435.3us
I20260709 12:25:26.842859 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.847433 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:26.817143 30186 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:26.816745 30135 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.852831 20884 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:26.852831 20876 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:26.820175 30198 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.837817 30132 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:26.852900 30194 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:26.878441 20733 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:26.872576 30186 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.885572 30109 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:26.852466 30150 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:26.866330 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.851945 30034 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:26.852401 29892 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:25:26.920840 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54584us
I20260709 12:25:26.921099 30034 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:26.921041 30150 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.922104 29892 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.922330 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.742861 30185 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.923249 30034 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:26.924469 30034 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:26.924816 30185 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:26.846576 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.875402 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:26.926589 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80100us
I20260709 12:25:26.891172 30230 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.927426 30230 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.930461 30230 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:26.930709 30187 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.458s	user 0.007s	sys 0.006s
I20260709 12:25:26.931465 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.932456 30197 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:26.932669 30142 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:26.933166 30142 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:26.933367 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3317) took 1733 ms. Trace:
I20260709 12:25:26.933571 30142 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:26.934088 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.934446 30109 consensus_queue.cc:1243] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:25:26.935027 30229 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:26.935491 30229 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:26.936432 30229 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.937938 30229 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:26.935027 30109 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:26.941629 30147 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:26.947664 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141443us
I20260709 12:25:26.949592 20876 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:26.880066 29939 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:26.902608 30228 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.151s
I20260709 12:25:26.902820 30224 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:26.933815 20732 rpcz_store.cc:276] 0709 12:25:25.200380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.363581 (+163201us) service_pool.cc:224] Handling call
0709 12:25:25.363874 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.364077 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.364102 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.372698 (+  8596us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.452137 (+ 79439us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:25.452178 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.452238 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.930107 (+1477869us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:26.933047 (+  2940us) raft_consensus.cc:1685] finished
0709 12:25:26.933081 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.933272 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54253952,"thread_start_us":20085,"threads_started":1,"wal-append.queue_time_us":57547}
I20260709 12:25:26.959493 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.963356 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:26.963624 21432 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:25:26.964092 20883 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:26.967941 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.969120 29756 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:26.969796 29756 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:26.886369 30135 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.982595 21433 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 12:25:26.982648 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.983101 21433 pending_rounds.cc:85] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 16
I20260709 12:25:26.983492 21433 pending_rounds.cc:107] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:25:26.983579 30135 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:26.983783 21433 raft_consensus.cc:2889] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:26.838478 30208 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.985208 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.986446 30200 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:26.875487 29735 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
W20260709 12:25:26.987247 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.987520 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.989070 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.989584 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:26.872033 29930 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:25:26.919073 30116 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:26.869791 30205 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.924598 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:26.837076 20801 raft_consensus.cc:2368] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33.
I20260709 12:25:26.911145 20880 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
W20260709 12:25:26.996287 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71760us
I20260709 12:25:26.818320 30169 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.952437 20876 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:26.952886 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:26.958712 30224 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:26.962702 30197 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:26.970728 29756 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:26.980232 30194 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:26.981073 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.886404 30198 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:26.922540 29892 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:26.992363 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.906461 30186 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:26.997848 20882 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
W20260709 12:25:26.913784 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83731.4us
I20260709 12:25:27.001650 21430 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:26.910336 30221 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.008881 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:27.011276 20880 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:27.012156 20878 raft_consensus.cc:2368] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30.
I20260709 12:25:26.999472 29735 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:27.039687 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:27.043686 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.040935 21400 leader_election.cc:400] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24.
W20260709 12:25:27.049986 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.002995 29875 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:27.005447 29932 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:26.961704 30213 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.080098 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:27.067471 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.003813 30224 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:27.083794 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:27.087643 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:27.052426 30222 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:27.089677 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.089876 20734 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:27.090298 30100 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:27.090826 30100 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.091873 30213 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.090269 30222 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.092897 30100 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.094520 30222 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:27.094894 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.036623 29541 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:25:27.095276 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:27.096060 29541 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:27.096158 20731 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:27.096529 29541 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:27.096931 30198 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.097426 30097 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:27.097648 30198 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:27.097944 30097 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:27.098809 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:27.102864 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.101616 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:27.106282 21430 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:27.052158 29930 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:27.114774 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.121598 21430 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:27.084352 29939 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:27.081586 30034 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:27.146070 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:27.162338 21430 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:25:27.163252 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48607.5us
I20260709 12:25:27.098515 30097 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:27.220291 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.053419 30099 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 13, Time since last communication: 0.071s
W20260709 12:25:27.171844 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134494us
I20260709 12:25:27.084244 29932 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:27.138706 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:27.124097 29930 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:27.131210 29939 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:27.189965 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:27.191812 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.280102 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.117765 30222 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:27.126410 30152 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:27.168052 29784 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:27.198688 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:27.288950 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 90366.6us
I20260709 12:25:27.289934 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:27.100683 30241 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.113s
I20260709 12:25:27.178190 30228 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:27.146871 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.205755 30034 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.271556 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.169173 29541 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.281647 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 142997us
I20260709 12:25:27.280593 29930 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:27.281154 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58161.3us
W20260709 12:25:27.342965 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.344839 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 115013us
I20260709 12:25:27.348366 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:27.351020 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 69627.3us
I20260709 12:25:27.284233 29939 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:27.339515 30229 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:27.286406 30185 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:27.371197 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:27.373391 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:27.380358 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109245us
I20260709 12:25:27.334084 30245 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:27.359529 30242 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:27.401667 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.363286 29784 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:27.288770 30222 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.398640 29541 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:27.345428 30228 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:27.358659 30134 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:25:27.403446 29900 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:27.403162 30155 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:27.422693 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:27.350049 30034 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:27.376483 30185 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:27.350303 30132 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.048s
I20260709 12:25:27.422598 20850 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.453521 30132 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:27.372954 30229 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:27.455873 30229 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:27.456355 30034 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:27.456964 30240 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:25:27.440721 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:27.437251 30208 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:27.406162 30169 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:27.368903 29932 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.420696 30221 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.399466 30135 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:27.475826 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53215us
I20260709 12:25:27.437484 30245 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:27.481900 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:27.406796 30152 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:25:27.420255 30205 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:27.500133 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 78504.4us
I20260709 12:25:27.468055 29900 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:27.458511 30185 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:27.440553 29875 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:25:27.526623 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:27.486971 29784 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:27.481812 30142 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.002s
I20260709 12:25:27.528141 30116 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:27.540815 30221 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:25:27.560262 30142 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:27.564419 30222 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.570768 30142 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:27.481242 30047 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.096s
W20260709 12:25:27.565999 30262 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:25:27.567866 30275 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:27.570278 30242 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:27.569249 30027 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.571846 29900 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.563337 30134 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:27.530751 30100 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:27.530471 30034 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:27.535284 30152 consensus_queue.cc:1243] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:25:27.560341 30266 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:25:27.584096 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 484780us
I20260709 12:25:27.500435 30135 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.585672 30134 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:27.580724 30275 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.595527 30250 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:25:27.597319 30135 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:27.579764 30142 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.586851 30100 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:27.590325 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 125124us
I20260709 12:25:27.608163 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:27.540316 29892 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.611704 30251 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:25:27.579012 30242 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:27.589359 30254 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:25:27.623040 30228 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:27.626029 21432 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:27.634476 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:27.639735 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 235516us
I20260709 12:25:27.641888 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:27.642771 30116 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:27.642760 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.610868 29939 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.626863 30100 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:27.648073 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:27.627081 30135 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:27.663123 30275 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:27.662710 30242 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.647374 30194 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:27.656251 30271 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:27.664778 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3343) took 1041 ms. Trace:
I20260709 12:25:27.665211 20735 rpcz_store.cc:276] 0709 12:25:26.623044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.623342 (+   298us) service_pool.cc:224] Handling call
0709 12:25:26.655630 (+ 32288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.655942 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.655970 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.943283 (+287313us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:26.943731 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.200926 (+257195us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:27.200974 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.201058 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.659525 (+458467us) raft_consensus.cc:1685] finished
0709 12:25:27.659548 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.664707 (+  5159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34320,"raft.queue_time_us":138872,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":7080,"spinlock_wait_cycles":12893056,"thread_start_us":256792,"threads_started":1,"wal-append.queue_time_us":503077}
I20260709 12:25:27.658571 30134 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.668195 30134 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:27.669602 30185 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.658387 30135 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.672142 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:27.663965 30194 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.628022 30142 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.673846 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.657685 30116 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:27.681149 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:27.681849 21428 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
W20260709 12:25:27.682124 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 156502us
I20260709 12:25:27.683490 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:27.684388 30228 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:27.685128 29903 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:27.685592 29903 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.685945 29903 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:27.688730 30228 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:27.691385 30275 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:27.625777 30245 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.693907 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.695960 20801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:25:27.704463 20801 raft_consensus.cc:606] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:27.698036 30211 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:27.701485 29875 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:27.716892 30185 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:27.719928 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.722934 30275 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.723881 30275 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.727152 30275 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:27.728840 30194 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:27.740548 29892 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.741871 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:27.747720 30142 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.748983 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:27.749876 21428 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:27.708119 30211 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:27.751319 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:27.752254 30142 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:27.752722 30142 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.752830 29939 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:27.753185 30142 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:27.759091 29875 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:27.761724 30249 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 12:25:27.762671 30142 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.764016 30239 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.665s	user 0.000s	sys 0.002s
I20260709 12:25:27.761739 29875 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:27.766487 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:27.769374 21428 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:25:27.769637 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1718) took 1001 ms. Trace:
I20260709 12:25:27.771540 30157 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:27.772202 30157 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:27.773149 30157 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.774825 30186 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.001s
I20260709 12:25:27.775442 30256 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:27.773660 30145 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:27.776825 21428 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:27.777505 30211 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:27.776827 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 184486us
I20260709 12:25:27.779899 30057 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:27.780046 30142 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:27.771620 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 245073us
W20260709 12:25:27.766860 30269 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 12:25:27.786818 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 250406us
W20260709 12:25:27.788977 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.769968 20803 rpcz_store.cc:276] 0709 12:25:26.768607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.820421 (+ 51814us) service_pool.cc:224] Handling call
0709 12:25:26.843207 (+ 22786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.843472 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:27.181108 (+337636us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.768523 (+587415us) spinlock_profiling.cc:243] Waited 8.85 ms on lock 0x7b18004d66e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:25:27.768749 (+   226us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.769104 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.769236 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.769273 (+    37us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:27.769319 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.769391 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.769572 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23583,"prepare.queue_time_us":372591,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":3409,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6565,"spinlock_wait_cycles":20349696,"thread_start_us":494852,"threads_started":2,"wal-append.queue_time_us":387814}
W20260709 12:25:27.789361 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 129516us
I20260709 12:25:27.788045 20776 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:27.810925 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1722) took 1288 ms. Trace:
I20260709 12:25:27.848483 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:27.828580 20805 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 24. Ops: []
I20260709 12:25:27.852907 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:27.830149 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:27.829960 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:27.844718 20802 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:27.848789 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60877.1us
I20260709 12:25:27.848649 21433 rpcz_store.cc:276] 0709 12:25:26.522790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.528724 (+  5934us) service_pool.cc:224] Handling call
0709 12:25:26.967278 (+438554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.984538 (+ 17260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.594729 (+610191us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b540049c854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:27.810678 (+215949us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b540049c850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:27.810853 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042238976}
I20260709 12:25:27.864369 20806 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:25:27.864158 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:27.866071 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:27.868641 30132 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:27.869441 30281 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:27.869441 30132 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:27.871158 30132 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.868496 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:27.874073 20809 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:27.808440 30145 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:27.745260 29930 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:27.880786 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.883556 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:27.890175 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.891543 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:27.892395 20729 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:27.886363 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:27.867714 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:27.898533 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.860672 29892 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:27.900022 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:27.900795 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:27.902307 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:27.911306 20732 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:27.911422 20803 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:27.911451 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:27.912876 20726 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:27.910257 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:27.913972 20727 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 30
W20260709 12:25:27.915535 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47897.3us
I20260709 12:25:27.916843 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:27.866637 30228 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:27.876015 29939 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:27.894590 30271 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.971967 30271 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.977412 30229 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 12:25:27.978132 30229 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:27.926520 30097 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.001s
I20260709 12:25:27.981927 30097 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:27.985497 30097 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:27.988776 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:27.990809 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:27.992013 21435 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:27.811679 30186 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.920523 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:27.894202 30197 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.036s	sys 0.071s
W20260709 12:25:27.993567 30197 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.036s	sys 0.071s
I20260709 12:25:27.993870 30197 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:27.994791 30197 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:27.864136 30222 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:27.879276 30142 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:27.997403 30222 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:27.998170 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3528) took 1262 ms. Trace:
I20260709 12:25:27.998534 20878 rpcz_store.cc:276] 0709 12:25:26.735416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.766065 (+ 30649us) service_pool.cc:224] Handling call
0709 12:25:27.997965 (+1231900us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:27.998112 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562881792}
I20260709 12:25:27.999812 30222 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.000002 30227 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:28.000538 30227 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:28.001629 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 81191.4us
I20260709 12:25:28.001549 30227 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:27.932647 20801 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:27.934543 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:27.937794 30155 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:27.938397 30275 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:27.896139 30276 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.074s
I20260709 12:25:27.939134 20729 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:28.017863 20878 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:27.952994 30135 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:27.956004 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
W20260709 12:25:27.940423 30224 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.827s	user 0.012s	sys 0.001s
I20260709 12:25:27.956519 29903 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:27.943367 30225 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:27.947270 30205 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:27.947443 29892 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:27.947650 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.034657 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:27.939361 20732 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:27.964867 20727 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:27.965698 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:27.966147 29541 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.001s
I20260709 12:25:27.966933 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:27.967295 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:27.967656 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:27.968712 30269 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:27.969432 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:27.969703 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:27.969684 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:27.970530 29930 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:27.993146 29932 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:27.997640 30142 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:27.875717 30211 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:28.007699 20735 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
I20260709 12:25:28.008056 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.022578 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.061964 20877 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:25:28.022495 30224 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.017117 30047 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:27.925366 30185 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:28.038169 21432 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:28.040359 20809 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:28.042800 21433 raft_consensus.cc:2376] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33. Re-sending same reply.
I20260709 12:25:28.019014 30271 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:28.047946 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 78339.7us
I20260709 12:25:28.043678 21427 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:28.057646 20803 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:28.074357 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:28.039171 30135 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.019325 30227 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:28.068238 20879 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:25:28.032191 30205 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.040063 30097 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:25:28.021708 30186 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.084336 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.018203 30228 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.084975 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.051631 30150 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:28.093974 21430 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:28.097553 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.098555 21427 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:25:28.100349 21400 leader_election.cc:400] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33.
I20260709 12:25:28.071177 30134 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:28.104247 30134 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.106040 30142 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.106690 30134 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.072309 30185 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.067085 30047 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.108053 30185 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:28.108191 30047 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:28.104281 30281 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:28.090494 30178 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.109539 30100 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.062248 29900 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.091379 20730 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:25:28.107549 30222 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.140017 30222 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.140875 30169 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:28.141327 30169 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:28.142377 30222 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.108282 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3542) took 1155 ms. Trace:
I20260709 12:25:28.061059 30275 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:28.143131 30275 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:28.091719 30245 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.143621 30100 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.143925 30245 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.145344 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:28.146147 30245 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.146205 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.147054 30097 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:28.147608 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.147845 30097 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:28.148741 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.149129 30097 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.149854 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.150488 20807 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:28.082312 30205 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:28.151355 29892 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.152505 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.153240 20810 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:28.153981 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.155237 30145 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.156232 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.157451 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.158437 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.116194 30279 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:28.117268 30047 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.117965 30228 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.088644 20808 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:28.119412 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.166638 20735 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:28.119599 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.120100 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.121156 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.168665 20729 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:28.123251 20701 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:28.142973 20881 rpcz_store.cc:276] 0709 12:25:26.952990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.953107 (+   117us) service_pool.cc:224] Handling call
0709 12:25:26.963351 (+ 10244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.971729 (+  8378us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:26.987826 (+ 16097us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b180028c0c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:25:27.260275 (+272449us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.582281 (+322006us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.590879 (+  8598us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.591046 (+   167us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:27.592486 (+  1440us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.765600 (+173114us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b180028c0c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e22b7358f 00007f9e1bbb2571 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd
0709 12:25:27.766279 (+   679us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:27.777793 (+ 11514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.777889 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.108062 (+330173us) raft_consensus.cc:1685] finished
0709 12:25:28.108083 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.108223 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":248709,"prepare.queue_time_us":325458,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":7158,"raft.queue_time_us":241201,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":17088,"spinlock_wait_cycles":31513728,"thread_start_us":472165,"threads_started":3,"wal-append.queue_time_us":234005}
I20260709 12:25:28.127460 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.128584 29903 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.172128 21432 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:25:28.128448 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.090574 30116 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:28.129462 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.131538 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.131419 30281 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:28.087380 30135 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.133726 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.156252 30145 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:28.136193 30229 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.139756 29900 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.142181 30169 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.143677 30275 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:28.089814 30271 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.026s
I20260709 12:25:28.111726 30224 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.160676 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.154992 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.203217 20810 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:28.161456 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.204712 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.176826 30228 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:28.158958 20806 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
W20260709 12:25:28.173328 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50176.2us
I20260709 12:25:28.173497 20727 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:28.130777 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:28.182761 20726 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [23.16-25.17]
I20260709 12:25:28.185777 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.213882 20735 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:28.193490 20808 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
W20260709 12:25:28.201819 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.214710 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.159096 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57639.4us
I20260709 12:25:28.215466 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.211943 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:28.216332 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85672.3us
I20260709 12:25:28.217249 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.218658 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.182081 30245 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.223918 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.078418 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.232465 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.109588 30178 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.242620 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.244135 20806 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:28.239950 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.252099 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.257118 20807 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:28.283365 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.174190 30279 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.285223 21435 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 12:25:28.285457 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45587us
I20260709 12:25:28.286635 20808 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:28.286880 30273 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:28.292112 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.292403 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.292474 20735 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:28.286630 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:28.215451 30157 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.211377 30228 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.298090 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.300091 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:28.243232 30282 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:28.307634 20809 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:28.314814 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.328725 20726 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:28.303768 20807 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:25:28.242856 29541 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:28.324097 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:28.319324 30157 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.292757 30169 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:28.283390 30203 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:28.341629 20808 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:28.334903 30157 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:28.338876 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.224424 30222 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.107s
I20260709 12:25:28.345327 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:28.317685 30228 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:28.303179 30273 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:28.358233 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141020us
I20260709 12:25:28.313058 30281 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.353902 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:28.312021 30100 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.257658 30241 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.083s
I20260709 12:25:28.328037 30198 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:28.360677 20805 raft_consensus.cc:2410] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:25:28.367592 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.328868 29735 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:28.328438 30282 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.369877 30271 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:28.403644 30271 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:28.404111 30271 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:28.389070 30194 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.019s
W20260709 12:25:28.407799 30292 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:28.389259 30100 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.410144 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.410820 20735 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:28.417573 30279 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.257652 30245 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.426101 20810 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:28.338577 30197 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:28.383236 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.431672 20877 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:28.433053 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.433818 20801 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:28.437566 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.438246 20807 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:28.428727 29903 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.439179 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.390611 30281 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.388890 29875 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:28.391386 30290 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:25:28.393375 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.402926 30282 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:28.427466 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:28.428120 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.439930 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.440090 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.441062 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:28.480856 20735 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:28.441800 20732 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:28.481410 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.442200 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:28.453972 20726 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:28.482311 20733 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:28.482729 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:28.455588 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:28.483096 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.462146 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.483690 21434 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:28.378830 20806 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:28.484380 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:28.484704 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.485239 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.485390 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.485891 20806 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 22.
I20260709 12:25:28.486168 20810 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:28.410993 30100 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:28.469959 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.481271 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:28.491878 20804 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
W20260709 12:25:28.477829 30288 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:25:28.497164 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.497464 20804 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:28.486913 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:28.482007 21431 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:28.382298 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:28.493432 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.431998 30197 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:28.493592 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.562s (SENT) [suppressed 22 similar messages]
I20260709 12:25:28.528935 20804 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:28.494895 20806 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:28.531237 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.531544 20806 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:28.531944 21435 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:28.533059 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:28.537223 20804 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:28.534400 20807 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:28.535565 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:28.420838 30198 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.546607 21434 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:28.555755 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.447409 29903 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.440357 30275 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.558770 30227 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:28.559342 30227 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.559814 30227 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:28.561165 30275 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.563828 30299 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:28.567986 30227 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.568456 20733 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
W20260709 12:25:28.572121 30301 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:25:28.573766 30227 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.577977 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.578820 21426 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:28.583632 30291 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:28.589075 21426 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:28.590795 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:28.592818 30227 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:28.479523 29892 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:25:28.595005 30227 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:28.596916 29939 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:28.597468 29939 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:28.598328 30227 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.598502 29939 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.494216 30047 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.620054 29939 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:28.621587 29903 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.624037 30281 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.601311 30205 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.627887 30205 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.636966 30197 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.642772 30305 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:28.643472 30305 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:28.645038 30198 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.644673 30305 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:28.651750 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 268572us
I20260709 12:25:28.651997 30305 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:28.548686 30100 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:28.489373 30245 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.477602 29875 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.646268 30203 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:28.646448 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 190950us
I20260709 12:25:28.602427 30135 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.675393 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.681605 20735 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:28.687027 20805 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
W20260709 12:25:28.691592 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.692716 21432 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:28.696853 20699 leader_election.cc:400] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30.
I20260709 12:25:28.702279 20801 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:28.702749 20699 leader_election.cc:403] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.697407 20807 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:25:28.698894 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:28.691016 30200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.702s	user 0.017s	sys 0.026s
W20260709 12:25:28.705410 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.705613 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:28.705468 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52878.2us
W20260709 12:25:28.692487 30185 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.000s
I20260709 12:25:28.720270 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.704402 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:28.696319 30194 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:28.691413 30100 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.698173 30203 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.732352 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:28.733163 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 239578us
W20260709 12:25:28.720717 30200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.724s	user 0.017s	sys 0.027s
I20260709 12:25:28.740381 30309 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:28.741376 30281 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:28.742657 30304 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:25:28.711494 30275 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:28.749215 29875 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.750038 30308 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.728835 30185 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.724005 30224 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.719215 30132 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:28.759697 30203 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:28.711418 30135 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:28.761071 30134 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:25:28.803354 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.803520 30134 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:28.729487 30100 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:28.731173 30222 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:28.805830 30222 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:28.806988 30222 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.721141 30279 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30.
I20260709 12:25:28.811581 30134 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.812886 30134 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:28.819808 30100 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:28.827960 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.831058 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.831916 21435 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:28.840860 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:28.841281 21427 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:28.845822 21430 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:28.845594 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:28.867708 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 378179us
W20260709 12:25:28.867592 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.890677 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:28.891039 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:28.795353 30308 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.896489 30308 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:28.900437 30221 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.900738 30185 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:28.901208 30221 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.902483 29900 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.904448 30221 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.876336 30208 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:28.905150 30221 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:28.905434 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3401) took 1047 ms. Trace:
I20260709 12:25:28.905707 30208 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:28.905911 20809 rpcz_store.cc:276] 0709 12:25:27.858386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.073024 (+214638us) service_pool.cc:224] Handling call
0709 12:25:28.905257 (+832233us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b54002b57d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:28.905374 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184697216}
I20260709 12:25:28.839497 30132 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.906693 30169 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.911609 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.911782 30169 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:28.913136 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.913939 30169 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:28.907830 29735 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.794478 30309 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.094s
W20260709 12:25:28.915364 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 487969us
I20260709 12:25:28.908368 29932 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:28.908804 30132 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.910815 30097 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.874306 30194 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:28.908159 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 525903us
I20260709 12:25:28.947805 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:28.948447 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:28.949322 20875 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:28.916122 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "761ea542665e419c8a63392e3014efa7"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
W20260709 12:25:28.946316 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55745.8us
I20260709 12:25:28.836953 30224 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.926970 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:28.927435 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:28.941617 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:28.942723 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.972411 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.973340 20883 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:28.973948 21429 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:28.974522 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:28.906718 30208 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.976126 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.977324 21429 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:28.977638 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:28.978250 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.979300 29932 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:28.980623 29932 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.914770 29735 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.985586 29735 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:28.905732 30221 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:28.987473 30132 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:28.988054 30132 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:28.988559 30132 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:28.988837 30221 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:28.921512 30228 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.003s
I20260709 12:25:28.945252 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.992445 30228 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:28.993211 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.994524 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.946316 20882 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:25:28.996055 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:28.996192 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.997081 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:28.997710 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:28.998128 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:28.998456 21426 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:28.952189 20878 raft_consensus.cc:606] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:25:28.998879 20878 raft_consensus.cc:614] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:28.999974 30315 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:29.002585 30197 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:25:29.003842 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 77224.1us
I20260709 12:25:29.004895 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:29.005231 29875 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:29.005743 29875 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:29.007417 30057 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:28.999992 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:29.008288 30057 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:29.009052 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.009498 30057 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.010253 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.006615 29875 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.011539 30186 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:29.012215 30186 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.012830 30186 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:29.013038 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.014358 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:28.953637 20880 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:25:29.015173 20881 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:29.016386 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.018072 20880 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:29.012355 29875 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:28.915683 30047 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:28.966202 30097 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.022387 30097 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:28.844133 30100 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:28.925621 30185 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:29.023126 20879 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:29.024852 30313 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:29.023495 30276 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:25:29.023518 30271 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.619s	user 0.012s	sys 0.000s
I20260709 12:25:29.026755 30271 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.028086 21429 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:29.029320 30271 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.030241 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:28.991779 30224 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:28.919888 20877 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:29.032296 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.032840 20884 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:29.003809 30197 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:25:29.034742 21426 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:25:29.035636 20882 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:29.015651 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.036603 20880 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:29.037163 20883 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:29.032944 20877 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:29.046495 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:29.052059 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:29.048338 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:29.032296 30145 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.848s	user 0.011s	sys 0.002s
I20260709 12:25:28.985659 30281 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:29.058450 20884 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:28.966308 30142 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:29.059674 20880 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:29.023942 30100 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:29.058629 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:29.041638 30316 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:29.087392 20880 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:25:29.064241 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:29.088857 20877 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:29.088899 21435 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:29.065241 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:28.986687 29735 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.055s
I20260709 12:25:29.085882 20884 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:29.041107 29900 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.024194 30047 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:29.024135 30185 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:28.888092 30282 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.106877 20878 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:25:29.114701 20883 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:29.102406 20807 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:29.102552 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:29.059688 30205 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:29.084496 30271 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:29.133893 20809 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:29.130276 21435 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:29.137449 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:29.102267 30273 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:29.107206 29875 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:25:29.107666 30274 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:29.096880 30027 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.045s
I20260709 12:25:29.145517 21426 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:29.087153 30299 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.052s
I20260709 12:25:29.133955 30276 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:29.152050 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:29.144299 21432 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:29.132211 30142 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:29.096319 30100 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.132592 30271 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:29.156733 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:29.140798 30205 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:29.142762 30273 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.157701 20807 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:25:29.115295 29900 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.110934 30099 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:25:29.108201 30047 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:29.185379 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 418888us
I20260709 12:25:29.186625 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:29.161742 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:29.162040 30311 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.166078 30186 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.145641 30185 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.167479 30205 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.176434 30271 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:29.170711 30178 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:29.127867 30198 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:25:29.171170 30299 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:29.171224 30142 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.173550 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:29.206046 21426 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:29.158077 30228 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:29.176992 20807 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:29.180862 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:29.183792 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.189767 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:29.213788 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.214092 20802 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:29.190734 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3616) took 1305 ms. Trace:
W20260709 12:25:29.170549 30305 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.007s
I20260709 12:25:29.178575 30273 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:29.219107 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 113168us
I20260709 12:25:29.215230 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:29.216243 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.203320 30242 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:29.227020 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.201090 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:29.214736 20734 rpcz_store.cc:276] 0709 12:25:27.885433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.885620 (+   187us) service_pool.cc:224] Handling call
0709 12:25:27.895474 (+  9854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.904050 (+  8576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.904106 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.745900 (+841794us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:28.746419 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.092103 (+345684us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:29.092149 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.092265 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.175839 (+ 83574us) raft_consensus.cc:1685] finished
0709 12:25:29.175864 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.190665 (+ 14801us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18845,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":87399,"spinlock_wait_cycles":18812672,"thread_start_us":345214,"threads_started":1,"wal-append.queue_time_us":409143}
I20260709 12:25:29.231019 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.223272 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:29.231853 20727 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:29.223090 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:29.232692 30178 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:29.233354 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:29.234719 29904 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:29.224818 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.232725 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:29.235610 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:29.237278 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.231019 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.238714 20730 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:29.231205 30227 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:29.240839 30227 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:29.205839 30185 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:29.243758 30327 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:29.244865 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:29.245622 20733 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:25:29.246558 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:29.227602 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.247821 20732 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:29.231912 20734 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:29.249243 20731 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:29.242123 30227 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.255355 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:29.256413 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:29.257198 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:29.256507 20727 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:29.263010 21433 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:25:29.263126 20732 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
W20260709 12:25:29.264635 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63664.6us
I20260709 12:25:29.265262 21433 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:29.266187 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:29.269608 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.270263 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:29.270761 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:29.273627 20729 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:29.278533 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:29.280684 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:29.267006 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.284382 20726 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:29.222980 30281 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:29.319499 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73044.9us
I20260709 12:25:29.282866 21426 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:29.324360 21426 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:25:29.323333 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:29.262221 30224 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:29.195588 30311 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:25:29.339427 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73312.8us
I20260709 12:25:29.341470 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:29.270218 30099 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:29.268517 30331 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:29.342847 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:29.330579 30322 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:25:29.292675 30150 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:29.329005 30225 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:29.268457 30178 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 12:25:29.275506 30134 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:29.275753 30280 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.091s
W20260709 12:25:29.382529 30325 raft_consensus.cc:670] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } [suppressed 4 similar messages]
I20260709 12:25:29.346040 29932 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.366235 30156 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:29.368690 30224 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.376412 30208 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:25:29.376799 30203 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 12:25:29.371515 30099 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:29.403606 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:29.366608 30197 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:29.334378 29939 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:25:29.377514 30225 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.427255 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:29.350622 30331 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:29.387635 30276 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:29.387753 30282 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.425563 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.430012 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.407877 30208 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:29.450057 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:29.429275 30326 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:25:29.426004 29939 consensus_queue.cc:1243] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:29.429616 30225 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:29.424026 30156 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:29.412528 29932 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.418186 30203 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.439745 30276 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:29.495157 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 153756us
I20260709 12:25:29.436421 30331 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:29.490394 20734 pending_rounds.cc:85] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 19
W20260709 12:25:29.497148 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 143711us
I20260709 12:25:29.453986 30208 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:29.431022 30197 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.503629 20734 pending_rounds.cc:107] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 29.20
I20260709 12:25:29.504134 20700 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:29.453389 30100 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:29.415812 30224 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:29.519541 20734 raft_consensus.cc:2889] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.493399 29900 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:29.523378 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:29.527160 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109298us
I20260709 12:25:29.537071 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:29.503249 30276 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:29.537192 21431 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:29.543582 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:29.522907 30186 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:29.519497 30333 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:29.565161 30347 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:29.567536 30345 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:29.568069 30345 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:29.568970 30345 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.518811 30099 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.542860 30100 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:25:29.550436 30280 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:29.574751 30280 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.575171 30280 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:29.576169 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.579591 29904 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:29.586681 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:29.599759 20802 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
W20260709 12:25:29.655441 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 113429us
I20260709 12:25:29.562634 30222 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:29.541239 30273 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.660751 30100 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:29.723394 30100 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.727254 30343 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:29.728511 30338 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:25:29.729975 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:29.730223 30349 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:25:29.732220 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:29.649672 30185 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.743806 20875 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:29.705626 30336 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:29.705420 29939 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.727010 30352 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:29.731261 30347 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:25:29.764206 30337 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 12:25:29.755331 30343 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:29.748566 30311 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.730610 30348 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.038s
W20260709 12:25:29.792343 30350 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:25:29.793190 30339 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:25:29.783725 30352 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:29.779299 30336 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.772213 29939 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.591593 30156 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.757153 30185 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:29.793874 30346 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 12:25:29.773103 30341 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.327s
I20260709 12:25:29.849287 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:29.827597 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:29.848234 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:29.858922 30328 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:29.866055 30328 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.870590 30342 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:25:29.871218 30335 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:25:29.871511 30100 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:29.872298 30330 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.385s	user 0.000s	sys 0.001s
W20260709 12:25:29.874257 30334 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:25:29.826864 29932 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:29.875152 30291 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:29.875707 30291 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:29.876138 30273 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:29.817173 30343 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.882872 21433 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:29.883739 30222 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:29.884150 30222 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:29.884984 30222 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.886015 29735 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.886205 30222 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:29.886579 29735 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:29.887293 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.888670 29735 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:29.889436 30271 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.673s	user 0.004s	sys 0.005s
W20260709 12:25:29.889688 30132 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.001s
I20260709 12:25:29.889837 30271 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.890133 30132 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:29.894690 30316 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.790s	user 0.008s	sys 0.000s
I20260709 12:25:29.864650 30197 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.895920 30352 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.897539 29939 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:29.898299 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.898422 30343 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:29.902526 21429 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:29.823542 30047 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:29.909878 20880 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:29.911859 30132 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.895968 30316 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:29.874967 30331 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.840199 30203 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:29.917388 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:29.918301 20730 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:29.906071 30328 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.922520 20727 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:25:29.922669 30271 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.925410 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2244) took 1035 ms. Trace:
I20260709 12:25:29.925838 21435 rpcz_store.cc:276] 0709 12:25:28.889622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.890787 (+  1165us) service_pool.cc:224] Handling call
0709 12:25:29.925241 (+1034454us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54006333d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:29.925347 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452408576}
I20260709 12:25:29.927641 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:29.930207 30185 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.927675 30224 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:29.931334 30358 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:29.931903 30358 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:29.933074 30357 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 12:25:29.934584 30242 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.685s	user 0.003s	sys 0.008s
W20260709 12:25:29.935329 30354 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:25:29.936015 29900 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.855942 30336 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:29.967491 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3642) took 2135 ms. Trace:
W20260709 12:25:29.999837 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.990643 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.026842 20802 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:29.994311 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:29.955245 21430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1702) took 1795 ms (client timeout 1818 ms). Trace:
W20260709 12:25:30.028287 21430 rpcz_store.cc:269] 0709 12:25:28.160201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.160447 (+   246us) service_pool.cc:224] Handling call
0709 12:25:29.954993 (+1794546us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:29.955152 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477895424}
I20260709 12:25:30.001183 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:29.951722 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:29.962924 21427 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:29.949302 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.031409 20804 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:30.025398 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.025637 20810 rpcz_store.cc:276] 0709 12:25:27.832224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.495905 (+663681us) service_pool.cc:224] Handling call
0709 12:25:28.496377 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:28.720813 (+224436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.720845 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:28.902104 (+181259us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:28.902767 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 12:25:29.200428 (+297661us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:29.320894 (+120466us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.346806 (+ 25912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.967145 (+620339us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:29.967219 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.967267 (+    48us) raft_consensus.cc:1685] finished
0709 12:25:29.967282 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.967428 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25186,"prepare.queue_time_us":172259,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":30300,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":9222,"spinlock_wait_cycles":523240704,"thread_start_us":394012,"threads_started":2,"wal-append.queue_time_us":433011}
I20260709 12:25:30.039040 30336 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:30.044701 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.044880 30331 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:29.939087 30178 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.045565 20802 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:30.045922 30178 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.942368 29903 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:30.047001 29903 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:29.942938 30274 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:30.022197 20728 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:30.023495 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:30.024619 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.066591 20875 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:29.963311 30319 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:30.048885 30178 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.050491 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:30.053267 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.073684 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.054627 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:29.952049 30280 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.056993 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:30.067436 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:30.018201 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:30.008589 30225 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:29.958580 30276 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:30.073813 30319 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:30.093719 30275 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.095119 30275 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.095419 30319 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.096807 30319 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:30.097896 30358 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:30.098205 30275 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.075085 20809 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:30.098955 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.100037 30291 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.100802 30225 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:30.101130 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.101543 20877 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:30.101656 20806 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:25:30.102180 30291 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.065064 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:30.069917 20804 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:30.035617 30242 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.103641 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3509) took 1136 ms. Trace:
I20260709 12:25:30.104643 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3503) took 1160 ms. Trace:
I20260709 12:25:30.104682 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:30.105381 20809 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:30.106141 30273 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:25:30.071242 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.003s	sys 0.013s
I20260709 12:25:30.071882 30274 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:30.095445 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1682) took 1097 ms. Trace:
I20260709 12:25:30.136169 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:30.136435 20880 rpcz_store.cc:276] 0709 12:25:28.998015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.104563 (+106548us) service_pool.cc:224] Handling call
0709 12:25:29.116921 (+ 12358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.910385 (+793464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.095163 (+184778us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b5400366854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:30.095382 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816201472}
I20260709 12:25:30.137459 21434 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 12:25:30.138612 20807 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:30.140231 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:30.118834 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53856.7us
I20260709 12:25:29.943739 30047 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.119701 20806 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:30.119892 21428 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:29.956609 30208 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:30.131639 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.006s	sys 0.013s
I20260709 12:25:30.148417 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:30.154140 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 811305us
I20260709 12:25:30.145624 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:25:30.131438 20882 rpcz_store.cc:276] 0709 12:25:28.944604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.997887 (+ 53283us) service_pool.cc:224] Handling call
0709 12:25:30.104483 (+1106596us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b5400059e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:30.104592 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84269312}
I20260709 12:25:30.158344 21428 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:30.158404 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:30.033972 29900 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.120826 30275 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:30.133909 21426 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:30.128553 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2249) took 1154 ms. Trace:
I20260709 12:25:30.135416 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:30.132763 20877 rpcz_store.cc:276] 0709 12:25:28.967722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.089935 (+122213us) service_pool.cc:224] Handling call
0709 12:25:29.875285 (+785350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.101938 (+226653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.102447 (+   509us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400366d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:30.103420 (+   973us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b5400366d50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:30.103573 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2325052416}
I20260709 12:25:30.113582 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:30.164968 21435 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:30.168828 21430 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:30.172662 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.174208 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.174773 20801 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:30.176294 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:30.176412 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.176905 20726 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:30.177892 20801 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:30.176924 20809 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:30.179517 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:30.182720 21428 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:25:30.182927 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:30.183878 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:30.121270 30097 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:25:30.188072 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:30.189911 20730 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:30.152426 30205 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:30.114671 20875 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:30.142304 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:30.109133 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:30.143126 21434 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:30.115962 30194 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:30.150619 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:30.153106 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:30.195549 20880 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:30.155788 30311 raft_consensus.cc:2764] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:25:30.161139 20731 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:30.163074 20732 pending_rounds.cc:85] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 12
I20260709 12:25:30.196587 20732 pending_rounds.cc:107] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:25:30.163252 20876 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:30.196907 20732 raft_consensus.cc:2889] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.190665 20808 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:25:30.129264 21433 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:30.140946 30242 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.175489 29904 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:30.156206 30316 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:25:30.191673 30274 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.192245 30241 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.193192 30367 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.308s	user 0.002s	sys 0.000s
I20260709 12:25:30.155916 30308 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:30.199827 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57614.3us
I20260709 12:25:30.199951 21433 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:30.164562 21431 rpcz_store.cc:276] 0709 12:25:28.974461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.027986 (+ 53525us) service_pool.cc:224] Handling call
0709 12:25:29.028311 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.556257 (+527946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.128224 (+571967us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400645ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:30.128479 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148189056}
I20260709 12:25:30.170186 29756 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:30.140659 20884 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:30.208465 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.209586 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.214704 20801 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:30.211511 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:30.217509 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:30.205706 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:30.218730 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:30.230654 20875 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:30.155614 29932 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:25:30.232404 21430 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:30.282706 20881 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:25:30.239456 20809 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:30.283640 30319 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.284258 30319 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.243396 30142 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:30.244494 30178 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:30.285604 30178 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.164791 29939 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:25:30.205025 30309 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:30.164530 30353 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 12:25:30.168953 30275 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.263409 20880 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:25:30.239925 30271 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:30.253304 30328 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:30.271178 30276 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:30.272571 30342 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:30.233834 30241 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.204711 30194 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.265964 29900 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:30.280184 20726 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:30.242256 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:30.310388 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68224.5us
W20260709 12:25:30.312834 30362 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.421s	user 0.004s	sys 0.000s
W20260709 12:25:30.314612 30364 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.418s	user 0.002s	sys 0.000s
I20260709 12:25:30.314747 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2243) took 1438 ms. Trace:
I20260709 12:25:30.315146 21427 rpcz_store.cc:276] 0709 12:25:28.876424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.876816 (+   392us) service_pool.cc:224] Handling call
0709 12:25:29.962730 (+1085914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.036269 (+ 73539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.314538 (+278269us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54005a4950. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:30.314690 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2496370560}
I20260709 12:25:30.320338 30208 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.323397 30309 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:30.324005 30309 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:30.324862 30309 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.327996 30185 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:30.329527 30185 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.330108 30275 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.330016 30185 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.331688 30299 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:30.332091 30299 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:30.332939 30299 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.333433 29903 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 25.21, Last appended by leader: 21, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.334213 30299 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:30.334465 29903 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:30.293826 30142 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.337512 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:30.287645 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.008s	sys 0.028s
W20260709 12:25:30.338963 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.008s	sys 0.028s
I20260709 12:25:30.252415 30203 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:30.215495 29904 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.289078 30186 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.344117 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:30.348296 21427 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:30.305100 30342 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:30.345333 21428 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
W20260709 12:25:30.344966 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.009s	sys 0.037s
W20260709 12:25:30.350804 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.009s	sys 0.037s
I20260709 12:25:30.361763 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:30.437314 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75657.8us
I20260709 12:25:30.346223 30203 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:30.441843 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 855171us
I20260709 12:25:30.345440 30328 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:30.345273 30271 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.449177 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:30.436839 30132 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.440205 30353 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:30.347062 30273 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.108s
I20260709 12:25:30.341591 30142 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.441282 30178 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.447840 30203 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:30.494390 30185 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:30.367774 30186 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.454751 30047 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:30.460392 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.499567 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.512542 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:30.516621 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67577.8us
I20260709 12:25:30.527722 20807 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
W20260709 12:25:30.495779 20735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.018s	sys 0.046s
W20260709 12:25:30.528981 20735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.019s	sys 0.046s
I20260709 12:25:30.486392 30241 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:30.529924 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:30.496783 30353 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.469167 30194 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.512007 30224 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:30.514171 30242 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.534248 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:30.553107 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.502223 30132 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.554863 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.584187 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.585785 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.586568 20809 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:30.556728 20801 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:25:30.467698 30328 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.556887 20802 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:30.557746 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.612573 20804 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:25:30.552188 30211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.031s	sys 0.037s
I20260709 12:25:30.564273 20807 raft_consensus.cc:2376] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26. Re-sending same reply.
I20260709 12:25:30.553231 30353 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:30.588279 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:30.551733 30352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.016s	sys 0.029s
I20260709 12:25:30.612731 20806 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:30.620586 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:30.622659 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.641131 20804 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
W20260709 12:25:30.641305 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.622468 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:30.467846 30271 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:30.621228 30211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.031s	sys 0.038s
I20260709 12:25:30.556303 30208 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:30.653797 21432 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
W20260709 12:25:30.719343 20732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.014s	sys 0.016s
W20260709 12:25:30.719848 20732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.014s	sys 0.016s
I20260709 12:25:30.638808 30047 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:30.583513 30194 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:30.739619 30211 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:30.740458 30211 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.742744 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3365) took 1517 ms. Trace:
I20260709 12:25:30.736320 20803 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:25:30.743134 20735 rpcz_store.cc:276] 0709 12:25:29.225251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.236889 (+ 11638us) service_pool.cc:224] Handling call
0709 12:25:29.237340 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.237702 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.237730 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.965633 (+727903us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:29.966216 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.529451 (+563235us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:30.529515 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.529624 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.742439 (+212815us) raft_consensus.cc:1685] finished
0709 12:25:30.742460 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.742679 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5835,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":133,"spinlock_wait_cycles":24930304,"thread_start_us":562724,"threads_started":1,"wal-append.queue_time_us":680651}
W20260709 12:25:30.744575 30359 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.861s	user 0.000s	sys 0.003s
I20260709 12:25:30.645165 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:30.745187 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3375) took 1319 ms. Trace:
W20260709 12:25:30.745783 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 100692us
I20260709 12:25:30.745546 20734 rpcz_store.cc:276] 0709 12:25:29.425499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.479930 (+ 54431us) service_pool.cc:224] Handling call
0709 12:25:29.490171 (+ 10241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.551497 (+ 61326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.551531 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.968008 (+416477us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:29.968384 (+   376us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.360938 (+392554us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:30.360978 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.361079 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.744914 (+383835us) raft_consensus.cc:1685] finished
0709 12:25:30.744934 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.745127 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23091072,"thread_start_us":392157,"threads_started":1,"wal-append.queue_time_us":744235}
I20260709 12:25:30.746440 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.637368 20809 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:30.739718 30194 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.580915 30291 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.740803 30371 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:25:30.744674 30211 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.747834 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:30.738979 30369 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.647969 30352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.016s	sys 0.029s
I20260709 12:25:30.754102 30225 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:30.770398 30369 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.774613 30369 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.759527 30291 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.769018 30211 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:30.777097 30211 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.777951 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.778899 20805 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:30.650902 29756 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:30.788170 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:30.788785 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:30.789088 20805 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:25:30.668287 30275 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:30.670083 30319 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.682588 30208 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:30.754102 30194 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:30.754228 30368 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:25:30.687565 30273 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.100s
I20260709 12:25:30.795418 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:30.841656 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46450.4us
W20260709 12:25:30.803609 29932 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.011s	sys 0.027s
W20260709 12:25:30.848805 29932 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.013s	sys 0.028s
I20260709 12:25:30.785885 30142 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.802260 30369 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:25:30.850682 30205 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:30.802765 30275 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:30.800537 30345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.012s	sys 0.016s
I20260709 12:25:30.846800 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:30.852146 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.801091 30145 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.010s	sys 0.027s
I20260709 12:25:30.851125 30205 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:30.855291 30345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.014s	sys 0.018s
I20260709 12:25:30.801959 30352 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:30.838639 29904 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:30.844074 30211 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.846696 30242 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:25:30.857096 30145 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.014s	sys 0.027s
I20260709 12:25:30.855096 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:30.793097 30224 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:25:30.777125 30291 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:30.858253 30227 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:30.857874 29932 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:30.861686 30352 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.753748 30047 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.860105 30132 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.862818 30328 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.877377 30242 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:30.879609 30227 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.880725 30345 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:25:30.905081 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144468us
I20260709 12:25:30.920902 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:30.889585 30352 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:30.923429 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:30.907590 30227 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:30.871084 30208 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.915025 30345 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:30.947680 30380 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:30.949498 30047 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:30.949667 30381 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:30.950189 30047 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:30.952126 30383 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:30.952673 30383 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:30.952850 30225 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:30.890059 30205 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.953871 30383 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:30.955049 30205 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:30.934870 30273 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:30.956657 30275 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:30.956904 30273 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:30.957858 30273 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:30.958495 30377 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:25:30.963440 30382 consensus_queue.cc:1175] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 22 status { last_received { term: 21 index: 17 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:25:30.966177 30379 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:25:30.967592 30378 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.182s	user 0.001s	sys 0.001s
I20260709 12:25:30.924984 29932 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:25:30.926433 30368 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:30.972553 30368 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.972940 30368 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:30.976327 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3565) took 1386 ms. Trace:
I20260709 12:25:30.976886 20877 rpcz_store.cc:276] 0709 12:25:29.589861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.165267 (+575406us) service_pool.cc:224] Handling call
0709 12:25:30.165573 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.165885 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.165910 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.181773 (+ 15863us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.640234 (+458461us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:30.640273 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.640343 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.976013 (+335670us) raft_consensus.cc:1685] finished
0709 12:25:30.976041 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.976251 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":587,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":14612,"thread_start_us":458150,"threads_started":1,"wal-append.queue_time_us":671912}
W20260709 12:25:30.979547 30386 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:25:30.982230 30385 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:25:30.986922 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.991449 20810 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:30.991545 20809 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:31.001657 30383 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:30.992003 30203 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:30.943953 30345 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:31.005419 30203 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.006219 21426 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:25:31.008714 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.009388 20878 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:31.009788 30225 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:31.010452 30225 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.011037 30291 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:31.012588 30225 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:30.963016 30342 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.000s
I20260709 12:25:31.017115 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:31.017221 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.017982 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.019551 21429 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:25:31.020483 20876 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:31.021129 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.026764 21432 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:25:31.027415 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.026150 20876 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:31.036804 30142 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:31.037463 30142 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.037899 30142 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.032853 20876 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.044914 20876 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:31.023448 30342 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.026300 30291 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
W20260709 12:25:31.085371 30299 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.005s
I20260709 12:25:31.035050 30353 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.093982 30227 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.026106 30242 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:31.105897 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.116536 30395 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.124176 30395 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.126680 30395 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.132467 30227 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.144471 30368 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:31.148399 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.148392 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:31.150084 20810 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:31.095041 30345 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.153362 20876 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 20. (term mismatch)
W20260709 12:25:31.155815 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.158460 20876 pending_rounds.cc:85] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 19
W20260709 12:25:31.157111 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 218908us
I20260709 12:25:31.161741 20876 pending_rounds.cc:107] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 25.20
I20260709 12:25:31.162165 20876 raft_consensus.cc:2889] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.163067 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3566) took 1573 ms. Trace:
I20260709 12:25:31.163484 20876 rpcz_store.cc:276] 0709 12:25:29.590125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.197954 (+607829us) service_pool.cc:224] Handling call
0709 12:25:30.198274 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.020377 (+822103us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b54002f8b20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca11c0 00007f9e1eca2231 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c
0709 12:25:31.162760 (+142383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.162997 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1496907648}
I20260709 12:25:31.178835 30203 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.179584 30203 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.182217 30203 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:31.191349 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 164377us
I20260709 12:25:31.191730 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:31.198060 20810 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:31.201894 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.176393 29735 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:31.191673 30368 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.198688 30205 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:31.223110 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:31.224102 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:31.225174 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.226871 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.229885 20875 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:31.230500 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.231987 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.208619 29735 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:31.211218 30211 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:31.238412 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.239884 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.237342 20876 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:31.255537 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.235502 30242 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.262079 30353 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:31.234773 30342 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.263741 30242 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.236093 30205 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.258302 30211 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:31.284581 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3656) took 1272 ms. Trace:
W20260709 12:25:31.286252 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.288566 30398 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:25:31.289353 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3564) took 1699 ms. Trace:
I20260709 12:25:31.289844 20878 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:31.291476 20803 rpcz_store.cc:276] 0709 12:25:30.012351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.100560 (+ 88209us) service_pool.cc:224] Handling call
0709 12:25:30.725039 (+624479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.743455 (+ 18416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.271699 (+528244us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54002b5cd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:31.284520 (+ 12821us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435405568}
I20260709 12:25:31.293779 20881 rpcz_store.cc:276] 0709 12:25:29.589509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.160112 (+570603us) service_pool.cc:224] Handling call
0709 12:25:30.282442 (+122330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.283114 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.289137 (+1006023us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b540005a850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:31.289293 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223458560}
I20260709 12:25:31.295085 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:31.295991 20809 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:31.302624 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.303333 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.303964 20880 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:31.304059 20876 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:25:31.305161 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.306155 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.307819 20877 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:31.308722 20878 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:31.309674 30400 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:31.310066 20881 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:31.310335 30400 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.311543 30368 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:31.312121 30368 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.312544 20882 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:31.307529 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.313517 20880 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:25:31.327615 20878 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:31.330118 20809 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:25:31.330974 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.331871 20879 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:31.332214 30411 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:25:31.292994 30342 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.335459 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3633) took 2106 ms. Trace:
I20260709 12:25:31.336833 20728 rpcz_store.cc:276] 0709 12:25:29.229128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.239870 (+ 10742us) service_pool.cc:224] Handling call
0709 12:25:29.240238 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.065937 (+825699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.335236 (+1269299us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54002977d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:31.335404 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793252736}
I20260709 12:25:31.337782 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.300674 30353 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:31.333535 30194 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 12:25:31.333961 30345 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:31.364068 30360 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.480s	user 0.003s	sys 0.009s
I20260709 12:25:31.340062 30342 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.356650 30224 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:31.345816 30353 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.331027 30211 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.346935 30194 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.390046 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3519) took 1639 ms. Trace:
I20260709 12:25:31.383649 30311 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:31.394009 30311 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:31.384375 30396 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
W20260709 12:25:31.385106 30409 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:25:31.386462 30368 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:31.387131 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.387286 30402 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:25:31.393083 30352 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.393096 30203 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.390480 20884 rpcz_store.cc:276] 0709 12:25:29.751079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.017907 (+266828us) service_pool.cc:224] Handling call
0709 12:25:31.389844 (+1371937us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b5400200b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:31.389988 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60104448}
I20260709 12:25:31.375479 30400 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.415441 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.416075 20807 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:31.389077 30224 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:31.417013 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.426157 30275 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:31.426759 30275 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.427091 30353 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.428999 30275 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.429179 30352 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.432381 29735 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.434145 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3643) took 1271 ms. Trace:
I20260709 12:25:31.434567 20730 rpcz_store.cc:276] 0709 12:25:30.163177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.163757 (+   580us) service_pool.cc:224] Handling call
0709 12:25:30.164080 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.196211 (+ 32131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.433886 (+1237675us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b540028d554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:31.434081 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46935808}
I20260709 12:25:31.395059 30311 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.436592 30311 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:31.444649 30194 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.445601 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1745) took 1108 ms. Trace:
I20260709 12:25:31.446004 20806 rpcz_store.cc:276] 0709 12:25:30.337092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.337255 (+   163us) service_pool.cc:224] Handling call
0709 12:25:31.445439 (+1108184us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:31.445545 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631278336}
I20260709 12:25:31.418321 30203 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.448246 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:31.448899 20808 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:31.450093 30203 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.421355 30211 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:31.425725 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:31.453872 30242 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.418578 30368 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.028s
I20260709 12:25:31.467938 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.468621 30368 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:31.468973 30368 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:31.455356 30205 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:31.456718 30224 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 33.16, Last appended by leader: 16, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:31.458112 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.473848 20808 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:31.475062 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.475991 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.459525 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:31.460294 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:31.461313 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.477452 20882 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:31.477555 20807 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:25:31.478578 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.461607 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.479137 30368 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.480516 30368 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:31.480549 20878 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:31.461983 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.482491 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:31.465404 30178 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.483354 20803 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:31.483527 30178 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.484488 20803 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:31.462257 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:31.462685 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:31.485898 20805 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:31.462981 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:31.458425 30342 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.489408 30342 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:31.489898 30342 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.464780 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.492069 30342 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.492820 30178 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:31.465548 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:31.493574 30387 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.595s	user 0.002s	sys 0.005s
I20260709 12:25:31.493738 20809 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:31.494910 20809 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:31.464812 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.496199 20804 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:31.452378 30142 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:31.466609 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:31.501726 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.501660 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:31.502380 20806 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:31.466960 20801 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:31.467077 30097 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:31.469142 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:31.503165 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43742.9us
I20260709 12:25:31.502409 20804 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.513047 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.513872 21427 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:31.514732 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.515378 21432 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:31.515751 21434 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:31.517650 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:31.518179 21434 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.518115 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.519218 21432 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:31.520933 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.522020 21432 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:31.465778 30203 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:31.464821 30413 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:31.478601 30276 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:25:31.452888 30211 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.524397 20880 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:31.525032 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:31.510115 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.526048 20882 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:31.528291 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.528717 20801 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:31.547642 20808 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:31.548532 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.549944 20804 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:31.523319 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.532073 20806 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:31.536355 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.537812 20805 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:31.542804 21434 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:31.552325 20802 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:25:31.509001 30194 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.523759 30275 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:31.523234 30369 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:31.510746 30342 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:31.563815 30418 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:25:31.567301 30194 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.569540 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.586607 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.546831 30211 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:31.596467 30352 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:31.597066 30352 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.597515 30352 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:31.600608 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.602536 30351 raft_consensus.cc:2764] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:25:31.603300 30351 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:31.585263 30342 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.604096 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1757) took 1115 ms. Trace:
I20260709 12:25:31.604646 21433 rpcz_store.cc:276] 0709 12:25:30.488387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.488503 (+   116us) service_pool.cc:224] Handling call
0709 12:25:31.007163 (+518660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.007403 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.007430 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:31.007473 (+    43us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:31.007505 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.007563 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.603903 (+596340us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400525150. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:31.604032 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182485376}
I20260709 12:25:31.606081 30319 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:31.606419 30342 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.577507 30142 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.583817 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.591578 29939 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:31.540108 30393 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:25:31.593111 30400 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:31.606518 30319 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:31.611030 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.620560 20880 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:31.611241 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.621249 20882 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:31.612003 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.622428 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.576269 30242 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:31.592693 30194 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.616356 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:31.604674 30351 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 12:25:31.623358 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:25:31.629012 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.632352 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.633270 20882 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:31.636435 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.585840 30369 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:31.550431 30413 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:31.641237 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52647.7us
I20260709 12:25:31.621454 30275 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.645617 30395 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 12:25:31.649310 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.649681 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.650720 20728 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:31.656100 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.627390 30242 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:31.664635 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.680819 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.682649 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.682823 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:31.685042 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.686302 20727 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
W20260709 12:25:31.687129 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.689179 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.690330 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:31.691094 20733 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.695659 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.696833 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.697513 20727 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:31.698232 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.699903 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:31.701627 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.703945 30423 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:31.704391 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 183059us
I20260709 12:25:31.704744 30423 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.707957 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.712930 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:31.713194 30425 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:31.714249 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:31.715296 30224 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:31.715691 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:31.716727 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.718080 30224 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:31.718483 30423 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.719313 30423 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:31.720201 30423 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.716406 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:31.723723 20727 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:31.725946 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:31.644856 30194 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:31.638806 30393 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:31.677495 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:31.731150 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 206209us
I20260709 12:25:31.733958 20733 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
W20260709 12:25:31.730621 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 220842us
I20260709 12:25:31.645601 30241 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.647419 30342 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.727948 20727 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:31.725688 30435 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:31.779278 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 101913us
I20260709 12:25:31.729993 30242 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:31.806274 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.922020 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.855860 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.785393 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:31.787949 20810 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:25:31.843075 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60081.6us
I20260709 12:25:31.675645 30275 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.928572 30275 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.675432 30271 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.815716 20727 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:31.795658 30351 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.914674 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 199259us
I20260709 12:25:31.806417 30157 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:25:31.888620 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.844507 30383 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.888244 30435 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.736974 30400 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:31.781263 30319 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.887969 30423 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:31.869293 30369 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:31.807145 30434 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:25:31.808838 30413 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.957563 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:32.006618 20881 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:25:31.958360 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:31.962132 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:31.944077 30402 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:31.729048 30156 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.191s
I20260709 12:25:31.962472 20727 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:31.964170 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:31.964183 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:31.965629 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.068045 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.068480 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.976414 21435 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:31.912130 30370 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:31.737887 30194 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:31.984218 29939 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:31.981773 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:31.957031 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:31.961820 30401 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.735s	user 0.000s	sys 0.003s
I20260709 12:25:31.962357 30271 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.960182 20879 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:31.958796 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:32.075688 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 151808us
I20260709 12:25:31.976552 30435 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:31.998994 29735 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:31.970889 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:32.000631 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:32.127811 20802 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:32.127880 21434 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
W20260709 12:25:32.078100 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152116us
W20260709 12:25:32.128301 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 128446us
I20260709 12:25:31.982856 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:32.128604 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:32.129079 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:32.129398 20875 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:32.014505 30311 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.003s
I20260709 12:25:31.946672 30308 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.842015 30241 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.808523 30342 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.962777 30351 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:31.967796 30319 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:32.086741 21432 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:32.113564 30280 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:31.951781 30440 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:31.968531 30423 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:31.970405 30425 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:32.166853 20735 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:32.085556 30208 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.054s
I20260709 12:25:32.129978 20808 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:32.128268 20802 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:31.865870 30393 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.131283 20810 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:25:32.083236 30275 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:32.135216 20809 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
W20260709 12:25:32.138348 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.087051 30157 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:31.965216 30383 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:31.937517 30395 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.145s
I20260709 12:25:32.086423 29939 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:31.965487 30400 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:32.150074 30186 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.121346 30205 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.134903 30308 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:32.168953 20730 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:25:32.115005 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 389101us
I20260709 12:25:32.174079 20802 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.134903 30271 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.171391 30423 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.144248 30382 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:32.164799 30440 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.088589 30370 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:32.205884 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 241772us
W20260709 12:25:32.212530 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84003.2us
I20260709 12:25:32.215917 20802 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:32.178226 30425 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.211508 30400 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:32.216460 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:32.186394 30156 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:32.239816 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1758) took 1751 ms. Trace:
I20260709 12:25:32.243912 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:32.205835 30438 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:25:32.231416 30430 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 12:25:32.231266 30436 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:32.174266 30353 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:25:32.244419 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 115425us
I20260709 12:25:32.248102 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:32.231729 30422 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:25:32.200717 30275 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.206092 30441 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:32.198412 30157 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.245648 30440 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:32.252516 30395 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.252777 21431 rpcz_store.cc:276] 0709 12:25:30.488704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.488826 (+   122us) service_pool.cc:224] Handling call
0709 12:25:30.489169 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.903137 (+413968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.903211 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.906752 (+  3541us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.077015 (+170263us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:31.077073 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.481667 (+404594us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400477a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:31.481718 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:31.961197 (+479479us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:32.239572 (+278375us) raft_consensus.cc:1685] finished
0709 12:25:32.239595 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.239747 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71610,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":2446,"spinlock_wait_cycles":411296384,"thread_start_us":371859,"threads_started":2,"wal-append.queue_time_us":268977}
I20260709 12:25:32.244406 30451 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.169946 30194 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.289690 30459 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:32.290169 30460 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:32.255965 30369 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.035s
W20260709 12:25:32.290668 30211 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.003s
I20260709 12:25:32.255218 30424 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.052s
I20260709 12:25:32.292725 30275 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:32.293098 30443 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:32.260618 30156 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.248981 30186 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:25:32.296157 30395 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.296492 30393 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:32.234735 30205 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.259399 29932 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:32.259087 30439 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:32.307632 30383 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:32.308528 30370 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.310058 30423 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:32.245585 30382 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:32.333978 30352 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.000s
I20260709 12:25:32.313750 30369 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:32.295943 30241 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:32.269235 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:32.330226 30351 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.331127 30395 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.331530 30205 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:32.331856 30439 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:32.304428 30441 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:32.305776 30460 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:32.320470 30353 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.025s
I20260709 12:25:32.320888 30443 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:32.315711 30211 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.331885 30156 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:32.345299 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76318.6us
I20260709 12:25:32.347363 30423 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:32.267650 29939 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.334424 30352 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.334488 30382 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.337160 30369 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:32.341094 30342 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:32.311869 30451 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.354743 30460 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.357977 30443 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:32.393648 30426 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:25:32.397330 30047 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:32.397859 30047 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:32.400544 30439 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.402310 30439 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:32.377210 30369 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.385664 30352 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:32.386883 30453 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:25:32.355140 30420 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.000s	sys 0.011s
W20260709 12:25:32.388661 30432 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:25:32.401347 30194 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:32.338650 30436 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.031s
I20260709 12:25:32.363626 30423 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:32.413323 30420 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.000s	sys 0.012s
I20260709 12:25:32.421265 30423 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:32.442929 30425 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.454562 30443 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.458164 30443 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:32.434517 30427 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:32.460777 30460 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.440563 30446 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:32.464378 30460 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:32.432124 30441 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:32.471750 30454 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:25:32.465196 30446 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:32.475790 30446 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.477490 30446 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:32.450390 30440 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.480520 30451 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:32.263013 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129660us
I20260709 12:25:32.523666 20726 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:32.524907 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:32.270011 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:32.529237 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 259154us
I20260709 12:25:32.532387 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:32.527532 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.032s	sys 0.062s
W20260709 12:25:32.535516 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.032s	sys 0.063s
I20260709 12:25:32.523842 20728 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:32.535727 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.002s	sys 0.006s
W20260709 12:25:32.537154 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.003s	sys 0.006s
W20260709 12:25:32.537670 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 597719us
I20260709 12:25:32.540555 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:32.319108 20802 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:32.551710 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:32.554858 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3673) took 1086 ms. Trace:
I20260709 12:25:32.555464 20803 rpcz_store.cc:276] 0709 12:25:31.468089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.501243 (+ 33154us) service_pool.cc:224] Handling call
0709 12:25:31.529778 (+ 28535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.530266 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.530314 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.544141 (+ 13827us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.734695 (+190554us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:31.734741 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.734805 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.554233 (+819428us) raft_consensus.cc:1685] finished
0709 12:25:32.554270 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.554656 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":211,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8141,"thread_start_us":157381,"threads_started":1,"wal-append.queue_time_us":705325}
W20260709 12:25:32.559599 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.564116 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:32.361181 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:32.569780 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 208501us
I20260709 12:25:32.571959 29903 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
W20260709 12:25:32.573522 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 356956us
I20260709 12:25:32.575711 29892 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:32.576339 29892 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:32.576709 21431 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 12:25:32.577281 21431 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 17
I20260709 12:25:32.577603 21431 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 12:25:32.578042 21431 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:32.573565 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:32.580508 21432 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:32.580366 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:32.582213 21434 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:32.584909 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:32.586895 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:32.587550 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:32.574421 29903 raft_consensus.cc:2764] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:25:32.588611 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:32.590466 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:32.592752 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:32.593920 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:32.578218 29892 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.596566 20624 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 22 to 24, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:32.601284 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:32.603900 21435 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:25:32.605785 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:32.606355 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.610740 20729 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:32.611342 29903 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:32.613912 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:32.614630 20731 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:32.615396 20726 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:25:32.616668 20728 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:32.612528 29903 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.620502 29903 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.574885 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:32.623117 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:32.624420 20731 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:32.628473 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.628123 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:32.628697 29875 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:32.629330 21426 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:32.632115 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:32.633167 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:32.633963 21433 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:32.398795 30047 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 33.16, Last appended by leader: 16, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.636513 30271 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:32.637251 30047 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:32.638432 30342 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:32.366503 29932 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.276s
I20260709 12:25:32.639500 30227 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:32.639868 29932 consensus_queue.cc:1243] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 31.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:32.382673 30178 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:32.641114 30178 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:32.641424 30186 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:32.641988 30186 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:32.637470 30271 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.413847 30369 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:32.638595 30224 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:32.624200 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49577.3us
I20260709 12:25:32.614969 20729 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 19.
I20260709 12:25:32.590277 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:32.669914 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.704182 30462 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.700336 29875 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
W20260709 12:25:32.704268 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 86049us
I20260709 12:25:32.707224 30462 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.708217 30443 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:32.709513 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:32.710664 30462 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:32.649333 30345 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:32.649613 30211 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:32.644599 30448 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:32.695370 30227 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:32.650053 30229 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:32.753342 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:32.648990 30323 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.109s
I20260709 12:25:32.647351 30271 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:32.775911 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 145694us
W20260709 12:25:32.787518 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.648914 30186 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.643146 30178 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.801189 20729 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:32.772855 30460 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:32.801115 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:32.806382 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:32.807610 29892 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:32.811400 20847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.014s	sys 0.046s
I20260709 12:25:32.817003 20624 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 19 to 22, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 22 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:32.818985 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:32.823952 20733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.004s	sys 0.016s
I20260709 12:25:32.827354 30413 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.072s
W20260709 12:25:32.804193 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75117.1us
I20260709 12:25:32.782289 30342 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:25:32.844149 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:32.854467 21426 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:32.844367 30099 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:32.865097 30242 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.918s	user 0.012s	sys 0.007s
I20260709 12:25:32.901664 30242 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.844290 30203 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:32.901536 30099 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.849087 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:32.870291 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:32.907298 20733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.005s	sys 0.016s
I20260709 12:25:32.844748 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:32.909708 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85086.2us
W20260709 12:25:32.843384 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.020s	sys 0.030s
I20260709 12:25:32.874472 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:32.876178 30469 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:25:32.915611 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:32.916164 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.916687 30242 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:32.916942 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 162182us
I20260709 12:25:32.917892 30178 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:32.859360 30274 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:32.853888 30400 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.572s	user 0.012s	sys 0.000s
I20260709 12:25:32.921231 29904 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.828923 30353 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.851303 30027 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:32.917114 20732 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:25:32.856535 30424 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:25:32.865782 30368 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.385s	user 0.016s	sys 0.004s
I20260709 12:25:32.870813 30425 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:32.851524 30460 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:32.936713 20731 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:25:32.940392 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 111245us
W20260709 12:25:32.897593 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 124532us
I20260709 12:25:32.956646 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3395) took 1261 ms. Trace:
I20260709 12:25:32.956233 30353 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.951903 20877 raft_consensus.cc:2410] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:25:32.938254 30400 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:32.959714 21434 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
W20260709 12:25:32.948853 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.951072 20633 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 293) took 134823 ms (client timeout 15000 ms). Trace:
I20260709 12:25:32.921905 30461 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:25:32.951845 30471 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.953228 30440 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:32.955161 20801 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:32.956044 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:32.932657 30027 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:32.932193 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 655948us
I20260709 12:25:32.955958 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:32.955907 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:32.963876 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:32.970928 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100860us
I20260709 12:25:32.932380 30424 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:32.959352 29904 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:32.957072 20733 rpcz_store.cc:276] 0709 12:25:31.695573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.942845 (+247272us) service_pool.cc:224] Handling call
0709 12:25:31.944446 (+  1601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.944693 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:25:32.245987 (+301294us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.947670 (+701683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.956230 (+  8560us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:32.956317 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.956405 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.956590 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":121247,"prepare.run_cpu_time_us":4696,"prepare.run_wall_time_us":26535,"raft.queue_time_us":4581,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":8871,"thread_start_us":895064,"threads_started":2}
I20260709 12:25:32.942138 29875 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:32.974174 21433 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:32.972101 21435 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 20.
I20260709 12:25:32.991205 30478 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:32.991380 20805 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.991930 30478 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:32.992408 20806 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:32.992457 30478 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:32.968698 30425 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:32.997483 30425 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:32.998281 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.001652 30400 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:32.969996 30438 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:33.003868 30438 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.006394 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1700) took 1390 ms. Trace:
I20260709 12:25:33.007781 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:32.934108 30047 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.009642 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.009932 29939 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:33.009394 30047 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.001123 30027 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.006959 20877 rpcz_store.cc:276] 0709 12:25:31.615790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.616053 (+   263us) service_pool.cc:224] Handling call
0709 12:25:33.006077 (+1390024us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54000a2dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:33.006287 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3026935168}
I20260709 12:25:32.985342 20624 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 18 to 23, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 23 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:33.034403 30099 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:33.034313 20624 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 14 to 18, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 18 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:32.961090 20633 rpcz_store.cc:269] 0709 12:23:18.000133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.000369 (+   236us) service_pool.cc:224] Handling call
0709 12:25:32.823067 (+134822698us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:23.357767 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:23.358587 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:23.358660 (+    73us) write_op.cc:435] Acquired schema lock
0709 12:25:23.358683 (+    23us) tablet.cc:662] Decoding operations
0709 12:25:23.438224 (+ 79541us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:23.438341 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:23.438374 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:23.438708 (+   334us) tablet.cc:701] Row locks acquired
0709 12:25:23.438764 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:25:23.438927 (+   163us) write_op.cc:270] Start()
0709 12:25:23.439087 (+   160us) write_op.cc:276] Timestamp: P: 1783599923438863 usec, L: 0
0709 12:25:23.439125 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:25:23.439989 (+   864us) log.cc:844] Serialized 541 byte log entry
0709 12:25:23.937305 (+497316us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:24.322390 (+385085us) op_driver.cc:464] REPLICATION: finished
0709 12:25:25.307649 (+985259us) write_op.cc:301] APPLY: starting
0709 12:25:25.327407 (+ 19758us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:25.473289 (+145882us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:25.473326 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:25.474078 (+   752us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:25.474356 (+   278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:25:25.474423 (+    67us) write_op.cc:312] APPLY: finished
0709 12:25:25.474733 (+   310us) log.cc:844] Serialized 25 byte log entry
0709 12:25:26.147584 (+672851us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1bb950ac 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b
0709 12:25:26.328141 (+180557us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:26.328227 (+    86us) write_op.cc:454] Released schema lock
0709 12:25:26.328289 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:26.328739 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:26.328767 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":640765,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":913116,"apply.run_cpu_time_us":51512,"apply.run_wall_time_us":1049388,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":473709,"prepare.run_cpu_time_us":31553,"prepare.run_wall_time_us":386561,"raft.queue_time_us":266553,"raft.run_cpu_time_us":19694,"raft.run_wall_time_us":502317,"replication_time_us":825307,"spinlock_wait_cycles":59702784,"thread_start_us":1741700,"threads_started":5,"wal-append.queue_time_us":1146899,"wal-append.run_cpu_time_us":45371,"wal-append.run_wall_time_us":2454582}]]}
I20260709 12:25:33.036345 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:33.036283 20624 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 20 to 23, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 23 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:33.037384 20881 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:33.038170 20624 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 18 to 21, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 21 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:33.047361 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.022984 30471 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:33.046453 20727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.013s	sys 0.014s
I20260709 12:25:33.023365 29904 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:33.064882 20727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.013s	sys 0.015s
I20260709 12:25:33.054540 20624 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 21 to 22, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 22 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:33.070533 30471 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:33.069902 20624 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 15 to 23. New cstate: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:33.072615 20624 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 20 to 22, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 22 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:33.073329 30424 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:33.076170 29735 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.948s	user 0.008s	sys 0.015s
I20260709 12:25:33.075139 30424 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:33.076486 30203 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:33.080677 30203 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:33.075529 20875 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:25:33.079636 30460 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.027952 30368 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:33.077211 20801 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 16. Preceding OpId from leader: term: 34 index: 17. (index mismatch)
I20260709 12:25:33.077266 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:33.086076 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:33.091881 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3394) took 1409 ms. Trace:
I20260709 12:25:33.091986 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:33.077279 20624 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:33.092417 20732 rpcz_store.cc:276] 0709 12:25:31.682461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.687126 (+  4665us) service_pool.cc:224] Handling call
0709 12:25:31.687487 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.948571 (+1261084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.090567 (+141996us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003b4054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:33.091793 (+  1226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2795372928}
I20260709 12:25:33.076840 30047 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:33.096379 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:33.099920 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.108373 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.100752 20728 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:33.106011 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3397) took 1410 ms. Trace:
I20260709 12:25:33.114063 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:33.116029 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:33.117082 30273 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:33.117810 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:33.118031 30273 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:33.099131 29735 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:33.119045 30470 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:25:33.119387 30319 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.953s	user 0.019s	sys 0.002s
I20260709 12:25:33.119846 20624 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 20 to 21, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 21 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:33.121735 20624 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 22 to 24, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:33.124528 30291 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:33.111682 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:33.125115 30480 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.000s	sys 0.004s
W20260709 12:25:33.128415 30439 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.725s	user 0.006s	sys 0.011s
I20260709 12:25:33.131456 30345 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:33.135421 30474 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:25:33.137048 20734 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:33.141136 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:33.147537 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:33.148526 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55026.3us
W20260709 12:25:33.150856 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.149662 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:33.147002 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 238508us
W20260709 12:25:33.155293 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.128837 30186 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:33.155794 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.111217 30435 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.947s	user 0.012s	sys 0.005s
I20260709 12:25:33.156836 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:33.154160 21427 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:33.149416 30271 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.104077 30353 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:33.163077 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45454.5us
I20260709 12:25:33.168905 30194 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.169317 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:33.191143 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:33.172348 30308 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.929s	user 0.009s	sys 0.011s
I20260709 12:25:33.189277 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.189504 30440 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.190156 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.191983 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.193683 30463 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:25:33.110610 20727 rpcz_store.cc:276] 0709 12:25:31.696079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.087500 (+391421us) service_pool.cc:224] Handling call
0709 12:25:32.123438 (+ 35938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.136322 (+ 12884us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:32.164401 (+ 28079us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.096048 (+931647us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.105546 (+  9498us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:33.105621 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.105743 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.105951 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":851,"prepare.queue_time_us":30381,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":2588,"raft.queue_time_us":5144,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":5532,"spinlock_wait_cycles":37659136,"thread_start_us":900534,"threads_started":1,"wal-append.queue_time_us":907344}
I20260709 12:25:33.202278 20810 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 29 because replica has last-logged OpId of term: 28 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:25:33.195528 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.214682 20730 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:25:33.212051 20881 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:33.158349 30027 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:33.222416 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75548.1us
I20260709 12:25:33.220134 20884 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:33.193054 29875 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:33.192993 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.193632 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.194442 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:33.237114 20878 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:33.106796 30368 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.236207 20624 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 14 to 27, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:33.259275 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:33.262633 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:33.223309 30443 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.033s
I20260709 12:25:33.195434 30319 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.090s
I20260709 12:25:33.263293 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:33.276888 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 82545.8us
I20260709 12:25:33.256276 30342 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:33.281076 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.283787 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:33.284054 30368 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:33.285146 20880 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
W20260709 12:25:33.289304 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64537.3us
I20260709 12:25:33.295450 30420 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.295636 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.296078 30420 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.298381 30420 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:33.299737 30423 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.845s	user 0.008s	sys 0.006s
I20260709 12:25:33.299459 20624 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 25 to 26, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 26 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:33.300312 30423 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.301445 30369 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.301265 20624 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 14 to 22, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 22 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:33.302194 30369 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.303575 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:33.273638 29875 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.272197 30345 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.307875 30345 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.308570 29875 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:33.309471 29904 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:33.309959 29904 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.302963 20624 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:33.310372 29904 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:33.310525 30345 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.311626 20624 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:33.313901 20624 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 23 to 27. New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:33.316342 30242 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.317107 30242 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.220429 30308 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.319048 30448 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.319808 30448 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.320741 30308 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:33.266413 30440 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.321915 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:33.323642 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:33.323979 30440 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:33.323750 30342 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.326006 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:33.337188 20734 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:33.336318 21431 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: [23.18-23.18]
W20260709 12:25:33.327454 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47949.3us
I20260709 12:25:33.343909 21434 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:33.221323 30194 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.350395 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:33.358357 20876 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:33.352396 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.290644 30203 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.217247 30435 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.365185 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.275861 30271 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:33.370904 20624 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 19 to 28. New cstate: current_term: 28 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:33.372732 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.373530 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.379287 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.303076 30423 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.375306 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:33.392405 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.382557 20727 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:33.378645 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.398149 20731 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:33.403873 20881 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:33.350277 30229 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:33.380573 30224 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:33.370539 30368 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:33.409960 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.425132 20624 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 18 to 25, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 25 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:33.425752 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.433562 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.420739 20804 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
W20260709 12:25:33.433308 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 214482us
W20260709 12:25:33.424609 30274 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.003s
W20260709 12:25:33.424362 30178 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.506s	user 0.004s	sys 0.006s
I20260709 12:25:33.422857 30291 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:33.450965 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:33.486102 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.433174 20883 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 18. (term mismatch)
I20260709 12:25:33.419987 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.490021 20883 pending_rounds.cc:85] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 17
I20260709 12:25:33.490375 20883 pending_rounds.cc:107] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 30.18
I20260709 12:25:33.490662 20883 raft_consensus.cc:2889] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:33.490392 20875 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:33.448957 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:33.494119 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:33.495114 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:33.474377 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:33.455267 20809 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:33.490690 20807 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:33.428714 29875 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:33.481233 20624 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index -1 to 12, term changed from 9 to 20, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to <none>, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 20 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:25:33.498788 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.458045 30224 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:33.419685 30229 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.523623 21425 tablet_service.cc:1558] Processing DeleteTablet for tablet 1047135f71834dbab6b869eb3c081b81 with delete_type TABLET_DATA_TOMBSTONED (TS 50c1ab13351d4427b836fbb449c6d7bb not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:42714
I20260709 12:25:33.525074 30298 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.525602 30298 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.526063 30229 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:33.526798 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 270234us
I20260709 12:25:33.439853 30481 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.527640 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.527563 30481 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.528980 20729 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:25:33.530786 30481 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.514290 20876 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:33.515805 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.414363 30420 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.078s
W20260709 12:25:33.439697 30477 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 12:25:33.523764 30369 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:33.404203 30463 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 12:25:33.413446 30475 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.092s
I20260709 12:25:33.535339 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:33.505090 29875 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.480619 29939 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:25:33.551062 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:33.517992 20624 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 16 to 19, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 19 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:25:33.449398 30368 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.539142 20877 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:25:33.544186 30047 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:33.433725 30203 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.545154 30439 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:33.427603 30319 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.547251 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52221.6us
W20260709 12:25:33.513028 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.434666 30342 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.553982 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:33.461050 30443 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:25:33.546172 30423 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:33.581192 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 99271.1us
W20260709 12:25:33.576565 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 114163us
W20260709 12:25:33.558360 30485 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 12:25:33.583230 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55634.1us
I20260709 12:25:33.583642 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:33.559610 30178 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:33.587010 20876 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:25:33.599820 21434 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:33.583760 30368 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:33.438318 30478 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.582912 30229 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:33.605386 20801 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:33.584153 30047 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.591521 30369 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:33.588606 29892 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:33.585466 30274 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.035s
I20260709 12:25:33.639575 20726 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:33.641194 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90214.9us
W20260709 12:25:33.635461 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 186639us
I20260709 12:25:33.620838 30047 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:33.626969 29892 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:33.600992 30319 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.655885 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.626173 30369 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.658874 30369 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:33.659162 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:33.650352 30353 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:25:33.664315 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:33.665208 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:33.619290 30439 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.674091 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:33.656936 30424 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.003s
I20260709 12:25:33.680616 20733 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 25 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:25:33.681814 20876 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:33.656031 30194 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:33.681108 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:33.685588 20877 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:33.674176 30435 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:33.701277 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58568.2us
I20260709 12:25:33.702437 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:33.710727 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46853.4us
I20260709 12:25:33.714313 29939 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:33.715035 29939 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:33.716179 29939 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:33.720532 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.727358 30463 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:25:33.727686 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73339.8us
I20260709 12:25:33.727346 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.730602 20876 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 21.
I20260709 12:25:33.699522 30423 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:33.700613 29875 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.743254 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:33.774617 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46184.6us
I20260709 12:25:33.688817 29904 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:33.778915 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104880us
W20260709 12:25:33.781026 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.736976 30481 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.737264 30470 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.683323 30224 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.796190 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.740283 30423 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:33.834396 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1737) took 1204 ms. Trace:
I20260709 12:25:33.792508 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:33.815631 30478 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:33.834964 21431 rpcz_store.cc:276] 0709 12:25:32.601289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.612521 (+ 11232us) service_pool.cc:224] Handling call
0709 12:25:32.613230 (+   709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.340163 (+726933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.802329 (+462166us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54005f3a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:33.805906 (+  3577us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632713472}
W20260709 12:25:33.817126 30496 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:33.823406 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.824117 30481 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.825883 30439 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:33.828112 30488 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:25:33.828881 29875 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:33.806187 30425 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:33.851670 20876 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
W20260709 12:25:33.830834 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140105us
I20260709 12:25:33.840407 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:33.843736 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141390us
W20260709 12:25:33.829296 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 130696us
I20260709 12:25:33.853350 30194 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:33.838683 30186 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.673s	user 0.015s	sys 0.000s
I20260709 12:25:33.831010 30353 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:25:33.837993 30423 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:33.845968 21399 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 2.244s (SENT) [suppressed 11 similar messages]
I20260709 12:25:33.876490 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:33.852162 30425 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.857018 30229 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:33.917884 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:33.867745 30495 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:25:33.838379 30470 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:33.916810 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 139805us
I20260709 12:25:33.863924 30274 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:33.921139 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47397.8us
I20260709 12:25:33.872202 30273 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.917201 30425 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:33.941263 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 143846us
W20260709 12:25:33.941005 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64665.2us
I20260709 12:25:33.957554 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:33.969352 30503 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:25:33.971107 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1546) took 1422 ms. Trace:
I20260709 12:25:33.971262 30229 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.971807 20730 rpcz_store.cc:276] 0709 12:25:32.548837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.587097 (+ 38260us) service_pool.cc:224] Handling call
0709 12:25:33.970818 (+1383721us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54000e74d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:33.970995 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249169664}
I20260709 12:25:33.975925 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:33.976686 20881 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:33.977921 20881 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:33.983629 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.984496 20732 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:33.949334 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:33.985857 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55708.2us
I20260709 12:25:33.985987 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.986855 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:33.987912 30194 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:33.988394 30194 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:33.988862 30194 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:33.990115 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.991097 20726 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 27 because replica has last-logged OpId of term: 25 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 19.
I20260709 12:25:33.992940 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.994040 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:33.994756 20730 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:33.994828 30368 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:33.997277 30368 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:33.998039 30368 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:33.927481 30186 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:25:34.000134 20881 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:25:34.001163 30186 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:34.001551 30186 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:34.001719 30298 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:33.932739 30319 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.007880 30435 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:25:34.008244 30446 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.530s	user 0.018s	sys 0.004s
I20260709 12:25:34.010689 20880 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:34.012784 30481 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.013983 30448 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.014739 30470 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:34.015064 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3679) took 2540 ms (client timeout 2306 ms). Trace:
W20260709 12:25:34.015520 20807 rpcz_store.cc:269] 0709 12:25:31.474941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.478318 (+  3377us) service_pool.cc:224] Handling call
0709 12:25:34.014823 (+2536505us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:34.014963 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4607357184}
I20260709 12:25:34.016170 29904 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.017356 30274 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.018092 30274 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:33.940809 30342 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:34.032771 30423 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:34.033999 30504 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:25:34.036710 29892 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.037220 30319 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:34.038467 20805 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
W20260709 12:25:34.051640 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.002524 30186 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:34.058257 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 129599us
I20260709 12:25:34.066871 20807 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: [30.18-30.18]
I20260709 12:25:33.931542 30273 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.041013 30178 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:34.071280 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 113819us
W20260709 12:25:34.070021 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62642.6us
I20260709 12:25:34.077543 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.078616 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:34.079276 21433 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:34.056999 30242 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.087240 30274 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.088176 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:34.088439 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95758.9us
I20260709 12:25:34.089154 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.089634 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.090171 21427 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:34.090842 30437 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:34.091406 30437 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.091794 30437 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:34.094199 20805 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:34.094749 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:34.095297 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.095715 29903 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:34.096163 29903 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:34.096498 29892 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:34.091913 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.097091 29903 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.098048 21428 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:34.098675 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.100607 29903 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:34.103932 20876 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:34.063403 30463 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.104883 30463 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.105713 30273 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:34.085075 30178 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.043321 29904 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.101701 30506 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:34.110306 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.102332 30319 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.101725 30502 tablet_replica.cc:333] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:25:34.101857 30501 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:34.077998 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.130110 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.132849 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.093669 30369 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.101186 30423 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.115427 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:34.116727 30186 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:34.117650 30506 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:25:34.135064 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.142377 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.140600 20801 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:34.145998 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.146261 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.148253 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.154860 20734 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:34.153249 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 307337us
I20260709 12:25:34.148257 20727 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:34.158159 20729 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:34.159108 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.159523 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.161298 20731 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:34.126767 30501 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.135174 30481 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:34.168390 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:34.157653 30509 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:25:34.172380 30047 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.001s
I20260709 12:25:34.172822 30369 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.174861 30319 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.175594 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.176189 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.126020 30274 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:25:34.184098 20805 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:34.185791 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.140823 30463 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.126480 29904 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.190172 20803 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 22: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:34.146806 30506 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:34.211282 21433 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:34.178344 30470 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.207891 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:34.208482 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:34.185288 30446 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.185633 30481 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:34.178426 30501 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:34.214321 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.225817 20879 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:34.214702 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.226207 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.226540 20876 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:34.178120 30425 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.203339 30353 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.228117 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:34.223495 20880 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:34.228780 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.240816 20875 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
W20260709 12:25:34.240602 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.242817 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.187130 30047 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.247221 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:34.249286 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.221545 30463 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:34.225703 30481 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:34.260293 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.248656 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.239831 30509 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:34.263847 20881 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:34.259863 30448 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:34.268653 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.270056 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.215888 30465 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.036s
I20260709 12:25:34.258299 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.273670 20882 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:34.249797 30342 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:34.258327 30463 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.276754 20877 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:34.227205 30513 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.259130 30027 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:34.261034 30435 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:34.269029 30509 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.287002 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.289582 20884 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:34.276057 30178 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.271814 30345 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:25:34.299618 30435 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:34.300144 30435 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:34.300575 30435 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:34.300724 30047 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.278563 30439 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:34.301849 30439 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.302424 29904 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.283746 30463 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:34.304533 30439 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.289024 30242 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:34.305668 30242 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:34.290966 30509 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:34.306823 30242 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.258106 30353 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.289055 30443 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:25:34.311452 30353 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.250048 30470 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.313717 30470 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.277520 30342 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:34.314453 30342 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:34.315651 20876 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:25:34.318285 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.318735 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.319128 20883 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:34.320303 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.320930 20883 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:34.321175 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.321990 20884 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:34.282685 30506 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.323755 30506 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:34.328734 30425 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:34.337653 20909 heartbeater.cc:646] Failed to heartbeat to 127.19.36.254:46039 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.36.254:46039 timed out after 15.000s (SENT)
I20260709 12:25:34.340471 30273 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:34.340955 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:34.342545 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.348232 20881 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:25:34.349915 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.259424 30298 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:25:34.225420 30446 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.358924 20877 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:25:34.282717 30517 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:34.356223 20884 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:34.287288 30027 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:34.380020 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.380391 20875 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:34.399310 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.401407 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.359223 30423 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:34.361322 30353 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:34.375910 30273 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:34.359388 30470 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:25:34.356168 30509 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.430748 30509 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.372290 30194 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.408319 30353 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:34.404055 29892 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:34.379220 30027 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:34.439152 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:34.453670 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44504.3us
W20260709 12:25:34.460793 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 203842us
I20260709 12:25:34.406776 30423 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:34.435273 30273 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.437350 30353 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:34.460289 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 217556us
I20260709 12:25:34.434442 30194 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:34.435375 30517 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:34.440032 29892 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:34.475088 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 260396us
I20260709 12:25:34.460513 30298 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:34.493938 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 218020us
I20260709 12:25:34.494238 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:34.490393 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 232140us
W20260709 12:25:34.490571 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51502.2us
I20260709 12:25:34.357951 30513 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.431174 30501 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.497000 21433 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:34.452903 30178 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:34.431147 30481 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.442698 30435 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.474421 30273 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:34.468976 30423 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:34.500878 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.432443 30463 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.511039 20884 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 19. (term mismatch)
I20260709 12:25:34.506186 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.475409 30099 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:34.501991 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.515246 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.508688 21430 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:34.509168 21432 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:34.419525 30437 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.509001 21428 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:34.508690 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.511570 20884 pending_rounds.cc:85] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 18
I20260709 12:25:34.526202 20884 pending_rounds.cc:107] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 27.19
I20260709 12:25:34.526576 20884 raft_consensus.cc:2889] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:34.498526 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.511868 30448 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:34.529204 21431 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:34.526460 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.508049 30194 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:34.530637 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:34.540243 30514 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:34.540709 30531 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:25:34.548947 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:34.549839 20729 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:34.550275 30435 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.550997 30435 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:34.551033 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.553102 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.542125 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:34.553977 30224 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:34.554931 21433 raft_consensus.cc:2368] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:25:34.555968 30502 raft_consensus.cc:2243] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEARNER]: Raft consensus shutting down.
I20260709 12:25:34.556735 30502 pending_rounds.cc:70] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Trying to abort 1 pending ops.
I20260709 12:25:34.557010 30502 pending_rounds.cc:77] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting op as it isn't in flight: id { term: 21 index: 15 } timestamp: 7305625245024886784 op_type: NO_OP noop_request { }
I20260709 12:25:34.557432 30502 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEARNER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:25:34.557832 30502 raft_consensus.cc:2272] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEARNER]: Raft consensus is shut down!
W20260709 12:25:34.559892 30519 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:34.559799 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.561810 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.562763 20807 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:34.564033 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:34.565507 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64697.5us
I20260709 12:25:34.566646 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.528782 30369 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.567649 30369 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:34.569948 30369 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.571504 29875 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:34.572024 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.572897 21431 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:34.574155 21431 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:34.575110 30368 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:34.532850 30099 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.575554 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.572217 29875 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.576311 29875 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.577289 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:34.577816 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.578007 21432 raft_consensus.cc:2368] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:25:34.578392 29875 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.578766 30526 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:34.579123 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.580245 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.580837 21429 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:34.580937 30342 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.575960 20731 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:34.585266 30501 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.536628 30194 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.587023 30194 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:34.592900 30465 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:34.543937 30446 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:34.576653 21435 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:34.536341 29892 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.554014 20726 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:34.577419 30099 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.578408 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:34.610498 30523 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.097s	user 0.002s	sys 0.002s
I20260709 12:25:34.611655 30437 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:34.612709 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.613461 21433 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:34.613600 29892 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:34.614616 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.619817 20731 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:34.620512 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:34.603875 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70209.5us
I20260709 12:25:34.623365 20883 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:34.576787 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:34.625731 21430 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:34.575686 30368 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:34.602993 21434 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:25:34.605156 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.628886 20876 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:25:34.629467 21433 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:34.629364 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.630136 20882 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:34.605201 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.631004 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.606181 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.632162 20808 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:34.546065 30514 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.635427 30514 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:34.635977 30342 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.531383 30298 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:34.605479 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.596396 30526 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.639794 30526 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:34.641245 30517 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:34.641779 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:34.642184 30517 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:34.642685 30517 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.544309 30481 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:34.545634 30463 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:34.546536 30448 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:34.578789 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:34.540709 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.627466 20879 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:34.656626 20810 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:34.629985 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.602856 30274 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.631492 20880 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:34.543476 30513 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:34.672849 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 132240us
I20260709 12:25:34.639827 20805 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:34.659483 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.673691 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.700657 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.707095 20803 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:34.706738 20881 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 16. (term mismatch)
I20260709 12:25:34.709283 20881 pending_rounds.cc:85] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 15
I20260709 12:25:34.709731 20881 pending_rounds.cc:107] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:25:34.710196 20881 raft_consensus.cc:2889] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:34.681957 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:34.710269 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 82097.3us
I20260709 12:25:34.713045 20876 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 16. Preceding OpId from leader: term: 34 index: 17. (index mismatch)
I20260709 12:25:34.711759 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:34.681051 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.715675 20802 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:34.711860 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.683370 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.719419 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.721983 20731 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
W20260709 12:25:34.720715 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.717304 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.727944 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.728339 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.729295 20734 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
W20260709 12:25:34.730329 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.732762 20882 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:34.643059 30517 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:34.672413 30178 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:34.739159 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:34.740391 20733 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:25:34.740442 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.741953 20878 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:34.734419 20883 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:34.733445 20882 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:34.752828 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:34.753073 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:34.753640 20728 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:34.741698 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:34.754885 21434 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:34.752142 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108475us
I20260709 12:25:34.770458 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:34.732309 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58698.2us
I20260709 12:25:34.735666 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:34.773483 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:34.774719 20732 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 26: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:34.781672 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 99837.1us
I20260709 12:25:34.784126 20734 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
W20260709 12:25:34.786494 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51339.2us
I20260709 12:25:34.787312 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:34.773751 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:34.788110 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:34.718036 30481 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:34.736533 29904 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.792682 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:34.780108 20882 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.783037 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:34.707145 30298 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.081s
I20260709 12:25:34.672483 30463 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.119s
I20260709 12:25:34.762004 30532 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.751355 30536 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:34.700160 30274 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:34.827756 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 206401us
I20260709 12:25:34.807884 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.806367 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.806339 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:34.785825 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 105391us
I20260709 12:25:34.833207 20882 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:34.763073 30178 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.848115 20805 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:34.759323 30520 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.111s
W20260709 12:25:34.870714 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 127634us
I20260709 12:25:34.838410 30481 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:34.888496 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:34.889195 30481 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.804085 30345 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.045s
I20260709 12:25:34.885938 20806 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:34.793140 30437 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.053s
I20260709 12:25:34.805483 30529 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:25:34.847901 30178 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:34.860285 30512 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:34.842659 30435 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:25:34.896180 30224 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:34.841365 29904 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.901067 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.853645 30242 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.907905 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:34.890381 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 107476us
I20260709 12:25:34.889894 30443 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.885409 30513 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:34.884514 30515 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:34.901304 30542 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:34.912281 30512 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.938253 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:34.931615 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 151960us
I20260709 12:25:34.929288 30047 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:34.884050 30423 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.844830 30532 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.957448 30502 ts_tablet_manager.cc:1916] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:34.964773 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.965428 20726 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:34.966616 20726 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:25:34.967612 30551 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:34.969070 20729 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:34.969158 20850 leader_election.cc:400] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:25:34.969635 20850 leader_election.cc:403] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:25:34.970367 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:34.971555 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.972944 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.950515 30512 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:34.977968 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:34.978663 20802 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:34.980830 30463 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:34.956171 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:34.935760 29903 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.832s	user 0.011s	sys 0.002s
I20260709 12:25:34.913234 30544 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:34.957283 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.954296 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.947607 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:34.991120 20732 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:34.961326 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.991762 20734 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:34.971524 30470 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:34.992226 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:34.897675 30531 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
W20260709 12:25:34.986488 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46851.6us
I20260709 12:25:34.997395 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.988610 20803 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
W20260709 12:25:34.989557 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.939842 30542 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:34.981846 30178 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:35.005424 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57892.9us
I20260709 12:25:34.926317 30448 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:25:34.951086 30047 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:34.991477 30503 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:34.936319 30242 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:35.006637 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68481.7us
I20260709 12:25:35.027432 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:35.008245 20728 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:35.029480 20734 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:25:35.018270 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:35.030846 20730 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:35.020767 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:34.987774 30551 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:34.991575 29939 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
W20260709 12:25:35.025564 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.995736 30544 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:35.071841 30353 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.000s
I20260709 12:25:34.999030 30470 consensus_queue.cc:1243] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:25:35.072017 30047 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.074195 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.058534 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.014565 30437 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:34.939245 30443 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:34.958676 30513 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.077471 20804 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:35.077317 21434 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:35.078398 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.086575 20805 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:34.986348 30027 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.003371 30536 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:35.091866 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:35.092478 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54784.8us
I20260709 12:25:35.090875 21427 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
W20260709 12:25:35.091888 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 321483us
I20260709 12:25:35.100605 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.011299 30509 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.080s
I20260709 12:25:35.091334 20810 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:35.090351 30541 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:35.117115 30437 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:35.164841 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:35.127305 20731 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:35.099805 30047 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:35.087356 30544 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:25:35.139741 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 112424us
I20260709 12:25:35.152575 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.177054 20805 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
W20260709 12:25:35.177186 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 389932us
I20260709 12:25:35.156231 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.156718 20880 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:35.178232 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:35.099292 30559 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:35.159358 21430 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:35.185674 21429 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:35.138252 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.159818 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.090211 30353 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.187429 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:35.124222 29892 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 12:25:35.168507 21434 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:25:35.153680 21433 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:35.189028 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.193142 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:35.194247 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:35.199685 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.200470 21427 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
W20260709 12:25:35.202342 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64172.6us
I20260709 12:25:35.206655 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.207947 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.212594 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:35.214435 20880 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:35.215862 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:35.221289 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.223836 21434 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:35.157459 30542 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.287815 30542 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:35.159811 30548 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:35.168424 30274 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:35.288340 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80490.9us
I20260709 12:25:35.286787 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:35.289352 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73576.6us
I20260709 12:25:35.318930 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:35.168088 30242 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:35.202512 30559 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:35.323450 30463 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:35.325956 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.330948 30274 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:35.331720 30551 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:35.331545 30437 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 25.21, Last appended by leader: 21, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:35.333451 30194 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.000s
I20260709 12:25:35.334067 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:35.333947 30194 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.334807 20877 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:35.335318 30559 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:35.338018 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:35.338665 20877 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 22.
I20260709 12:25:35.339612 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:35.340150 20877 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:35.341960 30536 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:35.343561 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 165398us
I20260709 12:25:35.345597 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:35.216580 30541 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:35.357659 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:35.358670 20805 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:35.360774 20882 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.001s
I20260709 12:25:35.314215 30446 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.210533 30560 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:35.368518 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.315919 29892 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:35.395917 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 215504us
W20260709 12:25:35.355821 30545 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.425s	user 0.000s	sys 0.001s
W20260709 12:25:35.349061 30273 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.820s	user 0.013s	sys 0.000s
I20260709 12:25:35.398001 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.346987 29904 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:35.390779 20882 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:35.403175 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:35.365820 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:35.407254 21427 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:35.409814 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44101.5us
I20260709 12:25:35.361519 30423 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:35.418200 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:35.423125 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 77570.7us
I20260709 12:25:35.424463 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:35.388856 30560 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.395879 29939 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:35.399497 30548 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:35.331578 30274 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:35.416461 30517 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.000s
I20260709 12:25:35.437997 20733 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:25:35.441092 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:35.442435 20729 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:25:35.417447 30515 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:35.417976 30546 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.417264 30178 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:35.473222 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48927.3us
I20260709 12:25:35.397363 30446 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.396973 30463 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.432634 30242 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:35.478165 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:35.485333 20805 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:35.477459 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.433507 30560 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:35.508431 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:35.508507 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:35.511675 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:35.451318 20878 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:35.441882 30529 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:35.497334 20735 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:35.511898 21433 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:35.411957 30273 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.517638 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:35.520569 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152161us
I20260709 12:25:35.521817 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.546475 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.489867 30541 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:35.511258 30353 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:35.506429 29903 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:25:35.512171 30543 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.397s	user 0.001s	sys 0.000s
I20260709 12:25:35.498970 30443 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:35.443315 30517 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.512620 30515 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.514333 30027 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:35.512249 30242 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.565862 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.553092 30548 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:25:35.566708 20808 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.000s
I20260709 12:25:35.652410 20808 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:35.569892 30463 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:35.570967 30567 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:25:35.571758 30535 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.581s	user 0.003s	sys 0.000s
I20260709 12:25:35.572734 30541 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:35.582192 30512 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.000s
W20260709 12:25:35.582682 20776 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.524s (SENT)
I20260709 12:25:35.549482 30557 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:35.549304 30194 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:35.628526 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106958us
I20260709 12:25:35.526938 29892 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:35.507757 30577 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:35.682097 30526 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.031s	user 0.010s	sys 0.002s
I20260709 12:25:35.664582 30186 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:35.664835 30273 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:35.651271 30353 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:35.567485 29903 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:35.672322 30514 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.036s	user 0.010s	sys 0.006s
I20260709 12:25:35.672766 30517 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:35.676271 30501 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:35.679615 30548 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:35.686353 20802 raft_consensus.cc:2368] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30.
W20260709 12:25:35.686949 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.003s	sys 0.006s
W20260709 12:25:35.680873 30549 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.701s	user 0.002s	sys 0.001s
I20260709 12:25:35.650815 30515 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:35.683409 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:35.697891 30027 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.516438 30546 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.745780 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1772) took 1007 ms. Trace:
I20260709 12:25:35.746166 20804 rpcz_store.cc:276] 0709 12:25:34.737983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.774233 (+ 36250us) service_pool.cc:224] Handling call
0709 12:25:35.745580 (+971347us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:35.745725 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530899328}
I20260709 12:25:35.747215 30546 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:35.747772 30552 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.581s	user 0.000s	sys 0.002s
I20260709 12:25:35.748227 20805 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 29 because replica has last-logged OpId of term: 28 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:25:35.749132 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3716) took 1136 ms. Trace:
W20260709 12:25:35.750036 30539 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.046s	user 0.000s	sys 0.003s
I20260709 12:25:35.749534 20807 rpcz_store.cc:276] 0709 12:25:34.612301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.617931 (+  5630us) service_pool.cc:224] Handling call
0709 12:25:34.618313 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.712274 (+ 93961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.712327 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.737794 (+ 25467us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.960454 (+222660us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:34.960496 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.960561 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.746976 (+786415us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:35.748840 (+  1864us) raft_consensus.cc:1685] finished
0709 12:25:35.748860 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.749070 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75327,"raft.queue_time_us":202190,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":9264,"spinlock_wait_cycles":9323776,"thread_start_us":129423,"threads_started":1,"wal-append.queue_time_us":293769}
W20260709 12:25:35.716895 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.003s	sys 0.006s
I20260709 12:25:35.658506 30178 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:25:35.715915 30553 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.013s	sys 0.052s
I20260709 12:25:35.721043 21431 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:35.727545 29892 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.691684 30512 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:35.756597 30553 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.013s	sys 0.053s
I20260709 12:25:35.728495 30575 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:35.729348 30368 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:35.728578 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 145919us
I20260709 12:25:35.732129 30274 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:35.735390 30548 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:35.737897 30554 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.597s	user 0.001s	sys 0.000s
W20260709 12:25:35.739646 30558 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.667s	user 0.002s	sys 0.000s
I20260709 12:25:35.708787 30501 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:35.708478 29903 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:35.704084 30463 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:35.744225 30446 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:35.716158 30514 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.699570 30186 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:35.745476 30027 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.744290 30506 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:35.656949 30242 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:35.632678 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 124313us
W20260709 12:25:35.733424 30047 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.003s
I20260709 12:25:35.753175 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3677) took 1205 ms. Trace:
I20260709 12:25:35.756009 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:35.849125 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.756095 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.850862 20807 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 25 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:35.768672 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3714) took 1145 ms. Trace:
I20260709 12:25:35.849292 20727 rpcz_store.cc:276] 0709 12:25:34.547376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.548069 (+   693us) service_pool.cc:224] Handling call
0709 12:25:34.548528 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.548919 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:34.548955 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.615412 (+ 66457us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:34.615936 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.727126 (+111190us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:34.727169 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.727280 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.749347 (+1022067us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:35.752974 (+  3627us) raft_consensus.cc:1685] finished
0709 12:25:35.752995 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.753119 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21595,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":90,"spinlock_wait_cycles":23590016,"thread_start_us":110766,"threads_started":1,"wal-append.queue_time_us":87687}
W20260709 12:25:35.774400 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59534.9us
I20260709 12:25:35.755144 30178 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.855970 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.703675 30526 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.819871 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1767) took 1191 ms. Trace:
I20260709 12:25:35.820034 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:35.876500 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56598us
I20260709 12:25:35.701184 30577 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.854004 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:35.823673 30368 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:35.851563 20806 rpcz_store.cc:276] 0709 12:25:34.611458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.614349 (+  2891us) service_pool.cc:224] Handling call
0709 12:25:35.756654 (+1142305us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54003b7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:35.756774 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568723712}
I20260709 12:25:35.821569 30546 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.837568 30551 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:35.858871 20884 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:25:35.866173 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 348553us
I20260709 12:25:35.848969 30506 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:35.894894 30506 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.883677 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:35.896798 20876 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:35.897691 30515 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.857772 30178 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:35.905958 30027 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:35.890590 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.910197 30194 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:35.910660 30194 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.911038 30194 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:35.857123 30274 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 12:25:35.912741 30274 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:35.913208 30274 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:35.914177 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:35.915298 30557 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:35.915494 29875 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.916301 30512 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:35.916218 29875 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.916778 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.918088 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3715) took 1306 ms. Trace:
I20260709 12:25:35.918568 30515 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:35.918515 20809 rpcz_store.cc:276] 0709 12:25:34.611917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.613404 (+  1487us) service_pool.cc:224] Handling call
0709 12:25:34.613768 (+   364us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:34.644328 (+ 30560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.644379 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:34.672265 (+ 27886us) log.cc:844] Serialized 52 byte log entry
0709 12:25:34.892244 (+219979us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:34.892354 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.892454 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.680139 (+787685us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:35.917729 (+237590us) raft_consensus.cc:1685] finished
0709 12:25:35.917753 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.918003 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45479,"raft.queue_time_us":58569,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":32053,"spinlock_wait_cycles":5746432,"thread_start_us":86472,"threads_started":1,"wal-append.queue_time_us":195750}
I20260709 12:25:35.920591 30514 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:35.921772 30501 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.923161 20809 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:35.876158 20808 rpcz_store.cc:276] 0709 12:25:34.646820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.659248 (+ 12428us) service_pool.cc:224] Handling call
0709 12:25:35.819675 (+1160427us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b540019aad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:35.819803 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636531584}
I20260709 12:25:35.846621 30186 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:35.886750 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:35.769646 30559 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.889369 20875 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:35.928727 30546 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:35.929852 30577 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:35.931164 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77246us
I20260709 12:25:35.925518 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.933465 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:35.890687 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:35.888988 30517 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:35.928386 30557 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:35.948848 21434 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:35.951123 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60502.5us
W20260709 12:25:35.966310 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 79652.6us
I20260709 12:25:35.845551 30463 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:35.978763 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:35.981395 30465 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.000s	sys 0.021s
I20260709 12:25:35.919061 29875 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:35.992107 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:35.998099 20809 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 12:25:35.986976 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53554.2us
I20260709 12:25:35.914273 30274 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.003087 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:35.891913 30583 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.114s
I20260709 12:25:36.029481 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:36.080168 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3632) took 1151 ms. Trace:
W20260709 12:25:36.038228 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46226.7us
I20260709 12:25:36.100644 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:35.963515 30536 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:35.982749 30027 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:36.105091 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 102084us
I20260709 12:25:36.098407 20884 rpcz_store.cc:276] 0709 12:25:34.928791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.928959 (+   168us) service_pool.cc:224] Handling call
0709 12:25:34.946668 (+ 17709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.889373 (+942705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.079812 (+190439us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b540022f6d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:36.080060 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089328384}
I20260709 12:25:35.947530 30517 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.002313 30515 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:25:36.061216 30582 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:25:36.001406 30546 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:35.965996 30587 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:25:35.993760 30465 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.000s	sys 0.021s
I20260709 12:25:36.113397 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:36.101812 20809 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:36.077404 30556 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.956s	user 0.000s	sys 0.014s
I20260709 12:25:36.000695 30501 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:36.166664 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:35.986797 30578 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:36.055738 30542 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.740s	user 0.012s	sys 0.000s
W20260709 12:25:36.061650 30579 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:25:35.963016 30557 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.079362 29875 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.176477 30557 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:36.176837 29875 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.136873 30581 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:36.061446 30586 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:36.137465 30585 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:25:36.044103 30584 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.136s
I20260709 12:25:36.122267 30517 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:36.067974 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:36.142282 30575 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:36.055059 30502 ts_tablet_manager.cc:1929] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.15
I20260709 12:25:36.110363 30536 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.003283 30580 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:36.147691 21430 pending_rounds.cc:85] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 14
I20260709 12:25:36.205124 21430 pending_rounds.cc:107] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:25:36.205490 21430 raft_consensus.cc:2889] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:35.983675 30577 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.139s
I20260709 12:25:36.131337 30515 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:36.159070 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.166853 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.191345 30586 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.168512 21433 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:36.169952 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:36.171648 30526 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:36.171926 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1764) took 1458 ms. Trace:
I20260709 12:25:36.166616 30027 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:36.173108 30559 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:36.173605 30274 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:36.245718 29875 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.173252 30242 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.248919 30242 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.003180 30560 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.250259 20807 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:36.251345 30560 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:36.253250 30273 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:36.253898 30273 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.179489 29903 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:36.255319 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3621) took 1566 ms (client timeout 2010 ms). Trace:
I20260709 12:25:36.255829 20801 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:36.255785 20875 rpcz_store.cc:269] 0709 12:25:34.688758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.713707 (+ 24949us) service_pool.cc:224] Handling call
0709 12:25:36.255101 (+1541394us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:36.255258 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571232000}
I20260709 12:25:36.256927 30273 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.181893 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3486) took 1146 ms. Trace:
I20260709 12:25:36.257604 20806 rpcz_store.cc:276] 0709 12:25:35.035662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.891906 (+856244us) service_pool.cc:224] Handling call
0709 12:25:35.892363 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.892893 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.892943 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.893561 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.994236 (+100675us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:35.994285 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.994366 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.181561 (+187195us) raft_consensus.cc:1685] finished
0709 12:25:36.181586 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.181805 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1905,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":100226,"threads_started":1,"wal-append.queue_time_us":165546}
I20260709 12:25:36.259419 29903 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.259593 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:36.260392 20806 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:36.260627 30513 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.178774 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3626) took 1472 ms. Trace:
I20260709 12:25:36.184532 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:36.261327 30298 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:36.261801 30298 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:36.262523 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:36.263379 20802 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:36.264261 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:36.264801 20802 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:36.067147 30544 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.753s	user 0.008s	sys 0.004s
I20260709 12:25:36.235343 20804 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:36.265573 30544 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.195219 20809 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:25:36.169888 30548 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.037s
W20260709 12:25:36.240069 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.235996 21432 rpcz_store.cc:276] 0709 12:25:34.713924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.725725 (+ 11801us) service_pool.cc:224] Handling call
0709 12:25:34.778666 (+ 52941us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.779207 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.779257 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.815730 (+ 36473us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.996851 (+181121us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:34.996888 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.996952 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.076556 (+1079604us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:36.171601 (+ 95045us) raft_consensus.cc:1685] finished
0709 12:25:36.171625 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.171858 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4686,"raft.queue_time_us":9692,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":16620,"spinlock_wait_cycles":114820352,"thread_start_us":139946,"threads_started":1,"wal-append.queue_time_us":198602}
I20260709 12:25:36.240990 30584 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.262784 30298 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.264747 30463 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:36.178462 30578 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.283912 30578 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:36.235469 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3484) took 1208 ms. Trace:
I20260709 12:25:36.284200 20806 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:36.285422 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:36.284462 20803 rpcz_store.cc:276] 0709 12:25:35.028265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.196185 (+167920us) service_pool.cc:224] Handling call
0709 12:25:35.196479 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.320925 (+124446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.320974 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.350073 (+ 29099us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.567156 (+217083us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:35.567228 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.567299 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.235214 (+667915us) raft_consensus.cc:1685] finished
0709 12:25:36.235239 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.235400 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117589,"raft.queue_time_us":126308,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":142,"thread_start_us":149345,"threads_started":1,"wal-append.queue_time_us":368427}
I20260709 12:25:36.286049 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:36.286688 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:36.254812 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2290) took 1153 ms. Trace:
I20260709 12:25:36.273300 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:36.295863 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:36.277369 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:36.204243 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 136317us
I20260709 12:25:36.210219 30515 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:36.273329 30178 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.267268 20810 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: [34.17-34.17]
I20260709 12:25:36.287604 30298 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:36.210258 30520 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:36.290375 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.207926 30536 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:36.238130 30502 log.cc:1199] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-3-root/wals/1047135f71834dbab6b869eb3c081b81
I20260709 12:25:36.296186 20804 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:36.288986 20803 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
W20260709 12:25:36.296489 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.261472 20881 rpcz_store.cc:276] 0709 12:25:34.706483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.759924 (+ 53441us) service_pool.cc:224] Handling call
0709 12:25:34.803223 (+ 43299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.895828 (+ 92605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.895883 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.898479 (+  2596us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.434394 (+535915us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:35.434436 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.434504 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.178440 (+743936us) raft_consensus.cc:1685] finished
0709 12:25:36.178475 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.178687 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21705,"raft.queue_time_us":26664,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":24285,"thread_start_us":542033,"threads_started":2,"wal-append.queue_time_us":772482}
I20260709 12:25:36.310171 20803 pending_rounds.cc:85] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 13
I20260709 12:25:36.310598 20803 pending_rounds.cc:107] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:25:36.311030 20803 raft_consensus.cc:2889] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:36.275048 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 105206us
I20260709 12:25:36.269089 30583 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:36.299954 30559 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.293202 30584 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.292939 30526 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:36.296530 21431 rpcz_store.cc:276] 0709 12:25:35.100981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.113034 (+ 12053us) service_pool.cc:224] Handling call
0709 12:25:35.128057 (+ 15023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.765850 (+637793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.254560 (+488710us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:36.254749 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363273088}
I20260709 12:25:36.203421 30194 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.254798 30186 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.318358 20850 leader_election.cc:400] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:25:36.327821 20881 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:36.328029 20805 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:36.330092 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.320415 30580 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.012s
I20260709 12:25:36.333370 30580 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:36.333739 30580 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:36.335281 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.336063 20875 raft_consensus.cc:2368] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:25:36.328979 30585 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.305480 29939 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.337141 30585 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.339576 20884 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:36.327836 30465 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 12:25:36.342406 30517 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.337244 29939 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.349210 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.331911 30520 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.239741 30586 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:36.351523 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.353770 20882 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:36.354212 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3485) took 1325 ms. Trace:
I20260709 12:25:36.354638 20810 rpcz_store.cc:276] 0709 12:25:35.028619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.287304 (+258685us) service_pool.cc:224] Handling call
0709 12:25:35.287645 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.300383 (+1012738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.353915 (+ 53532us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:36.354138 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252030336}
I20260709 12:25:36.355675 30186 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:36.359215 30178 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.359581 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.359416 30273 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:36.361140 30273 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:36.361538 30273 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:36.361130 30194 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:36.364485 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.336866 30502 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42714 (request call id 21) took 2848 ms. Trace:
I20260709 12:25:36.365190 20875 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:36.334534 30580 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.365293 30502 rpcz_store.cc:276] 0709 12:25:33.510226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.523319 (+ 13093us) service_pool.cc:224] Handling call
0709 12:25:33.953043 (+429724us) ts_tablet_manager.cc:1151] Deleting tablet 1047135f71834dbab6b869eb3c081b81
0709 12:25:34.059716 (+106673us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:35.581134 (+1521418us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:36.052486 (+471352us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:36.336784 (+284298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":371752,"tablet-delete.queue_time_us":402284,"thread_start_us":249498,"threads_started":1}
I20260709 12:25:36.367290 30580 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:36.368263 20620 catalog_manager.cc:5028] TS 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): tablet 1047135f71834dbab6b869eb3c081b81 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:25:36.368945 30546 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:36.369386 30546 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:36.370311 30546 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.362385 30273 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.371740 30532 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.372655 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.373314 20727 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.373569 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.374213 20884 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
W20260709 12:25:36.374593 30602 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:25:36.375178 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.375841 20875 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:36.377646 30553 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:36.378130 30553 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:36.383574 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.386662 20802 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:36.386816 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:36.383656 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.390866 20882 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:36.371852 30273 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:36.397034 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.312788 30583 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:36.383637 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.312186 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:36.321237 30571 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 12:25:36.325645 30512 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:36.413514 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101484us
I20260709 12:25:36.415115 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.427124 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:36.429103 20881 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:36.430498 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.438892 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:36.449401 20882 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
W20260709 12:25:36.450469 30470 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.005s	sys 0.021s
W20260709 12:25:36.450949 30470 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.005s	sys 0.021s
I20260709 12:25:36.439028 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:36.461951 21432 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:36.462486 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:36.463475 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.464243 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:36.466425 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:36.467519 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.321450 30478 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:25:36.460800 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.485302 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.484635 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.486335 30470 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 12:25:36.425426 30186 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 12:25:36.475613 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.439962 30541 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 12:25:36.472160 30587 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:36.439813 30194 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:36.461807 30548 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.032s
I20260709 12:25:36.484228 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.496119 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.328018 30559 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.491869 20732 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.499223 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:36.487875 21434 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:36.500188 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:36.472880 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.501760 21435 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:36.497210 20728 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:36.489487 20880 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:36.502548 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.495873 30509 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:36.498836 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:36.364898 29939 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.500365 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.505470 20731 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.497270 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.506817 20881 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:36.508239 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.508839 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3719) took 1079 ms. Trace:
I20260709 12:25:36.509260 20807 rpcz_store.cc:276] 0709 12:25:35.439163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.855706 (+416543us) service_pool.cc:224] Handling call
0709 12:25:36.508665 (+652959us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54003a0f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:36.508787 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904559488}
I20260709 12:25:36.438369 30178 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:36.511289 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2286) took 1503 ms. Trace:
I20260709 12:25:36.504422 30509 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.515281 20729 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:36.515610 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.515759 30517 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.516597 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.517077 30509 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.517231 20726 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:36.520701 30587 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.521214 30587 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:36.523712 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:36.505309 20804 raft_consensus.cc:2376] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28. Re-sending same reply.
I20260709 12:25:36.528631 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:36.460597 30526 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:36.529973 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:36.469036 30353 raft_consensus.cc:2764] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:25:36.540889 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:36.530764 20731 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:36.511802 21429 rpcz_store.cc:276] 0709 12:25:35.008267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.008418 (+   151us) service_pool.cc:224] Handling call
0709 12:25:35.185478 (+177060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.201914 (+ 16436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.511023 (+1309109us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400746b50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:36.511220 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406358400}
I20260709 12:25:36.542059 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:36.531649 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:36.533350 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.511042 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.536774 21427 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:36.537324 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.544991 21428 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:36.487267 30591 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.546917 20807 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:36.539041 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.545673 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.558524 21434 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:36.501627 30611 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:36.546663 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.544003 21435 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:36.548689 20878 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:36.499683 30609 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:36.561270 21426 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:36.365489 30557 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.558573 20733 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
W20260709 12:25:36.564749 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74579.6us
I20260709 12:25:36.534123 30368 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:36.528371 30178 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.565653 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:36.574313 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:25:36.577231 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.466285 30536 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.577960 20880 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:36.546456 30470 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:36.592634 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.589752 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:36.595400 30353 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:36.595404 30512 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:36.596053 30512 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.596122 30353 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:36.592365 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 95462.1us
I20260709 12:25:36.598209 21431 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:36.599267 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.600143 20726 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:36.571419 29903 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.651777 30353 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.652292 29903 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.653406 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3637) took 1461 ms. Trace:
W20260709 12:25:36.622061 30606 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.178s	user 0.002s	sys 0.000s
I20260709 12:25:36.629025 30536 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:36.578289 30611 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:36.633517 30557 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:36.588791 30585 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:36.637995 30513 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.583649 29939 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:36.641955 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:36.643734 20731 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
W20260709 12:25:36.687304 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45453.1us
I20260709 12:25:36.644268 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.688045 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:36.583338 30178 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:36.613112 30583 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.602707 30345 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.654151 30541 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.701929 30541 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.702785 30506 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:36.704250 30541 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:36.679900 30552 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:25:36.654443 29903 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.654953 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.655658 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1768) took 2008 ms. Trace:
I20260709 12:25:36.658797 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:36.659369 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.661495 30617 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:25:36.663445 30544 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:36.596541 30512 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.618651 30578 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.664527 20877 rpcz_store.cc:276] 0709 12:25:35.191552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.341600 (+150048us) service_pool.cc:224] Handling call
0709 12:25:35.341919 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.964549 (+622630us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:35.964597 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.984498 (+ 19901us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.457289 (+472791us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:36.457331 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.652743 (+195412us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54001d01d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:36.652816 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.652980 (+   164us) raft_consensus.cc:1685] finished
0709 12:25:36.653009 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.653329 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128158,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":472481536,"thread_start_us":780450,"threads_started":2,"wal-append.queue_time_us":582297}
I20260709 12:25:36.669390 20727 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:36.672540 20726 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
W20260709 12:25:36.707278 30612 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:36.677887 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:36.677887 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.725634 20810 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 28 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:25:36.726575 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.691670 20880 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:25:36.571991 30591 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.710556 20801 rpcz_store.cc:276] 0709 12:25:34.647554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.647692 (+   138us) service_pool.cc:224] Handling call
0709 12:25:35.756779 (+1109087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.256426 (+499647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.384441 (+128015us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54003b7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:36.655405 (+270964us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003b7250. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:36.655583 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3681322240}
I20260709 12:25:36.712842 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.729135 20731 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:36.713449 20878 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:36.698975 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:36.698046 29939 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:36.676354 20881 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
W20260709 12:25:36.724016 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.732394 20883 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.19-28.19]
I20260709 12:25:36.710109 30470 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:36.738912 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:36.727841 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.714834 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.680756 20732 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:36.750906 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.751391 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.726226 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.752540 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.744199 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.706754 30186 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:36.747885 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:36.746297 20878 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:36.753868 20808 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:36.754396 20726 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:36.762300 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:36.766860 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.768077 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.766834 20727 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:36.770735 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.772629 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.711124 30513 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.779495 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:36.735111 29939 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:36.713398 30345 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.786677 20878 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 21. Preceding OpId from leader: term: 27 index: 22. (index mismatch)
I20260709 12:25:36.780936 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.788679 20732 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:25:36.753189 30369 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.752681 30478 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:36.789583 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:36.796013 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3500) took 1112 ms. Trace:
W20260709 12:25:36.805783 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 366810us
I20260709 12:25:36.807255 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:36.810657 30194 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:36.811479 30194 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:36.811955 30194 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:36.812021 30613 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:36.812904 30194 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.813678 30611 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.819409 30559 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.818871 30526 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:36.820060 30559 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.820087 30526 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.820722 30559 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.823668 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.824509 21433 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:36.826489 30345 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.827127 20731 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:36.796633 20806 rpcz_store.cc:276] 0709 12:25:35.683794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.287074 (+603280us) service_pool.cc:224] Handling call
0709 12:25:36.293586 (+  6512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.304801 (+ 11215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.304862 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.313068 (+  8206us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.457450 (+144382us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:36.457532 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.457616 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.660918 (+203302us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:36.795627 (+134709us) raft_consensus.cc:1685] finished
0709 12:25:36.795670 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.795928 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3770,"raft.queue_time_us":4012,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":21143,"spinlock_wait_cycles":88692608,"thread_start_us":130424,"threads_started":1,"wal-append.queue_time_us":130035}
I20260709 12:25:36.834846 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:36.837576 30621 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.379s	user 0.002s	sys 0.000s
I20260709 12:25:36.757747 30501 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:25:36.822827 30559 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:36.845263 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3619) took 2187 ms. Trace:
I20260709 12:25:36.788432 29903 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:25:36.848587 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.856349 20734 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:36.843160 20733 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:25:36.785753 30186 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:36.864305 20879 rpcz_store.cc:276] 0709 12:25:34.658283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.660199 (+  1916us) service_pool.cc:224] Handling call
0709 12:25:34.719584 (+ 59385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.138157 (+418573us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:35.161738 (+ 23581us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.259202 (+1097464us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b18002acd00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:25:36.583379 (+324177us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.616699 (+ 33320us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.660231 (+ 43532us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.660879 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.661077 (+   198us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:36.661140 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.661341 (+   201us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.844933 (+183592us) raft_consensus.cc:1685] finished
0709 12:25:36.844964 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.845147 (+   183us) inbound_call.cc:177] Queueing success response
0709 12:25:36.852614 (+  7467us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b18002acd00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
Metrics: {"mutex_wait_us":98015,"prepare.queue_time_us":24099,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":3533,"raft.queue_time_us":120768,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":26436,"spinlock_wait_cycles":24850688,"thread_start_us":640679,"threads_started":2,"wal-append.queue_time_us":1459704}
I20260709 12:25:36.872601 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:36.865716 20732 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:36.870985 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:36.881705 21431 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:36.878741 20726 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
W20260709 12:25:36.883529 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48745.5us
I20260709 12:25:36.885031 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:36.891532 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.846319 30517 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.851284 30584 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:36.850708 30591 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:36.849192 30578 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.846606 30546 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:36.851828 30224 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:36.847755 30513 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:36.868387 29903 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.845395 30047 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:36.787104 30626 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.111s
W20260709 12:25:36.925247 30515 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
I20260709 12:25:36.952018 30591 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:36.890460 30572 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:36.894943 30501 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:36.853813 30369 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:36.932888 30298 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.937072 30517 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.956488 29903 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.956755 30047 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:36.961615 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76131.3us
I20260709 12:25:36.997815 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:36.980206 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:37.009091 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:37.010416 20774 leader_election.cc:400] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:25:37.010731 20774 leader_election.cc:403] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.012266 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.013430 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.017163 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.018167 20807 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:36.965548 30501 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:36.972431 30610 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.529s	user 0.003s	sys 0.000s
I20260709 12:25:36.961567 30515 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.024080 29904 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:37.024421 30578 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 29. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:25:37.064661 30583 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.028326 30099 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:37.030695 30584 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:37.054272 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:36.992467 30298 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:36.962720 30186 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.057000 30515 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.057049 30517 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.057960 30557 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.059969 30586 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.001s
I20260709 12:25:37.032232 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:37.104080 20879 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:37.037087 30501 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:37.107497 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:37.116400 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.106292 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:36.970288 30553 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.103459 30584 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.138185 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:37.131999 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 105171us
I20260709 12:25:37.157392 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.137639 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.137935 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.141530 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:37.106765 30559 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.158921 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:37.160054 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:37.167958 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.166455 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.169543 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:37.169701 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:37.134044 30578 raft_consensus.cc:2764] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:25:37.169821 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:37.172796 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.174077 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:37.139555 30517 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:37.168010 30553 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:37.174203 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 426393us
I20260709 12:25:37.159531 30584 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:37.158097 30515 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:37.179015 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.157651 30548 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:37.171753 30353 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:37.137367 30586 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.176225 30099 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:37.171481 30541 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:37.137723 30557 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.203734 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.214178 20806 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.206374 20879 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 29 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:37.178056 30560 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:37.177603 30580 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.187915 30224 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.159587 30178 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:37.212492 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 327572us
I20260709 12:25:37.132087 30047 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.172602 30559 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.171537 30604 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.034s
I20260709 12:25:37.216295 30353 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.236831 20805 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:37.243346 20880 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:37.250034 30368 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:37.250732 30368 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.251397 30604 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:37.251794 30604 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:37.252700 30604 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.253219 30368 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:37.254009 30604 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:37.255645 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.256327 30557 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:37.256616 30559 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:37.257200 30642 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:37.258324 30586 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:37.264159 30512 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.535s	user 0.005s	sys 0.006s
I20260709 12:25:37.264663 30512 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:37.265326 30644 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:25:37.267598 30178 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:37.274356 30506 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.001s
I20260709 12:25:37.234633 30047 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.275000 20732 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
W20260709 12:25:37.280435 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106393us
I20260709 12:25:37.232437 30353 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.288708 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:37.289880 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.290228 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:37.137302 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 343828us
W20260709 12:25:37.198765 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 391594us
I20260709 12:25:37.228812 30560 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:37.292150 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.196961 30517 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:37.201326 30194 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:37.294154 30517 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:37.294253 30194 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:37.295156 30517 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.295501 30194 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.221208 20875 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:37.242559 30587 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.721s	user 0.003s	sys 0.009s
I20260709 12:25:37.186695 30634 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:37.224866 30536 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:25:37.200745 30638 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:37.208580 30548 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.293025 20803 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:37.221206 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.300561 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:37.302388 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.307325 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.267010 30512 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:37.323175 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:37.283265 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.299348 30515 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:37.347806 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:37.349197 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 176209us
W20260709 12:25:37.349979 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.328620 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.330047 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.331900 20801 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:25:37.333388 20802 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 21. Preceding OpId from leader: term: 27 index: 22. (index mismatch)
I20260709 12:25:37.335366 30298 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.324930 30437 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:37.345942 20806 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
W20260709 12:25:37.347805 30649 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:25:37.305689 30529 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:37.354646 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.353826 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.357874 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.366216 20802 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:37.367086 20807 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:37.358260 20810 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:37.354219 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.359882 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.326400 30634 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:37.369184 21430 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:37.370172 21426 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:37.363437 21434 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:37.363965 21431 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:25:37.364236 30640 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:25:37.365538 21432 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:37.376056 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:37.377446 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:37.378058 30224 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:37.396621 20879 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:37.379936 30526 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:37.396920 29903 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.398270 20804 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:37.380100 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.381417 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3721) took 1941 ms. Trace:
I20260709 12:25:37.365268 30515 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:37.383773 30437 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:37.383355 30560 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.326170 30638 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:37.384397 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.399091 20808 rpcz_store.cc:276] 0709 12:25:35.440178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.963440 (+523262us) service_pool.cc:224] Handling call
0709 12:25:35.997914 (+ 34474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.755963 (+758049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.381148 (+625185us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54003cad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:37.381348 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1737859968}
I20260709 12:25:37.386344 30548 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:37.325122 30557 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:25:37.309789 30587 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.387833 30634 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.394434 29904 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:37.374855 29939 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 12:25:37.407706 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:37.407882 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 185002us
I20260709 12:25:37.411785 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.417409 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:37.421730 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.435053 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.433331 21431 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:37.439263 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:37.424717 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51919.5us
I20260709 12:25:37.426106 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:37.428793 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.431134 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.431851 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.440550 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:37.452440 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.456049 21426 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:37.441531 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.453063 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.454211 20878 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
W20260709 12:25:37.453878 20850 consensus_peers.cc:597] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.458813 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:37.458918 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:37.408464 30345 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:37.509558 20876 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:25:37.514066 30512 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:37.488756 30548 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:25:37.488453 30546 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:25:37.510851 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.491034 30178 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.517472 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 297871us
I20260709 12:25:37.524191 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.524956 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.464358 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:37.461346 21428 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:37.443987 30298 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:37.525298 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66564.3us
I20260709 12:25:37.526470 21432 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:25:37.524890 21431 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:37.527141 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:37.520514 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 61721.5us
W20260709 12:25:37.557837 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 93604us
I20260709 12:25:37.426025 30544 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.093s
I20260709 12:25:37.491330 29903 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:37.561094 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 128235us
I20260709 12:25:37.561635 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.442760 30583 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.560370 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:37.563679 21426 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:37.556308 20810 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:37.522666 30027 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.583904 20809 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:37.512027 30611 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:25:37.426640 30501 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.567512 30548 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:37.567818 30541 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:37.579859 30437 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.592633 30541 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.563592 30298 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:37.571676 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 168083us
I20260709 12:25:37.593741 30298 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:37.592296 30553 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:37.571519 30638 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.599735 30553 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.601969 30553 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:37.573175 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48341.4us
I20260709 12:25:37.533545 30515 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.573992 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:37.574196 30623 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:37.575420 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.604382 20808 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.578055 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.440104 29904 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.440459 29939 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.522497 30626 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:37.586833 30580 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:37.587165 20883 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:25:37.587714 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:37.561825 30546 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:37.558645 30368 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.592701 30353 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.572962 30591 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:37.603394 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.573736 30587 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:37.645449 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.619508 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.618605 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.638774 20803 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:37.652683 20877 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:37.649701 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:37.656996 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53712.9us
I20260709 12:25:37.657814 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:37.532209 30178 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.460798 30657 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.190s
I20260709 12:25:37.661263 20808 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:37.666271 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.634956 30546 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.636289 30572 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:37.645681 30654 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:37.639365 30583 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.677734 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.656381 30586 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:37.662086 29903 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:37.692142 29903 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:37.693465 29903 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.667564 30646 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:37.690070 30465 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.689844 30586 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:37.700487 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3503) took 1272 ms. Trace:
I20260709 12:25:37.701025 20805 rpcz_store.cc:276] 0709 12:25:36.427867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.523432 (+ 95565us) service_pool.cc:224] Handling call
0709 12:25:37.700257 (+1176825us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b5400126954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:37.700411 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632417664}
I20260709 12:25:37.707479 30526 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:37.708135 30526 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:37.716246 29904 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.718242 30517 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:37.721374 30541 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.672840 30546 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.723886 30586 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.683925 29939 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:37.685796 30665 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:25:37.690629 30591 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.728281 30501 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.728997 30526 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.671707 30572 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.733037 30572 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:37.691237 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.687778 30656 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:37.735703 21433 raft_consensus.cc:2410] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
W20260709 12:25:37.735842 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44746.6us
I20260709 12:25:37.736625 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.737607 30478 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:37.738368 30478 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:37.738763 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.674615 30654 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:37.739614 30630 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:37.739904 30654 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:37.741209 30584 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.541s	user 0.007s	sys 0.002s
I20260709 12:25:37.673501 30623 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:37.741773 30654 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.742319 30623 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:37.741815 30584 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.746127 30584 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:37.725709 30586 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:37.647284 30368 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.748390 30501 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:37.739566 30478 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:25:37.749574 30368 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.680330 30583 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.750854 30526 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:37.751573 30591 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.752355 30501 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:37.752969 30583 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:37.754675 30242 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:37.746959 30298 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.743440 30623 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.758157 30298 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.755352 30646 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:37.764828 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.766373 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:37.767066 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.771468 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.772413 21426 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:37.787303 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.789882 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.793073 20808 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:37.797358 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.798511 20808 raft_consensus.cc:2368] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30.
I20260709 12:25:37.801606 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.802505 20808 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:37.803864 30560 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:37.804723 30560 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.807683 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.808434 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:37.772670 30520 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.813652 20808 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 29 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:37.746886 30572 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.825606 30560 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.796625 30298 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.828364 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:37.840375 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.841343 20806 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:37.843978 20879 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:37.829892 30637 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:37.857164 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3431) took 1235 ms. Trace:
I20260709 12:25:37.857734 20731 rpcz_store.cc:276] 0709 12:25:36.621485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.743813 (+122328us) service_pool.cc:224] Handling call
0709 12:25:37.856930 (+1113117us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b54000c44d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:37.857102 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123321728}
I20260709 12:25:37.884083 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:37.886132 20700 leader_election.cc:400] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30.
I20260709 12:25:37.886574 20700 leader_election.cc:403] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.900931 20806 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
W20260709 12:25:37.916522 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 297959us
I20260709 12:25:37.836975 30298 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:37.923301 30604 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:25:37.925917 20731 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:37.926434 20735 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: [26.21-26.21]
I20260709 12:25:37.927170 30571 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.927665 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:37.927839 30571 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.928503 20732 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:37.930316 30571 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.931890 30604 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:37.933966 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.912388 20806 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:25:37.936220 20806 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:25:37.937911 30604 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:37.938937 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:37.942124 30604 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:37.942559 30604 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.942929 30604 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:37.950635 30178 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:37.951557 30604 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:37.952455 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:37.953131 20728 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:37.954118 30604 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:37.955621 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:37.956581 30529 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:37.957284 30529 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:37.954833 30604 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.004s
I20260709 12:25:37.963315 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:37.964206 20808 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:37.913815 29904 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.912343 30478 raft_consensus.cc:2764] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:25:37.911536 30506 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30.
I20260709 12:25:37.911782 30656 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:37.969205 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.979987 20731 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:37.981189 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.981398 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:37.965665 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.983176 20728 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:37.984737 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.985882 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:37.986596 20728 raft_consensus.cc:2368] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26.
I20260709 12:25:37.987486 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:37.988070 20729 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:37.988853 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.993070 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:37.993700 20728 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:37.994115 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.994908 20731 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:37.995523 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:37.999233 20808 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:38.005059 20730 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:38.005529 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:37.958540 30529 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.010324 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:38.010882 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:37.855916 30637 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.019326 20728 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:38.023974 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.026247 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:38.027220 21426 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:38.032899 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3580) took 1622 ms (client timeout 1813 ms). Trace:
W20260709 12:25:38.033372 20875 rpcz_store.cc:269] 0709 12:25:36.410221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.410469 (+   248us) service_pool.cc:224] Handling call
0709 12:25:38.032684 (+1622215us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:38.032838 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654212864}
I20260709 12:25:38.034713 20882 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:25:38.004478 30506 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.980513 29904 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.041394 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:38.051429 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.010434 30478 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:38.015467 30623 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:38.060113 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.063372 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:38.018157 30548 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:38.022359 20732 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:38.064536 30675 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:38.064723 30678 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:25:38.065186 30194 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:25:38.067049 30677 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:25:38.061605 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:38.070577 21427 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:38.087780 21427 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:38.088984 21427 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.090238 21427 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:38.095428 30369 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.095757 29904 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.100772 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1727) took 1588 ms. Trace:
I20260709 12:25:38.101292 20882 rpcz_store.cc:276] 0709 12:25:36.511944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.512781 (+   837us) service_pool.cc:224] Handling call
0709 12:25:38.034453 (+1521672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.036470 (+  2017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.100507 (+ 64037us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400200650. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:38.100695 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3498457472}
I20260709 12:25:38.103051 30546 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.106389 20801 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:38.107566 30194 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:38.108100 30194 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:38.108750 20801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:25:38.109647 20801 raft_consensus.cc:606] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.109125 30194 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.110059 20801 raft_consensus.cc:614] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:38.083952 30224 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:38.111696 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50173.9us
I20260709 12:25:38.111789 30224 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.114503 30224 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.114439 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:38.115399 30557 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:25:38.071406 30670 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:25:38.018299 30654 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:25:38.083263 30675 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:38.117385 30672 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:25:37.984534 30656 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.118760 30298 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.118584 30629 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:38.126561 20729 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:38.129256 20805 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
W20260709 12:25:38.131913 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 164233us
I20260709 12:25:38.031167 30646 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:25:38.084535 30478 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:38.206455 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 82878.8us
I20260709 12:25:38.209751 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.142292 30194 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.140808 30572 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.182657 30604 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:25:38.084854 30548 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.180871 30536 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:38.202833 30629 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:38.244901 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:38.272272 21435 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:38.272935 30478 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.293336 30643 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:38.294003 30643 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:38.238631 30674 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:25:38.300565 21430 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 20. (term mismatch)
I20260709 12:25:38.305641 21430 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 19
I20260709 12:25:38.305918 21430 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 29.20
I20260709 12:25:38.306161 21430 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:38.302198 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.302341 30551 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.309151 30682 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:25:38.309844 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.310636 20876 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:38.302671 30559 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.312251 30559 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:38.313489 30681 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
I20260709 12:25:38.314208 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.314864 20878 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:38.315546 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.295145 30643 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 34.17, Last appended by leader: 17, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.316239 20881 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:38.317140 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:38.317349 30559 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.317853 20881 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:38.318437 30637 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.318656 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.319931 30559 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.320542 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.320621 30559 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.246285 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.214007 20775 leader_election.cc:400] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26.
I20260709 12:25:38.328609 20775 leader_election.cc:403] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:25:38.229723 30224 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:38.229417 30583 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:38.264125 30676 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:25:38.249706 30571 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:38.309522 30551 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.343463 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.316730 30643 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:38.321621 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.245643 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:38.322058 30629 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:38.323433 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.351769 20880 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 20.
W20260709 12:25:38.323799 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.359812 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:38.362267 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.369153 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:38.323436 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1780) took 1007 ms. Trace:
I20260709 12:25:38.370774 20809 rpcz_store.cc:276] 0709 12:25:37.315568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.315830 (+   262us) service_pool.cc:224] Handling call
0709 12:25:37.316261 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.584355 (+268094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.323163 (+738808us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54003c9954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:38.323372 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615043840}
I20260709 12:25:38.245338 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.372552 20729 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:38.328135 20734 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:38.337208 20878 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:25:38.339329 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 127297us
I20260709 12:25:38.339493 30027 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26.
I20260709 12:25:38.350303 21429 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:38.345984 20810 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:38.374843 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.377908 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:38.377908 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.332604 30224 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:38.390416 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3702) took 1671 ms. Trace:
I20260709 12:25:38.393980 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1729) took 1881 ms. Trace:
I20260709 12:25:38.394526 20877 rpcz_store.cc:276] 0709 12:25:36.512609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.732544 (+219935us) service_pool.cc:224] Handling call
0709 12:25:36.766992 (+ 34448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.653162 (+886170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.393627 (+740465us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b540022ecd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:38.393899 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2036363008}
I20260709 12:25:38.403108 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.403610 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:38.370102 20803 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:38.405650 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.365089 30536 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:38.407243 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:38.407759 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:38.408440 29903 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.407086 30298 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:38.341741 30571 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:38.424045 30224 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.427850 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:38.498584 20806 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
W20260709 12:25:38.500958 20802 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.094s	sys 0.153s
W20260709 12:25:38.501386 20802 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.094s	sys 0.153s
I20260709 12:25:38.499876 30446 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:38.390996 20727 rpcz_store.cc:276] 0709 12:25:36.719036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.792828 (+ 73792us) service_pool.cc:224] Handling call
0709 12:25:36.798951 (+  6123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.866683 (+ 67732us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:36.892574 (+ 25891us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.260384 (+367810us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b18003ddec0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:25:37.522688 (+262304us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.572488 (+ 49800us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.572540 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:37.572585 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.010517 (+437932us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b18003ddec0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:38.390064 (+379547us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b54003b4a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:38.390131 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.390333 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":258591,"prepare.queue_time_us":23680,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":77,"raft.queue_time_us":244402,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":7660,"spinlock_wait_cycles":212421504,"thread_start_us":67217,"threads_started":1,"wal-append.queue_time_us":500374}
I20260709 12:25:38.428849 30548 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:38.436607 30559 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.437879 30675 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:38.439007 30676 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:25:38.439352 30463 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:38.440490 21427 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
I20260709 12:25:38.517232 21427 pending_rounds.cc:85] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 18
I20260709 12:25:38.517537 21427 pending_rounds.cc:107] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:25:38.517797 21427 raft_consensus.cc:2889] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:38.518850 30663 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 29 current term: 29 is higher.
I20260709 12:25:38.440793 30656 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.365320 30273 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.484465 29939 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:38.445734 30353 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:38.448614 21429 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
W20260709 12:25:38.450745 30586 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.000s
I20260709 12:25:38.401610 30583 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:38.474999 20879 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
W20260709 12:25:38.483180 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77608.4us
W20260709 12:25:38.483393 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75725us
I20260709 12:25:38.559885 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.561787 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.572521 29904 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:38.573553 29904 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:38.577744 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:38.578713 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:38.496920 30536 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.584635 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "49323f66286f4798b268f216bb62b3be"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:38.585457 20880 raft_consensus.cc:606] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Received request to transfer leadership to TS 49323f66286f4798b268f216bb62b3be
I20260709 12:25:38.585891 20880 raft_consensus.cc:614] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:38.480448 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:38.366601 30587 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:38.587505 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:38.591898 20884 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:25:38.504570 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:38.435006 30679 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 12:25:38.497112 30027 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:38.502518 20803 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:38.497399 30553 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.510553 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:38.598407 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:38.598722 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88259.1us
I20260709 12:25:38.598888 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:38.599486 20879 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:38.443276 30546 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.524806 21426 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
W20260709 12:25:38.525060 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:38.583827 30683 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:38.605712 21426 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:38.580643 30584 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:38.599767 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:38.604421 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:38.611608 21427 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.613548 30186 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:38.614421 30515 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:38.521816 30224 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.593816 20876 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
W20260709 12:25:38.606666 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.620352 30514 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:38.622131 30514 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:38.623235 30553 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.625022 20734 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:25:38.626099 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.626438 30553 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.627393 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:38.628155 20733 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:38.629913 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:38.520586 30446 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.608047 30536 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.632922 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:38.604032 30353 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:38.641254 20728 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:38.596045 29904 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:38.636162 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:38.637995 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.642508 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:38.638196 20803 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:25:38.544955 30273 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.638957 21427 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:38.544160 30463 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.061s
I20260709 12:25:38.642957 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.543291 29939 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.654397 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:38.655074 20806 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
W20260709 12:25:38.655223 30526 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.904s	user 0.007s	sys 0.004s
I20260709 12:25:38.642473 30353 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:38.656311 29939 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:38.656315 30353 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.658555 30274 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:38.645334 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:38.646685 20732 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:38.663228 20877 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:25:38.663511 20807 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:38.606957 30546 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:38.601080 30587 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.644469 20809 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:38.660344 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:38.650424 20730 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 19.
W20260709 12:25:38.670742 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.641222 30553 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:38.671036 20728 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:38.543733 30571 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:38.684468 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.653050 29904 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.686460 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:38.689461 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:38.680930 30553 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.703076 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.691957 30557 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.692163 30663 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.691032 29904 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:38.701643 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:38.711609 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1800) took 1194 ms. Trace:
I20260709 12:25:38.712134 21430 rpcz_store.cc:276] 0709 12:25:37.516809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.520991 (+  4182us) service_pool.cc:224] Handling call
0709 12:25:38.300076 (+779085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.306684 (+  6608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.711394 (+404710us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b54005302d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:38.711546 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790917376}
I20260709 12:25:38.685750 30675 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:25:38.700870 30571 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:38.676472 30656 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:25:38.680665 30526 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.026s
I20260709 12:25:38.672932 30583 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:25:38.700105 30548 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:38.689352 30546 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
W20260709 12:25:38.757256 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.718804 30663 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.730945 30273 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:38.704980 30194 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:25:38.744179 30584 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:38.744446 30691 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:25:38.707149 30553 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:38.759552 30697 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:38.778509 30584 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:38.788477 30697 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.789032 30697 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:38.778131 30194 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.011s
I20260709 12:25:38.793818 30629 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:38.738737 30298 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:38.751268 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:38.726011 30446 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:38.798084 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46920.1us
I20260709 12:25:38.798256 30587 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:38.801160 30646 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:38.801646 30646 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:38.801798 30663 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.802075 30646 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:38.802593 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1789) took 1197 ms. Trace:
I20260709 12:25:38.765857 30501 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:38.803524 30501 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.803109 20810 rpcz_store.cc:276] 0709 12:25:37.605464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.605710 (+   246us) service_pool.cc:224] Handling call
0709 12:25:37.606011 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.376501 (+770490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.802332 (+425831us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54003ca854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:38.802525 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687617792}
I20260709 12:25:38.752274 20809 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
W20260709 12:25:38.756093 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.762223 29904 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.719203 30638 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:38.778815 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:38.721896 30551 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:38.716351 30557 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.863746 29904 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.815486 30663 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.824376 30572 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:38.822016 30242 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:38.924194 30273 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.862937 30638 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:38.928905 30643 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.001s
W20260709 12:25:38.950515 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.018s	sys 0.017s
I20260709 12:25:38.967352 20726 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:38.924628 30520 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:38.936573 30465 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.977874 30369 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.937559 30663 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.988103 30369 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.989956 30663 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:38.990355 30369 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.950023 30242 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:38.993988 30242 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:38.995324 30520 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.956529 30572 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:38.998261 30520 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:38.998556 30572 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:38.999334 30638 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.004434 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:39.004761 29903 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.001s
I20260709 12:25:39.005142 29903 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:39.007678 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1794) took 1047 ms. Trace:
I20260709 12:25:39.008137 20801 rpcz_store.cc:276] 0709 12:25:37.960399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.111134 (+150735us) service_pool.cc:224] Handling call
0709 12:25:38.121345 (+ 10211us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:38.121879 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.121921 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:38.289994 (+168073us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:38.290821 (+   827us) log.cc:844] Serialized 52 byte log entry
0709 12:25:38.685028 (+394207us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:38.685075 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.685151 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.007321 (+322170us) raft_consensus.cc:1685] finished
0709 12:25:39.007357 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.007612 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":18754,"spinlock_wait_cycles":17099136,"thread_start_us":393658,"threads_started":1,"wal-append.queue_time_us":638476}
I20260709 12:25:38.955236 30273 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.012637 30273 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:39.015041 30557 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.025472 20877 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:25:38.980280 30536 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.052295 30186 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:39.054936 30186 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.065075 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.053288 30536 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.098234 30353 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.100437 30536 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.100394 30353 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:39.101579 30702 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:25:39.027635 30465 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.052728 30027 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:39.118196 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.015s	sys 0.055s
W20260709 12:25:39.119562 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.015s	sys 0.056s
W20260709 12:25:39.121088 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 496942us
W20260709 12:25:39.123093 20882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.009s	sys 0.034s
W20260709 12:25:39.126189 20882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.010s	sys 0.035s
W20260709 12:25:39.124538 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 485722us
W20260709 12:25:39.122527 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 518185us
I20260709 12:25:39.133637 20878 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:39.136202 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.136314 29904 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:39.137009 29904 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.137666 29903 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.138338 29903 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.135041 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:39.141531 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.142584 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:39.122290 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:39.143415 20876 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:39.134830 30515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.026s	sys 0.020s
I20260709 12:25:39.146505 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:39.147476 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:39.142915 20878 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:39.153821 29904 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:39.142378 30529 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.042s	user 0.021s	sys 0.033s
W20260709 12:25:39.157611 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 447072us
I20260709 12:25:39.161334 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:39.165627 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:39.167928 29903 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:39.162439 30529 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.021s	sys 0.034s
I20260709 12:25:39.151556 30583 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:39.151253 30587 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:39.250564 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:39.190321 30520 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:39.150667 30536 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.151345 30646 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.137245 30273 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
I20260709 12:25:39.182137 20876 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:39.269650 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.168715 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:39.271385 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102771us
W20260709 12:25:39.151150 30515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.026s	sys 0.020s
W20260709 12:25:39.234148 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68594.3us
I20260709 12:25:39.273802 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:39.166851 30557 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.007s
I20260709 12:25:39.237502 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:39.243364 20805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.012s	sys 0.029s
W20260709 12:25:39.296092 20805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.012s	sys 0.029s
I20260709 12:25:39.244756 20801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:32862
I20260709 12:25:39.297209 20801 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:39.297693 20801 raft_consensus.cc:614] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.188480 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.171953 29904 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:39.298919 29904 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.249979 30656 consensus_queue.cc:1243] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 29.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:39.171363 30027 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.276276 30529 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:39.280573 30583 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:39.278960 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.319414 20877 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:25:39.286182 30709 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.287063 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.320735 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.301154 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.306897 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 118512us
I20260709 12:25:39.322345 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:39.324812 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.316725 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.317989 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.327788 20876 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
W20260709 12:25:39.330014 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56306.6us
I20260709 12:25:39.334924 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.308247 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.336917 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.338260 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.339151 20728 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:25:39.340456 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:39.341159 30701 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:25:39.341797 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.343258 20732 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:25:39.344046 20731 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:39.344463 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.345101 20732 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:39.345012 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.346160 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.346508 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.347506 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.347888 20877 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:25:39.348321 20735 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:39.343290 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.352181 20734 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:39.352635 30604 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:39.353106 30604 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:39.354013 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 20.
I20260709 12:25:39.346689 20732 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:39.341123 20728 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
W20260709 12:25:39.338599 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.358940 30368 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:39.359402 30368 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:39.359623 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.360039 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.360229 30368 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.361006 30709 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.361567 30368 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:39.303710 30274 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.329618 30529 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.364392 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.370599 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:39.370909 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48656.8us
I20260709 12:25:39.360253 20810 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:39.378261 20726 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:39.354192 30604 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.269810 30587 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:39.397086 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:39.402186 21426 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 28 because replica has last-logged OpId of term: 26 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 20.
I20260709 12:25:39.402031 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.401151 20803 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:39.397567 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:39.421559 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46579.2us
I20260709 12:25:39.406958 20801 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 12:25:39.422874 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.423768 29903 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:39.424396 29903 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:39.426535 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:39.319707 30560 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:39.425302 29903 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:39.367058 30629 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.000s
W20260709 12:25:39.340521 30703 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:25:39.318964 30708 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.315315 30707 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:39.269305 30536 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.396659 30178 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:39.447275 20884 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.029s	sys 0.023s
W20260709 12:25:39.483680 20884 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.029s	sys 0.024s
I20260709 12:25:39.320046 30273 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:39.437218 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.411682 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.487432 20810 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:39.488646 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:39.490236 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:39.351128 30584 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.526s	user 0.004s	sys 0.004s
W20260709 12:25:39.444270 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60372.4us
W20260709 12:25:39.425634 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 117519us
I20260709 12:25:39.363173 30709 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.445298 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.316138 30675 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.112s
I20260709 12:25:39.495048 30536 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.495316 30584 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.462792 30706 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:39.363739 30583 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.498100 30584 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.364120 30274 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.366155 30529 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:39.466372 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.397125 30656 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.472829 21432 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:39.474229 30705 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.515923 20809 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:39.516171 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:39.517413 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:39.499334 30298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.012s	sys 0.035s
W20260709 12:25:39.477978 30694 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.696s	user 0.000s	sys 0.003s
I20260709 12:25:39.319921 30520 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.132s
W20260709 12:25:39.451395 30559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.051s	sys 0.033s
W20260709 12:25:39.431768 30553 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.641s	user 0.006s	sys 0.006s
I20260709 12:25:39.492746 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:39.493814 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:39.483019 30708 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.489560 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:39.421635 29904 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.503749 20810 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:32862
W20260709 12:25:39.559172 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66510.9us
I20260709 12:25:39.515137 30273 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:39.405490 30506 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 12:25:39.526170 30623 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:39.521642 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 76423.4us
I20260709 12:25:39.474880 20726 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:39.526436 20808 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:39.442564 30587 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.619211 30587 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.554659 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.620110 20803 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:39.620386 21435 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:25:39.621001 30698 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.527689 30298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.012s	sys 0.035s
I20260709 12:25:39.621501 30698 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.622493 30298 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:39.622915 21432 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:39.622984 30298 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.556214 21431 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 20.
I20260709 12:25:39.559544 20810 raft_consensus.cc:606] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.431329 30515 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:39.625453 30515 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.625784 30298 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:39.441205 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.020s	sys 0.042s
W20260709 12:25:39.626832 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.034s	user 0.020s	sys 0.042s
W20260709 12:25:39.627424 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.04e+06us
I20260709 12:25:39.628803 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:39.629437 29904 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.630336 29904 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.630002 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.632253 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.632577 29904 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.633289 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.633595 30646 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.634337 30650 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:39.634963 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:39.635790 30650 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:25:39.641180 30697 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.852s	user 0.010s	sys 0.002s
I20260709 12:25:39.566824 20801 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:39.568413 30708 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.564244 30553 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.564000 30706 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:39.570506 30536 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.665057 30708 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:39.665755 30708 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.665653 30536 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.669288 30536 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.669564 30708 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.671051 30584 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:39.671463 30708 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.671653 30584 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.671973 30708 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.522954 30705 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.602643 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1796) took 1640 ms. Trace:
I20260709 12:25:39.604406 30274 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.606128 30273 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.619531 30478 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:39.625891 30515 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:39.561163 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.626361 30178 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.642218 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:39.558986 30720 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.552155 30559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.053s	sys 0.035s
I20260709 12:25:39.699994 21432 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 20: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:39.565980 30710 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 12:25:39.646625 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.703931 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:39.704561 21399 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.584s (SENT) [suppressed 11 similar messages]
I20260709 12:25:39.705101 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.705715 20726 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:39.706610 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.646947 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:39.707160 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.707656 20807 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:39.650581 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3764) took 1022 ms. Trace:
I20260709 12:25:39.708592 20805 rpcz_store.cc:276] 0709 12:25:38.628412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.628557 (+   145us) service_pool.cc:224] Handling call
0709 12:25:38.628861 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.629219 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.629244 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.629771 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.296630 (+666859us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:39.296688 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.296783 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.650342 (+353559us) raft_consensus.cc:1685] finished
0709 12:25:39.650375 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.650523 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166451,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":666414,"threads_started":1,"wal-append.queue_time_us":928857}
I20260709 12:25:39.652772 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.653307 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.653707 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:39.711421 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.654546 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.712155 20734 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:39.659617 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:39.659004 20729 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:25:39.661052 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:39.664345 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:39.660871 29904 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.522729 30656 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:39.695467 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134412us
I20260709 12:25:39.713708 29904 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.714381 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.675812 20804 rpcz_store.cc:276] 0709 12:25:37.962263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.115289 (+153026us) service_pool.cc:224] Handling call
0709 12:25:38.115675 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.116342 (+   667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.116372 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.286048 (+169676us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:38.286715 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.575413 (+288698us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:38.575471 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.575592 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.477291 (+901699us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:39.602283 (+124992us) raft_consensus.cc:1685] finished
0709 12:25:39.602316 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.602555 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":93010048,"thread_start_us":273944,"threads_started":1,"wal-append.queue_time_us":480738}
I20260709 12:25:39.713272 20735 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
W20260709 12:25:39.715479 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54509.3us
I20260709 12:25:39.711591 20731 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:39.712235 20732 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 36 because replica has last-logged OpId of term: 34 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 16.
I20260709 12:25:39.663842 30298 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.717549 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:39.696537 30708 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.717723 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:39.678238 30186 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.021s
I20260709 12:25:39.745473 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:39.752308 29904 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.726369 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:39.754622 20882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.013s	sys 0.038s
W20260709 12:25:39.757299 20882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.013s	sys 0.038s
W20260709 12:25:39.756168 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.674122 30697 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.763938 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:39.764832 21432 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:25:39.751969 30186 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.013s	sys 0.021s
W20260709 12:25:39.774767 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57126us
I20260709 12:25:39.716478 30723 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:39.759232 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:39.771478 20731 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:39.748626 30463 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.767814 20807 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:39.725273 30706 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:39.725101 30643 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:39.717082 30224 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.797833 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.021s	sys 0.022s
W20260709 12:25:39.798609 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.780463 20805 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:25:39.747416 30656 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:39.781381 20809 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:25:39.831503 20733 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:39.805789 20807 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:39.727551 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:39.759270 30712 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:39.832990 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73874.8us
W20260709 12:25:39.833755 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106280us
I20260709 12:25:39.834503 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:39.815721 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.819453 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.022s	sys 0.022s
I20260709 12:25:39.692291 30478 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.827848 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:39.835309 20809 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:25:39.702199 30178 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.841514 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.849396 20880 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:39.844034 30551 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:39.836139 30707 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:39.845192 30587 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:39.845872 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:39.845991 30708 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:39.823495 30559 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:25:39.824574 30463 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.779348 30506 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:39.751329 30298 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.716588 30720 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.779479 30724 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.859000 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:39.818099 30501 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:39.856547 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:39.858626 30587 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:39.845923 30478 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.848188 30465 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.855324 30698 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:39.865803 30520 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:39.814173 30719 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:39.885769 30724 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.889243 30709 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:39.890565 30501 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.893594 30368 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.000s
I20260709 12:25:39.838406 30224 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.862191 30274 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:39.861828 30708 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:39.906620 20803 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:39.886987 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:39.914983 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.877745 30643 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.913712 30587 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:39.917608 30506 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.939688 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52803.5us
I20260709 12:25:39.944542 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:39.956183 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.966094 29903 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:39.966936 29903 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.976372 30705 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:39.983376 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 278818us
I20260709 12:25:39.983714 20727 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:39.941324 30708 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:39.979908 29904 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:39.985963 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:40.014410 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 230108us
I20260709 12:25:39.976608 30027 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:40.020730 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:39.977269 30560 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:40.024981 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.026237 21430 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:40.027329 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:40.041531 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:40.034765 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.035140 29903 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.037302 30536 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.010s
I20260709 12:25:39.950989 30274 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:39.939734 30501 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:39.983376 30465 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:25:39.951346 30643 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:39.976845 30709 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:25:39.976451 30506 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.023233 30027 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.066695 30274 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:40.066835 30733 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:25:40.068804 30726 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 12:25:40.069532 30731 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 12:25:40.070169 30727 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:25:40.070725 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3604) took 1512 ms. Trace:
I20260709 12:25:40.103703 20884 rpcz_store.cc:276] 0709 12:25:38.558127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.603268 (+ 45141us) service_pool.cc:224] Handling call
0709 12:25:38.633935 (+ 30667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.484068 (+850133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.484104 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.493704 (+  9600us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.858267 (+364563us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:39.858328 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.858445 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.070507 (+212062us) raft_consensus.cc:1685] finished
0709 12:25:40.070532 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.070666 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59558,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":6173,"thread_start_us":1213865,"threads_started":2,"wal-append.queue_time_us":511774}
I20260709 12:25:40.071178 30526 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:25:40.071780 30692 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:40.072826 30178 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.041776 30656 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:40.108788 30757 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.006s
I20260709 12:25:40.109488 30656 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.110057 30656 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:40.111828 30224 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.113826 30692 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:40.115015 30692 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.116551 30692 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:40.121238 30643 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.121428 30743 ts_tablet_manager.cc:933] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.082067 30027 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:40.086294 30729 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:25:40.130679 30724 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.131635 20881 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.19-28.19]
W20260709 12:25:40.134084 30529 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.002s
I20260709 12:25:40.135893 30463 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.140465 30587 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.142263 30298 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.142750 30587 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:40.143524 30720 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.144249 30708 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.146951 30708 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:40.026201 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 175836us
I20260709 12:25:40.035305 30560 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.040366 30584 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.015031 30478 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:25:40.041867 30697 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.060101 30557 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:40.084533 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:40.087865 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56670.5us
I20260709 12:25:40.064832 30536 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:40.083343 30559 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.093181 30730 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:40.093937 20878 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:25:40.134666 30529 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.160645 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.175066 21428 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:40.176338 30536 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.184935 30465 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.185297 30536 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:40.185567 30465 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.187930 30465 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.178373 30724 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:40.188838 30724 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.191802 30724 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.193732 30749 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:40.194386 30749 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.177165 30643 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.171164 30557 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.195688 30643 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:40.196710 30749 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.197561 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1742) took 1065 ms. Trace:
I20260709 12:25:40.165767 30560 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:40.196170 30557 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:40.203364 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.203162 21431 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:40.182605 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.204718 20802 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:40.197932 20882 rpcz_store.cc:276] 0709 12:25:39.132371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.135656 (+  3285us) service_pool.cc:224] Handling call
0709 12:25:39.136020 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.148972 (+ 12952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.149021 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.163871 (+ 14850us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.481439 (+317568us) spinlock_profiling.cc:243] Waited 8.7 ms on lock 0x7b4800240258. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:25:39.758940 (+277501us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:39.758986 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.759059 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.197249 (+438190us) raft_consensus.cc:1685] finished
0709 12:25:40.197268 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.197498 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12227,"raft.queue_time_us":17933,"raft.run_cpu_time_us":3488,"raft.run_wall_time_us":325089,"spinlock_wait_cycles":20002176,"thread_start_us":521777,"threads_started":1,"wal-append.queue_time_us":753829}
W20260709 12:25:40.206964 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46447.2us
I20260709 12:25:40.197876 30643 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:40.209283 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 150865us
I20260709 12:25:40.209635 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.216735 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.213347 30743 tablet_copy_client.cc:323] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:25:40.210400 20730 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:40.211386 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:40.211647 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.220655 20726 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:40.220726 20732 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:40.213248 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.210254 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.221393 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.169739 30515 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.218541 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:40.222065 20809 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:40.221832 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:40.224218 20809 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:40.225251 30273 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.208509 30724 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:40.227108 30724 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.227584 30724 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:40.234198 30637 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:40.234758 30637 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:40.235176 30637 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:40.235476 20729 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:40.237123 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.237740 20729 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:40.238024 30604 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:40.239086 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.239775 20729 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:40.242091 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:40.242687 20735 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:40.243366 30273 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:40.244165 30755 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.150s	user 0.003s	sys 0.000s
I20260709 12:25:40.245875 30273 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.248117 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1741) took 1119 ms. Trace:
I20260709 12:25:40.248579 30515 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.249004 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:40.250773 20735 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:40.251034 20807 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:40.248540 20881 rpcz_store.cc:276] 0709 12:25:39.128340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.131972 (+  3632us) service_pool.cc:224] Handling call
0709 12:25:40.131286 (+999314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.132196 (+   910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.247909 (+115713us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54000a2dd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:40.248052 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2269006720}
I20260709 12:25:40.255609 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:40.256201 20809 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:40.256645 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.184172 30559 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.257896 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.261870 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:40.261145 20726 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:40.265378 30529 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:40.265983 30553 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.261878 30515 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:40.266721 30515 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:40.267086 30515 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:40.275940 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:40.279659 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:40.282478 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.285187 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:40.284674 30623 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:40.290408 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:40.291402 20809 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:40.285805 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.288473 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:40.303090 30446 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:40.303426 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.304394 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.280779 30546 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.281805 30584 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.313860 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:40.308338 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45600.8us
I20260709 12:25:40.280230 30587 raft_consensus.cc:2764] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:25:40.280660 30551 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.030s
I20260709 12:25:40.308152 30643 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.305801 30446 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:40.312122 30553 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:40.284863 30478 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.301878 30273 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:40.334969 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.324045 20726 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:40.304787 30623 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.309201 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.325423 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.315941 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:40.326274 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:40.300778 30766 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:40.326576 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:40.343377 20730 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:40.339442 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.327826 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.330958 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:40.312602 30465 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:40.333927 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.334313 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.335248 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.335646 20728 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:40.340776 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:40.340143 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:40.328720 30768 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:40.354516 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.357066 20878 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:40.361071 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.403836 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.329006 30553 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.327143 30749 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:40.416805 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102248us
W20260709 12:25:40.432008 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.313771 30719 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:25:40.337743 30273 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.352082 30298 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:40.343848 30623 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:40.434870 20734 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:40.360736 20809 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:40.435534 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.315397 30546 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.436664 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.318902 30584 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.332862 30763 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:40.440706 20806 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:40.440632 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.443686 21429 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:25:40.444473 30274 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:40.421661 30027 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.434092 30553 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:40.362205 30762 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:40.446266 20809 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:40.357560 30465 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.404546 30709 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:40.509052 30027 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.523329 30709 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.523928 30709 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:40.524513 30724 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.526799 30724 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:40.454872 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.448190 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:40.542111 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 94043.5us
I20260709 12:25:40.543082 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:40.456194 20893 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 1047135f71834dbab6b869eb3c081b81 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:25:40.464416 30529 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.482470 30756 raft_consensus.cc:993] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:25:40.357941 30463 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.093s
W20260709 12:25:40.515838 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 97923.3us
I20260709 12:25:40.329442 30643 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.441097 30273 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:40.438045 30749 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.443110 30369 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:40.339490 30478 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.445653 30298 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:40.522732 30465 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:40.527316 30027 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.358698 30766 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.527580 30735 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:40.443620 30587 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:40.562142 30638 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:40.532331 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:40.590754 20735 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:40.532267 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 126731us
W20260709 12:25:40.547490 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.499806 30763 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.596139 30763 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:40.506143 30274 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.601154 30274 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:40.558542 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:40.562773 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:40.446003 30656 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.562458 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.603590 20734 raft_consensus.cc:2376] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27. Re-sending same reply.
I20260709 12:25:40.506307 30446 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.548969 30583 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.557698 30663 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:40.613644 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:40.614004 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:40.549139 30637 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:40.434124 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73134.1us
I20260709 12:25:40.583312 20893 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 1047135f71834dbab6b869eb3c081b81 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-1047135f71834dbab6b869eb3c081b81
I20260709 12:25:40.589210 30584 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:40.582871 30298 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:40.607439 29903 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:40.499178 30557 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.617791 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:40.618157 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:40.737440 20809 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:25:40.620918 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:40.620980 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:40.588425 30559 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:40.564213 30529 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.590631 30638 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:40.723964 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94806.9us
I20260709 12:25:40.724376 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:40.728858 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 21.
W20260709 12:25:40.737550 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 119840us
I20260709 12:25:40.605499 30656 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.575302 30756 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.059s
I20260709 12:25:40.746625 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:40.616283 30586 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:40.587531 30027 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.591234 30587 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.732354 30551 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:40.616083 30186 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.152s
I20260709 12:25:40.746608 30584 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:40.768106 30638 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:40.801286 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.633275 30583 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:40.801870 30692 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.685s	user 0.014s	sys 0.000s
I20260709 12:25:40.783987 30571 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:25:40.731050 20876 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:40.892892 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 285798us
I20260709 12:25:40.907732 21428 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 30 because replica has last-logged OpId of term: 28 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
W20260709 12:25:40.849040 30732 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:25:40.847537 30769 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.403s	user 0.000s	sys 0.001s
W20260709 12:25:40.871310 30730 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.080s	user 0.008s	sys 0.001s
W20260709 12:25:40.853592 30515 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
W20260709 12:25:40.913431 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 137672us
I20260709 12:25:40.920475 30520 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:40.921144 30520 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:40.921818 30560 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.000s
I20260709 12:25:40.922307 30560 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.923712 30546 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:40.924145 30773 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:25:40.927599 30749 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.928763 30587 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:40.929541 30587 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:40.930778 30587 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.911856 30463 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:40.933000 30224 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:40.912768 30766 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.933545 30224 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:40.934527 30587 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:40.892516 30761 raft_consensus.cc:1081] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:25:40.932288 30643 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:40.936246 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3766) took 1601 ms (client timeout 1835 ms). Trace:
W20260709 12:25:40.940969 20808 rpcz_store.cc:269] 0709 12:25:39.335202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.336624 (+  1422us) service_pool.cc:224] Handling call
0709 12:25:40.936034 (+1599410us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400126454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:40.936183 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433399168}
I20260709 12:25:40.937071 20810 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: [34.17-34.17]
I20260709 12:25:40.795647 30551 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:40.931614 30478 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:40.912734 29939 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:40.947327 20806 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:40.871588 30584 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:40.948513 30536 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.001s
I20260709 12:25:40.934590 30224 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.948336 29939 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.917498 30515 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:40.956493 30586 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:40.999887 20884 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:25:40.933547 30463 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:40.966794 30194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.002s	sys 0.002s
W20260709 12:25:41.012091 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.005s	sys 0.007s
W20260709 12:25:41.014910 30194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.002s	sys 0.002s
W20260709 12:25:41.015151 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.005s	sys 0.007s
W20260709 12:25:41.026211 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.015923 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 675794us
I20260709 12:25:40.969655 29939 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:41.091645 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.094702 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.097443 20884 raft_consensus.cc:2368] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21.
W20260709 12:25:41.027338 30731 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:25:41.104009 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3771) took 1676 ms. Trace:
I20260709 12:25:41.104432 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:41.120846 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 374245us
I20260709 12:25:41.127758 20801 rpcz_store.cc:276] 0709 12:25:39.427703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.436945 (+  9242us) service_pool.cc:224] Handling call
0709 12:25:41.103833 (+1666888us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b540006ccd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:41.103953 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183149312}
I20260709 12:25:41.129601 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:40.968045 30536 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.029364 30637 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.031919 30224 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:41.099643 30643 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:41.104252 30654 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.135318 30560 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.152204 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.153177 20883 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:41.032958 30551 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.169952 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.099148 30529 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:41.099462 30557 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.133785 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:41.143961 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:41.036280 30720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.007s	sys 0.006s
I20260709 12:25:41.162463 21434 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:41.210283 20875 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
W20260709 12:25:41.190341 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98448.3us
I20260709 12:25:41.194578 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:41.194902 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.211447 20729 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:41.129011 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:41.202598 30785 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:25:41.144042 30571 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:41.208374 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.091670 30178 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:41.245211 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.246332 20877 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:41.246850 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
W20260709 12:25:41.249118 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 115443us
W20260709 12:25:41.183122 30368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.003s	sys 0.017s
I20260709 12:25:41.158237 30027 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.259981 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.253014 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:41.193719 30650 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.194223 30698 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:41.032598 30584 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.208020 30629 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.190956 30654 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:41.243103 30720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.011s	sys 0.013s
I20260709 12:25:41.144222 30707 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.119s
I20260709 12:25:41.281858 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:41.255607 30368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.021s	sys 0.024s
I20260709 12:25:41.284401 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.289405 20732 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:25:41.255506 30583 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.256434 30789 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:41.309016 30784 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:25:41.345817 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.382345 20729 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:41.187242 30643 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.302651 20802 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:41.303982 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.280138 20893 tablet_copy_source_session.cc:215] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:41.296443 30629 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.306371 30654 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.338460 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.266937 30646 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:25:41.340191 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:41.340646 30787 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:25:41.343504 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.344365 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1756) took 1019 ms. Trace:
I20260709 12:25:41.260928 30186 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:25:41.244725 30586 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.257244 30178 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.426849 30178 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.393565 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.282895 30698 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:41.398500 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:41.361722 30553 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.832s	user 0.013s	sys 0.000s
I20260709 12:25:41.283035 30650 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:41.399394 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95095.7us
I20260709 12:25:41.339500 20883 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:25:41.437541 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.438326 20883 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:41.404150 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:41.338701 30709 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.814s	user 0.013s	sys 0.001s
I20260709 12:25:41.439904 30709 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:41.442452 30709 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.443811 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:41.444684 20802 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:41.445559 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:41.446193 20802 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:41.447619 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.448943 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:41.449065 30698 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.450842 30698 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:41.408301 20808 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:41.345881 30273 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.001s
I20260709 12:25:41.409513 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.410422 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.474725 20729 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:41.410482 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:41.362792 30604 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.124s	user 0.012s	sys 0.004s
I20260709 12:25:41.415433 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:41.415813 20893 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:53814 (request call id 1760) took 1078 ms. Trace:
W20260709 12:25:41.348907 30763 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.000s
I20260709 12:25:41.428505 20875 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 36 because replica has last-logged OpId of term: 34 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 16.
I20260709 12:25:41.363476 30793 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:41.477399 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:41.477742 30793 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:41.478302 20808 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:41.410996 20881 rpcz_store.cc:276] 0709 12:25:40.325366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.325528 (+   162us) service_pool.cc:224] Handling call
0709 12:25:40.337989 (+ 12461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.433722 (+ 95733us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:40.594333 (+160611us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.893056 (+298723us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.893561 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.893704 (+   143us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:40.893739 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.893805 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.344088 (+450283us) raft_consensus.cc:1685] finished
0709 12:25:41.344116 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.344294 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208716,"prepare.queue_time_us":156971,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3605,"raft.queue_time_us":161411,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":13019,"thread_start_us":398370,"threads_started":2,"wal-append.queue_time_us":354112}
I20260709 12:25:41.478859 30793 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.479961 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.481142 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.481921 20881 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:41.360363 30536 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:41.484397 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.485191 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69901.1us
I20260709 12:25:41.451876 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.452361 30629 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.454818 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:41.454595 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:41.347096 30800 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:25:41.429416 30623 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.917s	user 0.012s	sys 0.000s
I20260709 12:25:41.470669 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:41.364256 30515 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:41.495357 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:41.489732 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.500833 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:41.501868 20729 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 21: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:41.488991 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.491748 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:41.486825 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.491994 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:41.503993 20802 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:41.492992 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:41.498102 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:41.484138 30638 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.002s
I20260709 12:25:41.504923 20730 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:41.495415 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:41.496788 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:41.431910 30274 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.830s	user 0.014s	sys 0.000s
I20260709 12:25:41.505216 21434 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:41.507560 20881 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:25:41.508157 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.509830 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:41.515755 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:41.517395 20877 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:41.521620 20881 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:41.527467 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:41.528522 30789 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:41.528967 30789 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:41.529345 30789 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:41.531494 30515 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:41.532423 30735 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.532625 30515 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:41.532964 30735 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.533732 30551 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:41.534117 30551 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:41.534910 30551 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.536264 30551 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:41.539311 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.539783 20729 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:25:41.544011 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:41.544919 30799 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:41.545454 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:41.546175 30799 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:25:41.546926 30799 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:41.536264 30735 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.547845 30799 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 34.17, Last appended by leader: 17, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:41.513449 30520 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.000s
I20260709 12:25:41.497853 30791 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:41.497562 30723 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:41.585196 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.465538 30553 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.566159 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.484237 20893 rpcz_store.cc:276] 0709 12:25:40.438582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.455909 (+ 17327us) service_pool.cc:224] Handling call
0709 12:25:41.415708 (+959799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.515844 30692 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.064s
I20260709 12:25:41.486080 30273 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.579329 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.579368 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1595) took 2251 ms. Trace:
I20260709 12:25:41.519956 30629 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:41.580595 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85238.6us
I20260709 12:25:41.596796 20733 rpcz_store.cc:276] 0709 12:25:39.324289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.326993 (+  2704us) service_pool.cc:224] Handling call
0709 12:25:39.342874 (+ 15881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.839324 (+496450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.575781 (+1736457us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54001d7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:41.576029 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093069696}
I20260709 12:25:41.523964 30186 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:41.498262 30637 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:41.517165 30638 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.506965 30623 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.516279 30274 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.599478 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.598865 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:41.632959 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:41.589610 30557 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:41.602774 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:41.602643 30637 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:41.589857 20620 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 50c1ab13351d4427b836fbb449c6d7bb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:41.638139 20877 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:41.621937 30705 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:41.597692 30629 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.625728 20730 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:41.585923 30604 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:41.635120 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:41.601754 30186 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.488477 30763 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.568617 30520 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.641292 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:41.566927 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:41.642935 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:41.611274 30298 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.848s	user 0.013s	sys 0.000s
I20260709 12:25:41.663916 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.561765 30789 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.633723 20883 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 27 index: 21. (term mismatch)
I20260709 12:25:41.649386 20735 raft_consensus.cc:2368] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:25:41.667097 20883 pending_rounds.cc:85] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 20
I20260709 12:25:41.667423 20883 pending_rounds.cc:107] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 26.21
I20260709 12:25:41.667661 20883 raft_consensus.cc:2889] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:41.669569 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.670374 20883 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:41.672982 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.656044 20699 leader_election.cc:400] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21.
I20260709 12:25:41.638262 30557 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.652436 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:41.644127 30536 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:41.645344 30766 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:41.648427 30629 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:41.674010 20699 leader_election.cc:403] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:25:41.665627 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.682893 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:41.691169 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.676646 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:41.647415 20620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61686.1us
I20260709 12:25:41.684258 20728 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:41.657640 30186 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:41.684089 20875 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:41.664086 30786 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.767774 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 111918us
I20260709 12:25:41.769089 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:41.660267 30643 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.776440 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.777494 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.779816 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.782987 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.679247 30557 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:41.681794 30656 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:41.782389 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:41.791088 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:41.752729 30766 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.669099 30764 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.117s
I20260709 12:25:41.661005 30604 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.796208 20882 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:41.759168 30758 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:41.798629 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:41.651476 30705 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.677117 30298 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.784336 30650 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.806480 20805 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:41.778335 30797 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 20. Current term is 21.
I20260709 12:25:41.775789 30446 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:41.794402 30536 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:41.798771 30766 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:41.805138 30369 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:41.826061 30654 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:41.827986 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:41.816680 30806 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:25:41.793208 30656 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.797554 30529 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.824007 30789 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:41.880966 20808 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:41.889801 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:41.887125 30536 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:41.892271 30465 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.305s	user 0.015s	sys 0.000s
I20260709 12:25:41.892750 30465 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.888599 30529 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.897655 30529 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.898226 20726 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:41.890986 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:41.791060 30663 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.042s
I20260709 12:25:41.899130 20808 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:41.899339 30663 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:41.899778 30663 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:41.900601 30663 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.901110 20808 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:25:41.901791 30663 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:41.902987 20808 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:41.806913 30758 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:41.908623 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.890901 30536 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.906006 20732 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:41.794308 30749 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:41.817072 30802 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
W20260709 12:25:41.912664 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 114101us
I20260709 12:25:41.942474 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:41.829579 30637 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.817800 30298 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.885926 30654 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:41.826964 30763 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.878801 30446 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.920815 30758 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:41.921324 30746 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.830s	user 0.006s	sys 0.002s
I20260709 12:25:41.921844 30643 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:41.922335 30501 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.922694 30623 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:41.926363 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1807) took 2438 ms (client timeout 1917 ms). Trace:
I20260709 12:25:41.876453 30506 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.928580 30604 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:41.786784 30786 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:41.909072 29939 consensus_meta.cc:287] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.804s	user 0.002s	sys 0.010s
I20260709 12:25:41.932713 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3547) took 1178 ms. Trace:
I20260709 12:25:41.965458 20809 rpcz_store.cc:276] 0709 12:25:40.754774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.755750 (+   976us) service_pool.cc:224] Handling call
0709 12:25:41.103962 (+348212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.104148 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.104183 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:41.104228 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:41.104259 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.104315 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.932494 (+828179us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540006ccd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:41.932644 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799950080}
W20260709 12:25:41.966547 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 207323us
I20260709 12:25:41.967358 20809 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: [31.19-31.19]
I20260709 12:25:41.934424 30186 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:41.887261 30764 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:41.816407 30798 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:41.991931 30764 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:41.992175 30798 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.992908 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3681) took 1028 ms. Trace:
I20260709 12:25:41.993489 20882 rpcz_store.cc:276] 0709 12:25:40.964143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.964267 (+   124us) service_pool.cc:224] Handling call
0709 12:25:40.964588 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.799956 (+835368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.992598 (+192642us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:41.992830 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912041856}
I20260709 12:25:41.995096 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:41.938468 30723 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:41.995965 30723 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.827765 30273 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:41.818202 30638 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.828094 30520 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.947852 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:41.948673 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.001453 20877 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:42.001561 20876 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:42.002769 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.003588 20876 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:42.006271 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2351) took 1067 ms. Trace:
I20260709 12:25:42.007477 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:42.002806 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.006867 21427 rpcz_store.cc:276] 0709 12:25:40.939039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.939157 (+   118us) service_pool.cc:224] Handling call
0709 12:25:40.944662 (+  5505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.945054 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.945085 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.946830 (+  1745us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.037491 (+ 90661us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:41.037552 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.037647 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.005975 (+968328us) raft_consensus.cc:1685] finished
0709 12:25:42.006009 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.006188 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":504356,"raft.queue_time_us":1607,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7961,"thread_start_us":73806,"threads_started":1,"wal-append.queue_time_us":573001}
W20260709 12:25:42.021538 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.949688 20883 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:41.895076 30465 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:41.953949 30797 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:41.916776 30656 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:42.023021 30797 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:42.024050 30797 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:41.961823 21433 rpcz_store.cc:269] 0709 12:25:39.488278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.488396 (+   118us) service_pool.cc:224] Handling call
0709 12:25:41.926103 (+2437707us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:41.926237 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4611127168}
I20260709 12:25:41.960301 30643 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:42.027647 30643 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:42.032936 30656 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.969336 30604 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.042735 30604 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.969758 29939 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.986704 30654 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:41.995326 30798 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.937125 30274 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:41.977763 30557 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.035439 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.046515 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.046640 20876 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:42.058403 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.059881 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.060592 20876 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:42.047753 20882 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:42.064131 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:42.046208 30705 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.049489 30795 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:41.999681 30629 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:42.068922 20803 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3776) took 2445 ms (client timeout 2289 ms). Trace:
I20260709 12:25:42.070487 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:42.076951 20803 rpcz_store.cc:269] 0709 12:25:39.623572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.659277 (+ 35705us) service_pool.cc:224] Handling call
0709 12:25:42.068689 (+2409412us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540019a5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:42.068839 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445610368}
I20260709 12:25:42.066723 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:42.078255 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:42.073374 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.068612 20884 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:41.992864 30764 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.079816 21432 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28.
I20260709 12:25:42.084508 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.085758 21433 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: [30.20-30.20]
I20260709 12:25:42.082263 30797 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:42.086606 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.083460 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.087174 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.087320 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.087563 21427 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:42.088068 21433 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:42.083767 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.087749 21430 raft_consensus.cc:2376] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27. Re-sending same reply.
I20260709 12:25:42.079375 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:42.085398 21434 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:42.054795 30720 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:42.066879 30604 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.093909 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:42.074122 30572 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:42.094906 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3465) took 2418 ms. Trace:
I20260709 12:25:42.074301 30795 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.096294 20727 rpcz_store.cc:276] 0709 12:25:39.677034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.711151 (+ 34117us) service_pool.cc:224] Handling call
0709 12:25:39.822167 (+111016us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ae9c 00007f9e1bb7aa5b 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:25:39.822431 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.984214 (+161783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.094690 (+2110476us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540028c154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.094851 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372503808}
I20260709 12:25:42.092727 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.099684 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.089855 30797 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:42.102629 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.103307 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:42.104234 20803 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:42.104033 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.055732 30643 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.099081 30764 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:42.110462 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.112166 30643 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.113837 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.114703 21432 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 21: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:42.116117 30762 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.116633 30762 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:42.121245 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:42.122210 30819 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:42.124477 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:42.125330 30819 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:42.126827 30819 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:42.133634 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.099716 30795 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:42.101981 30572 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:42.135607 20883 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: [32.22-32.22]
I20260709 12:25:42.136495 30629 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.136853 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.166417 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.167124 20876 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:42.137715 30557 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.138145 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.138816 30186 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:42.140247 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3743) took 1519 ms. Trace:
I20260709 12:25:42.138937 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1809) took 2506 ms. Trace:
I20260709 12:25:42.182309 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:42.182554 21431 rpcz_store.cc:276] 0709 12:25:39.632049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.635326 (+  3277us) service_pool.cc:224] Handling call
0709 12:25:39.635586 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.203961 (+568375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.138697 (+1934736us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54004d9c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.138850 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304832384}
I20260709 12:25:42.142038 30353 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:42.181977 20726 rpcz_store.cc:276] 0709 12:25:40.621095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.621580 (+   485us) service_pool.cc:224] Handling call
0709 12:25:40.740577 (+118997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.898677 (+1158100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.140077 (+241400us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000e74d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.140202 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2661825280}
I20260709 12:25:42.144587 30656 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.144829 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3687) took 1072 ms. Trace:
I20260709 12:25:42.188370 20884 rpcz_store.cc:276] 0709 12:25:41.077491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.128656 (+ 51165us) service_pool.cc:224] Handling call
0709 12:25:42.144532 (+1015876us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b5400366854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:42.144743 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1895603072}
I20260709 12:25:42.145807 29939 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.099668 30654 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:42.103839 30797 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:42.131151 30793 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.159469 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.129916 30515 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.199481 30793 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.200222 30515 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.201860 30793 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:42.202369 30515 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.096243 30720 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.202872 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3786) took 1889 ms. Trace:
I20260709 12:25:42.203254 20806 rpcz_store.cc:276] 0709 12:25:40.313186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.500400 (+187214us) service_pool.cc:224] Handling call
0709 12:25:40.500784 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.948726 (+447942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.202597 (+1253871us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54003a0a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.202814 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026479488}
I20260709 12:25:42.204442 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.205494 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.206453 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.206528 30720 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:42.159898 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.162118 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:42.157473 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.171633 30572 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:42.175836 20728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 3741) took 1555 ms (client timeout 1847 ms). Trace:
I20260709 12:25:42.176981 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.207093 20876 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:42.192816 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.212546 20803 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:42.212666 20884 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:42.196297 30675 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:42.219074 30675 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:42.219992 30675 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.19, Last appended by leader: 19, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:42.211735 20728 rpcz_store.cc:269] 0709 12:25:40.620061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.620679 (+   618us) service_pool.cc:224] Handling call
0709 12:25:42.175611 (+1554932us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540028cb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:42.175770 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174695296}
I20260709 12:25:42.212774 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.211917 20729 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:42.216463 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.218676 20884 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:42.221879 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:42.223654 20806 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:42.238345 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:42.239094 20806 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:42.240342 20850 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28.
I20260709 12:25:42.221220 30675 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:42.240676 30713 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:42.263302 20850 leader_election.cc:403] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:42.264091 30793 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:25:42.241776 30795 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.242671 30763 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:42.243085 30463 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.243278 30515 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:42.243875 30830 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:25:42.265177 30713 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:42.186888 30353 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:42.272133 30713 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:42.273010 30353 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.274070 30353 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:40.340330 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:42.260130 20729 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:42.165427 30766 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.278842 30766 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:42.285154 30353 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.285727 30353 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.187668 30273 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.068s
I20260709 12:25:42.287658 20803 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:42.265939 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:42.287806 30353 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:42.289497 20803 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:42.289635 29939 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:42.289932 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:42.290465 30353 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:42.290848 30353 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:42.291260 30553 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.293120 30353 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.293673 30186 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:42.294432 30353 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:42.299638 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.314179 20806 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:42.282964 30713 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.318005 30795 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.341740 20806 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:42.320951 21433 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 33 because replica has last-logged OpId of term: 32 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 21.
W20260709 12:25:42.323416 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.287854 30763 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:25:42.323879 30794 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.232s	user 0.010s	sys 0.000s
I20260709 12:25:42.331161 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:42.341765 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3694) took 1150 ms. Trace:
I20260709 12:25:42.331579 30762 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.344409 20878 rpcz_store.cc:276] 0709 12:25:41.191133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.215479 (+ 24346us) service_pool.cc:224] Handling call
0709 12:25:41.255284 (+ 39805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.409343 (+154059us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.409380 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:41.409427 (+    47us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:41.409461 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.341539 (+932078us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.341584 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.341711 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353789824}
I20260709 12:25:42.332613 30724 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:42.355655 30724 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.357874 30724 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.358686 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3740) took 1739 ms. Trace:
I20260709 12:25:42.278385 30797 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.361049 30797 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.362525 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3523) took 2733 ms. Trace:
I20260709 12:25:42.362780 30719 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.362721 20775 connection.cc:256] RPC call timeout handler was delayed by -1.134327790000043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:25:42.363552 30719 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.364027 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:42.364847 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.365262 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:42.365403 30786 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.365594 30719 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.367627 30719 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:42.351246 30713 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.344002 30763 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:42.366171 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.366456 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:42.366740 30705 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:42.366868 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.363015 20810 rpcz_store.cc:276] 0709 12:25:39.629093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.634913 (+  5820us) service_pool.cc:224] Handling call
0709 12:25:40.936809 (+1301896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.945568 (+  8759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.362296 (+1416728us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400126450. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:42.362437 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2993331968}
I20260709 12:25:42.350886 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:42.291312 29904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.253s	user 0.014s	sys 0.028s
I20260709 12:25:42.372411 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.377766 20876 raft_consensus.cc:2410] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 31 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:25:42.378237 20875 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:42.379287 29904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.015s	sys 0.028s
I20260709 12:25:42.381356 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3533) took 2738 ms. Trace:
I20260709 12:25:42.381933 20807 rpcz_store.cc:276] 0709 12:25:39.643450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.834309 (+190859us) service_pool.cc:224] Handling call
0709 12:25:39.834579 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.252730 (+418151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.380940 (+2128210us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b540025d5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.381265 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956938112}
I20260709 12:25:42.359136 20734 rpcz_store.cc:276] 0709 12:25:40.619610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.619812 (+   202us) service_pool.cc:224] Handling call
0709 12:25:40.620297 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.740323 (+120026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.740356 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.763581 (+ 23225us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.937942 (+174361us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:40.956416 (+ 18474us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.956640 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.956723 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.830253 (+873530us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18004c81c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:42.323129 (+492876us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:42.358486 (+ 35357us) raft_consensus.cc:1685] finished
0709 12:25:42.358507 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.358629 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199748,"prepare.queue_time_us":21249,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":40,"raft.queue_time_us":118846,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":29312,"spinlock_wait_cycles":70024448,"thread_start_us":137692,"threads_started":1,"wal-append.queue_time_us":176480}
I20260709 12:25:42.388304 30663 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:42.388938 30663 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:42.389525 30705 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:42.390332 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.390789 30584 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:42.390482 30705 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.391314 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.391594 30553 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:42.392103 30553 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.394142 30553 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:42.395577 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44785.6us
I20260709 12:25:42.389854 30663 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:42.398322 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 232494us
I20260709 12:25:42.358975 29939 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:42.405236 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.405582 20849 leader_election.cc:400] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:25:42.406038 20849 leader_election.cc:403] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:42.402173 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.407613 21426 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:42.407124 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.402343 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:42.406776 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:42.401520 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.409755 21432 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:42.402882 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.402968 29903 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:42.408871 20808 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:42.403126 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.413215 21430 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:42.401141 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.403764 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.402410 21434 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:25:42.414438 21431 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:42.412226 29903 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.409171 20807 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:42.406008 20810 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:42.419499 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:42.421814 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.391353 30584 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:42.399842 30463 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.399560 30298 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:42.409363 29939 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.432709 21426 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:25:42.401612 30553 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.399597 30706 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.038s
I20260709 12:25:42.441370 30553 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.442168 30706 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:42.442715 30706 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:42.424913 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.395772 30515 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:25:42.424292 30766 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:42.427232 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:42.378847 30353 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:25:42.423921 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:42.446226 29903 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.404764 30646 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.036s
I20260709 12:25:42.421883 30692 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:42.435173 30799 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:25:42.422988 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:42.458818 20807 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:42.426625 30720 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:42.459576 21430 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:42.445447 29939 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:42.458485 30194 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:42.416452 30763 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.478518 30654 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.471678 30713 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:42.465763 30557 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:42.486833 30520 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:42.488987 30446 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.491755 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3785) took 2181 ms. Trace:
I20260709 12:25:42.498842 30584 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.503513 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.552608 20876 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:42.553679 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.554409 20876 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:42.504099 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:42.470400 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46515.7us
I20260709 12:25:42.514935 21429 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 17. (term mismatch)
I20260709 12:25:42.556826 21429 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 16
I20260709 12:25:42.556643 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.516444 30795 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.512317 30766 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:25:42.479620 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56702.2us
I20260709 12:25:42.534683 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.570473 20878 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:42.535542 21428 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2345) took 2074 ms (client timeout 2096 ms). Trace:
I20260709 12:25:42.551082 30572 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.559398 30643 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.571020 21428 rpcz_store.cc:269] 0709 12:25:40.440159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.440341 (+   182us) service_pool.cc:224] Handling call
0709 12:25:42.514323 (+2073982us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b540013d154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:42.514476 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936247040}
I20260709 12:25:42.571161 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:42.599455 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.601142 20876 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:42.476817 30692 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:42.519655 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 111938us
I20260709 12:25:42.443846 30706 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.557224 21429 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.17
I20260709 12:25:42.444959 30298 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.557865 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.620074 21426 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:42.556670 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:42.481550 30553 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.444845 30463 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.552281 20804 rpcz_store.cc:276] 0709 12:25:40.310027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.475564 (+165537us) service_pool.cc:224] Handling call
0709 12:25:40.475946 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.529003 (+ 53057us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:40.592753 (+ 63750us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.593335 (+   582us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.593391 (+    56us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:40.593434 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.491435 (+1898001us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.491500 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.491685 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":321509,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":374,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10119,"spinlock_wait_cycles":121463040,"thread_start_us":44028,"threads_started":1}
I20260709 12:25:42.561072 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:42.563805 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.605577 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.604363 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:42.543167 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.615048 21429 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:42.622330 30838 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:25:42.629086 30746 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:42.551689 30654 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.630682 30793 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28.
I20260709 12:25:42.602322 30720 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:25:42.607132 30763 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:42.569252 30520 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.599642 30557 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
W20260709 12:25:42.634845 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78245.8us
I20260709 12:25:42.638875 20734 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:42.639277 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:42.641000 21430 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:42.521696 30194 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.641527 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.643276 21435 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:42.637157 20881 raft_consensus.cc:2376] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29. Re-sending same reply.
I20260709 12:25:42.602996 30743 ts_tablet_manager.cc:1916] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 12:25:42.654405 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.694245 20727 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:42.655316 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.655956 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.607788 30766 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:42.685809 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.687081 21428 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:42.695991 20735 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:42.572436 30795 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.690300 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.699079 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.700417 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.598584 30572 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.717689 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:42.718475 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 79321.1us
I20260709 12:25:42.705742 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.604269 30643 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:42.711143 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.715999 20883 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:42.717000 20727 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:42.718587 20734 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:42.645129 30789 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:25:42.721472 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.644778 30675 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:42.725113 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.721930 30840 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 12:25:42.725421 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.768579 20730 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:42.727378 30298 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.736723 20733 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:25:42.737731 30572 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.739478 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.749704 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.750017 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.661242 30762 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.753054 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.755985 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:42.756233 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:42.757634 20726 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:42.766812 20875 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:42.705005 30798 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:42.750916 30720 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:42.778868 20731 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:42.780622 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.780867 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.782099 21428 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:42.757261 30675 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:42.771848 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:42.784538 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:42.802259 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.803176 20726 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:42.690770 21434 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:42.792753 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.793144 20804 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:25:42.794350 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.812124 20808 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:42.791940 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.813020 20732 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:42.785733 20734 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:42.779394 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 276936us
I20260709 12:25:42.803910 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.815907 20727 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:42.816073 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.816885 20734 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:42.816927 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.817878 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:42.818427 20727 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:42.819160 20734 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:42.806408 20802 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 16. Preceding OpId from leader: term: 35 index: 18. (index mismatch)
I20260709 12:25:42.808023 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.820410 20730 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:42.808130 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3790) took 1507 ms. Trace:
I20260709 12:25:42.818145 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:42.821758 20801 rpcz_store.cc:276] 0709 12:25:41.300496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.300664 (+   168us) service_pool.cc:224] Handling call
0709 12:25:41.301086 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:41.301544 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.301575 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:41.305004 (+  3429us) log.cc:844] Serialized 52 byte log entry
0709 12:25:42.352894 (+1047890us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b18003583c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:42.404228 (+ 51334us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:42.404506 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.404597 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.803047 (+398450us) raft_consensus.cc:1685] finished
0709 12:25:42.803068 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.808055 (+  4987us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":632,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":6834560,"thread_start_us":51096,"threads_started":1,"wal-append.queue_time_us":1169045}
I20260709 12:25:42.804291 20728 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:42.806123 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:42.825641 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:42.789784 30724 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:42.829948 29939 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.831655 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:42.837105 20802 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:42.768471 30793 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:25:42.827080 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:42.838611 20804 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:42.772723 30789 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:42.803227 30675 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:42.783497 30798 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:42.839686 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:42.808899 30766 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.040089 29903 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:42.802353 30720 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:42.962235 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3704) took 1009 ms. Trace:
I20260709 12:25:42.885288 29939 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.145731 20882 rpcz_store.cc:276] 0709 12:25:41.953117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.994753 (+ 41636us) service_pool.cc:224] Handling call
0709 12:25:42.961960 (+967207us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:42.962146 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30316288}
I20260709 12:25:43.145275 20726 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:42.848846 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.877308 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.071925 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:42.902642 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.149366 21428 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:43.150213 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.972882 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3791) took 1671 ms. Trace:
I20260709 12:25:43.152144 20805 rpcz_store.cc:276] 0709 12:25:41.301339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.301479 (+   140us) service_pool.cc:224] Handling call
0709 12:25:41.301781 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.807095 (+505314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.972534 (+1165439us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54003d68d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:42.972794 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152859904}
I20260709 12:25:42.982744 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:43.156091 20880 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:42.971372 29904 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.157687 29904 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:42.861776 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:42.809823 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.167972 20735 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:42.810156 30353 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:43.177515 20804 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:43.154232 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:43.034036 30836 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.523s	user 0.004s	sys 0.000s
W20260709 12:25:42.947273 30839 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.176s	user 0.001s	sys 0.001s
I20260709 12:25:42.832862 30713 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:42.840881 30186 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:43.203382 30789 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.021130 30751 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:42.924273 30274 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:43.271468 30851 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:25:43.271205 30720 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:42.900876 30799 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:42.840219 30724 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:42.845726 20732 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:42.981246 30707 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.851432 30463 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:42.933320 30520 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.939977 30692 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.037801 30795 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:42.952584 30654 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.039896 30557 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:43.071002 30656 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:43.047349 30643 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.016999 30194 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:43.055517 30723 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:43.249071 30844 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:25:43.258517 20802 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:25:42.852121 30572 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.361s
I20260709 12:25:43.260237 29904 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.271348 30274 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.283699 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:43.297420 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1825) took 1193 ms. Trace:
I20260709 12:25:43.327389 21433 rpcz_store.cc:276] 0709 12:25:42.103688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.103807 (+   119us) service_pool.cc:224] Handling call
0709 12:25:43.297119 (+1193312us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540050c8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:43.297334 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496087808}
W20260709 12:25:43.278767 30705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.010s	sys 0.010s
I20260709 12:25:43.269457 30751 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:43.272495 30186 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:43.332692 30186 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.275602 30799 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:43.274516 30713 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:43.279263 30724 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:43.332683 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:43.209746 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:43.478442 20880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3622) took 1996 ms (client timeout 2108 ms). Trace:
I20260709 12:25:43.478734 20726 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:43.479518 20880 rpcz_store.cc:269] 0709 12:25:41.482489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.486527 (+  4038us) service_pool.cc:224] Handling call
0709 12:25:42.982517 (+1495990us) spinlock_profiling.cc:243] Waited 922 us on lock 0x7b5800003698. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:25:43.478063 (+495546us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119552}
W20260709 12:25:43.481967 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.482194 20726 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:43.444069 20727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.029s	sys 0.024s
W20260709 12:25:43.483534 20727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.029s	sys 0.024s
W20260709 12:25:43.485603 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 306296us
I20260709 12:25:43.276648 20730 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: [31.19-31.19]
W20260709 12:25:43.490044 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.341790 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 878973us
I20260709 12:25:43.493324 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:43.300076 20732 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:25:43.459683 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.051s	sys 0.050s
W20260709 12:25:43.461649 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.007s	sys 0.016s
W20260709 12:25:43.495729 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.052s	sys 0.051s
W20260709 12:25:43.495869 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.008s	sys 0.016s
W20260709 12:25:43.496570 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.00058e+06us
I20260709 12:25:43.498196 20807 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:43.444433 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.036s
W20260709 12:25:43.498903 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 680774us
W20260709 12:25:43.499449 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.000s	sys 0.037s
W20260709 12:25:43.338197 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.500056 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 658013us
W20260709 12:25:43.462617 20806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.005s	sys 0.005s
W20260709 12:25:43.501916 20806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.005s	sys 0.005s
I20260709 12:25:43.503314 30465 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.503118 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.505030 21433 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:43.504444 30465 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.505703 30274 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.314978 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1808) took 1893 ms. Trace:
I20260709 12:25:43.363793 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:43.508823 30465 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.507676 20809 rpcz_store.cc:276] 0709 12:25:41.421144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.105028 (+683884us) service_pool.cc:224] Handling call
0709 12:25:42.105365 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.361409 (+256044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.361437 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.361893 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.444479 (+ 82586us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:42.444523 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.444602 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.314696 (+870094us) raft_consensus.cc:1685] finished
0709 12:25:43.314727 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.314907 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":269172,"raft.queue_time_us":256718,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":82142,"threads_started":1,"wal-append.queue_time_us":288423}
I20260709 12:25:43.500959 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:43.510335 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.512326 30186 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:43.513501 30186 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.323889 30557 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.515584 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1823) took 1344 ms. Trace:
I20260709 12:25:43.516134 20806 rpcz_store.cc:276] 0709 12:25:42.171486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.342651 (+171165us) service_pool.cc:224] Handling call
0709 12:25:42.343651 (+  1000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.344970 (+  1319us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:42.369387 (+ 24417us) log.cc:844] Serialized 14 byte log entry
0709 12:25:43.502651 (+1133264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:43.502831 (+   180us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:43.503000 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.503139 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.515475 (+ 12336us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":122,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":179255,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":53,"thread_start_us":1120571,"threads_started":2,"wal-append.queue_time_us":825491}
I20260709 12:25:43.519133 30557 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.520346 30557 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.521551 30557 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.525354 30557 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.527093 30557 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:43.527827 30557 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.531726 30572 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:43.566080 30557 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:43.566551 30572 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.569005 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.569872 30572 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.327239 30656 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:43.331720 30705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.011s	sys 0.010s
I20260709 12:25:43.300545 30707 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.378731 30713 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.538252 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:43.600050 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62197.5us
I20260709 12:25:43.340649 30723 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:43.602707 30723 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.603991 30643 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:43.604632 30643 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:43.606258 30723 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:43.342551 30799 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:43.609489 30656 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.542236 30758 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:43.385633 30855 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 12:25:43.613804 30763 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.930s	user 0.012s	sys 0.004s
I20260709 12:25:43.617594 30707 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.377029 30751 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:25:43.548453 30837 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.746s	user 0.000s	sys 0.004s
W20260709 12:25:43.330931 30843 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:25:43.379381 30852 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:25:43.622671 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3633) took 2019 ms. Trace:
I20260709 12:25:43.622936 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1821) took 1451 ms. Trace:
W20260709 12:25:43.557155 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62223.5us
I20260709 12:25:43.629055 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3482) took 1070 ms. Trace:
I20260709 12:25:43.562477 30274 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:43.564819 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:43.581609 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.539535 30186 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.588883 30515 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.636083 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:43.592305 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59742.2us
I20260709 12:25:43.623346 20879 rpcz_store.cc:276] 0709 12:25:41.603148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.785540 (+182392us) service_pool.cc:224] Handling call
0709 12:25:41.785901 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.965447 (+179546us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:41.985719 (+ 20272us) log.cc:844] Serialized 14 byte log entry
0709 12:25:42.093089 (+107370us) log.cc:844] Serialized 14 byte log entry
0709 12:25:42.097985 (+  4896us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:42.098041 (+    56us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:42.098086 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.486596 (+388510us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b1800320740. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:42.981617 (+495021us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b1800320740. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:25:43.622063 (+640446us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54000a2dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:43.622146 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.622417 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12720,"prepare.queue_time_us":20749,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":1448,"raft.queue_time_us":12800,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":4599,"spinlock_wait_cycles":437173504,"thread_start_us":96175,"threads_started":1,"wal-append.queue_time_us":94940,"wal-append.run_cpu_time_us":2018,"wal-append.run_wall_time_us":1091236}
I20260709 12:25:43.636150 20807 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:43.595922 20727 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:43.597281 20806 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:43.597913 30572 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:43.259932 30798 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:43.635010 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70568.9us
I20260709 12:25:43.629633 20729 rpcz_store.cc:276] 0709 12:25:42.558544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.558654 (+   110us) service_pool.cc:224] Handling call
0709 12:25:42.558923 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.696478 (+137555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.696508 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.717209 (+ 20701us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.127951 (+410742us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:43.128015 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.599953 (+471938us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54002e4fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:43.600033 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.628353 (+ 28320us) raft_consensus.cc:1685] finished
0709 12:25:43.628389 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.628799 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":493,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":31284,"spinlock_wait_cycles":315521920,"thread_start_us":401721,"threads_started":1,"wal-append.queue_time_us":484232}
I20260709 12:25:43.626896 30766 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:25:43.637131 30298 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:43.656888 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1813) took 1683 ms. Trace:
I20260709 12:25:43.657232 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.657464 21429 rpcz_store.cc:276] 0709 12:25:41.973264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.973417 (+   153us) service_pool.cc:224] Handling call
0709 12:25:42.514584 (+541167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.639522 (+124938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.655413 (+1015891us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540013d150. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:43.656667 (+  1254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1242412416}
I20260709 12:25:43.652233 30465 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:43.667519 30274 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:43.637195 30515 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:43.675925 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 76919.1us
I20260709 12:25:43.623519 20803 rpcz_store.cc:276] 0709 12:25:42.170898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.290670 (+119772us) service_pool.cc:224] Handling call
0709 12:25:42.290983 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.297160 (+  6177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.297188 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.297702 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.418155 (+120453us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:42.418209 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.418306 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.546970 (+1128664us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:43.622317 (+ 75347us) raft_consensus.cc:1685] finished
0709 12:25:43.622350 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.622738 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9322240,"thread_start_us":100600,"threads_started":1,"wal-append.queue_time_us":434128}
I20260709 12:25:43.681141 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:43.653644 30572 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.689755 30572 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.690536 30697 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:43.691996 30465 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:43.692888 30465 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.692749 30697 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:43.693719 30798 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:43.691148 30723 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:43.671092 30515 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:43.695792 30720 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.697124 30720 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.606734 30643 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.708860 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:43.695809 30723 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:43.713250 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.720034 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:43.720103 30723 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.717587 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.731133 30194 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:43.734216 21435 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:43.733017 30604 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:43.739334 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:43.740379 20880 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:43.716050 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.745257 30720 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:43.753701 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:43.755357 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50590.2us
I20260709 12:25:43.755980 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.756378 30706 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:43.756793 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:43.757440 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:43.757845 20807 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:25:43.758409 20805 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:43.759613 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:43.760434 20808 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:43.763811 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:43.729074 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:43.767758 30697 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:43.770644 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.771159 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.755410 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:43.767838 21429 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:43.779614 29903 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:43.781229 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.799630 20729 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:43.781623 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:43.782188 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:43.804411 20805 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:43.805431 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.806711 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.807641 20805 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:43.784109 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.809669 20727 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:43.786461 30751 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:43.756836 20804 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:25:43.808495 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.772264 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1612) took 1039 ms. Trace:
I20260709 12:25:43.774962 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.780359 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:43.818360 20808 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:43.791836 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:43.795228 20877 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:43.785486 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.824554 20877 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:43.829809 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:43.759632 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.787765 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:43.834062 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:43.834084 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:43.836076 20882 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:43.837209 20803 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:25:43.837803 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3764) took 1103 ms. Trace:
I20260709 12:25:43.840667 20877 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.787981 30724 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.842763 20877 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:43.781149 30840 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.062s
I20260709 12:25:43.812263 20734 rpcz_store.cc:276] 0709 12:25:42.732651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.821568 (+ 88917us) service_pool.cc:224] Handling call
0709 12:25:42.859642 (+ 38074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.859917 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:42.933905 (+ 73988us) log.cc:844] Serialized 14 byte log entry
0709 12:25:43.535669 (+601764us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b18002b11a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:25:43.771307 (+235638us) log.cc:844] Serialized 14 byte log entry
0709 12:25:43.771596 (+   289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:43.771675 (+    79us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:43.771775 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.771858 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.772072 (+   214us) inbound_call.cc:177] Queueing success response
0709 12:25:43.791490 (+ 19418us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b18002b11a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
Metrics: {"prepare.queue_time_us":102408,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":111,"raft.queue_time_us":10418,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":266,"spinlock_wait_cycles":40255104,"thread_start_us":43629,"threads_started":1,"wal-append.queue_time_us":721278}
I20260709 12:25:43.843751 30799 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.848912 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.849560 30799 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.849722 20807 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:43.844409 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:43.850337 30675 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.654s	user 0.005s	sys 0.015s
I20260709 12:25:43.851797 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:43.852029 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70590.6us
W20260709 12:25:43.772239 30663 consensus_queue.cc:1175] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 28 status { last_received { term: 27 index: 20 } last_committed_idx: 19 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:43.857885 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:43.860704 20808 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:25:43.862221 30763 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.863353 30697 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.863968 30697 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.864859 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.837837 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:43.866488 30697 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.870682 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.868323 30707 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:43.871757 20804 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:43.872572 20806 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:43.872808 30553 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:43.873376 30553 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:43.873915 30553 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:43.838455 20735 rpcz_store.cc:276] 0709 12:25:42.733814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.212140 (+478326us) service_pool.cc:224] Handling call
0709 12:25:43.250854 (+ 38714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.251110 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:43.260696 (+  9586us) log.cc:844] Serialized 14 byte log entry
0709 12:25:43.706366 (+445670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:43.715612 (+  9246us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:43.829345 (+113733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.837306 (+  7961us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.837548 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55445,"prepare.queue_time_us":4768,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":61578,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":2077,"thread_start_us":317576,"threads_started":1,"wal-append.queue_time_us":371827}
W20260709 12:25:43.877149 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.878022 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.879669 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:43.880076 30798 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:43.881366 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:43.881822 30845 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:43.885651 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:43.891485 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:43.892560 20880 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:43.893641 20809 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:25:43.862886 30763 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.895798 30553 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.844846 30724 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.897387 30763 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.898289 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:43.870450 30572 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:43.899111 30572 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:43.900519 30572 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.901347 20802 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:43.868896 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:43.869163 30520 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:43.890936 20877 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 21. (term mismatch)
I20260709 12:25:43.903095 20876 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:25:43.898383 30553 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.851074 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.911640 20807 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:43.910323 20803 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:43.912328 21430 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 22. (term mismatch)
I20260709 12:25:43.912865 21430 pending_rounds.cc:85] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 21
I20260709 12:25:43.907817 20877 pending_rounds.cc:85] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 20
I20260709 12:25:43.913223 21430 pending_rounds.cc:107] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 32.22
I20260709 12:25:43.913662 21430 raft_consensus.cc:2889] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:43.913671 20877 pending_rounds.cc:107] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 27.21
I20260709 12:25:43.914367 20877 raft_consensus.cc:2889] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:43.783862 30465 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:43.915498 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.916227 30663 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:43.916738 21433 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 32 because replica has last-logged OpId of term: 31 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 18.
W20260709 12:25:43.917702 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48922.1us
I20260709 12:25:43.918649 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:43.919512 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 87739.5us
I20260709 12:25:43.920229 29903 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:43.920810 29903 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:43.921141 29903 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:43.921864 30799 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:43.922217 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:43.923636 30465 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:43.923688 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:43.924335 30465 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:43.924728 20882 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:43.924865 30465 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:43.924971 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.925346 30799 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:43.925815 21435 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:43.919863 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:43.926268 30799 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:43.924765 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 137283us
I20260709 12:25:43.903470 20731 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:25:43.923547 30719 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:43.936439 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:43.927155 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.872808 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.850926 30675 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.950409 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:43.955138 30591 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:43.957862 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.958101 30795 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.959291 30663 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:43.962283 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.963546 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:43.964264 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.965303 20729 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:43.965824 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:43.967475 20733 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:43.939043 30859 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.969327 20730 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:25:43.969602 30859 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:43.958736 30795 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.971519 30697 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:43.972275 30859 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.972508 30697 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:43.927790 30840 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:43.974170 30840 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:43.974699 30840 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:43.978427 20876 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:43.980737 30675 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:44.023856 30763 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:44.024940 30763 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:43.980589 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:43.981952 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:43.968926 30707 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.987581 20628 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 24 to 25, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 25 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.989473 30643 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:44.028995 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.030200 30707 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.032914 30553 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:43.990645 30799 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:43.992213 30766 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.033466 30553 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:43.993681 21434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44460
I20260709 12:25:44.033903 30553 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:44.034210 21434 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:44.034613 21434 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:44.037607 30763 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:43.995458 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48494us
I20260709 12:25:43.995792 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:43.996201 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77683.7us
I20260709 12:25:44.039160 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:44.039389 20883 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:44.043295 30186 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:44.044728 30520 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:44.034618 30707 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:44.045198 30520 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:44.039736 30675 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:44.046294 30520 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.13, Last appended by leader: 13, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.046387 30675 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:43.996783 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98594.3us
I20260709 12:25:43.997990 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:43.998926 21432 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:44.000509 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:43.903291 30515 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.048544 30515 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.033908 30766 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.049412 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:43.928124 30646 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:44.050942 30697 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:44.041347 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:44.051676 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.051892 30646 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:25:44.052940 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.053766 30663 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:44.053975 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:44.001915 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.054625 30663 raft_consensus.cc:2764] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:25:44.055047 20882 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:43.998927 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.002970 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.056352 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:44.008643 21427 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:44.011034 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.058025 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:25:44.012006 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:44.012310 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.060357 20628 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:44.012989 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:44.062651 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49797us
I20260709 12:25:43.954092 20801 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:44.011976 30465 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.015697 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.064738 30763 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:44.016417 30719 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:44.006002 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:44.032796 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52363.1us
I20260709 12:25:44.067565 30465 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:44.051118 30766 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:44.068809 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.048935 30520 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:44.069581 30643 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:44.069957 30719 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:44.070566 30719 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:44.071535 30719 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.13, Last appended by leader: 13, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.073958 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:44.068642 30840 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.050918 30515 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.066586 30763 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.076220 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:43.991165 30795 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:44.077960 30720 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:44.081579 30857 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:44.082646 30845 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:44.079406 30707 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:44.082398 30720 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:44.047837 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:44.083873 30720 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:44.081480 30515 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:44.083395 30845 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:44.085031 30515 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:44.086126 30845 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.086283 30515 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.088546 30857 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:44.062232 20628 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 24, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 24 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:44.081671 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.018834 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.082506 29903 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:44.109107 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 154720us
I20260709 12:25:44.105526 21434 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.140209 20806 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:44.142797 21434 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:44.145352 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.146076 20880 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:44.149389 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:44.149644 30446 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.150059 20878 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:44.138733 30186 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:44.150226 30553 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.150837 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.151055 30186 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:44.153187 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 100653us
I20260709 12:25:44.153542 30186 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.153481 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.154639 20875 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:44.126597 30795 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:44.155057 30766 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:44.155450 30795 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.155897 30795 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:44.156147 20880 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:25:44.150305 30446 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.159408 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.160296 30446 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:44.160689 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.161023 30857 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:44.161087 20883 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
W20260709 12:25:44.161314 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 113564us
I20260709 12:25:44.161720 30857 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:44.162242 30857 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:44.162403 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:44.157984 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:44.159060 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.164168 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:44.165894 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.166996 30446 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:44.167528 30446 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.167695 30663 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:44.168044 30446 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:44.168170 30663 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:44.168378 20624 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 276) took 32566 ms (client timeout 15000 ms). Trace:
I20260709 12:25:44.168676 30663 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:44.175407 30845 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:44.175635 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.179001 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.048929 30675 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:44.182518 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.139679 30646 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.059s
I20260709 12:25:44.189769 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:44.196910 21430 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:44.198812 20628 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 31, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 31 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:44.223446 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:44.225760 20727 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:44.226696 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:44.179773 30465 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.232157 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:44.235235 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71144.8us
I20260709 12:25:44.199980 30274 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:44.236330 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:44.187013 30298 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:44.198625 30799 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:44.256558 20808 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:44.249696 20628 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 26 to 27, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 27 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:44.250855 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.259655 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.239910 30724 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:44.259140 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:44.264957 21432 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:25:44.168887 20624 rpcz_store.cc:269] 0709 12:25:11.602075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.648844 (+ 46769us) service_pool.cc:224] Handling call
0709 12:25:44.168148 (+32519304us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:34.399070 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:34.434957 (+ 35887us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:34.435012 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:25:34.435023 (+    11us) tablet.cc:662] Decoding operations
0709 12:25:34.436858 (+  1835us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:34.436955 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:34.436988 (+    33us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:25:34.438738 (+  1750us) tablet.cc:701] Row locks acquired
0709 12:25:34.438788 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:25:34.438967 (+   179us) write_op.cc:270] Start()
0709 12:25:34.439136 (+   169us) write_op.cc:276] Timestamp: P: 1783599934438900 usec, L: 0
0709 12:25:34.439152 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:25:34.512920 (+ 73768us) log.cc:844] Serialized 8781 byte log entry
0709 12:25:36.238927 (+1726007us) op_driver.cc:464] REPLICATION: finished
0709 12:25:36.571891 (+332964us) write_op.cc:301] APPLY: starting
0709 12:25:36.592479 (+ 20588us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:37.386610 (+794131us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b18003d0600. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e9de0ec 00007f9e1e9e0af7 00007f9e1e9e1fb0 00007f9e226f61bb 00007f9e226f6532 00007f9e2273db93 00007f9e225eade3 00007f9e225ea6ab 00007f9e225ebe77
0709 12:25:37.575020 (+188410us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:37.575070 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:38.421909 (+846839us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:38.422459 (+   550us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 12:25:38.422534 (+    75us) write_op.cc:312] APPLY: finished
0709 12:25:38.432811 (+ 10277us) log.cc:844] Serialized 196 byte log entry
0709 12:25:38.766450 (+333639us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:38.823342 (+ 56892us) write_op.cc:454] Released schema lock
0709 12:25:38.923314 (+ 99972us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:39.013247 (+ 89933us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.013285 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":681360,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":244544,"apply.run_cpu_time_us":79155,"apply.run_wall_time_us":2402428,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96087,"num_ops":22,"prepare.queue_time_us":476734,"prepare.run_cpu_time_us":58356,"prepare.run_wall_time_us":441252,"raft.queue_time_us":411646,"raft.run_cpu_time_us":5442,"raft.run_wall_time_us":287760,"replication_time_us":1754569,"spinlock_wait_cycles":29750016,"thread_start_us":1068142,"threads_started":5,"wal-append.queue_time_us":893739,"wal-append.run_cpu_time_us":17368,"wal-append.run_wall_time_us":3262359}]]}
I20260709 12:25:44.271828 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.272400 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:44.257445 30557 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.691s	user 0.015s	sys 0.001s
I20260709 12:25:44.243814 30446 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.252470 30572 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:44.272859 21435 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:44.277308 21434 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:44.269120 30763 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.231288 30859 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.046s
I20260709 12:25:44.280686 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:44.283288 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.284755 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.285506 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:44.281522 20628 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:44.248973 30465 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.276808 30705 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:44.280323 30572 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.305078 30789 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:25:44.305192 20624 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 298) took 16692 ms (client timeout 15000 ms). Trace:
W20260709 12:25:44.306595 20624 rpcz_store.cc:269] 0709 12:25:27.612556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.304944 (+16692388us) service_pool.cc:211] Skipping call since client already timed out
0709 12:25:44.305095 (+   151us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:25:44.307235 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60969.2us
I20260709 12:25:44.310619 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:44.313999 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:44.314647 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 124965us
I20260709 12:25:44.280043 30557 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.316205 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:44.302399 30515 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:44.323536 20801 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:44.324071 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.325420 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:44.327246 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 282648us
I20260709 12:25:44.327553 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.328512 21429 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:44.329124 30866 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:44.329757 30866 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.330583 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.320851 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:44.302244 30724 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:44.322556 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:44.332325 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.335180 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:44.335016 21432 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 32 because replica has last-logged OpId of term: 31 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 18.
I20260709 12:25:44.304193 30572 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:44.312798 30298 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:44.318168 30654 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:44.357538 20806 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:44.358067 20806 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:44.332590 20628 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:44.359694 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 116733us
I20260709 12:25:44.359948 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.359953 21428 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:44.361433 29903 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.335219 30675 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:44.335013 30697 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:44.364708 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:44.365944 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61288.4us
I20260709 12:25:44.349794 30298 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.367753 30298 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:44.368122 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:44.371634 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49161.2us
I20260709 12:25:44.365041 29903 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:44.370957 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:44.373796 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:44.377269 20882 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:44.375705 30894 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:44.375777 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:44.381853 20884 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 12:25:44.375877 30878 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:44.382681 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:44.356863 30654 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:44.401015 30654 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:44.401360 30724 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.353788 30646 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:25:44.411257 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:44.418239 30890 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:25:44.419713 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45998.5us
I20260709 12:25:44.420320 30557 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:44.421842 30465 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:44.387976 20806 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.373835 30840 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.364744 30697 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:44.434386 20878 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:44.413935 21426 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:25:44.381856 30894 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:44.426316 20628 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 20 to 24, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:44.458402 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 140214us
I20260709 12:25:44.464452 30705 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:44.401198 30706 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:44.425999 30707 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:44.440610 30520 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:44.481626 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:44.484803 20628 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 26. New cstate: current_term: 26 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:44.445366 30697 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:44.481009 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 145976us
I20260709 12:25:44.489938 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:44.489532 20628 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 28, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 28 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.492197 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 27 to 28, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 28 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:44.397055 30675 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 12:25:44.494118 20628 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 22 to 25, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 25 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:44.453907 30789 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:44.413997 30766 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:25:44.533995 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:44.538098 20731 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:44.466317 30894 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:44.481529 30706 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:44.484261 30707 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.538580 20729 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:44.529744 30751 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.005s
I20260709 12:25:44.482556 30840 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:44.546864 20628 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:44.440392 30194 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.055s
W20260709 12:25:44.555569 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 133606us
I20260709 12:25:44.452363 30720 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.466461 30572 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.540548 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:44.564095 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 184229us
W20260709 12:25:44.587085 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 216224us
I20260709 12:25:44.485235 30515 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:25:44.482887 30795 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:44.601287 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.601936 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 57636us
I20260709 12:25:44.591837 20876 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:44.487825 30274 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.077s
I20260709 12:25:44.610085 20628 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:44.649053 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 224199us
I20260709 12:25:44.658753 20628 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 14 to 20, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 20 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:44.666594 20628 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 25. New cstate: current_term: 25 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:44.556741 30706 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:44.559775 30707 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:44.587970 30840 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:44.691627 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 226383us
I20260709 12:25:44.689283 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:25:44.693070 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:44.649523 30656 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:44.674038 30274 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:44.710843 30901 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:44.714242 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.737397 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:44.737509 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45928.9us
I20260709 12:25:44.739101 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:44.686192 30553 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.736769 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:44.738368 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:44.697783 30840 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:44.764228 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:44.707165 30656 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:44.713477 30751 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.777725 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:44.793331 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100327us
I20260709 12:25:44.795559 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:44.768360 30465 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:44.841996 30465 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.845254 30465 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:44.824074 30903 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:25:44.813740 30572 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.849274 30697 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.852545 30697 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:44.854521 30663 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.000s
W20260709 12:25:44.817179 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47958.9us
I20260709 12:25:44.819952 30656 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.768378 30705 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.765779 30557 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:25:44.824048 30743 ts_tablet_manager.cc:1929] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 21.15
W20260709 12:25:44.824867 30891 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.400s	user 0.000s	sys 0.002s
I20260709 12:25:44.827993 30298 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.836217 30720 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:44.855070 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 205731us
I20260709 12:25:44.816133 30553 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:44.865957 30675 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:44.867436 30795 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:44.873695 30186 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.881224 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.785281 30274 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:44.880198 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:44.935796 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55667.3us
W20260709 12:25:44.894219 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 108071us
I20260709 12:25:44.898371 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:44.806151 30904 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:44.910363 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 484167us
I20260709 12:25:44.802506 30520 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:44.800742 30751 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.947067 30553 consensus_queue.cc:1243] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 31.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:44.947693 30902 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:44.949000 30751 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:44.949714 30751 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:44.952782 30743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047135f71834dbab6b869eb3c081b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:44.931962 30656 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:44.887526 30654 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.942551 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:44.955533 20877 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:44.945875 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:44.945943 20883 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:25:44.896435 30663 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:44.935092 30274 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:44.962215 20879 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:44.932189 30894 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:44.962249 30910 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:25:44.948290 30902 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:44.971932 20801 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:44.973879 30298 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:44.983305 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:44.998900 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 203444us
I20260709 12:25:45.015301 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:45.021986 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:45.138273 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:45.139165 20731 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:44.977960 30720 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:45.157482 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:45.144984 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.146103 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:45.146920 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:45.157358 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 136226us
I20260709 12:25:45.163333 21434 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:44.977196 30697 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:45.170917 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.171811 20880 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:44.958792 30553 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.200s
W20260709 12:25:45.201239 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 186150us
I20260709 12:25:45.143246 30707 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.197156 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:45.204286 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:44.954787 30186 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:45.197762 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 214583us
I20260709 12:25:45.191252 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:45.198618 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 53478.6us
I20260709 12:25:45.207382 20883 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:45.212569 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:45.213256 21434 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:45.213759 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:45.218619 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:45.225535 20801 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:45.227959 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:45.221235 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:45.224959 30764 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.234215 20735 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:45.204324 30857 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.042s	user 0.009s	sys 0.008s
W20260709 12:25:45.204793 30845 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.029s	user 0.009s	sys 0.005s
I20260709 12:25:45.165218 30720 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:45.176947 30697 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:45.197077 30591 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.243741 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:45.258869 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 1843) took 1381 ms. Trace:
I20260709 12:25:45.250413 20877 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:45.021479 30520 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:45.254562 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 322122us
I20260709 12:25:45.259492 21428 rpcz_store.cc:276] 0709 12:25:43.864367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.864557 (+   190us) service_pool.cc:224] Handling call
0709 12:25:45.245607 (+1381050us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:45.245753 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122275712}
I20260709 12:25:45.235077 30557 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:25:45.265161 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 317918us
I20260709 12:25:45.267131 30910 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:45.255965 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:45.276352 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:44.998554 30904 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.281718 20731 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:45.282533 20728 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:45.257799 30720 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:45.255993 30697 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:45.253562 30857 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:45.297350 20732 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:45.233995 30764 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:45.314170 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68666.7us
I20260709 12:25:45.320168 21434 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
W20260709 12:25:45.320557 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44296.3us
I20260709 12:25:45.291702 30643 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:45.291656 30705 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:45.284188 30591 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:45.279769 30902 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:45.304488 30924 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:45.345011 30924 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.345432 30924 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:45.349234 30764 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:45.295549 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 28 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:45.350548 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 184946us
I20260709 12:25:45.297281 30918 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:45.358558 30918 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:45.358991 30918 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:45.360661 20804 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 12:25:45.364178 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.839s (SENT) [suppressed 22 similar messages]
I20260709 12:25:45.303952 30194 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:45.338017 30643 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.278010 30795 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.039s
W20260709 12:25:45.354950 30884 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.574s	user 0.002s	sys 0.000s
W20260709 12:25:45.367151 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.391368 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1854) took 1154 ms. Trace:
I20260709 12:25:45.391659 30663 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.391889 21430 rpcz_store.cc:276] 0709 12:25:44.237223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.241382 (+  4159us) service_pool.cc:224] Handling call
0709 12:25:44.875894 (+634512us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b580010fe98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:25:44.876095 (+   201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.876387 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.876412 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.876831 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.173140 (+296309us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:45.173185 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.173253 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.391018 (+217765us) raft_consensus.cc:1685] finished
0709 12:25:45.391045 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.391302 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18028,"raft.queue_time_us":82329,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":12860,"spinlock_wait_cycles":11338496,"thread_start_us":295923,"threads_started":1,"wal-append.queue_time_us":507328}
I20260709 12:25:45.394726 30763 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:45.395164 30763 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:45.396078 30186 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:45.342067 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:45.396162 30763 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.303135 30743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44462 (request call id 1806) took 5830 ms. Trace:
I20260709 12:25:45.371500 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:45.397835 30840 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.618s	user 0.005s	sys 0.006s
I20260709 12:25:45.398092 30912 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:45.398703 30912 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:45.398900 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.374913 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.400157 30446 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:45.400283 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.400804 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.371799 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.401592 20884 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:45.402102 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.378512 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1853) took 1141 ms. Trace:
I20260709 12:25:45.301221 30520 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:45.402900 21433 rpcz_store.cc:276] 0709 12:25:44.236760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.237406 (+   646us) service_pool.cc:224] Handling call
0709 12:25:44.268122 (+ 30716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.295158 (+ 27036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.295223 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:44.298899 (+  3676us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:44.305677 (+  6778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.378267 (+1072590us) spinlock_profiling.cc:243] Waited 9.3 ms on lock 0x7b54007b2ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:45.378317 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.378450 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21389696}
I20260709 12:25:45.404194 20877 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: [28.17-30.18]
I20260709 12:25:45.405246 20735 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:45.373157 20806 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:45.405627 30897 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.692s	user 0.001s	sys 0.001s
I20260709 12:25:45.407411 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1847) took 1214 ms. Trace:
I20260709 12:25:45.399668 30186 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:45.407779 21431 rpcz_store.cc:276] 0709 12:25:44.193149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.199718 (+  6569us) service_pool.cc:224] Handling call
0709 12:25:44.245196 (+ 45478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.251586 (+  6390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.251619 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.274068 (+ 22449us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.448849 (+174781us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:44.481833 (+ 32984us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.695558 (+213725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.695636 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.404962 (+709326us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:45.407095 (+  2133us) raft_consensus.cc:1685] finished
0709 12:25:45.407216 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.407352 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":689,"prepare.queue_time_us":34079,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":242,"raft.queue_time_us":72602,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":6347,"spinlock_wait_cycles":51350400,"thread_start_us":344832,"threads_started":2,"wal-append.queue_time_us":270925}
I20260709 12:25:45.387609 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3569) took 1604 ms. Trace:
I20260709 12:25:45.299510 30857 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.410748 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:45.302320 30707 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:45.397413 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55484.2us
I20260709 12:25:45.411689 20879 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:25:45.412326 30894 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.399709 30912 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:45.410200 20810 rpcz_store.cc:276] 0709 12:25:43.782890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.852878 (+ 69988us) service_pool.cc:224] Handling call
0709 12:25:43.853351 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.863355 (+ 10004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.863384 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.864051 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.129953 (+265902us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:44.130012 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.130105 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.354494 (+1224389us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b24001686c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf109d 00007f9e1e82310e 00007f9e1e83a88c 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221
0709 12:25:45.387308 (+ 32814us) raft_consensus.cc:1685] finished
0709 12:25:45.387331 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.387550 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21601,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":138,"spinlock_wait_cycles":140137856,"thread_start_us":265390,"threads_started":1,"wal-append.queue_time_us":813952}
I20260709 12:25:45.413888 30912 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:45.372212 30643 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:45.416528 30706 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.002s
I20260709 12:25:45.417003 30706 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.419958 30766 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:45.420423 30766 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:45.421393 30766 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.422544 30766 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:45.383595 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:25:45.371637 30654 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.386137 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:45.368798 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.425173 20881 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:45.426805 20884 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:45.371472 30194 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.428125 30194 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:45.431402 30840 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:45.435263 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:45.436285 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69104.9us
I20260709 12:25:45.438328 30840 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.397410 30763 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:45.370473 30866 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.425911 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:45.434496 30591 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.397701 30743 rpcz_store.cc:276] 0709 12:25:39.487891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.488171 (+   280us) service_pool.cc:224] Handling call
0709 12:25:40.121900 (+633729us) ts_tablet_manager.cc:934] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
0709 12:25:42.407246 (+2285346us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:43.777189 (+1369943us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:44.690666 (+913477us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:44.952313 (+261647us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 12:25:45.301937 (+349624us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:45.302467 (+   530us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:45.302887 (+   420us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:45.303019 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":288,"tablet-copy.queue_time_us":605794,"thread_start_us":456973,"threads_started":2}
W20260709 12:25:45.439963 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.440135 20880 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:45.441109 30881 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.193s	user 0.000s	sys 0.010s
I20260709 12:25:45.426645 29903 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:45.448941 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 227806us
W20260709 12:25:45.460283 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.462725 20727 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:45.466154 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 222512us
I20260709 12:25:45.399030 20883 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:45.472656 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:45.474578 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.478955 20806 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:45.481817 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:45.481182 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69073.8us
I20260709 12:25:45.484804 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:45.484988 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1847) took 1180 ms (client timeout 1516 ms). Trace:
W20260709 12:25:45.485513 20806 rpcz_store.cc:269] 0709 12:25:44.299395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.299574 (+   179us) service_pool.cc:224] Handling call
0709 12:25:45.372219 (+1072645us) spinlock_profiling.cc:243] Waited 81.1 ms on lock 0x7b58005033d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca327c 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c
0709 12:25:45.479795 (+107576us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186455424}
W20260709 12:25:45.486032 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60238.5us
W20260709 12:25:45.483678 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 234452us
I20260709 12:25:45.495990 20804 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:45.502593 29903 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:45.480615 30840 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:45.462070 30934 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:45.454847 30706 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:45.509965 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.506949 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:45.521309 20883 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:45.480105 30724 raft_consensus.cc:2764] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:25:45.516651 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:45.529217 20802 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:45.524641 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:45.529712 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:45.530979 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.477178 30866 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:45.528558 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:45.547302 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74729.6us
W20260709 12:25:45.551851 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.590168 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55972.7us
I20260709 12:25:45.496176 30798 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.589869 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 105149us
W20260709 12:25:45.592665 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57767.1us
I20260709 12:25:45.459489 30194 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.595382 29903 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.597929 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82827.7us
I20260709 12:25:45.597853 29903 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.530408 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:45.527287 30719 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.024s	sys 0.020s
I20260709 12:25:45.600186 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:45.602079 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72467.5us
I20260709 12:25:45.588433 30764 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:45.532625 30845 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:45.607142 30520 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.593732 30646 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:25:45.609025 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 107958us
I20260709 12:25:45.607395 30923 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:25:45.604849 30719 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.024s	sys 0.021s
I20260709 12:25:45.495201 30894 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 12:25:45.600716 30743 tablet_copy_client.cc:806] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:45.614032 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 1846) took 1337 ms. Trace:
I20260709 12:25:45.599123 30866 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.643498 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:45.643708 30866 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.645527 20727 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 17. Preceding OpId from leader: term: 35 index: 18. (index mismatch)
I20260709 12:25:45.607234 30914 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:45.621441 20881 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.21-28.21]
I20260709 12:25:45.623087 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:45.624569 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 94225.9us
I20260709 12:25:45.613572 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:45.628957 29903 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:45.527637 30697 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:45.659310 30646 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:45.659473 30914 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.660358 30697 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.660542 30194 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:45.661845 30923 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:45.660394 30646 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.662216 30914 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:45.664441 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.665696 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.666252 20878 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:45.612066 30764 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:45.651678 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:45.656272 20809 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:25:45.619457 30915 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.178s
I20260709 12:25:45.657764 30274 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.640359 20808 rpcz_store.cc:276] 0709 12:25:44.277278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.278082 (+   804us) service_pool.cc:224] Handling call
0709 12:25:45.609992 (+1331910us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b540027bd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:45.613962 (+  3970us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270388608}
I20260709 12:25:45.620735 30795 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:45.669730 20727 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:25:45.611181 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:45.670625 20802 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:25:45.627378 30898 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.613224 30706 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:25:45.640551 30845 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:45.709340 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:45.694173 29903 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.717317 21430 raft_consensus.cc:2410] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:25:45.695228 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.601590 30798 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.695915 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:45.703823 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50749.9us
I20260709 12:25:45.696576 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:45.696934 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:45.703440 20728 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:45.711232 30663 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:45.711606 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:45.712231 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:45.720674 20734 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:45.712852 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:45.713205 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:45.713583 20804 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:45.714075 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:45.723798 21432 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:45.714902 21427 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:45.695281 30675 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:25:45.721273 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:45.613376 30924 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.737478 21435 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:45.725054 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:45.725231 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:45.725528 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:45.709374 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:45.659327 30928 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:45.738564 20733 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:45.692703 30194 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:45.718631 20881 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:45.692893 30866 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:45.728613 20810 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:45.748594 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:45.690410 20878 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:45.695276 30743 tablet_copy_client.cc:670] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:45.742466 20726 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:45.743450 29903 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:45.750137 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:45.628451 30520 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.753247 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:45.751004 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:45.756366 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:45.757467 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:45.757592 30921 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:45.758220 20810 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:45.758183 30921 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:45.759538 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144397us
I20260709 12:25:45.760387 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:45.760564 30921 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:45.699049 30909 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:45.767864 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51344.6us
I20260709 12:25:45.768319 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.753247 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:45.709004 30465 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:45.761740 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:45.662832 30923 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.746730 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:45.766462 20729 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:45.727833 30656 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.742864 30857 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:45.742916 30928 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.834153 20878 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:45.806824 20884 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:45.788020 21432 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:25:45.768625 20809 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
W20260709 12:25:45.828905 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 217764us
I20260709 12:25:45.808440 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:45.836426 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:45.935539 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:45.743119 30720 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:45.812265 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61354.9us
I20260709 12:25:45.773303 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:45.773352 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:45.846454 20727 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:25:45.848845 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:45.938623 20731 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:45.942641 29903 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:45.945736 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 171272us
I20260709 12:25:45.946832 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:45.947909 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 94269.6us
I20260709 12:25:46.013012 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:46.003906 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.988561 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:46.016661 20727 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:45.737417 30845 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:45.709975 30918 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.763512 30912 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:45.841131 30917 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.024963 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 251701us
W20260709 12:25:46.031971 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 270293us
W20260709 12:25:46.025228 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89770.5us
I20260709 12:25:45.987680 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:45.837622 30940 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:25:45.767895 30866 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:45.827430 30795 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:45.728137 30898 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.840620 30924 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:45.835932 30938 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:45.838662 30914 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.808229 30274 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:45.754196 30194 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:45.835635 30646 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:45.849315 30751 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.044164 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 286957us
I20260709 12:25:46.044210 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:46.037954 30719 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:45.985872 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:45.955461 30928 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:46.061615 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.053871 30857 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:46.077618 30751 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.081456 30751 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.083070 30515 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.069245 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.031741 30789 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:45.835757 30298 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:46.073567 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:45.987118 30446 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.064s
I20260709 12:25:45.840862 30916 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.064342 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:46.075635 20726 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:25:45.772634 30935 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.475s
I20260709 12:25:45.759714 30764 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:45.943385 30656 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.109418 20732 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:25:46.143870 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1850) took 1203 ms. Trace:
I20260709 12:25:46.086844 20731 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
W20260709 12:25:46.078428 30643 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.664s	user 0.014s	sys 0.000s
I20260709 12:25:46.086894 20727 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
W20260709 12:25:46.136372 30949 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:25:46.277909 20731 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
W20260709 12:25:46.137075 30942 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:25:46.272667 21431 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 12:25:46.280839 21431 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
W20260709 12:25:46.271167 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 197709us
W20260709 12:25:46.278321 21429 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.536s	user 0.003s	sys 0.007s
I20260709 12:25:46.281845 21429 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:46.279014 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:46.282109 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:46.148550 30520 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.226912 30898 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.070340 30914 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.280851 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:46.200021 30675 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:46.285043 20805 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.022s	sys 0.000s
I20260709 12:25:46.237133 20801 rpcz_store.cc:276] 0709 12:25:44.940028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.273153 (+333125us) service_pool.cc:224] Handling call
0709 12:25:45.290972 (+ 17819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.295892 (+  4920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.295935 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.296565 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.485724 (+189159us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:45.485764 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.485831 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.143501 (+657670us) raft_consensus.cc:1685] finished
0709 12:25:46.143532 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.143784 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10096,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":5347,"thread_start_us":188682,"threads_started":1,"wal-append.queue_time_us":352183}
W20260709 12:25:46.089861 30705 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.000s
I20260709 12:25:46.301520 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:46.302471 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 258341us
I20260709 12:25:46.326985 20881 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:46.305373 20878 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:46.075173 30917 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.123550 30751 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.161s
I20260709 12:25:46.143658 30921 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.191s
W20260709 12:25:46.319123 20805 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.023s	sys 0.000s
I20260709 12:25:46.271618 30859 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:46.236311 30706 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:46.347433 21431 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
W20260709 12:25:46.321683 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 28 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:46.333264 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 281883us
I20260709 12:25:46.331718 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.391232 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:46.310009 30186 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.901s	user 0.004s	sys 0.008s
I20260709 12:25:46.358762 29903 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:25:46.275709 30298 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:46.108476 30866 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.236173 30719 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:25:46.127328 30515 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.388988 20881 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:46.312116 30898 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.335119 30935 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:46.410307 29903 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:25:46.398293 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 451525us
I20260709 12:25:46.402626 20726 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:46.276685 30643 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.311697 30194 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:46.344069 30705 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:46.408079 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.403676 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.417002 20801 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:25:46.404109 30186 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:46.386189 30859 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:46.411723 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.418388 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.418205 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.349172 30924 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:25:46.411156 30840 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.882s	user 0.012s	sys 0.000s
W20260709 12:25:46.420397 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 192413us
I20260709 12:25:46.300830 30916 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.437408 20881 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:46.423264 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:46.408649 30706 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.410596 30952 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:25:46.415835 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94411.7us
I20260709 12:25:46.444710 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.445500 29903 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:46.527227 29903 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:46.394344 30465 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.446379 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:46.531136 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 107998us
I20260709 12:25:46.445853 30859 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:46.326443 30789 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.093s
I20260709 12:25:46.452102 20879 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:46.446738 30935 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:46.442481 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:46.412556 20727 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:46.443172 20733 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
W20260709 12:25:46.553959 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107680us
I20260709 12:25:46.345245 30795 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.344620 30895 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.345908 30798 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.437335 30758 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.108s
I20260709 12:25:46.540570 30918 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.526667 20801 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:46.311818 30857 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:46.440627 30751 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:46.438531 30924 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:46.472656 30933 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:25:46.444903 30720 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.452337 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:46.472291 30763 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.019s	user 0.007s	sys 0.006s
I20260709 12:25:46.451933 30840 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:46.551453 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:46.552273 30919 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:46.554999 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:46.529384 30766 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.106s	user 0.016s	sys 0.000s
I20260709 12:25:46.554898 30186 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:46.560161 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 169052us
I20260709 12:25:46.334483 30675 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.449990 30194 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.100s
I20260709 12:25:46.548285 29903 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.513496 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.627528 20883 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:46.515556 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.559494 30789 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:46.453760 30298 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.631789 30675 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:46.598384 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.635682 30928 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
I20260709 12:25:46.636353 30928 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.471869 30898 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:46.621838 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66952us
W20260709 12:25:46.622529 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 338185us
I20260709 12:25:46.639840 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:46.640712 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:46.641549 30274 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.003s
I20260709 12:25:46.662678 20728 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:25:46.546649 30465 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.591961 30952 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.604329 30643 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.626861 30446 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:46.655947 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:46.562647 30795 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:46.616772 30957 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
I20260709 12:25:46.605132 30914 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.592553 30515 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:46.671880 30764 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.594060 30706 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.621371 30857 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:46.621217 30919 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:46.562871 30902 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.693647 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:46.676373 30928 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:46.600279 30917 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.689982 30840 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:46.697396 29903 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:46.689113 20809 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:25:46.720022 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.568739 30895 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:46.727689 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.662586 30918 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:46.683772 30705 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:46.658535 20884 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:46.689141 30724 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:46.730845 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 112944us
I20260709 12:25:46.626632 30186 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.699805 30194 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:46.730845 30919 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:46.730648 30857 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:46.680116 30751 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.736356 30766 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:46.749743 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:46.751065 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:46.755981 30967 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:46.775892 30967 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.776355 30967 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:46.736346 30902 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.760092 30656 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.749007 30724 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.779562 30902 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.605672 30935 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.781347 30724 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:46.756441 30960 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:46.775910 30766 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:46.782226 30960 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:46.783473 30766 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.765583 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.787649 20802 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:46.788779 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:46.789458 20802 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:46.790369 30956 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.276s	user 0.002s	sys 0.002s
I20260709 12:25:46.732811 30952 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.792377 30937 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:46.792821 30937 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:46.763217 30719 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:46.794116 30719 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.750738 30970 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:46.796437 30970 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.798048 30465 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.798559 30970 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.800297 30795 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:46.801501 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53365.4us
I20260709 12:25:46.802466 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:46.806946 30845 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.801532 30763 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:46.740775 30764 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.766093 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.818081 20883 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:46.767665 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.768646 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:46.819455 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:46.772502 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2375) took 1035 ms. Trace:
I20260709 12:25:46.820237 21428 rpcz_store.cc:276] 0709 12:25:45.736900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.836085 (+ 99185us) service_pool.cc:224] Handling call
0709 12:25:46.772312 (+936227us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:46.772433 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":794296064}
I20260709 12:25:46.774062 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.821424 20801 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:25:46.786149 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.766176 30895 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.812884 21433 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:46.820775 20802 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:46.822224 20808 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:46.841006 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:46.843427 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:46.750288 30918 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:46.845696 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.848585 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3783) took 1005 ms. Trace:
I20260709 12:25:46.851161 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:46.851051 20735 rpcz_store.cc:276] 0709 12:25:45.843142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.846691 (+  3549us) service_pool.cc:224] Handling call
0709 12:25:45.847023 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.847508 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.847542 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.848063 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.200821 (+352758us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:46.200868 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.200941 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.789704 (+588763us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:46.839695 (+ 49991us) raft_consensus.cc:1685] finished
0709 12:25:46.839716 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.848522 (+  8806us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96678,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":39529,"spinlock_wait_cycles":32624512,"thread_start_us":352317,"threads_started":1,"wal-append.queue_time_us":539778}
I20260709 12:25:46.847285 30705 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.856734 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:46.854305 30705 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.869042 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:46.869218 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:46.845103 30298 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.869218 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.863692 30789 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:46.873935 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.845300 30724 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:46.747658 30840 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.865278 30446 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
I20260709 12:25:46.844769 30902 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:46.849247 30845 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.880162 20801 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:46.895573 30952 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.898484 30859 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:46.899298 30929 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.050s
I20260709 12:25:46.864701 30950 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:46.903434 30945 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.830s	user 0.002s	sys 0.000s
I20260709 12:25:46.907872 30982 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:46.910646 20884 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:25:46.911206 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.899348 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3510) took 1898 ms. Trace:
I20260709 12:25:46.932607 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.911970 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:46.884608 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:46.913767 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.935302 20727 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:46.936208 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.936894 20727 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:46.937493 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3827) took 1290 ms. Trace:
I20260709 12:25:46.937817 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.938628 20727 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:46.924189 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:46.906904 30840 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.850597 30928 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:25:46.940271 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.940936 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:46.942091 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:46.933804 20729 rpcz_store.cc:276] 0709 12:25:45.000808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.000944 (+   136us) service_pool.cc:224] Handling call
0709 12:25:45.157392 (+156448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.948611 (+791219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.899068 (+950457us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54002e4fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:46.899283 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400208640}
I20260709 12:25:46.944037 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.944784 20729 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:46.945868 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.946558 20729 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:46.955302 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:46.955571 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:46.934223 20732 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
W20260709 12:25:46.958314 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73807us
I20260709 12:25:46.959029 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:46.959801 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90671.5us
I20260709 12:25:46.960963 20875 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:46.962250 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:46.963075 20735 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:25:46.887630 30298 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.969720 30859 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.970134 30298 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.932864 30950 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:46.971343 30958 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:25:46.972761 30950 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:46.973471 30353 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:46.958348 30789 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:46.974855 20806 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:46.983857 30751 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:46.922722 30912 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:46.986835 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3753) took 1373 ms. Trace:
W20260709 12:25:47.005246 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46228.7us
I20260709 12:25:46.892439 30845 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.927995 30975 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:47.006460 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:46.910081 30724 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:46.961992 30982 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:47.002537 20876 rpcz_store.cc:276] 0709 12:25:45.613173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.757145 (+143972us) service_pool.cc:224] Handling call
0709 12:25:45.757507 (+   362us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:45.840135 (+ 82628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.840174 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:45.840907 (+   733us) log.cc:844] Serialized 78 byte log entry
0709 12:25:46.246058 (+405151us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:46.246121 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.246228 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.986587 (+740359us) raft_consensus.cc:1685] finished
0709 12:25:46.986607 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.986775 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15353,"raft.queue_time_us":91622,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":73806,"thread_start_us":393760,"threads_started":1,"wal-append.queue_time_us":803191}
I20260709 12:25:46.937849 20807 rpcz_store.cc:276] 0709 12:25:45.647124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.647288 (+   164us) service_pool.cc:224] Handling call
0709 12:25:45.647659 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.649486 (+  1827us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:45.754755 (+105269us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.936195 (+1181440us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.937183 (+   988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.937233 (+    50us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:46.937283 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.937343 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.937442 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1815423,"prepare.queue_time_us":877615,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":81,"raft.queue_time_us":150975,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":21806,"spinlock_wait_cycles":6794496,"thread_start_us":389353,"threads_started":3,"wal-append.queue_time_us":627537}
I20260709 12:25:47.049585 20804 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:46.999830 30950 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.012589 30353 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.035s
W20260709 12:25:47.052461 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 86031.6us
I20260709 12:25:47.010560 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.056185 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.063771 30353 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:47.064198 30353 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:47.065203 30353 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.22, Last appended by leader: 22, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.069494 30840 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:47.070747 30840 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:47.071009 30705 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.071117 30840 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:47.072050 30764 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.072242 30840 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.072625 30919 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.073668 30840 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:47.073781 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.074832 30919 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.077126 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.078266 30963 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:46.920778 30960 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.081552 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.081372 30857 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.083853 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.055698 30446 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:47.013377 30912 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.098237 30960 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:47.098898 30912 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.088201 30186 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:47.052994 30975 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.088904 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:47.001379 30918 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.096586 20804 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:25:47.100778 20801 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
W20260709 12:25:47.126616 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.138599 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.089342 30705 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:47.127988 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.128509 20807 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:47.142637 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:47.091112 30446 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:47.129217 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.143172 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3844) took 1022 ms. Trace:
I20260709 12:25:47.091085 30845 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.131081 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3659) took 1761 ms. Trace:
I20260709 12:25:47.131518 30975 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.146114 20879 rpcz_store.cc:276] 0709 12:25:45.369959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.412554 (+ 42595us) service_pool.cc:224] Handling call
0709 12:25:45.412844 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.544192 (+1131348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.130824 (+586632us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400305f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:47.131013 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389569792}
I20260709 12:25:47.146615 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.131930 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3740) took 2257 ms. Trace:
I20260709 12:25:47.132011 30994 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:47.141830 30705 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.136016 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.136091 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.100889 30963 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:47.125418 30912 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:25:47.137217 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3847) took 1015 ms. Trace:
I20260709 12:25:47.143944 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.143150 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:47.145066 20807 rpcz_store.cc:276] 0709 12:25:46.120951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.055454 (+934503us) service_pool.cc:224] Handling call
0709 12:25:47.089587 (+ 34133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.142909 (+ 53322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.143107 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.147492 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:47.165103 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:47.148770 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.160742 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:47.135238 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.143872 30936 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.161557 20884 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:47.168578 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:47.162990 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.147163 20882 rpcz_store.cc:276] 0709 12:25:44.874940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.875086 (+   146us) service_pool.cc:224] Handling call
0709 12:25:44.875439 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.157434 (+281995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.157467 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.219344 (+ 61877us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.494284 (+274940us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:45.494343 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.131546 (+1637203us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b540019bed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:47.131590 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.131634 (+    44us) raft_consensus.cc:1685] finished
0709 12:25:47.131647 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.131851 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27879,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":166376320,"thread_start_us":425386,"threads_started":2,"wal-append.queue_time_us":382864,"wal-append.run_cpu_time_us":10907,"wal-append.run_wall_time_us":1503124}
I20260709 12:25:47.172691 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:47.160259 30919 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:47.145850 30789 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.165474 20804 rpcz_store.cc:276] 0709 12:25:46.121923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.085608 (+963685us) service_pool.cc:224] Handling call
0709 12:25:47.096127 (+ 10519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.136968 (+ 40841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.137164 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.144690 30298 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:47.164305 30963 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.061877 30950 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.173194 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.170557 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.151360 30983 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.176973 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.174595 30975 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:47.178259 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.176652 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:47.162990 30705 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:47.161446 30994 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.136392 30758 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:25:47.198704 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:47.185684 30697 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:25:47.190722 30963 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:47.175925 30895 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:47.173659 30980 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:47.180245 30763 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:47.187806 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.174242 30706 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:47.180728 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.191177 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.192931 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.147881 30186 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.194591 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.185108 30919 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.200085 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.203361 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.216899 30895 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:47.209802 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.188396 30789 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.200562 30970 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:47.242200 21432 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:47.193681 30766 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:47.195628 30845 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:47.152552 30446 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.176465 30936 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.243106 21426 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.199553 30994 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:47.211002 30980 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.217166 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.237061 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:47.244398 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49881us
I20260709 12:25:47.217676 29903 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:25:47.237294 30995 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 12:25:47.246903 30990 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:25:47.247380 30988 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:47.247865 30789 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:47.248433 20877 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:47.236944 30895 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.167786 30982 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.197252 30967 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.235380 30758 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:47.275761 30758 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.276211 30758 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:47.277619 20877 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:47.217238 30795 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:47.279402 30795 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:47.280570 30795 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.282413 30795 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:25:47.236723 30991 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:25:47.287335 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3829) took 1638 ms. Trace:
I20260709 12:25:47.288465 30919 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.217391 30975 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:47.217689 20882 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:47.289885 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.289703 30975 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.290443 20882 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:47.257903 29903 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:47.257943 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.295464 29903 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:47.261596 30857 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.296350 30857 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:47.197244 30983 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.296757 30857 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.297124 30857 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:47.299275 30719 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.258452 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:47.256064 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 52971.4us
I20260709 12:25:47.237042 30763 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:47.252758 30766 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.252548 30840 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.242291 30186 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.244206 30912 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:47.251129 30706 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:47.264012 30845 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:47.320453 30845 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.320750 30186 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:47.320884 30845 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:47.265419 30970 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:47.323716 20883 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:25:47.325274 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.287878 20803 rpcz_store.cc:276] 0709 12:25:45.648483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.648679 (+   196us) service_pool.cc:224] Handling call
0709 12:25:45.691584 (+ 42905us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:45.715030 (+ 23446us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:46.810229 (+1095199us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b180029b120. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:25:46.810502 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.963441 (+152939us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:47.011743 (+ 48302us) log.cc:844] Serialized 52 byte log entry
0709 12:25:47.011887 (+   144us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:47.046049 (+ 34162us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.047204 (+  1155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.047294 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.286997 (+239703us) raft_consensus.cc:1685] finished
0709 12:25:47.287030 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.287244 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15834,"prepare.queue_time_us":1148587,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":2028,"raft.queue_time_us":19007,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":52345,"spinlock_wait_cycles":2513152,"thread_start_us":152412,"threads_started":1,"wal-append.queue_time_us":239327}
W20260709 12:25:47.314826 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56450.5us
I20260709 12:25:47.329128 29903 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.304289 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1637) took 1536 ms. Trace:
I20260709 12:25:47.259285 30980 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:47.337563 29903 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.318130 30763 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:47.265586 30988 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.331650 20734 rpcz_store.cc:276] 0709 12:25:45.763180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.773641 (+ 10461us) service_pool.cc:224] Handling call
0709 12:25:45.986010 (+212369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.151689 (+165679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.151722 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.238635 (+ 86913us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.562169 (+323534us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:46.562230 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.298760 (+736530us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54002990d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:47.298820 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.298878 (+    58us) raft_consensus.cc:1685] finished
0709 12:25:47.298895 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.299122 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61811,"raft.queue_time_us":61945,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":53290,"spinlock_wait_cycles":237640576,"thread_start_us":256043,"threads_started":1,"wal-append.queue_time_us":446684}
I20260709 12:25:47.336064 20882 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:47.319067 30789 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:47.301748 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:47.316980 30706 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:47.351775 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50113.9us
I20260709 12:25:47.352697 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.316243 30840 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.357494 30840 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.357584 30950 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:47.354480 20882 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:47.353489 30706 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:47.355446 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.372717 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.350333 30988 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:47.378453 30994 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.379702 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.379606 30970 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.335364 30912 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.352954 30789 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:47.381718 30954 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.840s	user 0.000s	sys 0.001s
I20260709 12:25:47.337884 30766 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.383471 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.373242 30936 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.389014 20734 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:47.378525 30705 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:25:47.390592 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 177824us
I20260709 12:25:47.393823 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.399575 20808 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:47.400767 20735 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:47.403121 20884 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:47.394446 30789 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:47.450228 30857 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.456382 30994 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:47.407683 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:47.457816 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50302.1us
I20260709 12:25:47.418121 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.419445 30705 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.359028 30963 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.433380 30912 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.465013 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:47.468412 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:47.471330 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.474054 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:47.460824 30795 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:47.478068 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.476267 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.479877 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:47.483461 20808 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:47.478018 31001 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:47.488955 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.491609 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:47.492225 20734 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:47.478404 30795 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.486554 30845 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.499899 20731 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:25:47.498940 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.512048 20808 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:47.497234 30795 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:47.513648 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.522365 30845 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.527419 30763 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:47.528110 30763 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:47.529091 30763 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.531050 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.532563 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.487725 30988 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.533229 30446 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:47.544404 20878 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1798) took 1478 ms (client timeout 1891 ms). Trace:
W20260709 12:25:47.533982 31005 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:25:47.537933 30963 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.538126 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:47.551472 30916 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.551859 20728 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:47.553308 30979 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:47.541491 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:47.544847 20878 rpcz_store.cc:269] 0709 12:25:46.065711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.068925 (+  3214us) service_pool.cc:224] Handling call
0709 12:25:47.544209 (+1475284us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b5400171e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:47.544344 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504894720}
I20260709 12:25:47.540376 30845 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:47.549129 30186 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:47.558024 20808 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.553467 30520 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:47.562372 30520 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:47.563356 30520 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.564452 30520 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:47.498864 30706 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.539445 30857 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.578794 20882 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:47.584203 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:47.587801 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1857) took 2098 ms. Trace:
I20260709 12:25:47.576136 30446 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.588284 20805 rpcz_store.cc:276] 0709 12:25:45.489181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.489352 (+   171us) service_pool.cc:224] Handling call
0709 12:25:45.512547 (+ 23195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.668867 (+156320us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:46.347734 (+678867us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.879481 (+531747us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.910272 (+ 30791us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.910334 (+    62us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:46.910397 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.587507 (+677110us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54003cad54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:47.587561 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.587704 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":288915,"prepare.queue_time_us":1375268,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":158312,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":59380,"spinlock_wait_cycles":358909184,"thread_start_us":933216,"threads_started":3,"wal-append.queue_time_us":344767}
I20260709 12:25:47.590396 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.590821 30446 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.591449 30646 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:47.594766 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.595305 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.579047 30186 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.599974 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3517) took 1284 ms. Trace:
I20260709 12:25:47.553766 30979 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:47.600281 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.590984 20805 raft_consensus.cc:2376] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27. Re-sending same reply.
I20260709 12:25:47.604648 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.604772 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.605170 20882 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:47.609738 20805 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 36 because replica has last-logged OpId of term: 35 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 34 index: 17.
I20260709 12:25:47.610930 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.611624 20878 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:47.611696 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.612720 20808 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29.
I20260709 12:25:47.612968 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.614975 30988 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.616724 30706 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.576318 30980 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.619135 30980 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.599004 30758 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.560938 30789 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.621476 30979 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.622120 20882 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:47.622560 30979 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:47.623232 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.623844 20882 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:47.632555 30789 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:47.637061 20728 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:47.638517 30967 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:47.590487 30937 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.640009 30937 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:47.634922 30646 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:47.644469 20884 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:47.644902 20882 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:47.600431 20726 rpcz_store.cc:276] 0709 12:25:46.315453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.421511 (+106058us) service_pool.cc:224] Handling call
0709 12:25:46.421961 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.743454 (+321493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.743507 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.765599 (+ 22092us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.880073 (+114474us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:46.880126 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.596987 (+716861us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54003b4a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:47.597035 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.597080 (+    45us) raft_consensus.cc:1685] finished
0709 12:25:47.597094 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.599911 (+  2817us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35676,"raft.queue_time_us":17875,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":1207,"spinlock_wait_cycles":727852288,"thread_start_us":114080,"threads_started":1,"wal-append.queue_time_us":198336}
I20260709 12:25:47.658143 20882 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.14-29.14]
I20260709 12:25:47.660360 20882 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:25:47.660996 20734 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:47.662083 20882 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.16-29.16]
I20260709 12:25:47.664116 20882 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:47.579126 30845 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.672856 20700 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29.
I20260709 12:25:47.673424 20700 leader_election.cc:403] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:25:47.675689 20884 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:25:47.678992 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.682850 20808 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.683771 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.685578 30845 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:47.686457 20808 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 32 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 20.
I20260709 12:25:47.687733 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.688978 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.689785 20808 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.690663 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.692068 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.696552 30724 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.699204 20808 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 30 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
W20260709 12:25:47.703801 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.705008 30186 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.706936 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.717065 30983 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:47.718709 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51343us
I20260709 12:25:47.718936 30976 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29.
I20260709 12:25:47.719611 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:47.719966 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.720659 30859 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:47.722378 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.723565 20803 raft_consensus.cc:2368] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26.
W20260709 12:25:47.732161 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 28 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:47.733840 20775 leader_election.cc:400] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:25:47.734268 20775 leader_election.cc:403] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:47.743311 20882 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:25:47.744168 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:47.748800 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.749409 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.753819 21432 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:47.754429 30914 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:47.754865 30914 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:47.755897 30914 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.757315 30914 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:47.759895 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2370) took 2083 ms (client timeout 2086 ms). Trace:
W20260709 12:25:47.760376 21433 rpcz_store.cc:269] 0709 12:25:45.676379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.676504 (+   125us) service_pool.cc:224] Handling call
0709 12:25:47.759697 (+2083193us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54007602d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:47.759829 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087817472}
I20260709 12:25:47.762360 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3846) took 1640 ms. Trace:
I20260709 12:25:47.762778 20806 rpcz_store.cc:276] 0709 12:25:46.121602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.941772 (+820170us) service_pool.cc:224] Handling call
0709 12:25:47.762114 (+820342us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b540019aad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:47.762268 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73511296}
I20260709 12:25:47.764179 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:47.765013 20728 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.766523 30924 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:47.767921 30924 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:47.768601 30904 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:47.771625 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.774145 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:47.776074 20775 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.777881 20733 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:47.778035 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.778756 20882 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
I20260709 12:25:47.780457 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:47.782267 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.782980 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.784802 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:47.784868 21400 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:47.812956 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:47.816965 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.794999 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.820565 20805 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.819636 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.825877 20803 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.827910 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.824602 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:47.842607 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.843964 29903 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:47.846247 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.846902 20878 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:47.846973 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.849515 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:47.834697 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.830883 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.752071 30900 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25.
I20260709 12:25:47.849129 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:47.853034 21426 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:47.877249 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:47.847978 30840 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:47.880611 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.854776 21427 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.785965 30904 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:47.882530 30904 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.882926 30904 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:47.818867 30912 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:47.883306 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.850977 30960 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:47.850781 30705 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:47.885452 21430 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:47.876753 21433 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:25:47.882948 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.887362 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:47.884096 21429 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:47.814457 30979 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:47.886857 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.784482 30857 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:47.908051 30857 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:47.912177 30900 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:47.913439 30900 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:47.914103 30900 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:47.926474 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77424.3us
I20260709 12:25:47.929991 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:47.930843 20882 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:47.921527 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3519) took 1605 ms. Trace:
I20260709 12:25:47.860100 20805 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:47.881899 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:47.856809 20733 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:47.851120 30972 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:47.876842 30921 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.892024 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.896116 21431 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:47.944252 21429 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 23.
W20260709 12:25:47.896564 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.944788 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:47.945300 20802 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:47.945398 21433 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:47.830117 30937 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:47.946122 20732 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:47.932420 20731 rpcz_store.cc:276] 0709 12:25:46.316065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.556810 (+240745us) service_pool.cc:224] Handling call
0709 12:25:46.557176 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.500310 (+943134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.921165 (+420855us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54003b4f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:47.921439 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164366720}
W20260709 12:25:47.937971 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95492us
I20260709 12:25:47.927290 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:47.942288 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3852) took 1285 ms. Trace:
I20260709 12:25:47.887411 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.943022 20879 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.21-28.21]
I20260709 12:25:47.904348 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.948537 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.937191 30950 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:47.947927 20801 rpcz_store.cc:276] 0709 12:25:46.656885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.958774 (+301889us) service_pool.cc:224] Handling call
0709 12:25:47.942035 (+983261us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540027c254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:47.942206 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324056960}
I20260709 12:25:47.952203 20801 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 21. (term mismatch)
I20260709 12:25:47.952533 20801 pending_rounds.cc:85] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 20
I20260709 12:25:47.952752 20801 pending_rounds.cc:107] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 32.21
I20260709 12:25:47.952981 20801 raft_consensus.cc:2889] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:47.955364 20803 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:25:47.968008 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:47.969090 20729 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:25:47.977695 20881 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:47.945560 30997 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:47.999644 20881 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: [22.18-24.19]
I20260709 12:25:48.004905 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:48.008037 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59627.5us
I20260709 12:25:48.009431 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.968307 30898 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.024935 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:48.025213 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.947402 30960 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:48.042603 30912 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:48.041633 20801 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:47.854769 30643 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.134s
I20260709 12:25:47.961571 30859 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:47.953179 30705 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:48.057054 21433 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:48.032318 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:47.943774 30915 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.081s
I20260709 12:25:48.063175 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.027549 30997 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.064126 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:47.954480 30921 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.003633 30937 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:48.026273 30663 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:48.070125 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:48.070817 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:47.954082 30950 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.073746 20879 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:48.065414 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:48.087342 20884 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:25:48.089089 21433 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:47.955776 30900 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.065089 31020 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:48.064462 31010 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
W20260709 12:25:48.077804 30795 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.538s	user 0.013s	sys 0.001s
I20260709 12:25:48.102841 20881 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:48.104055 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:48.070974 30997 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:48.070297 30898 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:48.105710 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:48.109851 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.062371 30983 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:25:48.118144 20878 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:48.128316 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.089793 30912 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:25:48.115103 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3837) took 2102 ms. Trace:
I20260709 12:25:48.097108 31004 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 12:25:48.104621 30723 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:48.130779 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.153684 31025 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:48.154187 21431 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:48.154111 30723 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.155211 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.156395 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.157152 21431 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:48.157976 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.158782 21431 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 12:25:48.131536 30743 tablet_copy_client.cc:538] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:48.115950 30915 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:48.116287 30994 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.132530 31016 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:25:48.133405 30705 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.174916 30994 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.177160 30994 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.134517 30937 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.126748 30960 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 27.22, Last appended by leader: 22, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.119470 30898 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:48.140064 31017 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:48.114287 20882 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:25:48.143638 20809 rpcz_store.cc:276] 0709 12:25:46.012770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.029456 (+ 16686us) service_pool.cc:224] Handling call
0709 12:25:46.060230 (+ 30774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.737887 (+677657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.113094 (+1375207us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540027b854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:48.115026 (+  1932us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445418368}
I20260709 12:25:48.115113 30795 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.104420 30967 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.013s
I20260709 12:25:48.102241 30663 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.192165 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:48.152880 30950 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:48.152917 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.201398 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.162150 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.131886 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:48.203891 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.164335 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:48.167150 30983 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:48.167443 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:48.206058 21431 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:48.171558 30915 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:48.173581 30921 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:48.059689 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 207601us
I20260709 12:25:48.157876 30723 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:48.207655 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 150333us
W20260709 12:25:48.214067 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49790us
W20260709 12:25:48.195007 30895 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.006s	sys 0.017s
I20260709 12:25:48.188732 30994 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.218873 21426 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 21. (term mismatch)
I20260709 12:25:48.224974 21426 pending_rounds.cc:85] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 20
I20260709 12:25:48.225314 21426 pending_rounds.cc:107] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.21
I20260709 12:25:48.225625 21426 raft_consensus.cc:2889] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:48.166728 30904 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.218334 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:48.207178 30724 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.231279 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3800) took 1089 ms. Trace:
I20260709 12:25:48.222100 30859 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:48.222991 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 395144us
I20260709 12:25:48.234694 21433 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:48.180717 30997 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.213665 30967 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:48.236351 20882 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:48.217371 31010 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:48.220692 30964 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:48.253904 30964 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:48.254341 30964 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:48.231812 30994 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.259642 30994 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.237929 31021 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.262593 31021 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.240178 31020 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.263635 31020 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.264926 31021 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:48.266037 31015 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.502s	user 0.004s	sys 0.011s
I20260709 12:25:48.234222 30894 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.032s
I20260709 12:25:48.237905 20881 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:48.220456 30895 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.006s	sys 0.017s
I20260709 12:25:48.227950 29903 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:48.231225 30724 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.242959 20806 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:48.231702 20727 rpcz_store.cc:276] 0709 12:25:47.141883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.299934 (+158051us) service_pool.cc:224] Handling call
0709 12:25:47.300287 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.314225 (+ 13938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.314276 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.316777 (+  2501us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.694468 (+377691us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b18003ec5c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:47.829281 (+134813us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:47.829324 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.829397 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.227543 (+398146us) raft_consensus.cc:1685] finished
0709 12:25:48.227561 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.231209 (+  3648us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":852,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":13057,"spinlock_wait_cycles":23352192,"thread_start_us":134585,"threads_started":1,"wal-append.queue_time_us":622118}
I20260709 12:25:48.243673 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.244207 30859 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 28.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.237780 30921 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:48.247085 31002 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.007s
I20260709 12:25:48.247874 30898 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:48.252990 30972 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:48.262876 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.227979 30982 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.272392 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 128890us
I20260709 12:25:48.294440 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.295061 20805 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:48.236243 20776 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:48.275147 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:48.253023 30663 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.280697 29903 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.281471 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.281808 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.283679 30724 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:48.283890 30795 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.315084 20879 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.14-29.14]
I20260709 12:25:48.282158 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:48.317847 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:48.288239 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:48.293618 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50163.2us
I20260709 12:25:48.300621 31025 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:48.302831 20878 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:48.304059 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:48.308480 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.305408 21426 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:48.291011 30929 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.030s
I20260709 12:25:48.265693 31020 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.324463 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:48.325249 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 89065.5us
I20260709 12:25:48.367624 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:48.326067 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:48.243670 30904 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.334079 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.335896 20808 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: [22.18-24.19]
I20260709 12:25:48.285455 30921 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:48.293165 30972 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:48.316396 30186 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:48.331913 30697 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:48.341053 30975 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:48.333397 30663 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:48.372864 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.339078 31031 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.376787 20801 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:48.341166 20803 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
W20260709 12:25:48.390074 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 65679.5us
I20260709 12:25:48.361874 31027 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.315882 31021 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:48.332114 30916 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.039s
I20260709 12:25:48.335273 30983 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:25:48.315517 30982 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.337594 30724 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
W20260709 12:25:48.394011 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98417.3us
I20260709 12:25:48.385401 30921 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.353085 30705 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:25:48.447685 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.389319 30904 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:48.393666 31036 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:48.447978 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.391353 31035 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.393988 31010 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:48.397900 30975 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.397451 30789 raft_consensus.cc:2764] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:25:48.396953 30663 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:48.503683 20805 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:48.391721 31020 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:25:48.436631 30900 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:48.479079 20804 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:48.472878 30929 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:25:48.396684 30697 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.503381 20882 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:48.508430 30904 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:48.509450 31040 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:25:48.511806 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64231.4us
I20260709 12:25:48.512210 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.512120 30921 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:48.512848 31024 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:25:48.513264 31033 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:48.533485 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:48.468941 30914 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.711s	user 0.004s	sys 0.014s
I20260709 12:25:48.518998 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.456180 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:48.508488 31038 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:48.526079 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:48.526818 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 75121.2us
I20260709 12:25:48.489423 31002 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:48.439532 31031 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.489181 30795 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:48.534178 29903 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:48.513377 31041 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:48.529263 30997 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.514304 31022 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.074s
I20260709 12:25:48.466877 31021 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.521377 30975 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:48.523434 30663 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:48.542235 20807 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:48.561479 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 227474us
I20260709 12:25:48.467161 31027 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.535857 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:48.551388 29903 consensus_queue.cc:1243] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 31.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:48.541600 30705 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:48.556298 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3607) took 1200 ms. Trace:
I20260709 12:25:48.573107 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.574486 29903 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:25:48.549901 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 61142.2us
I20260709 12:25:48.548404 31041 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:48.571273 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 245269us
I20260709 12:25:48.516486 31035 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.575084 29903 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:48.550599 31044 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.576977 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:48.573230 20802 rpcz_store.cc:276] 0709 12:25:47.355960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.947550 (+591590us) service_pool.cc:224] Handling call
0709 12:25:47.977692 (+ 30142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.977911 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:48.181135 (+203224us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.555923 (+374788us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:48.555983 (+    60us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:48.556033 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.556104 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.556233 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":213833,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":53,"raft.queue_time_us":6326,"thread_start_us":530898,"threads_started":2}
I20260709 12:25:48.584441 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.585494 20802 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:48.578945 20808 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:48.586553 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.587606 20802 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:48.586638 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.589290 20808 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:25:48.552721 31029 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.581334 30840 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.002s
I20260709 12:25:48.572582 30859 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:48.578336 30705 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:48.605010 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.600350 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:48.560405 30997 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:25:48.547246 31002 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.632288 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.626619 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.582674 31041 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:48.649184 31021 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.571908 30904 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.593070 31036 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.062s
I20260709 12:25:48.628124 20878 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:48.626288 30840 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:48.645252 30697 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.647477 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.661607 20802 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
W20260709 12:25:48.627269 31037 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:25:48.662552 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.663218 20802 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:48.664083 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.664774 31002 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:48.628952 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52030.6us
I20260709 12:25:48.666419 31027 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:48.667408 31042 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:25:48.668015 30921 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:48.669128 31035 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:48.669741 31046 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:48.664749 20802 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:48.575477 30795 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.594972 29903 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 29, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.595440 30186 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.049s
I20260709 12:25:48.679702 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:48.677517 20878 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:48.682920 20808 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:48.673453 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.690776 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:48.628615 30705 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:48.703493 20801 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:48.705605 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.708144 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.709177 20802 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 31 because replica has last-logged OpId of term: 30 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:25:48.710266 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:48.711174 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.711663 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.712416 20802 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:25:48.713892 20802 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:48.600298 31044 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.717759 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.718556 20801 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:48.720041 20801 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
W20260709 12:25:48.723428 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.724467 20801 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:25:48.728982 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.729626 20801 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
W20260709 12:25:48.733536 31043 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.165s	user 0.000s	sys 0.002s
I20260709 12:25:48.599625 31029 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.737596 31029 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.741879 30743 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:25:48.630411 30859 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.746462 30663 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.748122 30859 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.751705 31041 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.752420 31044 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.758546 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.778569 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.764310 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:48.764557 30967 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.766162 30964 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.000s
I20260709 12:25:48.756078 31035 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.782013 31024 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:25:48.793469 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 319043us
I20260709 12:25:48.794854 21435 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:48.795331 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:48.797029 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:48.776297 30982 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.797802 30937 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:48.799073 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3761) took 1646 ms. Trace:
I20260709 12:25:48.802639 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.803622 20883 rpcz_store.cc:276] 0709 12:25:47.152769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.155313 (+  2544us) service_pool.cc:224] Handling call
0709 12:25:47.167735 (+ 12422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.324120 (+156385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.798803 (+1474683us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54003f6e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:48.798983 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356136832}
I20260709 12:25:48.805392 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:48.810352 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.815306 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:48.786494 30967 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.807097 31057 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:48.806218 30840 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:48.788653 30964 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.813082 31033 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:48.789862 31035 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:48.820158 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 240744us
I20260709 12:25:48.811297 30937 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:48.821673 30859 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:48.819511 31031 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.841969 31048 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:25:48.832463 31033 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.829833 31057 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:48.855926 31057 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.863457 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1868) took 1670 ms. Trace:
I20260709 12:25:48.863940 21428 rpcz_store.cc:276] 0709 12:25:47.192507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.192640 (+   133us) service_pool.cc:224] Handling call
0709 12:25:47.200779 (+  8139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.323312 (+122533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.323353 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.323407 (+    54us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:47.323454 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.863232 (+1539778us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400525154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:48.863275 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.863394 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280319744}
I20260709 12:25:48.866806 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3678) took 1799 ms. Trace:
I20260709 12:25:48.902268 20876 rpcz_store.cc:276] 0709 12:25:47.067560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.073510 (+  5950us) service_pool.cc:224] Handling call
0709 12:25:48.862922 (+1789412us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540022fbd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:48.866729 (+  3807us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009335040}
I20260709 12:25:48.829785 30972 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.906528 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.850934 31033 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:48.848796 30859 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:48.908165 30795 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.922777 31041 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.902684 31020 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.904927 30960 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.937582 30972 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.916954 31036 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:48.940368 30924 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:48.919855 31060 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.932699 30663 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.940485 30967 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:48.966195 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:48.967367 20801 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:48.968019 30859 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.968690 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:48.966389 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:48.949059 30924 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:48.973493 20876 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:48.934748 31020 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:48.980191 21426 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:48.980540 21434 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:48.941370 30960 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:48.982483 31058 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:25:48.951679 31036 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.997761 30353 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:48.998052 30705 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:48.998857 30960 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:48.999475 31035 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:48.980253 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:48.991360 21432 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:25:48.992730 21434 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:48.993455 30964 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:48.956535 31060 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:48.977553 31055 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:25:49.009068 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:48.984109 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.012010 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:49.014210 30975 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.003608 21429 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:49.004601 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:48.935423 30937 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.002266 21434 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 27.22, Last appended by leader: 22, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.019862 21429 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:49.021205 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:49.047983 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:49.093532 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.136693 20735 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
W20260709 12:25:49.136698 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 132146us
I20260709 12:25:49.141707 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:49.136785 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80280.2us
I20260709 12:25:49.202164 21434 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:48.998884 31036 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:49.206157 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64595.6us
I20260709 12:25:49.209118 20732 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:25:49.207991 21429 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.218113 21429 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:49.220409 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 386979us
I20260709 12:25:49.203428 30789 raft_consensus.cc:2764] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
I20260709 12:25:49.221709 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.207580 30983 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:49.039434 30960 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.180s
I20260709 12:25:49.020444 31021 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.200s
W20260709 12:25:49.233282 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.238612 21434 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 22. Preceding OpId from leader: term: 28 index: 23. (index mismatch)
I20260709 12:25:49.240981 21429 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 18. (term mismatch)
I20260709 12:25:49.241520 21429 pending_rounds.cc:85] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 17
I20260709 12:25:49.241814 21429 pending_rounds.cc:107] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:25:49.242180 21429 raft_consensus.cc:2889] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.223521 31036 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.019s
I20260709 12:25:49.247875 31072 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.240s
I20260709 12:25:49.253906 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:49.255120 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 13 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:49.255810 31073 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
I20260709 12:25:49.256299 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:49.257082 20809 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 30 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:25:49.257198 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3835) took 3482 ms. Trace:
I20260709 12:25:49.257853 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:49.261447 31070 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:49.263091 30988 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:49.263515 30988 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:49.263360 31031 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:49.262850 29903 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.002s
I20260709 12:25:49.264374 30988 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.266893 30988 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:49.272403 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.272854 20809 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:49.277225 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:49.257714 20810 rpcz_store.cc:276] 0709 12:25:45.775136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.789982 (+ 14846us) service_pool.cc:224] Handling call
0709 12:25:45.790477 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.032861 (+242384us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:46.878497 (+845636us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.014189 (+135692us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.015141 (+   952us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.015238 (+    97us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:47.015309 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.123033 (+107724us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b1800479620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:48.180036 (+1057003us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b1800479620. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:25:49.256879 (+1076843us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54003c7154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:49.256945 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.257115 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":822927,"prepare.queue_time_us":842119,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":5828,"raft.queue_time_us":914,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":4672,"spinlock_wait_cycles":627391488,"thread_start_us":129060,"threads_started":1,"wal-append.queue_time_us":179946,"wal-append.run_cpu_time_us":1394,"wal-append.run_wall_time_us":1001417}
I20260709 12:25:49.223817 30964 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.018s
W20260709 12:25:49.278731 31019 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.201s	user 0.000s	sys 0.003s
I20260709 12:25:49.222461 30924 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.327795 20810 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:49.328332 20810 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:49.355459 31020 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.356132 30975 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:49.357250 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.357926 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.358516 30924 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:49.361846 31071 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 12:25:49.362653 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.363329 30520 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.363919 30520 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:49.365572 31075 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:49.344628 20810 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.368583 20810 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.369302 30758 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.374406 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1870) took 1067 ms. Trace:
I20260709 12:25:49.375012 20807 rpcz_store.cc:276] 0709 12:25:48.307340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.322188 (+ 14848us) service_pool.cc:224] Handling call
0709 12:25:48.340173 (+ 17985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.571648 (+231475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.374030 (+802382us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:49.374315 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464046848}
I20260709 12:25:49.380596 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:49.384518 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:49.391129 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.395788 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:49.400017 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:49.404098 20808 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:49.418339 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.439579 20807 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:49.441953 20808 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:25:49.388238 30758 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.456948 20801 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
W20260709 12:25:49.462680 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 255916us
I20260709 12:25:49.470528 20810 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 16. (term mismatch)
I20260709 12:25:49.471433 20810 pending_rounds.cc:85] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 15
I20260709 12:25:49.474148 20810 pending_rounds.cc:107] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 29.16
W20260709 12:25:49.468508 31086 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:25:49.472064 31033 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:25:49.475014 20810 raft_consensus.cc:2889] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.463135 30967 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.484915 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 273896us
I20260709 12:25:49.459456 30758 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.488584 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:49.489398 20801 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:49.490757 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:49.471110 31022 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:49.497933 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:49.499395 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.500353 20879 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:49.500432 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:49.494210 31060 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.486526 31033 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.499056 30967 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.499650 31022 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.520711 31094 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.521603 31018 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.427s	user 0.000s	sys 0.009s
I20260709 12:25:49.521361 31094 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.522789 31022 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.523537 31094 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:49.524107 31094 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.524323 30758 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.524891 20807 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:49.525583 30967 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:49.524518 31033 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.526656 30967 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.524554 31094 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.527387 30967 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:49.529740 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:49.529559 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.528883 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.544370 31078 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.002s
W20260709 12:25:49.550364 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.531859 31088 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:49.545360 31072 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
W20260709 12:25:49.573261 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.576740 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.577298 20875 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:49.578428 20876 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:49.578575 20878 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: [22.18-24.19]
I20260709 12:25:49.578814 20876 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:49.578984 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:49.579838 20884 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:49.579885 20878 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:49.579681 20876 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.586258 20884 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 22. Preceding OpId from leader: term: 28 index: 22. (term mismatch)
I20260709 12:25:49.584292 20878 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:49.589701 20884 pending_rounds.cc:85] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 21
I20260709 12:25:49.590165 20884 pending_rounds.cc:107] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 27.22
I20260709 12:25:49.587828 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.590598 20884 raft_consensus.cc:2889] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.594906 20876 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:49.585875 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.575156 31078 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.596937 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:49.580314 31088 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:49.587116 31092 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:49.588106 30697 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:49.588099 30967 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:49.574065 30859 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:49.605868 31030 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 12:25:49.589924 30988 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:49.588042 31095 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:49.582922 31010 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.021s
I20260709 12:25:49.601392 31078 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.596448 31057 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.019s
I20260709 12:25:49.616998 30914 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:49.618366 21432 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:25:49.626456 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.629380 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 92002.1us
I20260709 12:25:49.633939 30923 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:49.636343 30923 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:49.637519 31097 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:49.637909 31097 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:49.638818 31097 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.641203 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:49.593596 30937 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:25:49.609750 30697 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:49.629108 30859 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:49.630832 31095 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:49.632606 30914 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:49.662539 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.664865 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:49.628957 31088 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.667826 31030 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:49.662444 31095 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:49.673977 31052 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.030s	user 0.003s	sys 0.000s
I20260709 12:25:49.679448 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3699) took 1391 ms. Trace:
I20260709 12:25:49.679926 20882 rpcz_store.cc:276] 0709 12:25:48.287748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.287899 (+   151us) service_pool.cc:224] Handling call
0709 12:25:49.679171 (+1391272us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54000a2dd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:49.679362 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424249728}
W20260709 12:25:49.666396 30994 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 34 current term: 34 is higher.
I20260709 12:25:49.678819 31103 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:49.684149 31103 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.684585 31103 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:49.632835 30988 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.666033 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.689498 31078 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.695632 30697 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.705154 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.705570 20876 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:49.757366 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.778296 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:49.779071 20876 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:49.774840 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:49.778213 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.781513 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.782586 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.780586 20879 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:49.805514 20876 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 24.
I20260709 12:25:49.807034 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.806939 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:49.808064 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:49.809965 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144012us
I20260709 12:25:49.670936 31088 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.664912 31104 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:49.812739 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.823246 29903 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.815447 20879 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:49.825387 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:49.824198 29903 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.832261 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.834782 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:49.780846 30937 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:49.792727 31044 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.799808 30975 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.843256 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:49.834082 30764 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.799273 31033 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:49.824798 31078 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:49.808138 20882 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:49.854892 29903 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.859716 29903 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:49.838717 30936 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.860387 29903 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.861958 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:49.867542 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 29 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:49.847739 31104 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:49.874464 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.876191 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:49.876926 21431 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:49.853797 30697 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:49.850805 30795 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:25:49.850610 31030 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
W20260709 12:25:49.887316 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.808863 30914 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.888825 30914 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:49.891616 30520 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.852665 31044 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.852453 30937 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.856596 31033 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:49.901718 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.908507 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.869015 30646 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:49.872583 31022 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.832482 30859 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.854117 30975 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.915524 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.929803 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.895134 31044 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.882573 31078 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:25:49.934535 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.920625 29903 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.918560 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.930706 21428 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:49.930717 21434 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:49.931443 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.899608 31033 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.897142 30937 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.932507 21431 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
W20260709 12:25:49.938973 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.935739 21430 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:49.913045 31109 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.946858 21433 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:49.957762 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.959748 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.962985 30520 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.963711 30520 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.965912 30520 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.967296 31104 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.971872 30988 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:49.973502 30975 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:49.978291 31092 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:49.979084 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3603) took 2652 ms (client timeout 3063 ms). Trace:
W20260709 12:25:49.979555 20806 rpcz_store.cc:269] 0709 12:25:47.327086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.816589 (+489503us) service_pool.cc:224] Handling call
0709 12:25:49.978736 (+2162147us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54002b6bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:49.978980 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835083648}
I20260709 12:25:49.981666 20802 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: [33.19-33.19]
I20260709 12:25:49.982187 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:49.983057 21433 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:49.983423 30859 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:49.983759 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3872) took 1278 ms. Trace:
I20260709 12:25:49.984126 20802 rpcz_store.cc:276] 0709 12:25:48.704930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.913856 (+208926us) service_pool.cc:224] Handling call
0709 12:25:49.981407 (+1067551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.982110 (+   703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.983549 (+  1439us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002b6bd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:49.983701 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454394752}
I20260709 12:25:49.930213 31022 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.978876 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:49.987624 21431 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:49.959115 21430 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:49.990162 30979 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:49.955708 31109 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:49.990576 30979 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:49.991468 30979 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:49.992543 30979 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.992764 31109 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:49.999665 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.010346 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.016752 21433 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:50.015610 31022 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.012861 31103 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.030131 30950 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:50.030655 30764 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:50.033361 30857 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.038753 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1867) took 3192 ms. Trace:
I20260709 12:25:50.039966 20804 rpcz_store.cc:276] 0709 12:25:46.846653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.197663 (+351010us) service_pool.cc:224] Handling call
0709 12:25:47.942809 (+745146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.525328 (+582519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.305880 (+780552us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b540027c254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:50.038470 (+732590us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b540027c250. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:50.038670 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2916567168}
I20260709 12:25:50.030707 30950 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:50.042050 30950 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.043285 30950 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:50.044952 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 482271us
I20260709 12:25:50.051448 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.052233 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.053059 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.053292 20728 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:50.053967 20726 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
W20260709 12:25:50.054688 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.055605 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.057559 20882 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:50.060158 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.060415 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.061022 20726 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:50.061679 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.062697 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.063907 20728 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:50.064409 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.064668 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.065442 20801 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 24: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:50.066943 31103 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.067863 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.068800 20806 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:25:50.069721 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.070395 20802 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:50.057878 30976 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.073773 30976 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.075881 30935 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:50.057993 30764 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.076350 30935 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:50.076738 30764 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.077415 30935 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.078511 30935 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:50.080016 20732 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:25:50.080849 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.081615 20730 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:50.085835 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.086628 20807 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:25:50.087350 30936 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:50.088183 20734 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:50.089279 30764 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.062088 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.089701 31095 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.090469 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.092724 20730 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:50.086100 31097 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:50.096262 20804 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:50.097544 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.098330 20877 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:50.096149 20730 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:50.100862 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.101331 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.102001 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.105599 31095 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.110394 20884 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:50.118832 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.121057 20808 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:50.099996 20730 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:50.123720 20730 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:50.123075 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.075883 30976 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.126410 20884 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 29 because replica has last-logged OpId of term: 26 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 20.
I20260709 12:25:50.161245 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2401) took 1160 ms. Trace:
I20260709 12:25:50.166594 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.158811 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.155761 20726 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:25:50.131563 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.127235 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:50.178339 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.126787 20877 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:50.181036 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.153910 30840 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:50.183037 30840 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:50.184640 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.186203 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.187052 21433 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:25:50.163401 21430 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:25:50.178568 20804 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 25: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:50.176723 21432 rpcz_store.cc:276] 0709 12:25:49.001668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.001816 (+   148us) service_pool.cc:224] Handling call
0709 12:25:49.002219 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.631618 (+629399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.160932 (+529314us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54005f4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:50.161170 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415980288}
I20260709 12:25:50.159025 20882 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
W20260709 12:25:50.194591 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.156853 20733 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:50.200794 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.202278 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.196503 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:50.204408 20804 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:50.207042 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.212942 20803 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:50.214711 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.217396 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.145509 30921 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.144845 30915 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.225723 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.220525 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.219228 30845 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:50.184099 30840 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.227136 30845 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.231505 30937 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.232151 30937 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.235285 30937 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.222160 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.236076 30921 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.236688 30840 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:50.238441 30921 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.239390 20735 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:50.239682 21431 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:25:50.240248 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.240871 20726 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:50.247393 30915 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.249439 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.249672 30915 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:50.249745 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.190486 30859 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.250491 20728 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:50.250654 30859 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.251529 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.252503 20804 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:50.252733 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.253357 30859 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.254204 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:50.219399 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 145574us
I20260709 12:25:50.258301 21426 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:50.260447 20726 raft_consensus.cc:2410] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 31 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
W20260709 12:25:50.264607 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.266821 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.270371 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.272665 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.272814 20882 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:25:50.277215 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.254313 30924 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:50.281921 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46363.3us
I20260709 12:25:50.282279 30924 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:50.269685 31045 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 28 current term: 28 is higher.
W20260709 12:25:50.283646 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103292us
I20260709 12:25:50.284230 20802 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:50.272348 30935 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:50.289014 31095 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:50.292971 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:50.293569 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.272421 30724 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:50.295058 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:50.263965 31004 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:25:50.298305 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:50.306772 31029 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:50.309827 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.309908 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:50.310739 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.299321 30764 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:50.319890 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:50.322469 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.302140 30789 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:50.327473 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.326143 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:50.377579 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50226.9us
I20260709 12:25:50.378322 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:50.313498 30935 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:50.382274 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:50.381086 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64135.8us
I20260709 12:25:50.309020 31029 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:50.423985 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 97960.4us
I20260709 12:25:50.321422 30859 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:50.425873 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:50.316411 31095 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:50.403589 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53685.7us
I20260709 12:25:50.361455 31097 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:50.324947 30976 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:50.382262 31116 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:50.379730 31060 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:50.379491 31115 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:25:50.431504 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.432935 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.425438 30723 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:50.437413 21400 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.517s (SENT) [suppressed 16 similar messages]
W20260709 12:25:50.444033 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61859.4us
I20260709 12:25:50.324494 30697 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.107s
I20260709 12:25:50.426458 31029 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:50.321130 30751 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:50.476447 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:50.430919 30789 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:50.379973 30915 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.058s
I20260709 12:25:50.430011 30935 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:50.475173 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 91685.9us
I20260709 12:25:50.459525 31053 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:50.435698 31116 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.494681 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:50.477643 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.329196 30950 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.103s
I20260709 12:25:50.430598 31095 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.464493 31115 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:50.500408 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:50.498590 21432 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
W20260709 12:25:50.501588 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.502156 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.502175 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.501030 21433 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:50.499650 31116 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:50.493719 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.476737 31061 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.486440 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 167860us
I20260709 12:25:50.476277 30976 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:50.453868 30982 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.030s
I20260709 12:25:50.501605 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.521481 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.505501 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.505076 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:50.513602 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.486406 30789 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:50.518464 21433 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
W20260709 12:25:50.530550 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.503594 30921 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:25:50.534823 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58492.6us
I20260709 12:25:50.500070 31095 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:50.537184 20805 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:50.503952 31115 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:50.548918 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.549803 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.561548 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.550606 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.552484 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.563170 20730 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:25:50.545162 31126 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:25:50.534143 30789 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:50.575811 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.576855 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.577894 20805 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:50.544236 30763 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:50.539047 30921 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:50.597617 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.598479 20727 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:25:50.600008 21427 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:25:50.553076 30970 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:50.602105 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.582193 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.603304 20805 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25. Ops: [22.18-24.19]
W20260709 12:25:50.604282 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.605901 20805 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:50.607160 20729 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:50.518499 31061 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.560678 30972 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:50.610913 30970 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:50.608074 30924 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.615090 30976 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.611510 30763 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:50.651052 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1860) took 2775 ms. Trace:
W20260709 12:25:50.654825 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.663497 20883 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:50.658422 21435 rpcz_store.cc:276] 0709 12:25:47.875652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.876196 (+   544us) service_pool.cc:224] Handling call
0709 12:25:47.892049 (+ 15853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.795345 (+903296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.650748 (+1855403us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54004da8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:50.650954 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075822592}
W20260709 12:25:50.668555 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.575770 30950 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.048s
W20260709 12:25:50.639672 31128 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:25:50.685818 20805 pending_rounds.cc:85] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 15
I20260709 12:25:50.693130 20805 pending_rounds.cc:107] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 27.16
W20260709 12:25:50.654805 30916 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 30 current term: 30 is higher.
W20260709 12:25:50.625658 31122 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 12:25:50.626251 31133 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:25:50.625262 31131 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:50.582183 30935 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.652045 30643 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.058s
I20260709 12:25:50.672263 30979 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:50.695838 20805 raft_consensus.cc:2889] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:50.682086 31125 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:25:50.685611 30898 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:50.682792 31112 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:50.706861 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.656876 31061 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.666180 30960 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:50.701035 30840 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.614899 30937 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:50.721498 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:50.715796 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 229612us
I20260709 12:25:50.685302 30998 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:25:50.720805 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.624226 30972 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.753031 20802 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:25:50.719978 30997 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:50.751643 20727 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:25:50.704403 31103 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:25:50.743253 30898 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:50.764794 20727 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:25:50.756222 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:50.709671 30916 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.039s
I20260709 12:25:50.759790 30997 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:50.767441 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:50.768509 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.720297 30979 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:50.772434 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.773943 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.774801 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 21.
I20260709 12:25:50.707576 30921 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.751163 30960 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.746454 31112 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:50.753945 30840 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.758846 30929 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:25:50.724707 30763 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.790841 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.810067 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.808193 20735 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:50.778905 30988 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:50.813596 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.783963 30724 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:50.783902 30898 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.823014 31112 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.824769 30960 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.825307 30724 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:50.825915 30724 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:50.790002 30921 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:50.815124 30988 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.811506 30997 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:50.827885 30988 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:50.820222 30972 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.831949 31053 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:50.822048 30764 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:50.839316 30928 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:50.839879 30928 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:50.840814 30928 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:50.842191 30928 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:50.843340 30840 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.844336 30979 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:50.828507 30997 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:50.864315 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.870852 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:50.875103 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 313603us
I20260709 12:25:50.877290 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.879235 20802 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:25:50.878643 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.877549 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.877581 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.878247 20883 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 22. Preceding OpId from leader: term: 28 index: 23. (index mismatch)
I20260709 12:25:50.882287 20882 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:50.882694 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.883275 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.884377 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.884138 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:50.885412 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.878933 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.900878 30982 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:50.902479 20878 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.14-29.14]
I20260709 12:25:50.903379 30840 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:50.903625 30929 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:50.904456 31116 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.905831 20730 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:50.906888 31094 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.908614 31115 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:50.911052 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.911554 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.903913 30840 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:50.912695 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.912781 30840 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:50.913251 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:50.913861 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.914675 20729 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:50.916145 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:50.915066 30898 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:50.917232 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.919031 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.919699 20877 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:50.917577 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.920490 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.921069 21430 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:50.918212 21428 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:50.921967 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:50.918111 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.922647 20882 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:50.917263 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:50.923143 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.923960 21433 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:25:50.925274 30960 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:50.927330 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.927732 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:50.929706 20882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:50.930457 20882 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
W20260709 12:25:50.933915 31129 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.509s	user 0.004s	sys 0.000s
I20260709 12:25:50.934790 31097 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:50.937649 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:50.948649 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60853.5us
I20260709 12:25:50.949463 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.951479 20884 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:50.951583 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.940511 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:50.953572 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:50.950930 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73477.2us
I20260709 12:25:50.957620 20876 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:25:50.955940 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 262272us
I20260709 12:25:50.902601 30789 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.939909 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.974162 21426 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:50.941457 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.942957 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.945210 21428 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:25:50.963074 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:50.971714 31137 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:50.979712 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:50.981065 21428 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:50.981918 20728 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:50.982751 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.964278 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.939877 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:50.960387 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:50.984547 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.973134 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:50.986090 21434 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
W20260709 12:25:50.989866 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50073.2us
I20260709 12:25:50.958242 30924 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:50.991223 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:50.979125 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:50.906939 31116 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:50.963757 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:50.962395 31112 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:50.960951 30929 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:50.971776 30982 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:50.958077 30960 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 12:25:50.992342 30994 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:51.013566 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:51.026204 20732 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:51.026957 20735 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:51.000082 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:50.988715 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:51.024187 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.985620 31140 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:50.974308 31095 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:50.972733 30898 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:25:50.961189 31094 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:25:50.986876 31097 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
W20260709 12:25:51.035125 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46476.6us
I20260709 12:25:50.993191 30963 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:50.994297 31139 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:51.040132 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3634) took 1547 ms. Trace:
I20260709 12:25:51.011466 31137 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:50.976039 31113 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 12:25:51.044356 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:51.060521 20734 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:51.013736 30929 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.049990 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:51.038684 31112 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:25:51.041085 31029 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 12:25:50.992144 30840 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.052441 20808 rpcz_store.cc:276] 0709 12:25:49.492252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.492401 (+   149us) service_pool.cc:224] Handling call
0709 12:25:49.492764 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.121518 (+628754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.039809 (+918291us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540019afd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:51.040049 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444491008}
I20260709 12:25:51.070016 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:51.053956 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.031343 30916 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:50.991789 30988 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.024093 31093 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
W20260709 12:25:51.084192 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 120545us
W20260709 12:25:51.087167 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 123000us
W20260709 12:25:51.081142 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.050860 30963 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.064662 30929 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:51.064247 31112 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.051208 30895 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:51.045711 30924 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.026s
I20260709 12:25:51.090555 20727 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:51.063633 31145 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:51.047935 30994 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:51.106487 20881 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:51.094414 20878 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
W20260709 12:25:51.094360 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 105140us
W20260709 12:25:51.098104 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 135086us
I20260709 12:25:51.110606 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:51.124941 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 133817us
I20260709 12:25:51.086151 31115 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:51.122319 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:51.121217 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 128711us
I20260709 12:25:51.092785 30963 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:51.133666 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:51.070242 31001 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:25:51.134552 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:51.089535 30845 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.126710 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.101819 30895 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.102452 31112 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:51.128044 31139 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:51.136452 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:51.136837 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:51.125661 30960 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:51.091336 30705 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.073570 31029 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.126571 30898 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:51.154827 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:51.152017 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53616.5us
I20260709 12:25:51.137570 30915 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.167738 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3823) took 2150 ms. Trace:
I20260709 12:25:51.168340 21434 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:51.159046 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:51.160410 31146 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:51.157436 30998 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:51.187228 31004 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:51.187706 31004 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:51.188573 31004 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:51.189369 30859 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.752s	user 0.010s	sys 0.000s
I20260709 12:25:51.189852 30859 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:51.144299 30895 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:51.168208 20731 rpcz_store.cc:276] 0709 12:25:49.008092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.008225 (+   133us) service_pool.cc:224] Handling call
0709 12:25:49.202179 (+193954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.269488 (+ 67309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.269509 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.269854 (+   345us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.377902 (+108048us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:49.377938 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.157721 (+1779783us) spinlock_profiling.cc:243] Waited 8.13 ms on lock 0x7b54001d7c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:51.157770 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.157813 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:51.157826 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.158030 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23182,"raft.queue_time_us":38263,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":113,"spinlock_wait_cycles":18703232,"thread_start_us":107675,"threads_started":2,"wal-append.queue_time_us":70928,"wal-append.run_cpu_time_us":3302,"wal-append.run_wall_time_us":1175173}
W20260709 12:25:51.171983 31143 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:25:51.171821 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3641) took 1008 ms. Trace:
I20260709 12:25:51.191674 20806 rpcz_store.cc:276] 0709 12:25:50.163738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.163880 (+   142us) service_pool.cc:224] Handling call
0709 12:25:50.211416 (+ 47536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.211898 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.211931 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.216093 (+  4162us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.381010 (+164917us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:50.429726 (+ 48716us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.470358 (+ 40632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.470454 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.161117 (+690663us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b18002b8e80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:51.163738 (+  2621us) raft_consensus.cc:1685] finished
0709 12:25:51.171558 (+  7820us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.171745 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40190,"prepare.queue_time_us":94875,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2367,"raft.queue_time_us":45803,"raft.run_cpu_time_us":446,"raft.run_wall_time_us":50641,"spinlock_wait_cycles":7487744,"thread_start_us":127071,"threads_started":1,"wal-append.queue_time_us":244438}
I20260709 12:25:51.173739 31029 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.156333 30960 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:51.190897 30898 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.217094 30894 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:51.218366 31095 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.218837 30898 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.163529 31153 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.181217 30915 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.158138 30994 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.200263 30988 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:51.200482 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.163384 30789 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.204602 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:51.204679 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45742.6us
I20260709 12:25:51.219022 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3548) took 1167 ms. Trace:
I20260709 12:25:51.219707 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:51.220712 20810 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.21-28.21]
I20260709 12:25:51.151041 31139 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:51.218405 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:51.226214 20805 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:25:51.155467 30763 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:25:51.219022 30914 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.231837 21434 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:51.183737 30840 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.232331 30960 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:51.232925 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:51.267815 20806 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:51.235913 21428 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
W20260709 12:25:51.246205 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 127134us
I20260709 12:25:51.190433 31004 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:51.252163 21433 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
W20260709 12:25:51.252748 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 126092us
I20260709 12:25:51.271831 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:51.272264 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67729.4us
I20260709 12:25:51.256868 20733 rpcz_store.cc:276] 0709 12:25:50.051834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.051983 (+   149us) service_pool.cc:224] Handling call
0709 12:25:51.218837 (+1166854us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400298bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:51.218938 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239074688}
I20260709 12:25:51.256323 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:51.264914 21427 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:51.269150 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:51.272017 21435 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
W20260709 12:25:51.253069 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 119538us
I20260709 12:25:51.278765 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.278781 20883 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:25:51.279829 20881 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:25:51.280335 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.281608 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.273200 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.284672 20701 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:51.286262 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:51.288276 21433 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:51.287142 21434 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:25:51.281176 20883 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:25:51.256421 30898 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:51.262674 31046 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:51.294435 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.294297 31001 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:51.297117 31001 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.303716 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.299134 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:51.302690 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.303607 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.286363 30894 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:51.310433 20882 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:51.252372 30724 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.317435 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:51.318347 21433 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:51.286528 31113 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:51.319794 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:51.315239 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 29 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:51.292662 30840 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:51.323567 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:51.322608 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 340174us
W20260709 12:25:51.332033 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63305.4us
I20260709 12:25:51.297497 31001 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:51.271083 30914 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.335732 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.254384 31153 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.343995 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:51.305450 31048 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:51.366649 31048 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:51.367489 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52353.1us
I20260709 12:25:51.367707 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:51.376307 21433 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
W20260709 12:25:51.391745 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47926.7us
W20260709 12:25:51.393134 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57525.6us
I20260709 12:25:51.394264 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:51.395731 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59400.7us
I20260709 12:25:51.402158 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:51.392592 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:51.408780 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:51.303355 31116 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.067s
I20260709 12:25:51.420338 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:51.421386 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:51.357045 31053 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:51.424256 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.424938 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:51.356218 30979 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.428622 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:51.332459 30898 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:51.357241 31158 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:51.434989 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:51.435649 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:51.388573 31048 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:51.318872 30916 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.086s
I20260709 12:25:51.449714 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:51.450569 20731 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:51.395714 31113 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:51.454849 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:51.469348 20810 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:51.356801 31061 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.075s
I20260709 12:25:51.460456 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:51.433403 30935 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:51.397320 31140 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:51.481125 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79035.2us
I20260709 12:25:51.364452 30929 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.475512 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:51.496696 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:51.499365 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.496385 30935 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:51.508764 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.510213 20877 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:51.478454 30916 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:51.517010 20731 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:51.479828 31046 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.357303 31139 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.532248 20881 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.22-28.22]
W20260709 12:25:51.532379 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 108187us
I20260709 12:25:51.497738 30845 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:51.443055 30979 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.442543 31112 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.480019 31162 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:51.541827 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1879) took 1376 ms. Trace:
I20260709 12:25:51.514159 31113 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:51.542850 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.514267 30935 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:51.588521 31179 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:25:51.551966 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 112765us
I20260709 12:25:51.596558 21431 rpcz_store.cc:276] 0709 12:25:50.165664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.165811 (+   147us) service_pool.cc:224] Handling call
0709 12:25:50.166174 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.240035 (+ 73861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.541585 (+1301550us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:51.541768 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167904512}
I20260709 12:25:51.545710 20810 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:51.555759 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:51.603261 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 142925us
I20260709 12:25:51.604022 20809 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:51.604760 20880 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:51.603082 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:51.606949 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64180us
I20260709 12:25:51.607690 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:51.609090 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:51.604367 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.579352 30353 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:51.613936 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:51.615232 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:51.617534 20883 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:51.617898 20883 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:51.619095 20801 raft_consensus.cc:2368] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29.
I20260709 12:25:51.523569 31001 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.620641 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:51.533614 30937 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.067s
W20260709 12:25:51.604633 31164 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 12:25:51.541451 31140 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:51.595083 31162 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.556074 30845 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.602833 30916 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:51.626432 20731 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:51.616111 31093 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:51.602329 30915 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:51.578301 30789 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.055s
I20260709 12:25:51.633318 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:51.632243 20809 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
W20260709 12:25:51.672703 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.689153 20802 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 34 because replica has last-logged OpId of term: 32 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 19.
I20260709 12:25:51.604233 31179 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:51.624467 20883 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.636968 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1649) took 1087 ms. Trace:
I20260709 12:25:51.541357 31156 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.095s
I20260709 12:25:51.604395 31053 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:51.612201 31180 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:51.622335 31174 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.026s
I20260709 12:25:51.642668 31153 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.644418 31183 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:51.551731 31046 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.623775 31113 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.647655 30724 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.595337 30998 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:25:51.631964 20810 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: []
W20260709 12:25:51.658402 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.629406 31188 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:51.632788 31095 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:51.625682 30751 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:51.690379 20728 rpcz_store.cc:276] 0709 12:25:50.549155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.553384 (+  4229us) service_pool.cc:224] Handling call
0709 12:25:50.555821 (+  2437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.991834 (+436013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.636751 (+644917us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54002274d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:51.636913 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979458944}
I20260709 12:25:51.622608 31175 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:51.611655 31056 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:25:51.649416 30845 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:51.649292 30916 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:51.705852 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 277313us
I20260709 12:25:51.709683 20733 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:25:51.648370 31187 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:51.640396 31162 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:51.715763 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:51.739961 31095 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:51.740240 20735 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 25: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:25:51.740764 31165 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:25:51.627831 30353 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.741051 31095 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.742581 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.716656 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:51.743453 20734 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:25:51.693974 31179 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:51.697795 30914 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:51.747671 30970 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:51.713514 30764 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.748796 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:51.717412 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:51.726251 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:51.751978 30998 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:51.752594 30998 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:51.753539 30998 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:51.755851 31186 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:51.730252 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.731884 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:51.761097 20884 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:51.752231 20726 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:51.758580 20880 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:51.765389 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:51.767683 20776 consensus_peers.cc:597] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:51.768456 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:51.707654 31153 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.031s
I20260709 12:25:51.786062 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:51.736371 30916 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:51.710583 31094 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.049s
I20260709 12:25:51.780859 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:51.781468 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.748799 30764 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.762985 30914 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:51.729038 31159 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.055s
W20260709 12:25:51.783656 31169 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 12:25:51.794656 30895 consensus_meta.cc:287] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.000s
I20260709 12:25:51.795609 31001 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.796715 30894 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:51.778733 30929 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:51.798558 31004 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.000s
I20260709 12:25:51.783878 31046 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:51.782718 31170 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:25:51.767297 31178 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.713518 31113 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.823589 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:51.849972 20734 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:51.776096 31188 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.785302 30751 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:51.785132 31115 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:51.817567 31173 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
W20260709 12:25:51.817826 31166 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:25:51.837401 30895 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.847680 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:51.862164 20880 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 24: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:51.820560 31175 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:51.824295 31174 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:25:51.866456 31174 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:51.867440 31156 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.867754 30915 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:51.868049 31156 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.868301 30915 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:51.861454 30963 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.002s
I20260709 12:25:51.828958 31192 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:51.859125 30751 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.876343 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:51.847513 31112 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:51.870541 30915 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:51.887887 20726 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:51.784672 31053 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.932956 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:51.925733 20880 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:51.931759 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:51.936185 21430 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: [30.20-32.21]
I20260709 12:25:51.847265 30914 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.079s
W20260709 12:25:51.937255 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 332939us
I20260709 12:25:51.876798 31192 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.855739 30979 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.942816 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:51.891456 31175 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.941066 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:51.932989 31170 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:25:51.855651 31193 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.092s
W20260709 12:25:51.949400 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 127430us
I20260709 12:25:51.959517 20726 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [26.21-28.22]
W20260709 12:25:51.958992 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 131637us
I20260709 12:25:51.858186 31188 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.960980 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:51.858505 31178 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.862783 31115 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.964871 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:51.883785 30963 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:51.936360 30950 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.028s
I20260709 12:25:51.942077 31192 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:51.958738 30979 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:51.980854 20808 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
I20260709 12:25:51.981230 20726 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:51.979314 30764 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:51.961305 31140 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:51.965894 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:51.980826 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:51.977942 30963 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:51.981652 20726 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:51.996809 30915 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:51.996737 20726 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.997821 20726 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:51.999503 30751 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.999729 31090 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:52.000320 31090 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:51.981197 30979 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:52.001816 30979 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:51.977152 31123 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.513s	user 0.000s	sys 0.005s
I20260709 12:25:51.942685 31112 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:25:51.988121 30960 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.002s
W20260709 12:25:51.989020 31158 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
I20260709 12:25:51.988665 30845 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.990312 31093 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:51.991442 31113 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:51.992236 30895 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:52.019431 31194 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:52.019750 31205 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:52.020376 31205 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:51.963143 31175 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.006924 31159 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:52.014479 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2403) took 2745 ms. Trace:
I20260709 12:25:52.023597 31159 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.024075 31159 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:52.015657 31156 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.004065 31180 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:52.016671 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1657) took 1394 ms. Trace:
W20260709 12:25:52.017151 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.033210 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.007858 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:52.007169 31204 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:52.023636 21429 rpcz_store.cc:276] 0709 12:25:49.269252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.269921 (+   669us) service_pool.cc:224] Handling call
0709 12:25:49.270268 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.322765 (+ 52497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.322793 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.072364 (+749571us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:50.072785 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.259496 (+186711us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:25:50.259549 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.259642 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.976542 (+1716900us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:52.014251 (+ 37709us) raft_consensus.cc:1685] finished
0709 12:25:52.014274 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.014419 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52056,"raft.queue_time_us":65904,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":35921,"spinlock_wait_cycles":10286208,"thread_start_us":186272,"threads_started":1,"wal-append.queue_time_us":306812}
W20260709 12:25:52.027148 31200 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:52.025748 31158 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:52.003178 30979 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:52.032076 20732 rpcz_store.cc:276] 0709 12:25:50.622169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.622430 (+   261us) service_pool.cc:224] Handling call
0709 12:25:51.025964 (+403534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.026816 (+   852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.016416 (+989600us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400226ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:52.016593 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844536576}
W20260709 12:25:52.041579 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106311us
I20260709 12:25:51.988160 31188 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.043478 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.007942 31112 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:52.050539 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.002930 31090 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.043566 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:52.046715 21433 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:52.052479 30724 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:52.057785 30724 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:52.017457 30960 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.058404 31115 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.059365 20810 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:52.060111 31178 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.060276 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.061305 21434 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:52.061584 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.062625 30982 raft_consensus.cc:993] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:25:52.062697 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.040783 30895 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.056137 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:52.069051 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:52.070396 31045 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:52.071456 31045 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:52.071802 31045 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:52.071869 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3843) took 1133 ms. Trace:
I20260709 12:25:52.072738 31045 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.073887 31045 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:52.063124 31112 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.075315 31112 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:52.058717 30724 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 22, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:52.054765 31048 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.002s
I20260709 12:25:52.076983 31048 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:52.052392 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44654.9us
I20260709 12:25:52.072316 20727 rpcz_store.cc:276] 0709 12:25:50.938602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.961588 (+ 22986us) service_pool.cc:224] Handling call
0709 12:25:50.961847 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.095989 (+134142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.071633 (+975644us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54003b3b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:52.071809 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295485696}
I20260709 12:25:52.039991 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.065963 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.089027 20884 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:52.089272 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.090656 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:52.090865 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51001.7us
I20260709 12:25:52.062726 31188 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:52.103874 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.103559 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:52.105077 20810 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:52.064112 30982 raft_consensus.cc:1081] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:25:52.108454 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:52.110018 31183 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:52.078127 30950 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:52.112280 20810 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:52.113164 30845 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.113682 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.049924 31205 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.113827 31048 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.113180 31056 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.092234 31204 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:25:52.139355 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.117475 31202 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:25:52.078387 30895 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.117921 30950 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:52.143855 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49992us
I20260709 12:25:52.105563 31201 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:52.136610 31176 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:52.167994 20731 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:25:52.167367 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1656) took 1566 ms. Trace:
W20260709 12:25:52.160846 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53520.2us
I20260709 12:25:52.169377 20730 rpcz_store.cc:276] 0709 12:25:50.601430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.601820 (+   390us) service_pool.cc:224] Handling call
0709 12:25:52.167105 (+1565285us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54002495d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:52.167300 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696307584}
I20260709 12:25:52.156594 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:52.159140 31152 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.976s	user 0.000s	sys 0.003s
I20260709 12:25:52.152074 30950 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:52.176386 30916 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.182508 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:52.175657 30935 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.000s
I20260709 12:25:52.185786 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:52.188575 21433 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:52.175055 30845 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:52.193081 30845 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.193624 30845 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:52.177829 30895 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.193821 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.197278 21429 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:52.159508 31179 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.176450 30960 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.168365 31176 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.218704 21433 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:25:52.221843 31175 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.145303 31056 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.226248 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3649) took 2057 ms. Trace:
I20260709 12:25:52.226713 20804 rpcz_store.cc:276] 0709 12:25:50.168879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.253347 (+ 84468us) service_pool.cc:224] Handling call
0709 12:25:50.253679 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.270711 (+ 17032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.270741 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.271251 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.921433 (+650182us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b1800164860. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:25:51.071818 (+150385us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:51.084043 (+ 12225us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.084235 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.084318 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.158497 (+1074179us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:52.226034 (+ 67537us) raft_consensus.cc:1685] finished
0709 12:25:52.226056 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.226186 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23675,"prepare.queue_time_us":14129,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":21024,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":8811,"spinlock_wait_cycles":107584256,"thread_start_us":150193,"threads_started":1,"wal-append.queue_time_us":871367}
I20260709 12:25:52.230274 31056 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.230093 31192 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.232507 31179 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.234022 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:52.237037 31159 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.198360 30935 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.240499 21431 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:52.242136 30935 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.242537 31188 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:52.243778 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.245251 31139 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:52.219343 30895 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.252688 21430 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:52.256440 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3889) took 1040 ms. Trace:
I20260709 12:25:52.261147 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.261389 20808 rpcz_store.cc:276] 0709 12:25:51.216291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.219410 (+  3119us) service_pool.cc:224] Handling call
0709 12:25:52.256193 (+1036783us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54002aa8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:52.256362 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662454016}
I20260709 12:25:52.247156 31176 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.251946 30895 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:52.260192 31210 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 12:25:52.275952 30979 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:52.278076 31159 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:52.282418 30916 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.299886 30915 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:52.301343 31192 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:52.304152 31130 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.779s	user 0.007s	sys 0.004s
I20260709 12:25:52.306170 30988 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:52.306285 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3640) took 2142 ms. Trace:
I20260709 12:25:52.306684 30988 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:52.308840 31161 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.137s	user 0.000s	sys 0.003s
I20260709 12:25:52.310779 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3820) took 1245 ms. Trace:
I20260709 12:25:52.311305 20876 rpcz_store.cc:276] 0709 12:25:51.065427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.065621 (+   194us) service_pool.cc:224] Handling call
0709 12:25:51.066080 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.066364 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.066402 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.066912 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.212638 (+145726us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:51.212678 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.212748 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.308309 (+1095561us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:52.310472 (+  2163us) raft_consensus.cc:1685] finished
0709 12:25:52.310503 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.310714 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73876736,"thread_start_us":109351,"threads_started":1,"wal-append.queue_time_us":80598}
I20260709 12:25:52.306885 20807 rpcz_store.cc:276] 0709 12:25:50.163393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.191485 (+ 28092us) service_pool.cc:224] Handling call
0709 12:25:50.191885 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.198685 (+  6800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.198715 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.199223 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.382395 (+183172us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:50.713860 (+331465us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.717017 (+  3157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.717133 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.971085 (+253952us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:25:52.303525 (+1332440us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:52.306005 (+  2480us) raft_consensus.cc:1685] finished
0709 12:25:52.306029 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.306209 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76066,"prepare.queue_time_us":122477,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":10630,"raft.run_cpu_time_us":2893,"raft.run_wall_time_us":783750,"spinlock_wait_cycles":28982528,"thread_start_us":372557,"threads_started":2,"wal-append.queue_time_us":270970}
I20260709 12:25:52.282011 31176 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.010s
I20260709 12:25:52.298607 30845 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.326087 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:52.314945 31162 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.546s	user 0.015s	sys 0.000s
I20260709 12:25:52.307619 31138 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:25:52.321846 30840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.009s	sys 0.012s
I20260709 12:25:52.337183 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:52.343384 20876 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:52.323812 31192 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:52.325495 31178 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:52.330335 31162 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.329890 31138 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:52.341830 30840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.010s	sys 0.013s
I20260709 12:25:52.366820 21434 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
W20260709 12:25:52.372401 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.348968 30895 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.349011 31178 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:52.376457 31178 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.377781 31178 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:52.378681 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.379431 20732 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:52.356722 31176 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.384534 20876 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:25:52.388613 20876 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:52.403803 31162 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:52.404476 31176 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.406214 20883 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:52.406526 30743 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=1 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:52.408236 30845 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:52.409073 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.411259 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:52.412441 31162 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.413033 31162 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.414103 30950 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.416393 21433 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:52.416774 20880 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:52.418901 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:52.420248 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:52.419849 21426 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 12:25:52.421881 21426 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 18
I20260709 12:25:52.422149 21426 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 12:25:52.422436 21426 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.373694 31138 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.423262 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.423727 31138 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:52.345014 30988 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.422905 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:52.425804 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:52.426473 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:52.430333 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.419857 31176 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.430540 30936 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.443567 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.450090 31053 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.447906 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.451669 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:52.431545 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.352837 31192 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.402908 31090 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.456804 30915 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:52.457988 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:52.460156 30935 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:52.424023 31162 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:52.463984 20732 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:52.492193 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.492475 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.431154 31188 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.493376 20732 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:52.462643 21433 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:52.469231 20735 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 17
I20260709 12:25:52.487392 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.494825 20735 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:25:52.469767 20733 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:52.494884 20728 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:52.484241 21426 raft_consensus.cc:2410] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 29 because replica has last-logged OpId of term: 28 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 21.
I20260709 12:25:52.495497 20735 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.468079 20803 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:52.490672 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:52.493561 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:25:52.497110 20727 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:25:52.478930 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.498070 20729 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:52.498250 20726 raft_consensus.cc:606] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.498663 20726 raft_consensus.cc:614] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:52.499161 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.499691 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.500562 21426 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:52.489836 30743 tablet_bootstrap.cc:492] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:25:52.506129 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.506299 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.508579 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:52.486250 30857 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.516240 30743 ts_tablet_manager.cc:1403] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 3.775s	user 0.129s	sys 0.113s
I20260709 12:25:52.493391 30923 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.520310 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.523445 31197 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:52.501417 31188 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.529431 30353 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.501598 31179 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:52.530139 30937 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:25:52.486860 30898 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:52.517827 21433 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:25:52.499823 30988 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:52.501271 31183 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:52.492746 31090 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.542786 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:52.591415 21429 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:52.485301 30936 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.507296 30845 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:52.540678 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:52.539078 20883 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:52.581182 30898 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.576941 31179 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:52.582680 30937 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:52.584709 30970 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:52.585626 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.490447 31053 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:52.588604 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.582808 31155 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.385s	user 0.002s	sys 0.001s
I20260709 12:25:52.547930 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.530792 31215 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:25:52.530969 30960 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:25:52.536096 30935 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:52.574160 30914 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:52.589313 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.577976 31180 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:52.592823 30859 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:52.601169 31184 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:52.601593 31179 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.612519 20726 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:52.603041 20730 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:52.602476 30898 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:52.599360 31101 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 29 current term: 29 is higher.
I20260709 12:25:52.536800 30857 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.608752 21433 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
W20260709 12:25:52.600728 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60150.9us
I20260709 12:25:52.615584 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.597426 30937 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:52.587898 31103 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.020s
I20260709 12:25:52.617532 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:52.548360 30923 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.605921 30840 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:52.622563 20728 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:52.593255 31204 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:25:52.626143 21429 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:52.607928 31180 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:52.624785 30840 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:52.636549 31090 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.638191 30936 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:52.639333 31149 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.471s	user 0.008s	sys 0.000s
I20260709 12:25:52.619724 30859 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:52.640350 31151 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.460s	user 0.004s	sys 0.000s
I20260709 12:25:52.628533 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:52.619961 30898 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:52.648713 31113 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:52.649235 20730 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:52.623833 31179 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:52.657056 20729 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:52.653385 20731 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:52.655089 31158 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:52.645783 30840 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:52.652752 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.652479 30914 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:52.656960 31153 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.004s
I20260709 12:25:52.655877 31113 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.653553 30845 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.010s
I20260709 12:25:52.664410 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:52.665161 21429 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:52.667388 20731 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:52.655779 31204 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:25:52.662581 31192 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.663324 31158 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:52.663403 30914 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:52.671073 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:52.666471 31113 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:52.668561 31153 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 30.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:52.654363 30743 raft_consensus.cc:359] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:52.670821 30963 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:52.677811 31158 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.684482 31153 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:52.687589 30935 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:52.688599 30935 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.690846 30914 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.691918 30916 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:52.692366 30916 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:52.692711 30914 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:52.675976 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:52.662057 31180 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.677148 30743 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:25:52.702245 31193 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:52.693252 30916 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.702891 31193 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:52.703732 30916 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:52.705943 30743 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:52.711091 31053 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:52.715672 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2406) took 1581 ms. Trace:
I20260709 12:25:52.716149 21435 rpcz_store.cc:276] 0709 12:25:51.134727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.135603 (+   876us) service_pool.cc:224] Handling call
0709 12:25:51.269745 (+134142us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.276001 (+  6256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.715467 (+1439466us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54005a4950. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:52.715607 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262412672}
W20260709 12:25:52.717154 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.717469 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 541311us
I20260709 12:25:52.660794 30937 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.723565 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1828) took 1446 ms. Trace:
I20260709 12:25:52.724037 20882 rpcz_store.cc:276] 0709 12:25:51.277018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.277134 (+   116us) service_pool.cc:224] Handling call
0709 12:25:51.277451 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.322115 (+ 44664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.723290 (+1401175us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:52.723505 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75280256}
I20260709 12:25:52.718390 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.674538 31103 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:52.674516 31188 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.735338 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.745246 20877 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:25:52.738418 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.675552 31204 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:52.733858 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:52.748281 20880 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:52.748203 20882 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:52.709342 30743 ts_tablet_manager.cc:1434] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.172s	user 0.008s	sys 0.000s
I20260709 12:25:52.772172 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.772957 20880 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:52.772928 20876 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:52.774217 31093 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:52.776793 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.778380 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.779158 20880 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:52.723156 31205 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.020s
I20260709 12:25:52.804320 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:52.726927 31153 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:52.772481 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.695021 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:52.806973 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:52.784535 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.814332 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.819542 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.822381 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.823314 20883 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:52.815095 20882 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:52.820451 20875 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:52.732296 30963 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:52.754730 30789 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:52.827302 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 132373us
I20260709 12:25:52.703840 31193 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:52.833525 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.835397 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:52.767724 31176 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.781718 31204 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:52.843995 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:52.774741 31093 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.752942 31103 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.818677 30937 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:52.847362 20735 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.766850 31180 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:52.825310 31045 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:52.852310 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134048us
I20260709 12:25:52.836349 31112 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:52.880867 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:52.862221 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.869735 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:52.870564 30724 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:52.871577 20876 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:25:52.872087 20808 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:25:52.840821 30857 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:52.861467 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.888067 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:52.888384 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.888835 20730 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:52.889022 20727 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:52.883368 20875 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:52.818943 31153 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:52.893226 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.894331 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:52.895242 20729 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:52.855170 31157 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.898913 20806 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:52.841651 30963 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.900969 20883 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:25:52.900255 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3836) took 1139 ms. Trace:
I20260709 12:25:52.904013 20804 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.21-28.21]
I20260709 12:25:52.904865 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.881145 31045 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.919564 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:52.842036 30789 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:52.920202 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51154.4us
I20260709 12:25:52.925659 20808 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:52.908946 20879 rpcz_store.cc:276] 0709 12:25:51.760801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.010274 (+249473us) service_pool.cc:224] Handling call
0709 12:25:52.010642 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:52.011113 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:52.048814 (+ 37701us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.152334 (+103520us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:52.155431 (+  3097us) log.cc:844] Serialized 52 byte log entry
0709 12:25:52.155613 (+   182us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:52.473455 (+317842us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.810352 (+336897us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.899974 (+ 89622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.900043 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.900082 (+    39us) raft_consensus.cc:1685] finished
0709 12:25:52.900092 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.900204 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":548228,"prepare.queue_time_us":437666,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":115,"raft.queue_time_us":387291,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":4701,"thread_start_us":378259,"threads_started":2,"wal-append.queue_time_us":129289}
I20260709 12:25:52.926101 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:52.926116 20808 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:52.926889 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.924455 20875 raft_consensus.cc:2368] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29.
I20260709 12:25:52.927857 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.928630 20727 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:52.904311 20893 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-1047135f71834dbab6b869eb3c081b81 received from {username='slave'} at 127.0.0.1:53814
I20260709 12:25:52.922852 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:52.847167 31176 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:52.932819 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:52.930938 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:52.900818 31112 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:25:52.980296 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54269.6us
I20260709 12:25:52.990724 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:52.992772 20882 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
W20260709 12:25:52.961618 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.938271 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:52.995772 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57564.8us
I20260709 12:25:52.998066 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:52.930012 20893 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-1047135f71834dbab6b869eb3c081b81 on tablet 1047135f71834dbab6b869eb3c081b81 with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:52.936777 31029 raft_consensus.cc:2764] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:25:53.120036 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.120781 20807 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:52.904119 31001 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.085s
I20260709 12:25:53.121639 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.122291 20807 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:53.123680 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.124917 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.125784 31159 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:53.126381 31159 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.128671 31159 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.128811 30963 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.134191 20810 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:25:53.134370 31176 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.136132 31139 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:53.136448 31113 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:52.999264 31154 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:52.940764 31061 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:53.138806 31061 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.140347 30970 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.141180 31061 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.140981 30970 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.141969 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:53.092607 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:53.100930 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 110307us
I20260709 12:25:53.093191 30997 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:53.104008 31093 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.105424 20808 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.108237 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.109035 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:52.903630 31157 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.112421 31112 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.144013 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:53.144670 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:53.148563 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.193681 21427 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
W20260709 12:25:53.191326 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 194779us
I20260709 12:25:53.153887 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.196075 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:53.196949 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.197520 20807 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 24.
I20260709 12:25:53.198247 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.155762 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.159126 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.160136 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:53.199800 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.200333 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:52.940130 30936 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:53.179611 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.208725 20730 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:53.208775 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:53.180555 20733 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:25:53.145752 20802 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:25:52.982173 31153 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:52.926499 31045 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:53.177855 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.201646 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:53.207335 20801 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 30 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:25:53.198769 20807 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:53.191712 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.211463 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:53.211727 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 67803.8us
I20260709 12:25:53.228226 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.229097 20849 leader_election.cc:400] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29.
I20260709 12:25:53.229462 20849 leader_election.cc:403] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:52.978893 30789 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:52.986922 30857 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:52.984894 31103 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.176854 30997 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.215636 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.242631 20802 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:53.216112 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.139030 31113 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.211474 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.231700 20809 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:53.232052 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.247133 20876 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:53.178483 31217 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:53.220255 20810 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:53.242198 31139 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:25:53.255369 31103 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:25:53.259896 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1835) took 1981 ms. Trace:
I20260709 12:25:53.221581 31154 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:53.240972 20807 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:53.243582 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:53.243549 20726 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:53.239511 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.248977 30963 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:53.265899 21427 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:53.202507 31093 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.202649 31029 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:53.244817 30997 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:53.241765 30970 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.235663 20849 leader_election.cc:400] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
I20260709 12:25:53.278712 20849 leader_election.cc:403] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:25:52.933465 31192 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.281s
I20260709 12:25:53.220057 30845 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.287597 30845 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.290539 30845 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.266767 20807 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:53.260398 20881 rpcz_store.cc:276] 0709 12:25:51.279012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.280561 (+  1549us) service_pool.cc:224] Handling call
0709 12:25:51.280829 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.539796 (+258967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.259609 (+1719813us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540005a854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:53.259842 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577499904}
I20260709 12:25:53.294435 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.268930 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:25:53.295904 31232 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:25:53.296871 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:53.298419 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.244880 30902 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:53.300014 30902 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:53.300892 30902 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:53.301646 30902 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:53.302081 30902 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.302493 30902 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:53.303732 20881 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:53.250036 31176 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:25:53.272835 31004 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.008s
I20260709 12:25:53.273162 31001 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:53.262882 31217 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:53.258594 31153 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:25:53.309378 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73902.9us
I20260709 12:25:53.313774 31001 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.335071 20810 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:53.273036 30963 consensus_queue.cc:1243] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 30.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:53.337479 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.338097 20810 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:25:53.338498 20733 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:53.333364 31001 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:53.345304 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.343129 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:53.347910 30997 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.350630 30997 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.277910 31154 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:25:53.319082 30840 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.654s	user 0.007s	sys 0.003s
I20260709 12:25:53.354400 20881 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:53.358390 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.365517 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.317873 31153 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.371827 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.372702 20881 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:53.328358 31029 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:53.366125 20733 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:25:53.277611 31093 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:53.388206 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 460382us
I20260709 12:25:52.959350 31204 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:53.398571 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.364346 30186 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
W20260709 12:25:53.399286 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.399986 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:53.377331 31153 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:53.405252 20730 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:25:53.384392 30353 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:53.370378 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.406695 30353 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:53.407022 31137 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:53.407625 31137 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.408018 31137 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:53.408118 30353 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.411439 31157 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.417552 21431 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:53.417794 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47499.4us
I20260709 12:25:53.420423 21427 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:53.371778 20876 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:53.370366 30840 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.406674 31193 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:53.385516 30988 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:25:53.431444 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:53.417582 30936 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.440955 21433 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:53.441267 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.447337 20809 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:53.394054 31029 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.467568 20810 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:53.469112 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.457670 30936 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:53.429773 30789 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:25:53.426936 30845 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:25:53.491667 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.464437 20881 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:53.473243 30935 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:53.466295 31193 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.027s
I20260709 12:25:53.492430 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.497512 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:53.495379 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 196344us
I20260709 12:25:53.502772 20876 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:53.510236 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:25:53.512596 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 301175us
I20260709 12:25:53.514737 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:53.521479 30988 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:53.524412 30988 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:53.527868 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.530048 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.530115 30988 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.531668 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.533722 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.534431 31243 raft_consensus.cc:2764] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:25:53.535071 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.535880 31184 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:53.536381 31184 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.536834 31184 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:53.540048 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.505481 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.534431 20804 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 21.
I20260709 12:25:53.543150 31193 consensus_queue.cc:1243] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 33.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
W20260709 12:25:53.551779 31180 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.001s
I20260709 12:25:53.552229 31029 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.552256 31001 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.556859 31001 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.557655 20732 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:53.506650 31101 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:53.485317 31154 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.576088 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:53.573536 31241 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 12:25:53.571877 31237 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:25:53.502696 30935 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.598588 31056 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.603240 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3576) took 1123 ms. Trace:
I20260709 12:25:53.605291 20731 rpcz_store.cc:276] 0709 12:25:52.479429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.585388 (+105959us) service_pool.cc:224] Handling call
0709 12:25:53.603008 (+1017620us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b54003b4a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:53.603165 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132810112}
I20260709 12:25:53.588697 20876 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:25:53.610654 21433 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:25:53.610996 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.612145 20876 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:53.607038 31101 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.639448 30840 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.642917 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:53.643657 21435 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 23.
I20260709 12:25:53.643541 30902 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.644935 31101 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.645035 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.645884 20806 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:53.647459 30923 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:53.647995 30937 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.785s	user 0.010s	sys 0.001s
I20260709 12:25:53.647886 31045 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.608093 31056 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.650184 31045 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.610508 31093 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:53.651882 30914 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.959s	user 0.012s	sys 0.000s
I20260709 12:25:53.654354 31184 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.656325 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:53.660208 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.676714 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:53.661109 31217 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.662766 30935 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:53.664551 31233 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.732s	user 0.001s	sys 0.000s
I20260709 12:25:53.665539 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.684610 21428 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 22. (term mismatch)
I20260709 12:25:53.684964 21428 pending_rounds.cc:85] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 21
I20260709 12:25:53.685288 21428 pending_rounds.cc:107] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.22
I20260709 12:25:53.685529 21428 raft_consensus.cc:2889] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:53.666520 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:53.667783 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.669194 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.670571 30902 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.670680 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:53.671376 31112 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:53.672262 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.661145 20802 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
W20260709 12:25:53.673215 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 167814us
I20260709 12:25:53.674281 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:53.674918 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.705607 21430 raft_consensus.cc:1217] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Deduplicated request from leader. Original: 31.20->[32.21-32.21]   Dedup: 32.21->[]
I20260709 12:25:53.705999 21430 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:25:53.677690 21426 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 22: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:53.677508 21434 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:53.674748 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.679270 31004 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:53.679883 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.707790 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.680608 30950 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.681130 31215 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:53.712795 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.681213 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.683092 30186 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:53.683147 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:53.716063 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:53.716342 30186 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:53.683526 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:53.689498 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:53.689839 20729 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:25:53.718016 30353 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:53.718312 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:53.690335 20876 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:53.692373 30859 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.693188 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.692981 31137 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.721575 30923 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:53.722154 30923 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.722522 30923 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:53.694489 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:53.694689 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:53.724072 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:53.695305 31236 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.716s	user 0.005s	sys 0.000s
I20260709 12:25:53.724509 20806 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:53.724676 20728 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:53.688905 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:25:53.684337 31246 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:53.704763 20880 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:53.704737 20802 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:53.718495 30353 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.719326 20875 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:53.720214 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.774444 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.775000 20876 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:53.721091 30935 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:53.720966 30994 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:53.721640 31029 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:53.795284 31029 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:53.795789 31029 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:53.696302 31235 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.767s	user 0.001s	sys 0.000s
I20260709 12:25:53.693879 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.739665 31250 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:53.745258 20877 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:53.747766 31137 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.746068 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:53.750203 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:53.750415 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1838) took 1746 ms. Trace:
I20260709 12:25:53.715852 30857 raft_consensus.cc:2764] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
W20260709 12:25:53.750916 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60476us
I20260709 12:25:53.712265 31004 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.759531 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:53.804060 20806 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:53.723424 31215 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:53.682173 31056 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.725497 20732 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:53.811239 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39562 (request call id 11) took 1884 ms. Trace:
I20260709 12:25:53.762296 20728 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:25:53.811802 20734 rpcz_store.cc:276] 0709 12:25:51.915037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.915301 (+   264us) service_pool.cc:224] Handling call
0709 12:25:53.799008 (+1883707us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001d6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca5735 00007f9e1ec9d9fa 00007f9e23c654c9 00007f9e1d1e1c54 00007f9e1d1e1be2 00007f9e1d1e1b11 00007f9e1d1e1a8c 00007f9e1d1e0d42 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:53.799285 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348289280}
I20260709 12:25:53.745837 31045 raft_consensus.cc:2764] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:25:53.772581 30186 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:53.763535 30997 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
I20260709 12:25:53.782526 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:53.783077 20876 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:53.816082 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.751960 31217 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:25:53.816743 20876 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:53.817101 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.817657 20883 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:53.784610 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1839) took 1779 ms. Trace:
I20260709 12:25:53.803637 31243 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:53.799953 20884 rpcz_store.cc:276] 0709 12:25:52.004197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.099945 (+ 95748us) service_pool.cc:224] Handling call
0709 12:25:52.100323 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.100870 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.100901 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.693959 (+593058us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:52.694365 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.761491 (+ 67126us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:52.761527 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.761600 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.663976 (+902376us) spinlock_profiling.cc:243] Waited 61.4 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:53.750210 (+ 86234us) raft_consensus.cc:1685] finished
0709 12:25:53.750230 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.750359 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5665,"spinlock_wait_cycles":153683584,"thread_start_us":66806,"threads_started":1,"wal-append.queue_time_us":199156}
I20260709 12:25:53.821112 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:53.819674 20878 rpcz_store.cc:276] 0709 12:25:52.005013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.333777 (+328764us) service_pool.cc:224] Handling call
0709 12:25:52.334167 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.740053 (+405886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.740121 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.740748 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.805544 (+ 64796us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:52.805589 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.805658 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.575002 (+769344us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:53.769885 (+194883us) raft_consensus.cc:1685] finished
0709 12:25:53.769918 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.784525 (+ 14607us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":494676,"raft.queue_time_us":418092,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":14168,"spinlock_wait_cycles":48533376,"thread_start_us":64438,"threads_started":1,"wal-append.queue_time_us":181305}
I20260709 12:25:53.782032 20802 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.823071 20802 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:53.802788 31139 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.547s	user 0.004s	sys 0.008s
I20260709 12:25:53.829906 20876 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:53.834161 20878 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:53.807205 31004 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:53.838918 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:53.839833 20880 raft_consensus.cc:606] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:25:53.840283 20880 raft_consensus.cc:614] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:53.805059 31175 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:53.820597 31183 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.820425 31250 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:53.846661 20878 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:53.844751 30353 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:53.846609 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1929) took 1175 ms. Trace:
I20260709 12:25:53.825367 30935 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:53.830060 30950 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:53.848677 20878 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:25:53.847810 21429 rpcz_store.cc:276] 0709 12:25:52.671611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.671738 (+   127us) service_pool.cc:224] Handling call
0709 12:25:52.700912 (+ 29174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.719494 (+ 18582us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:52.719526 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.719992 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.787904 (+ 67912us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:52.787945 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.788013 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.695681 (+907668us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:53.842967 (+147286us) raft_consensus.cc:1685] finished
0709 12:25:53.842989 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.846541 (+  3552us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17990,"raft.queue_time_us":22412,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":16122,"spinlock_wait_cycles":160315136,"thread_start_us":67591,"threads_started":1,"wal-append.queue_time_us":145130}
W20260709 12:25:53.825531 30845 consensus_queue.cc:1175] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 33 status { last_received { term: 33 index: 22 } last_committed_idx: 20 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.21-32.21]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:53.841543 31139 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29.
I20260709 12:25:53.846333 31153 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.849340 30935 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:53.885838 31094 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:53.886423 31094 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:53.878734 30936 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.859807 30950 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:53.847505 31175 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.850441 31183 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:53.891335 31256 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:25:53.880398 30935 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:53.892244 31257 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:25:53.897780 31250 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.911864 31183 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:53.912510 31184 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.914180 31153 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:53.853240 31215 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.914321 30950 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:53.914898 31094 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:53.915375 31215 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:53.915935 31094 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:53.909482 31175 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:53.801858 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55870us
W20260709 12:25:53.968178 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.980962 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:53.990721 20802 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
W20260709 12:25:53.996934 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 158748us
W20260709 12:25:54.001231 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 250935us
I20260709 12:25:54.008275 20732 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:25:54.003006 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:53.982738 20878 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:54.016575 30929 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.018321 31192 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:54.018079 31004 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.019863 31046 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.030115 31264 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.030019 30929 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.033030 31262 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:54.035706 31192 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.091702 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.092965 21427 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:54.093012 31264 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.095180 31046 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:54.097395 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94737.4us
I20260709 12:25:54.107959 31183 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:54.110061 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:54.111214 31175 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.112299 31175 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.112998 31029 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.111490 21427 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:54.115520 31262 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.118732 20808 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.119817 31175 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.121999 31192 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.124155 31264 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:54.128566 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:54.180145 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51974.6us
I20260709 12:25:54.181388 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.129565 30789 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:25:54.128607 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.130470 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.131803 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.131758 20883 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:25:54.184122 30929 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:54.185376 20726 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:54.187599 31137 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:54.188293 20883 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
W20260709 12:25:54.135934 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.135914 31029 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.136698 31262 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.191073 31262 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.191200 31175 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.191493 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:54.191838 31262 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.192126 31175 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:54.139688 30916 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 33 current term: 33 is higher.
I20260709 12:25:54.195083 31262 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:54.196674 31004 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.141782 30935 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.198148 31175 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.198278 30929 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.141054 30186 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:54.195760 21429 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
W20260709 12:25:54.143229 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 353073us
I20260709 12:25:54.200084 30935 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:54.102487 20807 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.130918 20884 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:54.210486 30929 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.212208 31175 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.212947 31175 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.213721 30929 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.201222 30186 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:54.098701 31046 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.222469 30186 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.224915 30186 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:54.228372 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.228025 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:33470 (request call id 3907) took 1336 ms. Trace:
I20260709 12:25:54.281893 30845 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:54.282992 30845 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:54.302579 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.308701 20878 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
W20260709 12:25:54.304551 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 194633us
I20260709 12:25:54.259379 20808 rpcz_store.cc:276] 0709 12:25:52.869700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.871910 (+  2210us) service_pool.cc:224] Handling call
0709 12:25:54.116844 (+1244934us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b54003c7154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec96a3d 00007f9e23c66cd0 00007f9e1d1eb254 00007f9e1d1eb1e2 00007f9e1d1eb111 00007f9e1d1eb08c 00007f9e1d1ea342 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:54.202577 (+ 85733us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91766016}
I20260709 12:25:54.319324 31153 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:54.320238 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3744) took 1443 ms. Trace:
I20260709 12:25:54.320377 31153 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.320751 20882 rpcz_store.cc:276] 0709 12:25:52.877182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.833641 (+956459us) service_pool.cc:224] Handling call
0709 12:25:53.837072 (+  3431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.837276 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:53.852053 (+ 14777us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.319410 (+467357us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:54.319544 (+   134us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:54.319692 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.319807 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.320006 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":182392,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":73,"thread_start_us":456987,"threads_started":1,"wal-append.queue_time_us":466616}
W20260709 12:25:54.323400 20806 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.010s
I20260709 12:25:54.324014 20806 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:25:54.324548 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.325030 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.325539 20881 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:54.326187 21429 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:54.330575 30928 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:54.320643 30950 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.309062 31137 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:54.327549 30840 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:54.338407 30950 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.342171 30950 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:54.343372 20875 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.535s	user 0.017s	sys 0.001s
I20260709 12:25:54.343637 31153 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.344208 20875 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:54.350121 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.336954 30840 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:54.351914 30840 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:54.352051 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.353194 31153 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.353766 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:54.353998 31153 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.356938 30960 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:54.358036 30960 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.357051 31175 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.358758 30960 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.360800 31153 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.362712 30960 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.364353 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.365691 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.366711 31153 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:54.380081 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.380743 31153 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:54.381953 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51030.4us
I20260709 12:25:54.367048 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.383288 30929 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:54.383452 31157 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:54.368305 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.385239 20802 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:54.369242 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.370708 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.373394 31215 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.375509 30923 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.627s	user 0.016s	sys 0.004s
I20260709 12:25:54.349421 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.388903 20878 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:54.313877 31183 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:54.411420 20805 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:54.423287 30950 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.426111 30950 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.426554 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3745) took 1439 ms. Trace:
W20260709 12:25:54.427634 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 74069.1us
I20260709 12:25:54.426990 20876 rpcz_store.cc:276] 0709 12:25:52.877482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.836377 (+958895us) service_pool.cc:224] Handling call
0709 12:25:53.837544 (+  1167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.838330 (+   786us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:25:53.965481 (+127151us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.258662 (+293181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:54.295371 (+ 36709us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:25:54.301863 (+  6492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.307425 (+  5562us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.317099 (+  9674us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29056,"prepare.queue_time_us":181840,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":74,"raft.queue_time_us":36998,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":51063,"thread_start_us":251451,"threads_started":1}
I20260709 12:25:54.428532 30950 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.428865 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:54.443527 31250 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.444276 31250 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.445264 30845 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.447785 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.448954 21434 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:54.451309 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.428675 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:54.454975 20882 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:25:54.451941 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.463925 31279 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:54.464629 31279 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.468465 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.469635 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.466955 31279 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.413265 30923 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.450124 31153 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.470296 20805 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
W20260709 12:25:54.472332 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.453827 31250 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.471642 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3590) took 1306 ms. Trace:
I20260709 12:25:54.522485 20730 rpcz_store.cc:276] 0709 12:25:53.164733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.200073 (+ 35340us) service_pool.cc:224] Handling call
0709 12:25:54.458342 (+1258269us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b5400298bd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:54.471404 (+ 13062us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17820416}
I20260709 12:25:54.524711 20732 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
W20260709 12:25:54.479504 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 516958us
I20260709 12:25:54.481941 31183 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.532301 20880 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 13. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:25:54.480285 31184 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.462831 30840 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.492893 30859 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:54.493603 31153 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:54.458005 31215 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.474066 31046 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:25:54.495873 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 670880us
I20260709 12:25:54.497933 30789 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:54.486685 31171 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.501863 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50335.6us
I20260709 12:25:54.545673 20807 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:54.546047 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:54.511821 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:54.511889 30923 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.518218 20802 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:54.548523 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 136460us
I20260709 12:25:54.517831 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.540169 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:54.555519 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.555967 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:54.557197 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.557387 30859 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.558037 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.559389 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.557217 31274 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 34.23, Next index: 24, Last known committed idx: 21, Time since last communication: 0.050s
I20260709 12:25:54.560940 31274 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:54.561946 31274 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:54.562399 20883 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:54.563511 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:54.563423 31274 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.561944 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:54.559670 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 144665us
I20260709 12:25:54.568295 30186 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:54.569294 30186 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.569715 30186 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:54.565511 31274 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:54.571250 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.572578 20808 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:54.578665 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:54.580639 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152096us
I20260709 12:25:54.578716 20730 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:25:54.582242 20882 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:54.587348 31276 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.050s	user 0.000s	sys 0.002s
W20260709 12:25:54.594025 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87759.9us
I20260709 12:25:54.598037 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:54.598563 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.601063 31177 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:54.601121 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.571866 31184 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.616844 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3736) took 1749 ms. Trace:
I20260709 12:25:54.617380 31184 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.617300 20881 rpcz_store.cc:276] 0709 12:25:52.867625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.509962 (+642337us) service_pool.cc:224] Handling call
0709 12:25:54.616601 (+1106639us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:54.616749 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086333824}
I20260709 12:25:54.621707 30789 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:54.623107 30789 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:54.615044 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.627777 30186 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.624857 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:54.648356 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.649214 21434 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:25:54.623624 30789 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:54.654160 30186 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:54.656558 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54260.3us
I20260709 12:25:54.603849 31177 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.051s
I20260709 12:25:54.597199 31171 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.665738 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:54.666625 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65677.7us
I20260709 12:25:54.667451 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.667997 30936 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.653412 31004 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:54.663978 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.671681 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:54.672793 31094 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.005s
I20260709 12:25:54.674682 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.675473 20806 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:54.678234 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.678982 20810 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:54.679291 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.679803 21434 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 12:25:54.679952 20803 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:54.680087 21430 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:54.681130 30845 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:54.681715 30845 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:54.682749 30845 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.683975 30840 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.684284 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.684821 30994 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:54.686677 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.655781 31158 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:54.687520 21431 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:54.688968 30789 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.689636 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.690557 30789 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:54.669396 31215 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.691798 30928 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:54.608386 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:54.695837 20878 raft_consensus.cc:606] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:54.696702 31274 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.696959 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.697666 20801 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:54.697396 31274 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.699808 31274 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:54.700614 21431 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 LEARNER]: Advancing to term 26
I20260709 12:25:54.702219 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:54.703542 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:54.704217 21434 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:54.704588 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.705508 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.707094 31215 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:54.707536 31215 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.707890 31215 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:54.708662 30859 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.711722 31274 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:54.712415 31274 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.712962 21435 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:54.713852 20878 raft_consensus.cc:614] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:54.716109 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:54.696753 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:54.724428 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 266904us
I20260709 12:25:54.724699 31004 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:54.730263 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.731415 31153 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:54.731961 31153 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:54.732400 31153 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:54.734498 31056 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:25:54.735224 31056 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:54.736678 31056 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.739554 20809 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:54.740204 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43596.2us
I20260709 12:25:54.724498 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:54.744196 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79385us
I20260709 12:25:54.747121 20806 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:54.758502 20802 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:54.758742 20803 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:25:54.763609 20801 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:54.775121 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:54.776687 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:54.778479 30937 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.779335 20808 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:54.791361 20809 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:54.804325 20805 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:25:54.807003 31277 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.052s
I20260709 12:25:54.809779 20807 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 19. (term mismatch)
I20260709 12:25:54.755954 31274 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.808905 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:54.817180 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 130489us
I20260709 12:25:54.820662 20807 pending_rounds.cc:85] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 18
I20260709 12:25:54.825762 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.840237 20801 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
W20260709 12:25:54.845574 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 120198us
I20260709 12:25:54.839774 20807 pending_rounds.cc:107] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 29.19
I20260709 12:25:54.850102 20810 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:25:54.851323 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.855474 30840 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:54.855932 20801 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:54.856411 20801 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:54.856690 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.847450 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:54.857501 20801 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.858418 31113 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:54.859344 31113 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:54.859827 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.858594 20801 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:54.861073 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.848064 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:54.849949 20807 raft_consensus.cc:2889] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:54.770957 31279 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.074s
I20260709 12:25:54.862461 20726 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:25:54.856289 31158 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:25:54.856698 31283 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:25:54.850955 31137 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.876020 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3674) took 1762 ms. Trace:
I20260709 12:25:54.884690 20810 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:25:54.881126 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 201550us
I20260709 12:25:54.849592 30789 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:54.883100 20807 rpcz_store.cc:276] 0709 12:25:53.110230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.240400 (+130170us) service_pool.cc:224] Handling call
0709 12:25:53.240817 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.101267 (+860450us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b5400357120. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1d06f 00007f9e1ec95f60 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62
0709 12:25:54.503135 (+401868us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b58007c10d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:25:54.866304 (+363169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.872263 (+  5959us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409542784}
W20260709 12:25:54.887981 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.931499 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.887730 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:54.876327 30840 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:54.871097 31158 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.945472 31158 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:54.945818 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:54.946528 20810 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:25:54.951812 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.953985 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:54.957973 30840 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.959542 30186 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:54.959487 31094 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:54.973587 31153 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.976203 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:54.886894 31137 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.980355 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.981614 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.982362 31137 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:54.912528 30789 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.983875 31286 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:54.984076 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:54.984599 31286 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:54.985205 30789 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:54.985356 31291 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:54.860466 31113 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:54.936501 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 181215us
I20260709 12:25:54.988722 20880 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:54.859308 30937 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.943802 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:54.936997 30705 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 30 current term: 30 is higher.
I20260709 12:25:54.992583 31288 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:54.993088 31288 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:55.001574 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47751us
I20260709 12:25:54.981647 30859 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:55.008540 20802 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:54.988448 31291 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:55.001214 31288 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:54.985844 31286 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:54.998834 30929 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.014803 20801 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 16. (term mismatch)
I20260709 12:25:55.021282 20808 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:55.025683 20801 pending_rounds.cc:85] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 15
I20260709 12:25:55.035302 20801 pending_rounds.cc:107] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 28.16
I20260709 12:25:55.036079 20801 raft_consensus.cc:2889] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.037736 31292 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:55.038208 31291 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.039001 31264 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:55.037789 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 172275us
I20260709 12:25:55.040061 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.040928 20805 raft_consensus.cc:2410] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:55.042092 20805 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:55.043308 30929 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:54.998994 31274 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.017s
I20260709 12:25:55.052982 31029 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:55.055864 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:55.059878 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.013877 31157 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:25:55.064477 30937 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.049525 31171 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.033943 30859 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:55.065523 30929 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:55.066407 30859 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:55.067453 31154 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:55.067982 31154 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.068423 31154 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:55.066787 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.071473 31153 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.072157 20732 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 32 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:25:55.066408 30928 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:25:55.078094 31158 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.079603 20809 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:55.081634 31158 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.085793 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.087013 20808 raft_consensus.cc:2410] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 34 because replica has last-logged OpId of term: 32 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 20.
I20260709 12:25:55.088178 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.127846 31288 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:55.130266 31288 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:54.859026 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:55.136905 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.135914 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 232012us
I20260709 12:25:55.131362 31288 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 12:25:55.174345 31288 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:55.174714 31288 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:55.175639 31288 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.176890 31288 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:55.178399 31274 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:55.179019 31274 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.139765 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.090821 31137 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:25:55.181219 31137 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:55.181653 31137 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:55.183688 30936 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:55.153029 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.189002 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:25:55.187209 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 171243us
I20260709 12:25:55.120134 31046 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:25:55.181244 31274 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.192173 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.182693 31137 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:55.189312 31183 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:55.192610 30937 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:55.204797 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90458.5us
I20260709 12:25:55.206171 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:55.208382 30994 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:55.208786 30994 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.209129 30994 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:55.200928 31292 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:25:55.218820 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 206164us
W20260709 12:25:55.219357 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 197218us
I20260709 12:25:55.196661 30936 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:55.223693 30936 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:55.221086 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.202445 31183 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.205008 31275 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:55.233464 31264 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:55.240715 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:55.242635 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.243747 31291 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:55.244344 31291 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:55.245285 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:55.245929 21431 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:55.245419 31291 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.192028 31215 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.210444 30937 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.243481 31307 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:55.247159 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:55.235837 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:55.233794 31183 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:55.201970 30789 consensus_queue.cc:1175] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "657bc62d20374e539b23641715f22745" responder_term: 27 status { last_received { term: 27 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 27 index: 16 } }
I20260709 12:25:55.236912 31153 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:55.270484 20803 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:55.205497 31154 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:55.270236 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:25:55.274358 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.274507 21427 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:55.263446 30937 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:55.230618 31292 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:55.276650 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.287369 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.277531 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73799.4us
I20260709 12:25:55.292438 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:55.292927 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.296237 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.297036 20809 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:55.296981 31154 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:55.266525 31275 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:55.298220 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.268339 31277 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:55.252506 31264 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.309758 20884 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:55.272637 30859 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:55.266101 30705 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:25:55.314649 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.268904 31286 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.020s
I20260709 12:25:55.321326 21431 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:55.334760 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.335891 20808 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:25:55.322551 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:55.337546 20805 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:25:55.338785 20880 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:55.329263 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.340204 20882 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:25:55.311152 31056 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:55.290104 31153 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.334481 31274 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.334250 31171 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:55.365993 31223 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:55.366636 31223 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:55.393399 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 84163.3us
I20260709 12:25:55.402525 20880 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:25:55.412427 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.413506 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:55.425714 31223 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.345975 31316 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:55.427098 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:55.427567 31316 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:55.428138 31315 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:25:55.428689 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61868.1us
W20260709 12:25:55.429473 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 137154us
I20260709 12:25:55.429615 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:55.429885 31316 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.430289 21431 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:55.341507 31277 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.431243 31139 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.431943 31139 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.427120 31215 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.427966 21426 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:55.433113 31277 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.438690 20805 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:25:55.454061 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:55.454950 20806 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:55.368053 30859 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:55.480492 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49352.3us
I20260709 12:25:55.435822 31250 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:25:55.463409 31304 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:55.492798 21431 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:55.463778 31184 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.463486 31277 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:55.503722 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:55.367611 31275 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.501276 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:55.514645 21426 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:55.470793 31274 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.516286 20733 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:55.523396 31112 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:25:55.524184 31313 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:25:55.525236 31184 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.525841 31205 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:55.526262 31205 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.526595 31205 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:55.526813 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.527415 31139 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.527616 31184 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.542913 31264 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:55.544315 31183 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.545199 31153 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.546696 31183 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:55.546880 31323 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:55.508225 31277 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.549599 31277 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.514639 31304 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.553555 31304 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.558926 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.559373 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.560158 20806 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:25:55.525482 31308 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.567685 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:55.568514 21426 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:25:55.568250 31308 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:55.578454 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.580893 31307 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:55.581393 31307 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:55.571022 31308 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.509450 30859 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:25:55.608860 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 118841us
I20260709 12:25:55.614241 30859 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.616072 20805 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:25:55.556540 31274 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.628662 31177 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:55.631740 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:55.634835 20730 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:25:55.634995 20732 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:25:55.635389 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.636416 20733 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:55.636644 20729 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:25:55.640234 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:55.641044 20734 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:55.642370 20726 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:25:55.643858 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.644778 20734 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:55.582397 31307 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.647688 31307 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:55.648221 21426 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:25:55.651856 31324 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.652495 31324 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.656349 30859 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.657006 30859 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.657480 31205 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.616400 31323 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:55.677456 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.680117 31153 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:55.680790 31153 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:55.683753 20629 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 14 to 17, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 17 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:55.681953 31153 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.691375 30789 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:55.692160 30789 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.694628 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.696190 31250 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:55.696810 31250 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:55.697947 31250 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:55.731609 20775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.515s (SENT) [suppressed 11 similar messages]
I20260709 12:25:55.747264 31205 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:55.748682 30994 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.001s
I20260709 12:25:55.749349 30994 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:55.567400 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 103438us
I20260709 12:25:55.820192 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.821141 30994 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.823524 30705 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:55.824546 30705 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:55.825727 30705 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:55.654913 31324 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.755752 30929 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.770864 30859 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.835520 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.836576 20881 raft_consensus.cc:2368] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
W20260709 12:25:55.837443 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.837459 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.838738 20880 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:55.778618 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.778039 31323 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:55.775883 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:55.840731 21426 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:25:55.780270 31318 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.784057 30789 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:55.844221 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.784581 31137 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:55.788445 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.788341 31205 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:55.794574 20807 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:25:55.796308 20806 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:25:55.847436 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.797359 20801 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:55.852453 20884 raft_consensus.cc:2368] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:25:55.853111 20876 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:25:55.799297 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.800505 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.800777 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.802289 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:55.802493 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.803270 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.803984 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.804744 20735 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:25:55.805653 30937 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.815625 31157 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.867141 20729 raft_consensus.cc:2368] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:25:55.565821 21427 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:55.848343 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.798489 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:55.864501 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.855392 20733 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:55.854533 20726 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
W20260709 12:25:55.877040 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 88660.8us
I20260709 12:25:55.864181 30898 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:55.879494 30898 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:55.880594 30937 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:55.882192 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.883037 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.883795 20884 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:55.885052 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.885751 20878 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:55.886128 30859 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:55.887616 30845 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:55.888000 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.888819 20883 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:55.888967 20731 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:55.831704 30929 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.890039 20774 leader_election.cc:400] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:25:55.888123 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.890554 20774 leader_election.cc:403] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.883625 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:55.891220 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:55.888043 30845 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:55.892108 30845 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:55.889174 20877 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:25:55.895872 20880 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:25:55.881701 20629 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:55.898339 20884 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:55.883651 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:55.900027 20882 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:55.897567 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:55.917691 21426 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:55.933698 20629 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 17 to 19, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 19 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:55.938892 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:55.938922 20881 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
W20260709 12:25:55.928227 21398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.004s	sys 0.004s
W20260709 12:25:55.943470 21398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.004s	sys 0.004s
W20260709 12:25:55.944473 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.946385 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.003s	sys 0.006s
W20260709 12:25:55.946822 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.003s	sys 0.006s
I20260709 12:25:55.901350 31205 consensus_queue.cc:1243] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 27.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:55.914876 31323 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:25:55.928097 31215 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:25:55.963670 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:55.964408 21427 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:55.901450 30840 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.053s
I20260709 12:25:55.938118 30789 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:55.968019 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.009s
W20260709 12:25:56.034277 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.002s	sys 0.010s
W20260709 12:25:55.995219 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.965813 20731 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
W20260709 12:25:55.967685 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.000s	sys 0.012s
W20260709 12:25:56.035856 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.005s	sys 0.018s
W20260709 12:25:56.036288 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 641521us
I20260709 12:25:56.054200 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.056291 20726 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:25:56.057912 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:55.918500 30859 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:25:55.897261 31318 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.917280 31157 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:55.957096 30994 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:56.055167 20729 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:56.072413 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:55.963517 31205 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:25:56.065483 20810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.005s	sys 0.026s
W20260709 12:25:56.076406 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 186601us
I20260709 12:25:56.078665 21429 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:56.084369 20810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.006s	sys 0.027s
I20260709 12:25:56.080188 20726 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:56.095621 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:56.098277 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:55.965359 31056 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:56.106427 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:56.121330 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:56.123422 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:56.131227 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46298.2us
I20260709 12:25:56.096302 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
W20260709 12:25:56.137714 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54796.3us
I20260709 12:25:56.139056 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:56.139148 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:56.124382 20884 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:56.146811 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:56.147519 21430 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 22.
I20260709 12:25:56.155375 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:56.168006 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:56.076188 31331 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.086011 31275 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.077314 30929 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:56.167865 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.163620 31056 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.178121 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:56.186277 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:56.187295 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:56.185427 21430 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:56.190186 20884 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 36 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 22.
I20260709 12:25:56.190363 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:56.191129 20730 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:56.151415 31046 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:56.179456 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:56.231029 20734 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:25:56.102293 30994 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.078512 30789 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.234803 20735 raft_consensus.cc:2368] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:25:56.239588 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:56.241782 20629 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 17 to 23, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 23 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:56.173018 31250 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.255578 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 166147us
I20260709 12:25:56.256526 20774 leader_election.cc:400] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:25:56.256951 20774 leader_election.cc:403] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:56.258960 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:56.270798 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:56.265604 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:25:56.221546 31309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.001s	sys 0.005s
W20260709 12:25:56.288250 31309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.001s	sys 0.007s
W20260709 12:25:56.290879 31327 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:25:56.293226 30840 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:25:56.287111 21430 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:56.287369 30937 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:56.272166 31334 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:56.184073 31331 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.286562 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.301513 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:56.311997 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:56.293256 31328 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
I20260709 12:25:56.191603 31275 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.316318 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:56.300403 30937 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.355517 30937 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:56.304091 31334 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.294032 30840 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:25:56.246646 31324 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.073s
I20260709 12:25:56.233548 31046 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:56.308087 31094 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.378456 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.321795 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:56.359561 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 305) took 38197 ms (client timeout 15000 ms). Trace:
W20260709 12:25:56.379994 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 482486us
W20260709 12:25:56.400231 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.273998 31250 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.342060 31215 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:56.384476 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:56.409286 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:56.376690 31334 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:56.424665 20884 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:56.391884 31316 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:56.430454 30994 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:56.434602 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 266106us
I20260709 12:25:56.441895 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:56.496615 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54832.9us
I20260709 12:25:56.497084 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1927) took 1208 ms. Trace:
I20260709 12:25:56.493866 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:56.527864 21429 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:56.497515 20802 rpcz_store.cc:276] 0709 12:25:55.288526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.288696 (+   170us) service_pool.cc:224] Handling call
0709 12:25:55.315729 (+ 27033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.323058 (+  7329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.323090 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.333258 (+ 10168us) log.cc:844] Serialized 26 byte log entry
0709 12:25:55.494536 (+161278us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:55.494575 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.494635 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.496851 (+1002216us) raft_consensus.cc:1685] finished
0709 12:25:56.496877 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.497017 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7431,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":9206,"thread_start_us":113133,"threads_started":1,"wal-append.queue_time_us":500164}
I20260709 12:25:56.533136 20877 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:56.405184 31094 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:56.497764 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:56.570011 20629 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 15 to 20, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 20 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:56.572764 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75107us
W20260709 12:25:56.578141 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 410177us
I20260709 12:25:56.587141 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:56.590812 20882 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.003s
W20260709 12:25:56.571727 30845 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.004s
I20260709 12:25:56.591553 20882 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:56.529637 30936 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election won for term 35
W20260709 12:25:56.597337 20733 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.003s	sys 0.006s
W20260709 12:25:56.599682 20733 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.005s	sys 0.007s
I20260709 12:25:56.597146 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:56.600652 20629 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 20 to 23, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:25:56.526424 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.620098 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 298372us
I20260709 12:25:56.577647 31304 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.579025 31323 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:25:56.569694 31205 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:56.590895 30840 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:56.624787 31324 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:56.530444 31316 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.626792 30936 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:56.593827 31337 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.067s
I20260709 12:25:56.655678 30928 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:25:56.600759 30845 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:56.673671 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 273480us
I20260709 12:25:56.693775 20629 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 10 to 18. New cstate: current_term: 18 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:56.703482 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:56.675376 31331 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:56.693810 31223 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:56.708091 31223 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:56.708384 31347 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:25:56.710450 31223 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:56.689677 31339 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.093s	user 0.001s	sys 0.001s
W20260709 12:25:56.714262 31307 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.066s	user 0.013s	sys 0.008s
I20260709 12:25:56.715178 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
W20260709 12:25:56.396981 20628 rpcz_store.cc:269] 0709 12:25:17.958109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.774377 (+4816268us) service_pool.cc:224] Handling call
0709 12:25:56.138155 (+33363778us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:45.422369 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:45.423078 (+   709us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:45.423135 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:25:45.423154 (+    19us) tablet.cc:662] Decoding operations
0709 12:25:45.424623 (+  1469us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:45.424719 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:45.424748 (+    29us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:25:45.426151 (+  1403us) tablet.cc:701] Row locks acquired
0709 12:25:45.426212 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:25:45.438945 (+ 12733us) write_op.cc:270] Start()
0709 12:25:45.439109 (+   164us) write_op.cc:276] Timestamp: P: 1783599945438864 usec, L: 0
0709 12:25:45.439170 (+    61us) op_driver.cc:348] REPLICATION: starting
0709 12:25:45.440006 (+   836us) log.cc:844] Serialized 6385 byte log entry
0709 12:25:46.454507 (+1014501us) op_driver.cc:464] REPLICATION: finished
0709 12:25:46.873983 (+419476us) write_op.cc:301] APPLY: starting
0709 12:25:46.892741 (+ 18758us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:47.360191 (+467450us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:47.360315 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:47.551939 (+191624us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:47.552322 (+   383us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:25:47.552365 (+    43us) write_op.cc:312] APPLY: finished
0709 12:25:47.552808 (+   443us) log.cc:844] Serialized 148 byte log entry
0709 12:25:47.581534 (+ 28726us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:47.609169 (+ 27635us) write_op.cc:454] Released schema lock
0709 12:25:47.664203 (+ 55034us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:47.709986 (+ 45783us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:47.710017 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":93971,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":344614,"apply.run_cpu_time_us":49018,"apply.run_wall_time_us":835977,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71007,"num_ops":16,"prepare.queue_time_us":458337,"prepare.run_cpu_time_us":135545,"prepare.run_wall_time_us":219338,"raft.queue_time_us":558054,"raft.run_cpu_time_us":40648,"raft.run_wall_time_us":393730,"replication_time_us":961490,"thread_start_us":860144,"threads_started":5,"wal-append.queue_time_us":240503,"wal-append.run_cpu_time_us":22263,"wal-append.run_wall_time_us":2596369}]]}
I20260709 12:25:56.703608 30928 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:56.720736 30928 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:56.690096 31355 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:56.722573 31355 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:56.656942 31304 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.723767 31205 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.725210 31304 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:56.726518 20804 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:56.723042 31355 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:56.737402 20629 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:56.744127 20730 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.004s
I20260709 12:25:56.744664 20730 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
W20260709 12:25:56.744099 20726 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.000s
I20260709 12:25:56.745772 20726 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:56.663646 30840 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:56.757514 21434 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
W20260709 12:25:56.746009 31336 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:25:56.766386 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 126726us
I20260709 12:25:56.746371 31177 raft_consensus.cc:2764] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
W20260709 12:25:56.761358 31335 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.368s	user 0.000s	sys 0.002s
I20260709 12:25:56.772210 21427 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:25:56.755378 31046 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:56.766525 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 113866us
I20260709 12:25:56.785075 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.767745 31275 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:56.799809 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.778374 30936 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:56.802651 31250 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:56.803395 31094 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:56.804320 30845 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:56.806612 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3933) took 1990 ms. Trace:
I20260709 12:25:56.787079 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.807081 20810 rpcz_store.cc:276] 0709 12:25:54.816123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.947237 (+131114us) service_pool.cc:224] Handling call
0709 12:25:54.947624 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:55.218750 (+271126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.218781 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:55.234729 (+ 15948us) log.cc:844] Serialized 52 byte log entry
0709 12:25:56.092885 (+858156us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:56.092945 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.093038 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.806347 (+713309us) raft_consensus.cc:1685] finished
0709 12:25:56.806381 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.806550 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":287082,"raft.queue_time_us":270782,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5307,"thread_start_us":821984,"threads_started":1,"wal-append.queue_time_us":1020215}
I20260709 12:25:56.808686 20726 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:25:56.790458 30898 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:56.790684 31316 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:56.838546 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.846292 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:56.847831 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:56.849304 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:56.774951 31223 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:25:56.776494 31353 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:25:56.855217 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:56.866425 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.855849 31177 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:25:56.871652 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 30 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:56.845360 30845 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:56.855309 31288 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:56.889454 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.872416 31323 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:25:56.905355 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:56.909812 31334 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:56.913208 31349 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:25:56.917589 31334 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:56.914832 31355 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:56.928332 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 158568us
I20260709 12:25:56.896814 30845 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:56.925426 30937 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
I20260709 12:25:56.947666 21429 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:25:56.949659 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:56.954674 21435 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:56.950601 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:56.956470 21434 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:56.908237 31250 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:56.977969 21435 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:25:56.979676 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:56.986622 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:56.985643 31348 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:56.987522 31357 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:25:56.967576 31250 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.021440 31250 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:56.986145 31223 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.036s
I20260709 12:25:56.984454 31355 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:56.998250 21435 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:56.965195 30937 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.034133 30937 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:57.035578 31343 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.505s	user 0.004s	sys 0.000s
I20260709 12:25:57.034967 31348 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:57.038542 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3605) took 1236 ms. Trace:
I20260709 12:25:57.037874 31348 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.004179 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:57.039909 21433 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:57.004855 31376 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 12:25:57.007376 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:57.008636 31370 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:25:57.009074 30845 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:57.014075 31342 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.405s	user 0.002s	sys 0.000s
I20260709 12:25:57.015450 21430 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:57.016566 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 1917) took 1064 ms. Trace:
W20260709 12:25:57.017446 31360 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:25:57.048931 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3950) took 1008 ms. Trace:
I20260709 12:25:57.050025 30840 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.049408 20803 rpcz_store.cc:276] 0709 12:25:56.040236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.040375 (+   139us) service_pool.cc:224] Handling call
0709 12:25:56.075612 (+ 35237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.423642 (+348030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.423675 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.433041 (+  9366us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.684373 (+251332us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:56.684430 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.684525 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.048692 (+364167us) raft_consensus.cc:1685] finished
0709 12:25:57.048715 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.048864 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15414,"raft.queue_time_us":108995,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":28556,"thread_start_us":488954,"threads_started":2,"wal-append.queue_time_us":279981}
I20260709 12:25:57.053987 31377 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 12:25:57.055698 31376 consensus_queue.cc:1243] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 27.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:57.045006 21431 rpcz_store.cc:276] 0709 12:25:55.951964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.952098 (+   134us) service_pool.cc:224] Handling call
0709 12:25:55.952521 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:55.958326 (+  5805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.958374 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:56.049273 (+ 90899us) log.cc:844] Serialized 52 byte log entry
0709 12:25:56.482720 (+433447us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:56.523621 (+ 40901us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.526002 (+  2381us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.973946 (+447944us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b1800241d80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:57.016049 (+ 42103us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b54001fbe20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1a733 00007f9e1ec9fdbf 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c
0709 12:25:57.016121 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.016224 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.016279 (+    55us) raft_consensus.cc:1685] finished
0709 12:25:57.016296 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.016461 (+   165us) inbound_call.cc:177] Queueing success response
0709 12:25:57.033058 (+ 16597us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b1800241d80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
Metrics: {"mutex_wait_us":1893,"prepare.queue_time_us":88044,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"raft.queue_time_us":135397,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":9757,"spinlock_wait_cycles":43071744,"thread_start_us":527079,"threads_started":2,"wal-append.queue_time_us":598087}
I20260709 12:25:57.057932 21431 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:25:57.062062 21433 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:57.003525 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:57.038947 20732 rpcz_store.cc:276] 0709 12:25:55.802852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.802991 (+   139us) service_pool.cc:224] Handling call
0709 12:25:57.038363 (+1235372us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54000c44d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:25:57.038476 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974878592}
W20260709 12:25:57.029278 31137 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.163s	user 0.012s	sys 0.002s
I20260709 12:25:57.031245 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:25:57.071869 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68445.4us
I20260709 12:25:57.074333 31056 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.080538 21429 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:57.103741 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.114661 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.112382 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:57.115777 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.116586 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:25:57.118806 20876 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25.
W20260709 12:25:57.119347 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.122120 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.122833 21433 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:25:57.119349 20881 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 23: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:57.088511 31376 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:57.125165 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.125984 21429 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 36 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 22.
I20260709 12:25:57.131700 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.132416 20803 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:57.133204 20804 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:57.133301 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.134776 20805 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:57.133814 20804 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:57.136592 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.139487 21433 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:57.140990 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.141789 21427 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:57.142470 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.143150 21435 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:57.109814 31377 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:25:57.124146 31205 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.147329 31205 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.149437 31205 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:57.151055 20808 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:25:57.152092 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:57.154476 20775 leader_election.cc:400] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:25:57.155973 30840 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:57.156076 31376 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:57.156522 30840 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:57.156980 30840 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:57.157112 31376 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:57.157593 31376 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:57.158396 31376 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.159989 21430 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:25:57.136791 20804 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.164345 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.164855 20804 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:57.164966 20808 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:57.146903 21434 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 25.
I20260709 12:25:57.166582 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.167263 21426 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:57.168177 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.168401 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:57.169157 21429 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:57.169298 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:57.154915 31347 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:25:57.173163 31347 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:57.173890 31372 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:25:57.173686 31347 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.163055 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.176462 31347 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.177583 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:57.179139 20808 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:57.179030 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:57.178598 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.195180 20775 leader_election.cc:336] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.525s (SENT)
I20260709 12:25:57.195897 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:57.208806 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.215366 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.220497 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.221231 20730 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:25:57.221764 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.221872 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:57.224104 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 61204us
I20260709 12:25:57.224262 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:57.235742 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.237347 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:57.238029 20727 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:57.240782 20803 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:57.194892 31029 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.245846 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.246433 20734 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:57.243654 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.244164 20805 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:25:57.244987 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.256280 20802 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 18. (term mismatch)
I20260709 12:25:57.256747 20802 pending_rounds.cc:85] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 17
I20260709 12:25:57.256985 20802 pending_rounds.cc:107] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:25:57.257237 20802 raft_consensus.cc:2889] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:57.259124 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.270429 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.260871 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 22.
I20260709 12:25:57.222922 31376 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:57.272717 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3721) took 1354 ms. Trace:
I20260709 12:25:57.272807 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.277220 20806 rpcz_store.cc:276] 0709 12:25:55.917856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.918007 (+   151us) service_pool.cc:224] Handling call
0709 12:25:55.918316 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.152479 (+234163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.152529 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.194945 (+ 42416us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.652899 (+457954us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:56.652944 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.244487 (+591543us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003c7654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:57.244536 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.244590 (+    54us) raft_consensus.cc:1685] finished
0709 12:25:57.244604 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.272652 (+ 28048us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17444,"raft.queue_time_us":120557,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":16662,"spinlock_wait_cycles":497114368,"thread_start_us":356753,"threads_started":1,"wal-append.queue_time_us":523802}
I20260709 12:25:57.289075 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.289315 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.269908 31029 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.303164 31347 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:57.289075 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.314772 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.315766 20733 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:25:57.325472 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.313843 31355 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.336031 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.329959 31347 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.273993 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.274724 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:57.331271 31288 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:57.273734 20804 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
W20260709 12:25:57.347838 31112 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:25:57.348390 30789 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.347718 31288 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.337523 31355 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:25:57.364753 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 169611us
I20260709 12:25:57.279218 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.352895 31153 raft_consensus.cc:1064] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:25:57.373322 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.376550 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:57.377209 20726 raft_consensus.cc:2376] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34. Re-sending same reply.
I20260709 12:25:57.374835 20733 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
W20260709 12:25:57.382469 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.385239 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.388240 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.344772 31347 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.396816 20730 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:25:57.399012 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.406386 31029 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:57.407836 30840 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.410111 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.412700 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.380146 31215 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.415959 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:57.420454 20727 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:57.422240 20729 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:25:57.422297 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:57.423059 20733 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:57.423897 20808 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:57.402351 30705 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.430191 20804 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:57.434255 20809 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:25:57.439878 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.440876 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.442788 31377 raft_consensus.cc:1064] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:25:57.444885 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.446687 31288 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.447340 31157 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.448254 31355 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:57.450403 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:57.439069 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50646.7us
I20260709 12:25:57.470335 21433 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:57.484694 30840 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.486409 31347 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:57.426626 31215 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.489056 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.489053 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.486867 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.487645 21428 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:25:57.433352 30705 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.502647 21433 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:25:57.504781 30840 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.505398 30840 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.507675 31318 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.509032 31366 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:57.517179 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1875) took 1278 ms. Trace:
I20260709 12:25:57.518244 31352 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:25:57.519793 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:57.522823 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.476401 31346 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.045s
I20260709 12:25:57.517654 20884 rpcz_store.cc:276] 0709 12:25:56.243744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.530881 (+287137us) service_pool.cc:224] Handling call
0709 12:25:56.592498 (+ 61617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.592723 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:25:56.612589 (+ 19866us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.785549 (+172960us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.173355 (+387806us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:57.336967 (+163612us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b1800284aa0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:25:57.516638 (+179671us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b1800284aa0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:25:57.516803 (+   165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.516859 (+    56us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:25:57.516917 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.516986 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.517115 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12991,"prepare.queue_time_us":31547,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":91,"raft.queue_time_us":13908,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":130465408,"thread_start_us":149548,"threads_started":1,"wal-append.queue_time_us":231017}
I20260709 12:25:57.528539 20810 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:25:57.530907 20805 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 20. (term mismatch)
I20260709 12:25:57.531345 20805 pending_rounds.cc:85] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 19
I20260709 12:25:57.531548 20805 pending_rounds.cc:107] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 34.20
I20260709 12:25:57.531716 20805 raft_consensus.cc:2889] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:57.550665 30705 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.552549 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.554608 20733 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 23.
I20260709 12:25:57.558799 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:57.559863 31352 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:57.560259 31352 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.560596 31352 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:57.533771 31347 consensus_queue.cc:1175] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 26 status { last_received { term: 26 index: 18 } last_committed_idx: 18 last_received_current_leader { term: 26 index: 18 } }
I20260709 12:25:57.569598 31215 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.574179 31352 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.578089 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.582641 21427 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:57.595082 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:57.594003 31153 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 17, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.550796 30840 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:57.601953 31250 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:57.610031 31250 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.610405 31250 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:57.612130 30994 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.612334 31223 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:57.612831 30994 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.612953 31223 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.616941 21427 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:57.617906 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.618551 21432 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:25:57.618881 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:57.619956 20879 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:57.621273 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.622459 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:57.621994 20884 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:57.631668 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.630336 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:57.635346 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.636234 20884 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:25:57.638860 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.639310 20883 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:25:57.639536 20878 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
W20260709 12:25:57.667999 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.632611 20881 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:25:57.672447 30840 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:57.675772 30840 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:57.676206 30840 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:57.677323 30705 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.677883 30705 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.640357 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.639557 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.639865 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.618814 30928 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.684299 30928 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.686549 30928 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.687605 30994 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.681093 31356 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:57.689175 31356 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.689965 30705 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:57.627594 31334 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.627511 31223 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.697685 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.698684 20875 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:57.634370 31154 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.695664 30840 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.603585 30936 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.080s
I20260709 12:25:57.634258 31346 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.068s
I20260709 12:25:57.672596 31137 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:57.708412 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:57.682493 31046 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.695624 31356 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.735270 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:57.743556 30840 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.702270 31334 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.758688 20877 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:57.759380 31056 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:57.759601 31324 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.760183 31324 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.760694 31334 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:57.761565 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:57.762332 31324 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.762629 31334 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:25:57.763231 31334 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.763702 31334 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:25:57.766008 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 57717us
I20260709 12:25:57.713677 31137 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.771039 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:57.774794 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:57.734444 30928 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.705821 31154 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.753203 31407 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:57.780683 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:57.792240 20730 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 23: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:57.734742 31046 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.782224 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.777217 30789 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:57.783751 31352 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.784318 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.778677 30928 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.787073 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:57.770601 31411 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.788774 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.788920 20883 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 21. (term mismatch)
I20260709 12:25:57.804054 20883 pending_rounds.cc:85] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 20
I20260709 12:25:57.792346 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:57.796859 20807 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:57.781484 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.794960 20803 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:25:57.797256 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.796300 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:57.797039 20878 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:25:57.804412 20883 pending_rounds.cc:107] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 28.21
I20260709 12:25:57.807286 20883 raft_consensus.cc:2889] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:57.809072 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:57.810014 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.801326 30994 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:57.797004 31337 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:57.799849 30928 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:57.797228 31154 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:57.797351 31324 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.823246 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:57.785555 30929 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:25:57.824157 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:57.824491 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 217535us
I20260709 12:25:57.838308 20877 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:57.823769 31331 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:57.816910 30994 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.871349 20877 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:57.881839 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:57.880553 30929 consensus_queue.cc:1243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:57.892140 31137 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:57.892508 31404 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 12:25:57.871666 31403 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:25:57.794111 31407 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:57.871306 31347 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:57.910344 20877 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:25:57.914254 20883 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:25:57.892194 31406 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:25:57.838014 31334 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.811455 31411 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.917436 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:57.880409 30994 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:57.932329 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109182us
I20260709 12:25:57.935880 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:57.871088 31331 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:57.903554 31410 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:25:57.840770 31324 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:57.944290 31046 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:57.946043 31324 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:57.926831 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 125923us
I20260709 12:25:57.947039 31331 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:57.947613 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:57.951296 31250 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:57.925732 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:57.954671 20879 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:25:57.937788 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:57.955461 20807 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:57.955756 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:25:57.951872 31416 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:57.957016 31422 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:57.959098 31422 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:57.960073 20803 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:57.969324 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.003410 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.005128 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:57.973810 20877 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:25:57.980505 20807 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:25:57.926821 31333 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.020045 31333 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.021274 30923 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:25:57.932111 31418 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.023664 31418 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.024504 31250 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.972805 31422 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 12:25:57.917277 31368 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 12:25:57.938962 31352 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.950332 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:57.990159 31332 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:57.991375 31419 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.099s	user 0.000s	sys 0.002s
I20260709 12:25:57.992986 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:57.986990 30937 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:58.019244 31424 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:57.928881 31347 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.062309 31424 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.062844 31424 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.064677 31347 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:57.986105 31413 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:58.065717 31413 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.068164 31413 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.026930 31309 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:25:58.068642 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 118389us
I20260709 12:25:58.069350 31056 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:58.069551 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.069787 31056 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.070096 31334 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:58.026233 31418 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.026604 31407 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:58.016657 31324 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:58.072402 31324 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:58.072820 31324 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:58.074503 31408 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.258s	user 0.000s	sys 0.003s
W20260709 12:25:58.074635 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 81741.4us
I20260709 12:25:58.076162 31250 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:58.076443 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.077167 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.077323 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:58.077631 31411 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:58.077813 20877 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:58.077852 31353 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:58.079136 31353 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.079562 31353 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:58.054169 31352 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.081899 31352 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:58.061158 31417 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 12:25:58.040040 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:58.086355 21433 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:58.070182 31056 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:58.096496 31337 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:58.097018 31337 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:58.023612 30923 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 8 to 18, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:58.068557 31425 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.037s
I20260709 12:25:58.100656 31373 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:58.060479 31422 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:58.101430 31422 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:58.102262 31422 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.070917 31334 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:58.103547 31334 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.103729 31422 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:58.103945 31334 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:58.101069 31373 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:58.105172 31373 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.106410 31373 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:58.107515 31415 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.180s	user 0.000s	sys 0.002s
W20260709 12:25:58.109563 31412 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.189s	user 0.002s	sys 0.000s
I20260709 12:25:58.023025 31333 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.114797 30705 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.115533 30840 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:58.115362 30705 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.115988 20774 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.116008 30840 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.116864 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:58.117172 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 186223us
I20260709 12:25:58.117456 30705 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.117997 30705 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:58.118191 30840 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:58.118549 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.118042 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.118816 30840 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:58.123061 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.125723 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.135767 21433 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:58.137012 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.137790 20877 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:25:58.140681 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:58.141762 30840 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:58.118745 30705 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:58.172642 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:58.173650 21434 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.154943 31424 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.118692 31223 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:58.142623 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:58.142036 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.161669 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.159307 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.166270 31250 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:58.165840 31056 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.172705 30840 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.182992 30840 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:58.186120 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:58.186879 21433 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:58.190971 20879 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:58.189006 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46508us
I20260709 12:25:58.174734 30994 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.196610 31424 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:58.197091 30994 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:58.197388 31414 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:58.198379 31414 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:58.198735 31414 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:58.199229 20881 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:58.199623 31414 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.200084 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:58.200841 31414 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.201113 21399 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25.
I20260709 12:25:58.201462 21399 leader_election.cc:403] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:58.202031 31332 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:25:58.202397 31332 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.202775 31332 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:58.183243 31425 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:58.204294 31425 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:58.205137 31425 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.181948 31223 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.207566 31223 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.175302 30705 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:58.208169 30705 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.210777 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.213666 21434 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:58.211316 31029 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:58.188581 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:25:58.232981 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44514.4us
I20260709 12:25:58.229779 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.232594 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:58.236881 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:25:58.238991 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 212450us
I20260709 12:25:58.175277 31324 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.225914 31250 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:58.246101 31346 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:58.212208 31332 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.268430 31056 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:58.269377 31346 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:58.270526 31346 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.257169 31418 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:25:58.272624 31337 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.274755 31324 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:58.269548 31332 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.274778 21434 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:58.230968 31353 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.277400 31353 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.277257 31401 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:58.280038 31352 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:58.281033 20884 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:25:58.233737 31368 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 24. Current term is 25.
I20260709 12:25:58.285851 20882 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:58.235558 31112 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:58.245740 31029 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:58.287772 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:58.290416 20879 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:58.287809 31333 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.292411 31333 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.293371 31223 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.296195 31333 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.296403 31177 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:58.297118 31177 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.297921 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.298642 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:25:58.300143 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:58.304299 30840 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.274973 31337 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:58.319826 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 383970us
I20260709 12:25:58.321731 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.300480 31223 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:58.352945 20775 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.631s (SENT)
I20260709 12:25:58.288908 31112 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:25:58.366487 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44870.7us
I20260709 12:25:58.368737 30840 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.310954 31308 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.401190 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:58.406158 31426 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:58.406350 31288 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:58.406771 31426 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:58.407394 31426 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:58.323654 30928 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:58.407059 31288 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:58.413246 20775 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.693s (SENT)
I20260709 12:25:58.416386 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.418318 30840 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:58.371753 31332 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:58.313022 31333 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:58.433049 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:25:58.434123 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66616.4us
I20260709 12:25:58.368996 31112 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:58.371507 31368 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.051s
I20260709 12:25:58.324888 31334 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.439445 30937 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:58.422405 31177 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:58.425870 31333 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.434983 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.455741 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:58.457098 31333 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.426123 31308 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:58.464505 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 456011us
I20260709 12:25:58.466097 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:58.462239 31332 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:58.498570 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.499075 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:58.500063 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.507552 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:58.500229 20735 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:25:58.512221 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.513665 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.515717 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:58.516007 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:25:58.516781 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61097.3us
I20260709 12:25:58.517194 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.516686 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:58.494105 31356 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:58.561817 20731 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:25:58.563226 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:58.557682 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:58.565789 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:58.567962 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:58.570636 20728 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:25:58.571575 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:58.479727 31056 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.085s
I20260709 12:25:58.555235 31316 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
W20260709 12:25:58.579762 20775 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.858s (SENT)
I20260709 12:25:58.591490 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:58.593223 30840 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:58.579965 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:58.555961 30936 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.574649 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 286925us
W20260709 12:25:58.589615 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 334110us
I20260709 12:25:58.594928 30936 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.511828 31422 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.062s
I20260709 12:25:58.600193 20731 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:58.601365 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:58.609666 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:58.610575 20728 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:58.567319 31424 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.617622 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46119.1us
W20260709 12:25:58.620021 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 372092us
I20260709 12:25:58.624548 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:58.624359 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.574026 31157 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.626911 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69316.9us
I20260709 12:25:58.630290 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.586874 30789 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:58.575368 31275 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:58.639277 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:58.636829 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.600329 31332 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:58.638645 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:25:58.683396 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:58.623955 31316 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:25:58.684823 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65803.9us
I20260709 12:25:58.635933 30937 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:58.689875 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 449866us
I20260709 12:25:58.627286 31356 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:25:58.692732 20775 leader_election.cc:336] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.950s (SENT)
I20260709 12:25:58.685849 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.633828 31250 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:25:58.694738 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:58.696014 20728 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
W20260709 12:25:58.694294 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 97520.3us
I20260709 12:25:58.700134 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:58.681766 31348 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:58.700981 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:58.651407 31332 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:58.642004 31353 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:58.687508 31368 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:58.628810 31424 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.687989 30923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39562 (request call id 12) took 5962 ms. Trace:
I20260709 12:25:58.687901 31288 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:58.704296 31430 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:25:58.699604 31427 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:58.690632 30936 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:58.647805 31223 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.055s
I20260709 12:25:58.653211 31275 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.687357 31215 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:25:58.649082 31157 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:58.702098 31334 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:25:58.734944 20775 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.976s (SENT)
I20260709 12:25:58.707471 31348 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.705821 31332 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:58.749846 20775 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.977s (SENT)
I20260709 12:25:58.756477 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:58.752982 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:58.748085 31368 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:58.754824 31348 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:58.729347 31353 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.808981 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:58.762337 31436 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:25:58.811666 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 81263.4us
I20260709 12:25:58.759923 31153 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:58.827100 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:58.769984 31449 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:58.773911 31308 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:58.765717 31447 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
W20260709 12:25:58.858898 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 169447us
I20260709 12:25:58.859215 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:58.861577 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 161019us
I20260709 12:25:58.825444 31153 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:58.862728 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:25:58.869819 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 159078us
I20260709 12:25:58.870628 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:58.872768 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.873799 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.874796 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.815013 31425 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:25:58.884997 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.862910 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.896742 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:58.893126 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:58.871097 20883 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 8 to 18, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:58.901137 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:58.772740 31316 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:58.906322 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.778452 31250 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:58.868609 30705 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.001s
I20260709 12:25:58.911180 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.862761 31215 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:58.952131 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:58.952090 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:58.894263 31056 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:58.955026 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:58.885279 31422 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:58.957194 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:58.960362 30840 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:58.961011 20882 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:25:58.900693 31433 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 12:25:58.868381 31153 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:58.857664 31308 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.076s
I20260709 12:25:58.852628 31446 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.113s
W20260709 12:25:58.959239 31442 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 12:25:58.930160 31177 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.857594 31288 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.098s
I20260709 12:25:58.904995 31356 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:59.003423 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 235945us
W20260709 12:25:59.006675 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 135426us
I20260709 12:25:59.019659 31424 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:58.964205 31422 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:59.011519 20775 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.994s (SENT)
I20260709 12:25:58.909446 30994 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.086s
I20260709 12:25:59.029393 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:59.035645 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142590us
I20260709 12:25:59.036708 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:59.018867 31429 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.448s	user 0.000s	sys 0.002s
I20260709 12:25:59.037675 31056 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:59.038090 31056 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:59.038472 31056 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:59.038651 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:25:59.039517 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3746) took 1115 ms. Trace:
I20260709 12:25:59.039872 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.040102 20805 rpcz_store.cc:276] 0709 12:25:57.924601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.924773 (+   172us) service_pool.cc:224] Handling call
0709 12:25:57.925137 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.925504 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.925547 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.926120 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.238341 (+312221us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:58.238405 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.238509 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.018075 (+779566us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:59.039084 (+ 21009us) raft_consensus.cc:1685] finished
0709 12:25:59.039115 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.039435 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2948,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8545,"spinlock_wait_cycles":66707328,"thread_start_us":311832,"threads_started":1,"wal-append.queue_time_us":566476}
I20260709 12:25:58.986897 30789 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:58.733741 30923 rpcz_store.cc:276] 0709 12:25:52.719952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.736106 (+ 16154us) service_pool.cc:224] Handling call
0709 12:25:52.928120 (+192014us) log.cc:844] Serialized 501 byte log entry
0709 12:25:58.637870 (+5709750us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4392,"raft.run_cpu_time_us":1355,"raft.run_wall_time_us":1354,"thread_start_us":387679,"threads_started":1,"wal-append.queue_time_us":586707,"wal-append.run_cpu_time_us":2122,"wal-append.run_wall_time_us":1058227}
W20260709 12:25:59.049031 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 152355us
I20260709 12:25:59.052245 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:25:59.059073 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47681.2us
I20260709 12:25:58.977237 30705 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.030234 31422 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:59.051373 30789 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.952692 31368 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.057634 31438 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:25:59.065147 31373 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:59.024274 31177 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.105258 20802 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
W20260709 12:25:59.114573 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73018.6us
I20260709 12:25:59.095166 30789 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:59.096159 31437 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:59.099511 31157 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.134583 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2493) took 1577 ms. Trace:
I20260709 12:25:59.132560 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.135051 21430 rpcz_store.cc:276] 0709 12:25:57.557657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.557908 (+   251us) service_pool.cc:224] Handling call
0709 12:25:57.558377 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.983580 (+425203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.983615 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.989003 (+  5388us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.227923 (+238920us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:58.227958 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.228023 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.134349 (+906326us) raft_consensus.cc:1685] finished
0709 12:25:59.134374 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.134516 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":315911,"raft.queue_time_us":368055,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7916,"thread_start_us":333690,"threads_started":2,"wal-append.queue_time_us":521400}
W20260709 12:25:59.157512 31459 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:59.130461 31438 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:59.160339 31438 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:25:59.164161 31439 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 12:25:59.164726 31426 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.757s	user 0.011s	sys 0.002s
I20260709 12:25:59.165203 31426 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.137434 31373 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:59.172354 31464 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:59.172940 31464 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:59.172636 31373 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.173388 31464 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:59.166939 31112 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.000s
I20260709 12:25:59.120572 31454 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:59.185032 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68977.4us
I20260709 12:25:59.189523 31466 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:59.191352 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3896) took 1239 ms. Trace:
I20260709 12:25:59.190120 31467 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.217845 31467 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.151672 31474 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:59.219748 31414 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.018s	user 0.011s	sys 0.005s
I20260709 12:25:59.220440 31414 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:59.220602 31426 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:25:59.221308 31455 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.265s	user 0.000s	sys 0.000s
W20260709 12:25:59.223570 31453 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:25:59.193630 20878 rpcz_store.cc:276] 0709 12:25:57.937115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.937254 (+   139us) service_pool.cc:224] Handling call
0709 12:25:57.976327 (+ 39073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.996143 (+ 19816us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:57.996189 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.017919 (+ 21730us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.316691 (+298772us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:58.316753 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.316857 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.176108 (+859251us) raft_consensus.cc:1685] finished
0709 12:25:59.176137 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.176327 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21786,"raft.queue_time_us":10930,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":14998,"thread_start_us":240768,"threads_started":1,"wal-append.queue_time_us":831497}
I20260709 12:25:59.227577 31373 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:25:59.230499 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.233179 20775 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.881s (SENT)
I20260709 12:25:59.241897 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.242146 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.242698 20733 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
W20260709 12:25:59.229211 31441 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.344s	user 0.000s	sys 0.000s
I20260709 12:25:59.229838 31112 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.252453 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.253175 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2491) took 1742 ms. Trace:
I20260709 12:25:59.254197 31112 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:59.253572 21428 rpcz_store.cc:276] 0709 12:25:57.511176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.513678 (+  2502us) service_pool.cc:224] Handling call
0709 12:25:57.514019 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.931942 (+417923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.931975 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.103180 (+171205us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:25:58.109560 (+  6380us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.575151 (+465591us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:58.575225 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.575512 (+   287us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.228374 (+652862us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:25:59.252975 (+ 24601us) raft_consensus.cc:1685] finished
0709 12:25:59.252995 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.253121 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":417574,"raft.queue_time_us":454729,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11274,"spinlock_wait_cycles":22776448,"thread_start_us":465210,"threads_started":1,"wal-append.queue_time_us":710924}
I20260709 12:25:59.251304 31332 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.244496 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:25:59.288404 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44023.7us
W20260709 12:25:59.179718 31448 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:25:59.253145 20728 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
W20260709 12:25:59.266669 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 192056us
I20260709 12:25:59.281642 20809 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:59.281844 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.229290 31475 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:59.298146 20801 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 35 because replica has last-logged OpId of term: 32 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 18.
I20260709 12:25:59.298821 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.300575 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.304590 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.305207 21434 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:59.314389 21428 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:59.314790 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.255110 31112 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:59.288869 31377 raft_consensus.cc:1068] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Unable to promote non-voter 50c1ab13351d4427b836fbb449c6d7bb: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }.
  Pending config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.320835 21430 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:25:59.321410 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 308085us
I20260709 12:25:59.316104 31157 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:25:59.333396 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 182113us
I20260709 12:25:59.334668 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.338276 31461 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:59.242980 31454 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.347177 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.348748 31486 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:59.348851 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.350054 31486 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:59.350569 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.350415 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.321471 31426 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 12:25:59.305577 31475 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.358592 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.358829 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.359897 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.360324 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.361469 31476 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:59.361678 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:59.362089 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.362534 31488 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.363394 31471 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:25:59.365945 31353 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:59.368043 30705 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:59.320097 31474 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:25:59.391297 31474 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:59.391943 31474 consensus_queue.cc:1243] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 26.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:59.341100 31157 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.411803 31332 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.386888 31476 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:59.412781 31476 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:59.413579 31157 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.385931 20877 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:25:59.416182 31353 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:25:59.416922 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:59.417326 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:59.417558 20877 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:25:59.388567 31488 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.418717 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:59.387218 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:59.369976 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:59.385931 20729 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:59.395503 31475 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.395798 30923 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:59.424834 30923 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:25:59.424921 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.425184 31420 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:25:59.398003 31348 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.000s
I20260709 12:25:59.398636 31275 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.425733 30923 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.434079 30923 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:59.323170 31464 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.435312 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.324883 31112 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.399802 31177 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:59.362535 31180 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:59.401160 31056 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:25:59.355412 31425 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:25:59.455302 31112 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:59.402001 31483 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:59.355093 31463 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:59.403311 31454 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:59.405201 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.398973 31153 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.407998 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:42714 (request call id 24) took 1179 ms. Trace:
I20260709 12:25:59.410874 31324 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:59.416692 31353 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.466264 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.467530 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.419950 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:59.424268 31347 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:59.469036 31347 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:25:59.469148 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49309.9us
I20260709 12:25:59.469509 31347 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:59.393682 31474 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:59.438753 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.439311 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.443921 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:59.446979 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.448377 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.425725 31420 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.449343 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.445611 31180 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:59.452814 31464 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.481946 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.458904 20801 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:59.482369 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:25:59.440860 31333 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.969s	user 0.013s	sys 0.000s
I20260709 12:25:59.431486 31422 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.464243 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.465067 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.465902 31324 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:25:59.467705 31353 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:59.386770 20733 raft_consensus.cc:2376] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30. Re-sending same reply.
I20260709 12:25:59.425662 20809 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:25:59.429082 20728 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:59.475366 20876 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:59.479823 20726 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:25:59.480213 20877 raft_consensus.cc:2410] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 32 because replica has last-logged OpId of term: 31 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 14.
W20260709 12:25:59.484097 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 151102us
I20260709 12:25:59.462471 21433 rpcz_store.cc:276] 0709 12:25:58.228849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.234187 (+  5338us) service_pool.cc:224] Handling call
0709 12:25:59.407713 (+1173526us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b54007b2ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca5735 00007f9e1ec9d9fa 00007f9e23c654c9 00007f9e1d1e1c54 00007f9e1d1e1be2 00007f9e1d1e1b11 00007f9e1d1e1a8c 00007f9e1d1e0d42 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:25:59.407901 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134924160}
I20260709 12:25:59.483537 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.495872 20733 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:25:59.444087 31348 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.496415 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:59.497237 20880 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:25:59.477979 31482 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:59.511441 20884 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:59.509652 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:59.503108 20731 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:59.507107 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.516920 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.520193 20730 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:59.521921 20733 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:25:59.521268 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:25:59.524066 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.526751 20804 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:59.528426 20806 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:59.528990 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.529736 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.534042 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.534804 21430 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:25:59.534175 21435 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:25:59.530746 21428 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:25:59.537427 21426 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:59.539968 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.540143 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:59.540894 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.541724 20730 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:59.519342 31482 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:59.493834 31318 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.038s
I20260709 12:25:59.542277 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:59.490679 31333 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.543881 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.522257 31424 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:59.548058 21429 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:59.548671 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:59.549155 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.549646 21426 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:25:59.536350 31491 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:59.558140 31491 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:59.558575 31491 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:59.560268 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:59.563367 31318 consensus_queue.cc:1243] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 33.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20260709 12:25:59.564093 31153 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:59.565251 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.566253 31333 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:59.566339 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.566694 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.567798 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.568056 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.568972 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.569137 31420 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:59.569275 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.570502 31422 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.571336 30937 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:59.571971 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.572535 21433 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:59.572763 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.570161 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.573705 21427 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:25:59.570513 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.545697 31482 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:59.574774 20731 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:25:59.563041 31476 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.575169 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.576056 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:25:59.572561 21430 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:59.581562 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.576045 20731 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:59.592288 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:25:59.597580 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.598272 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
I20260709 12:25:59.598766 31348 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:25:59.597678 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.599290 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.600257 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.600371 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.602123 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.604326 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.605947 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.607725 20878 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:59.578703 31476 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.609158 20878 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:59.554418 31424 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.609684 31424 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:59.597884 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.612255 21429 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:59.614588 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.615259 20734 raft_consensus.cc:2376] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26. Re-sending same reply.
I20260709 12:25:59.615370 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.615749 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.616333 20729 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
W20260709 12:25:59.618280 31495 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.225s	user 0.003s	sys 0.000s
I20260709 12:25:59.619104 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.600458 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.620183 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:25:59.607587 20883 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:59.621610 20876 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:59.607003 31056 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:25:59.624925 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:25:59.628937 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.629191 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.630966 20734 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:25:59.632923 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.633376 20730 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:25:59.634215 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.634385 20726 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:25:59.634877 20730 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:25:59.635671 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:25:59.627908 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.631965 31476 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:25:59.636397 20729 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:25:59.769520 21429 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:25:59.775118 21429 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:59.777477 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:25:59.779682 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151832us
I20260709 12:25:59.780270 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:25:59.782104 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.788173 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.792012 20806 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:25:59.810460 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:25:59.818835 20731 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:59.830878 31510 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:25:59.834686 31505 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:59.835644 31505 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:25:59.841816 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.840430 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:25:59.847361 21430 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:25:59.849356 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:25:59.850916 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:25:59.853296 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 227693us
I20260709 12:25:59.853593 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:59.854240 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.854262 21432 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:25:59.784744 31488 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:25:59.836913 31505 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.884415 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:59.886612 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:59.886551 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:25:59.783401 31476 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:25:59.897820 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:25:59.893290 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:25:59.895323 31462 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:59.892511 21430 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:25:59.857096 31474 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:59.883324 31337 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:59.823232 31316 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.067s
I20260709 12:25:59.888424 31464 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:59.896399 31420 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:59.890874 31056 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:59.907163 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:25:59.908643 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:25:59.925599 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.904311 31426 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:25:59.937786 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:25:59.903172 31474 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:25:59.904968 31486 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:59.906486 31422 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:59.882366 31501 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.068s
W20260709 12:25:59.948781 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 31 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:25:59.926944 31483 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:59.902335 31373 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.033s
I20260709 12:25:59.953330 20804 pending_rounds.cc:85] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 17
I20260709 12:25:59.967885 20804 pending_rounds.cc:107] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:25:59.968276 20804 raft_consensus.cc:2889] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:59.936784 31420 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:59.935103 31056 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:59.947724 31474 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:59.906955 31462 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:25:59.947110 31422 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:25:59.940093 31485 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:59.967315 31480 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:59.946439 31157 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:26:00.008399 31509 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 12:26:00.009438 31512 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:26:00.012418 31443 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.253s	user 0.004s	sys 0.000s
I20260709 12:26:00.029110 31401 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.030057 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:25:59.959796 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:00.030777 31418 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.071943 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 112273us
I20260709 12:26:00.072989 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:00.031831 31513 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:26:00.033807 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.039604 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.041040 31476 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.043121 30840 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:00.055264 20803 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [27.19-28.20]
I20260709 12:25:59.981760 31420 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:59.980465 31056 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:59.951504 31333 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.050s
I20260709 12:26:00.067404 20881 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:00.030176 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.004079 31422 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:59.996747 30928 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:00.093307 30928 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:00.078205 20729 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:00.081005 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 145297us
I20260709 12:26:00.081089 30840 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:00.048842 31485 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:00.088481 20730 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:00.089409 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.030879 31482 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.096693 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.082661 31337 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:00.089200 31157 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:00.039635 31491 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.064973 31424 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.089154 31476 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.098135 31480 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:00.144562 30840 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.143721 20730 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:00.076884 31401 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.148995 31401 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.152621 31482 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.153935 31476 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.156798 31476 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.162245 31056 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.165446 31056 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:00.165879 31347 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.684s	user 0.008s	sys 0.002s
I20260709 12:26:00.180739 31462 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:00.100950 31485 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:00.181288 20802 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3749) took 2061 ms (client timeout 2155 ms). Trace:
W20260709 12:26:00.181805 20802 rpcz_store.cc:269] 0709 12:25:58.120178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.125400 (+  5222us) service_pool.cc:224] Handling call
0709 12:26:00.181070 (+2055670us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b54003d5ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:00.181192 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2230497536}
W20260709 12:26:00.187623 31516 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:26:00.194264 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3811) took 1306 ms. Trace:
I20260709 12:26:00.194936 20882 rpcz_store.cc:276] 0709 12:25:58.888000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.962792 (+ 74792us) service_pool.cc:224] Handling call
0709 12:25:58.963151 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.312626 (+349475us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:25:59.353207 (+ 40581us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.596237 (+243030us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.596751 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.596890 (+   139us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:59.608890 (+ 12000us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.609380 (+   490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.609451 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.193923 (+584472us) raft_consensus.cc:1685] finished
0709 12:26:00.193967 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.194177 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":116995,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":57501,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":220,"thread_start_us":577318,"threads_started":3,"wal-append.queue_time_us":438007}
I20260709 12:26:00.199685 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2505) took 2001 ms. Trace:
I20260709 12:26:00.200307 21431 rpcz_store.cc:276] 0709 12:25:58.198299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.200724 (+  2425us) service_pool.cc:224] Handling call
0709 12:25:58.201060 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.201427 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:58.201456 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.201944 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.593793 (+391849us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:58.593855 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.593957 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.906267 (+1312310us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:00.199206 (+292939us) raft_consensus.cc:1685] finished
0709 12:26:00.199241 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.199595 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13241,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":13928,"spinlock_wait_cycles":333278464,"thread_start_us":372653,"threads_started":1,"wal-append.queue_time_us":501031}
I20260709 12:26:00.083010 31418 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.166425 31347 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.234918 20880 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:00.224066 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:00.239144 20882 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:00.241118 31347 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.229599 20729 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:00.131259 31337 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:00.245496 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:00.247164 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:00.247620 31418 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:00.249377 30789 consensus_meta.cc:287] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.109s	user 0.012s	sys 0.000s
I20260709 12:26:00.255564 20882 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:00.256758 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:00.258414 30840 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:00.303009 20875 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
W20260709 12:26:00.306068 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 266523us
I20260709 12:26:00.261065 30789 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.306068 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:00.307157 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.311096 20875 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
W20260709 12:26:00.317660 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.318982 20878 raft_consensus.cc:606] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:00.319669 20878 raft_consensus.cc:614] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:00.320832 20879 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:26:00.318627 20882 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:00.329447 31157 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:00.339561 31418 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.340583 31418 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:00.347883 31520 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:26:00.350283 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:00.351066 20882 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:00.349483 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:00.353729 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 131807us
I20260709 12:26:00.352829 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:00.357158 31420 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.359169 31474 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.352344 31418 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.360411 31420 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.308527 31337 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.353884 31056 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:00.368566 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:00.363606 31424 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:00.370009 20875 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:00.374732 20882 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:00.373793 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.375824 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.381698 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.382401 20728 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:00.369426 31491 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.370599 31500 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:00.386574 31500 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:00.387616 31500 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.387987 30928 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.392105 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3765) took 1075 ms. Trace:
I20260709 12:26:00.392340 20802 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:00.392428 20806 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:00.392558 20805 rpcz_store.cc:276] 0709 12:25:59.316881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.391013 (+1074132us) service_pool.cc:224] Handling call
0709 12:26:00.391512 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.391736 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:00.391765 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.391828 (+    63us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:00.391865 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.391917 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.392041 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.399870 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:00.401255 31496 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.013s	user 0.000s	sys 0.002s
I20260709 12:26:00.402763 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:00.403712 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:00.404242 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.404763 20809 raft_consensus.cc:2410] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 27 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:26:00.407881 20806 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:00.409368 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.409842 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:00.410181 20732 raft_consensus.cc:2368] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29.
I20260709 12:26:00.410425 20802 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:00.410485 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:00.411489 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:00.414413 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:00.414629 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:00.415284 20802 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:00.415297 20801 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:00.416949 31474 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:00.419605 30789 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:00.419967 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:00.420699 31526 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:26:00.421726 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3910) took 1553 ms. Trace:
I20260709 12:26:00.421943 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:00.422118 20884 rpcz_store.cc:276] 0709 12:25:58.868005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.875429 (+  7424us) service_pool.cc:224] Handling call
0709 12:26:00.415142 (+1539713us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54001d01d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:00.421671 (+  6529us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236289792}
I20260709 12:26:00.423267 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:00.423887 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:00.432672 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.433530 20733 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:00.439318 20802 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:00.439795 20882 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:00.444659 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:00.445456 20729 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 26: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:00.445574 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:00.445941 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:00.447100 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:00.382885 31056 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:00.448611 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:00.450053 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.452006 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:00.465145 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:00.468291 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:00.469116 20806 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:00.471275 20730 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:26:00.471829 20730 pending_rounds.cc:85] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 16
I20260709 12:26:00.472108 20730 pending_rounds.cc:107] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:26:00.472437 20730 raft_consensus.cc:2889] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.472259 20728 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:00.476392 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:00.449219 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.478034 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.478055 31467 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:00.414853 31485 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.474236 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:00.481885 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.486564 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.489192 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:00.491652 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.501505 31418 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:00.509262 20806 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:00.514186 20733 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
W20260709 12:26:00.480813 20775 connection.cc:256] RPC call timeout handler was delayed by -1.1349274389999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 12:26:00.527037 20775 connection.cc:256] RPC call timeout handler was delayed by -1.102253077000114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
W20260709 12:26:00.529343 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.530287 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.543056 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:00.496791 31467 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:00.526165 31373 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:00.548902 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:00.542552 31288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.478s	user 0.018s	sys 0.045s
I20260709 12:26:00.538785 31476 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:00.570689 30840 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:00.576141 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:00.546125 31368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.001s	sys 0.023s
I20260709 12:26:00.577160 30840 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:00.536406 31491 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:00.577754 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:00.538628 31527 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:00.588496 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:00.591642 30840 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:26:00.574241 31288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.510s	user 0.019s	sys 0.045s
I20260709 12:26:00.548604 31334 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.583123 31368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.001s	sys 0.023s
I20260709 12:26:00.544632 31414 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:26:00.571960 31466 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:00.546262 31029 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:26:00.568395 30923 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:26:00.548799 31479 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:26:00.588021 31337 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:26:00.617758 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68906.4us
I20260709 12:26:00.593144 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:00.592239 31462 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:00.626579 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.615101 31334 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.630884 31334 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.629634 31180 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.594585 31491 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.634969 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:00.598547 31422 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.643112 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:00.645212 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:00.645879 30840 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:00.631245 31347 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:00.646953 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:00.648504 20733 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 23.
I20260709 12:26:00.619649 31466 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.631889 31337 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:00.650919 31462 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:00.651382 31482 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:00.650856 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.653051 30840 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:00.653474 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:00.625258 31373 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:00.598337 31056 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:26:00.679932 31434 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.914s	user 0.005s	sys 0.012s
I20260709 12:26:00.678160 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:00.687719 31539 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:00.689339 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1973) took 1060 ms. Trace:
I20260709 12:26:00.689760 21432 rpcz_store.cc:276] 0709 12:25:59.628728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.826984 (+198256us) service_pool.cc:224] Handling call
0709 12:25:59.848296 (+ 21312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.857009 (+  8713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.689119 (+832110us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b54007b2ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:00.689274 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26002176}
I20260709 12:26:00.678895 31462 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:00.691883 31373 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.692991 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:00.693796 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 3982) took 1249 ms. Trace:
I20260709 12:26:00.694125 20808 rpcz_store.cc:276] 0709 12:25:59.444188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.692730 (+1248542us) service_pool.cc:224] Handling call
0709 12:26:00.693736 (+  1006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.681093 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:00.663327 31180 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:00.695354 31485 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:00.721946 31491 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.694878 31538 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:00.723760 31538 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.724138 31538 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:00.699978 31480 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:26:00.728498 31532 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:00.723273 31422 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.732206 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:00.734092 31422 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:00.734645 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.678591 30923 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.808633 31533 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
W20260709 12:26:00.767376 31530 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:26:00.817612 31539 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:00.818307 31534 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:26:00.819563 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:00.820291 20877 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:00.819267 30923 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.705473 30928 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:00.820884 31308 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:00.848603 31466 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:00.848196 31482 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:00.818301 31539 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:00.906422 31444 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 2.057s	user 0.000s	sys 0.005s
W20260709 12:26:00.908921 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.890290 30928 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.885461 31308 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.867669 31337 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.934386 31481 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:00.899024 31482 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.945442 31352 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:00.948987 31352 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:00.953130 31449 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:00.952826 31352 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:00.953743 31449 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:00.954097 31352 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:00.954988 31449 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.957885 31308 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:00.958555 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:00.935096 31481 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:00.973843 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.926168 30928 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:00.975550 31449 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:00.978737 20806 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 12:26:00.979786 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 883118us
I20260709 12:26:00.990453 31424 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:00.980341 31333 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:26:00.983705 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 585327us
I20260709 12:26:00.983819 31483 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:00.993848 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:00.993448 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:00.996054 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:00.996791 20733 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.000623 20734 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:26:01.000844 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:01.001552 20732 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.011456 31333 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.011819 30928 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:01.012036 31333 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.014289 31333 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.014783 31333 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.015368 31485 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:01.001423 31505 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.015919 31481 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.016887 31505 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.017536 31485 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:01.017912 31485 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:01.017970 31481 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.020601 31309 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.020814 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:01.021181 31309 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:01.022797 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.023062 20699 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 2.155s (SENT) [suppressed 33 similar messages]
I20260709 12:26:01.008759 31428 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.023823 31428 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.024044 31474 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 34.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:01.025853 31474 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:01.025977 31428 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.026562 31428 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:01.026904 31428 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:01.010982 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:01.028571 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:01.029166 20881 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.030004 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41760 (request call id 37) took 1192 ms. Trace:
I20260709 12:26:01.030508 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.031307 21432 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:01.032662 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:01.027341 31483 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:01.038105 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:26:01.032753 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.030587 20878 rpcz_store.cc:276] 0709 12:25:59.837554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.840245 (+  2691us) service_pool.cc:224] Handling call
0709 12:26:01.029706 (+1189461us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54000a32d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec99853 00007f9e23c6760a 00007f9e1d1ed7d4 00007f9e1d1ed762 00007f9e1d1ed691 00007f9e1d1ed60c 00007f9e1d1ec8c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:01.029873 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085804928}
I20260709 12:26:01.036629 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:00.848080 31468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.535s	user 0.001s	sys 0.012s
I20260709 12:26:01.035030 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:01.036270 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:01.038249 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:01.042034 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 388610us
I20260709 12:26:01.041224 21431 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:01.042798 20733 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:01.018790 31485 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.036490 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.048725 21429 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:01.027599 31333 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.040620 21427 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:01.044829 21435 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:01.036384 31427 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:01.041041 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:01.045923 20775 leader_election.cc:400] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29.
I20260709 12:26:01.100256 20775 leader_election.cc:403] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:26:01.100339 21433 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:01.051946 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:01.054138 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:01.055055 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:01.055485 20805 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
W20260709 12:26:01.103967 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 61208.3us
I20260709 12:26:01.056746 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:01.104763 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:01.104775 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 82414us
I20260709 12:26:01.106384 31548 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:01.106978 31548 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:01.107334 31540 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:01.107998 31548 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.107031 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:01.110807 30840 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:01.111171 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:01.105497 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48869.5us
I20260709 12:26:01.111442 30840 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:26:01.111505 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:01.111714 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.088232 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:01.116503 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 61574.3us
I20260709 12:26:01.117188 20806 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:01.119649 20733 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:26:01.121860 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.122407 21431 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:01.122761 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:01.123405 20734 raft_consensus.cc:2376] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28. Re-sending same reply.
I20260709 12:26:01.124171 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.019115 31505 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:01.041039 31468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.737s	user 0.002s	sys 0.012s
I20260709 12:26:01.125830 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.126188 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.126569 20734 raft_consensus.cc:2368] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29.
I20260709 12:26:01.126801 20732 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:01.131294 31500 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:01.131896 31500 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.132339 31500 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.134330 30923 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:01.134716 21433 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:26:01.037580 31483 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:01.142638 20731 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:26:01.137516 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.135706 20732 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: [29.20-31.21]
I20260709 12:26:01.141398 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:01.138444 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1974) took 1504 ms. Trace:
I20260709 12:26:01.146299 20802 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:01.146559 20804 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:01.144308 21428 rpcz_store.cc:276] 0709 12:25:59.634011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.827354 (+193343us) service_pool.cc:224] Handling call
0709 12:25:59.858334 (+ 30980us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:59.888304 (+ 29970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.888339 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:59.899910 (+ 11571us) log.cc:844] Serialized 52 byte log entry
0709 12:26:00.273621 (+373711us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:00.296888 (+ 23267us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.128089 (+831201us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.131448 (+  3359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.131547 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.131604 (+    57us) raft_consensus.cc:1685] finished
0709 12:26:01.131621 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.138389 (+  6768us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":694636,"prepare.queue_time_us":153146,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":712123,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":4534,"thread_start_us":478863,"threads_started":2,"wal-append.queue_time_us":422350}
I20260709 12:26:01.148752 30840 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:01.149120 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.150216 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:01.046321 31309 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:01.152678 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2512) took 1856 ms. Trace:
I20260709 12:26:01.157307 20728 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 12:26:01.157989 30840 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:01.158569 30840 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.153239 21434 rpcz_store.cc:276] 0709 12:25:59.295960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.296189 (+   229us) service_pool.cc:224] Handling call
0709 12:25:59.296503 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.305626 (+  9123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.152374 (+1846748us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:01.152596 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19536256}
I20260709 12:26:01.105556 31445 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:01.175072 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.159793 30840 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.203620 30840 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:01.200429 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
W20260709 12:26:01.205766 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56065.1us
I20260709 12:26:01.206503 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:01.207490 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:01.210141 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.210834 20882 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:01.146809 31485 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:01.212952 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3631) took 1756 ms. Trace:
I20260709 12:26:01.216382 20775 leader_election.cc:400] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29.
I20260709 12:26:01.214659 20735 rpcz_store.cc:276] 0709 12:25:59.448217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.460111 (+ 11894us) service_pool.cc:224] Handling call
0709 12:25:59.460533 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.479240 (+ 18707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.479289 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.490823 (+ 11534us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.913800 (+1422977us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b1800321520. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:01.165178 (+251378us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:01.165251 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.165366 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.165438 (+    72us) raft_consensus.cc:1685] finished
0709 12:26:01.165459 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.204661 (+ 39202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31084,"raft.queue_time_us":31339,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":13375,"spinlock_wait_cycles":6448896,"thread_start_us":251085,"threads_started":1,"wal-append.queue_time_us":1597630}
I20260709 12:26:01.215060 31347 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:01.209882 31215 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:01.232089 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.232959 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:01.233750 21433 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 25: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:01.222366 31466 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:01.242638 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.247545 21434 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:01.242681 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.249029 20732 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:01.244874 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:01.253301 31347 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.254305 31347 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.258167 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:01.123565 31180 consensus_queue.cc:1175] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 29 status { last_received { term: 29 index: 20 } last_committed_idx: 18 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.20-28.20]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:01.212554 31428 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.272082 31428 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:01.245685 31250 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29.
W20260709 12:26:01.266965 31538 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.000s
I20260709 12:26:01.276535 31538 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.279587 20877 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:01.283614 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.226336 31483 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.288751 31482 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.291421 20733 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:01.252406 31466 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:01.301517 31466 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.307971 31539 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:01.332949 20880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3924) took 1888 ms (client timeout 1779 ms). Trace:
W20260709 12:26:01.333576 20880 rpcz_store.cc:269] 0709 12:25:59.445770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.521825 (+ 76055us) service_pool.cc:224] Handling call
0709 12:26:01.332646 (+1810821us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54001ff254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:01.332862 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595678592}
I20260709 12:26:01.340004 31482 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.340828 31482 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:01.341589 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.343390 31368 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:01.347239 31466 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.362613 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.363394 21427 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:26:01.364354 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.365173 21427 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 27: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:01.365990 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.366619 21427 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:01.370862 21433 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:01.376852 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:01.378068 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 355042us
I20260709 12:26:01.381031 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:01.381325 20802 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:26:01.381835 20803 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:01.381035 20806 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:01.383769 20804 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:01.384510 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.383813 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:01.387048 30923 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.387308 31250 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:01.387660 30923 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.388304 31540 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:01.388646 31540 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.389747 31540 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.391310 31449 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.392158 31449 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.355764 31368 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.397239 31368 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.403301 31486 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:01.404206 31466 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.405385 21431 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:01.407436 31215 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:01.409332 30840 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:01.409701 30840 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.407528 31466 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:01.410869 31462 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:01.410467 30840 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.413017 31462 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.413609 30840 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.413975 31462 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.404441 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:01.416347 31324 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.416846 31288 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:01.417162 31462 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.417654 31288 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.418406 31288 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.420085 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:01.425485 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.426332 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.428368 21434 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:01.429862 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.430802 20875 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:26:01.432199 31537 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.789s	user 0.004s	sys 0.006s
I20260709 12:26:01.434801 31318 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:01.395025 31449 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:01.435773 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52767.8us
I20260709 12:26:01.440346 30936 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:01.456548 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:01.460587 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:01.422631 31482 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:01.481652 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.417698 31324 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:01.475054 20729 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.475893 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:01.477303 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.477849 30936 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:01.478205 21435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1945) took 1881 ms (client timeout 2124 ms). Trace:
I20260709 12:26:01.482466 20732 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:01.477921 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:01.444017 31500 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.487077 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:01.485368 21435 rpcz_store.cc:269] 0709 12:25:59.596393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.600204 (+  3811us) service_pool.cc:224] Handling call
0709 12:26:01.477991 (+1877787us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400746b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:01.478120 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043224448}
I20260709 12:26:01.489761 21399 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:01.490995 20875 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:01.494273 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.499642 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:01.500762 31505 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:01.501178 31505 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.501585 31505 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:01.501747 31468 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:01.502344 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.503432 31468 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:01.504122 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.504941 20877 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:01.507310 30840 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:01.508006 30840 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.510715 30840 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.512704 30840 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.515599 31485 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:26:01.512913 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 232054us
I20260709 12:26:01.516577 31427 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:01.517171 31427 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:01.518606 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.521361 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:01.466223 31352 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:01.496479 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:01.482394 31540 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.518265 31427 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.488641 31474 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:01.495424 30936 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:01.548820 31414 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:01.549784 31465 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:01.550223 31465 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.495965 31463 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:01.551064 31465 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.552232 31465 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.555768 31418 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:01.556231 31418 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.557025 31418 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.558079 31418 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:01.558204 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.549319 31414 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.568216 31414 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.569306 31414 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.498821 31481 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:01.467820 31318 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.543609 31522 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.528992 31352 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.545817 31485 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:01.587461 31485 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.589993 31485 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.519968 31368 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:01.542872 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:01.591351 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48580.7us
I20260709 12:26:01.592180 31491 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:01.482544 31288 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.578091 20729 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:26:01.591985 31413 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:26:01.600214 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.529728 31478 raft_consensus.cc:2764] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:26:01.520850 31351 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.060s
I20260709 12:26:01.558555 31505 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.597579 30840 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:01.524458 31482 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.076s
I20260709 12:26:01.525681 31486 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.070s
I20260709 12:26:01.636682 31486 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:01.637084 31486 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.637938 31486 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.528054 31531 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.078s
I20260709 12:26:01.640216 31531 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:01.640615 31531 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.641392 31531 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.642659 31531 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:01.630806 31351 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:01.644320 31351 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.644537 31480 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.645066 31480 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:01.645365 31351 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.619491 31420 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.647377 31420 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.602672 31474 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:01.651034 31474 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.652370 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:01.630618 31491 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:01.632812 31288 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:01.634198 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.634689 31352 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.644076 30936 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.717053 20876 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:01.624358 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.719683 20877 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:01.653859 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:01.611390 31368 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:01.722807 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.675405 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 173650us
I20260709 12:26:01.724583 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:01.646941 31351 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:01.693780 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102417us
W20260709 12:26:01.667382 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.672816 21428 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:01.592025 31467 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.096s
W20260709 12:26:01.727294 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.687474 31466 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:01.688271 31486 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:01.677238 31491 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.736702 20728 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:01.595979 31522 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.744117 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 1970) took 2117 ms. Trace:
I20260709 12:26:01.744596 21426 rpcz_store.cc:276] 0709 12:25:59.626844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.626988 (+   144us) service_pool.cc:224] Handling call
0709 12:25:59.764833 (+137845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.808482 (+ 43649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.808542 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.838066 (+ 29524us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.005879 (+167813us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:00.070414 (+ 64535us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.070766 (+   352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.070881 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.979147 (+908266us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b1800346c00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e22b7358f 00007f9e1bbb2571 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd
0709 12:26:01.734273 (+755126us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b1800346c00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:01.743865 (+  9592us) raft_consensus.cc:1685] finished
0709 12:26:01.743890 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.744030 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2169,"prepare.queue_time_us":59682,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3299,"raft.queue_time_us":17360,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":43620,"spinlock_wait_cycles":18609408,"thread_start_us":107898,"threads_started":1,"wal-append.queue_time_us":235101}
I20260709 12:26:01.750775 31318 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:01.752279 31177 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:01.752504 31478 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:01.752959 31177 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.754091 31478 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:01.754624 31478 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.752265 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:01.759542 31467 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.762966 31505 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.763978 31478 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.765141 31482 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:01.768046 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:01.768743 20875 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:01.769491 31427 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:01.771327 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:01.772003 31468 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:01.772264 20880 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:26:01.772454 31468 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.772859 31468 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:01.773437 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:01.774084 20732 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:01.779846 20735 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:01.766360 31522 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.768702 31482 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.777894 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:01.780517 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:01.785229 20850 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.604s (SENT)
I20260709 12:26:01.785086 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:01.785816 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:01.787025 20730 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:01.788672 20735 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:01.788574 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:01.790208 20803 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:01.736634 31466 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.812062 20732 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 24.
I20260709 12:26:01.815543 20875 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
W20260709 12:26:01.803539 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 272936us
I20260709 12:26:01.783962 31177 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:01.812682 31466 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:01.812587 31318 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:01.839555 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 245831us
I20260709 12:26:01.819299 31548 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.822455 31538 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:01.817909 31157 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:01.820516 31324 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:01.820689 31505 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:01.851920 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.847484 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:01.852690 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:01.854543 31505 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.826068 31491 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.856971 31505 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:01.857642 31491 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:01.858403 31483 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:01.859519 31565 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:01.861079 31565 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:01.861431 31565 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:01.862358 31565 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.863018 31480 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.842105 31368 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.845583 31318 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.866930 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 3992) took 1443 ms. Trace:
I20260709 12:26:01.852856 31157 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:01.871483 20801 rpcz_store.cc:276] 0709 12:26:00.423458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.423599 (+   141us) service_pool.cc:224] Handling call
0709 12:26:01.866232 (+1442633us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b5800004b98. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:26:01.866430 (+   198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.866597 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.866625 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.866663 (+    38us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:01.866691 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.866747 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.866869 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3917696}
I20260709 12:26:01.859725 31420 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:01.873754 31420 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:01.874219 21429 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:01.876973 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:01.878777 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:01.883659 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:01.884533 21435 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.886868 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.887738 21434 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:01.848044 31548 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.875932 20805 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
W20260709 12:26:01.898124 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.898998 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:01.901335 21427 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: [25.16-25.16]
W20260709 12:26:01.904292 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51713.6us
I20260709 12:26:01.907608 21426 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 20. (term mismatch)
I20260709 12:26:01.908226 21426 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 19
I20260709 12:26:01.908514 21426 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.20
I20260709 12:26:01.908797 21426 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:01.923746 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:01.925012 21428 raft_consensus.cc:2410] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 37 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 22.
I20260709 12:26:01.943439 31565 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:01.947906 31157 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:01.949002 31557 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:26:01.891908 31540 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:01.898427 31559 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:01.898691 31250 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:01.965888 21435 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:01.926182 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:01.958550 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:01.852103 31482 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:01.952639 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 298857us
I20260709 12:26:01.920542 31501 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:26:01.980988 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54915.3us
W20260709 12:26:01.993865 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.995820 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.996438 31568 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:01.874264 31420 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.996869 31559 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:01.997298 31445 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.031410 31445 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.033576 31333 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:01.999662 31562 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:26:02.033854 31466 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:01.966540 31570 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:02.035991 31570 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:02.036955 31570 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.039613 31445 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:01.963765 31157 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.042465 31157 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.044715 31157 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.048175 20801 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:02.049542 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3819) took 2781 ms. Trace:
I20260709 12:26:02.049794 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:02.049952 20879 rpcz_store.cc:276] 0709 12:25:59.268057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.596516 (+328459us) service_pool.cc:224] Handling call
0709 12:25:59.597069 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.325153 (+728084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.049295 (+1724142us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54000a32d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:02.049474 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663635840}
I20260709 12:26:02.050731 31500 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.022166 31468 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.051151 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.051523 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.053067 31468 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.053404 20808 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:26:02.054244 31480 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.049660 30923 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.013535 31548 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.021066 31420 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.004518 31413 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:02.000370 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3832) took 2111 ms. Trace:
I20260709 12:26:02.023573 20726 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:26:02.025421 31531 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:02.057898 31413 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.058904 31531 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:02.059314 31531 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:02.060588 31318 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.021903 30936 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.061842 20805 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:26:02.041641 31568 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:02.057919 20876 rpcz_store.cc:276] 0709 12:25:59.888683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.889218 (+   535us) service_pool.cc:224] Handling call
0709 12:25:59.902892 (+ 13674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.717580 (+1814688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.999942 (+282362us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54001fe854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:02.000242 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4171220224}
I20260709 12:26:02.027375 31446 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:02.063356 31446 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:02.062240 31478 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:02.064241 31446 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.065644 31446 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:02.062959 31568 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.067708 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.040701 31559 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.060191 31531 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:02.075407 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66285.9us
I20260709 12:26:02.075649 20628 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.076565 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:02.077275 20628 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 23, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 23 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:02.079072 31215 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:02.080133 31215 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:02.080231 30840 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:02.080605 30840 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:02.081310 30840 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.081975 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.082605 20805 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:02.082765 30840 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:02.083806 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 125348us
I20260709 12:26:02.083858 20802 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:02.086730 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:02.081324 20808 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:02.088917 21430 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:02.089180 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3934) took 2036 ms. Trace:
I20260709 12:26:02.078668 20628 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 22 to 27, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 27 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.090224 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.090750 20628 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 18 to 21, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 21 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:02.093007 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:02.093235 20628 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 35. New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.096129 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:02.060736 31413 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.097692 20879 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
W20260709 12:26:02.097857 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.100131 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 93157.7us
I20260709 12:26:02.101897 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.106041 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:02.089577 20883 rpcz_store.cc:276] 0709 12:26:00.053124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.055338 (+  2214us) service_pool.cc:224] Handling call
0709 12:26:00.055884 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.056473 (+   589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.056544 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.057197 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.466331 (+1409134us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b180038f200. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:01.628990 (+162659us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:01.821795 (+192805us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.035704 (+213909us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b180038f200. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:02.088860 (+ 53156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.088965 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.089006 (+    41us) raft_consensus.cc:1685] finished
0709 12:26:02.089020 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.089127 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5490,"prepare.queue_time_us":220820,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":79550,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":18653,"spinlock_wait_cycles":109572608,"thread_start_us":618888,"threads_started":3,"wal-append.queue_time_us":1658886}
W20260709 12:26:02.107515 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 382984us
I20260709 12:26:02.109933 20805 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:02.114143 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.115041 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.115125 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:02.116135 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:02.078758 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:02.120393 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:02.123394 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.126078 21432 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:02.018960 31467 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:02.126024 21427 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:02.088511 31486 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:02.108462 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:02.127810 20628 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 25, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 25 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.098744 31531 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:02.133419 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54758.7us
I20260709 12:26:02.139930 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.142560 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.139456 21426 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:02.143409 21429 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:02.143016 31413 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:02.146718 31413 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:02.147104 31413 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:02.140020 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.149566 21435 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:02.150096 21435 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:02.120545 30936 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:02.157469 20804 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:26:02.155045 20628 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 23 to 29, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 29 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:02.155786 31445 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:02.124773 30937 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:02.177502 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:02.177834 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.133215 30840 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:02.165207 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:02.165809 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:02.166503 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:02.179044 20883 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:02.179507 20879 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:02.167227 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:02.180182 20880 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:02.180764 20880 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:02.155845 31501 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:02.178426 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:02.182284 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.148936 31465 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.149178 31505 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.160420 20628 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.120672 30923 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:26:02.176378 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:02.200278 20726 raft_consensus.cc:606] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:02.200801 20726 raft_consensus.cc:614] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:02.178519 20875 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:02.167085 31548 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:26:02.172484 20629 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 297) took 60168 ms (client timeout 15000 ms). Trace:
I20260709 12:26:02.191416 20804 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:02.206460 30840 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:02.192265 20877 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:02.196952 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.202767 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.208397 21427 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:02.211143 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:02.213137 21435 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:02.150288 31427 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:26:02.224843 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.236887 21428 raft_consensus.cc:2410] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:26:02.225629 20774 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:02.241148 30840 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:02.241777 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:02.241982 30840 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:02.243014 30840 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:02.244056 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:02.226929 20883 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:02.156632 31483 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:26:02.181870 31337 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:02.186959 31418 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.016s
I20260709 12:26:02.182494 31501 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:02.228356 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.230821 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:02.181591 31324 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:02.225368 20628 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.251747 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:02.254756 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.255765 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.256721 30840 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:02.258258 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:02.267808 20628 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.269301 20628 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 26 to 30. New cstate: current_term: 30 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.270056 21435 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 12:26:02.270721 20628 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 31, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 31 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.203267 31318 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:02.184543 31578 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.075s
I20260709 12:26:02.270406 21435 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 18
I20260709 12:26:02.184895 30937 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.277364 21435 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 26.19
I20260709 12:26:02.277683 21435 raft_consensus.cc:2917] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Aborting config change with OpId 26.19: Aborted: Op aborted by new leader
I20260709 12:26:02.277796 20806 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:02.278080 21435 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.278834 31427 consensus_queue.cc:1243] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 32.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:02.279376 31324 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.282207 31324 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.210892 31541 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:02.225034 31368 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:02.225198 31414 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:02.247172 31180 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:02.330061 20883 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
W20260709 12:26:02.212894 20629 rpcz_store.cc:269] 0709 12:25:02.003868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.828360 (+1824492us) service_pool.cc:224] Handling call
0709 12:26:02.172384 (+58344024us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:57.472039 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:57.539482 (+ 67443us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:57.539539 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:25:57.539560 (+    21us) tablet.cc:662] Decoding operations
0709 12:25:57.540622 (+  1062us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:57.540811 (+   189us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:57.540848 (+    37us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:25:57.541708 (+   860us) tablet.cc:701] Row locks acquired
0709 12:25:57.541759 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:25:57.541957 (+   198us) write_op.cc:270] Start()
0709 12:25:57.542142 (+   185us) write_op.cc:276] Timestamp: P: 1783599957541890 usec, L: 0
0709 12:25:57.542193 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 12:25:57.651896 (+109703us) log.cc:844] Serialized 3223 byte log entry
0709 12:25:58.689147 (+1037251us) op_driver.cc:464] REPLICATION: finished
0709 12:25:59.219366 (+530219us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1bb950ac 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e226d5ef0
0709 12:25:59.452613 (+233247us) write_op.cc:301] APPLY: starting
0709 12:25:59.452718 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:59.601865 (+149147us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:59.601907 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:00.417600 (+815693us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:00.417949 (+   349us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 12:26:00.418008 (+    59us) write_op.cc:312] APPLY: finished
0709 12:26:00.418342 (+   334us) log.cc:844] Serialized 81 byte log entry
0709 12:26:00.613277 (+194935us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:00.619732 (+  6455us) write_op.cc:454] Released schema lock
0709 12:26:00.624270 (+  4538us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:00.656562 (+ 32292us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:00.656599 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":870335,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":750986,"apply.run_cpu_time_us":69776,"apply.run_wall_time_us":1191922,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37567,"num_ops":8,"prepare.queue_time_us":275494,"prepare.run_cpu_time_us":22583,"prepare.run_wall_time_us":458506,"raft.queue_time_us":508092,"raft.run_cpu_time_us":5558,"raft.run_wall_time_us":665677,"replication_time_us":1085373,"spinlock_wait_cycles":289664768,"thread_start_us":1293136,"threads_started":5,"wal-append.queue_time_us":422692,"wal-append.run_cpu_time_us":17749,"wal-append.run_wall_time_us":2697430}]]}
I20260709 12:26:02.260207 31501 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:02.260372 31250 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:02.210247 31465 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.303280 21427 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:02.268661 31484 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:02.210026 31505 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.272624 20628 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 20 to 24, leader changed from <none> to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 24 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:02.286257 31351 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:02.349740 31351 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:02.289930 31318 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.304571 31482 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:02.306906 31522 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:02.305369 31177 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:02.310370 31427 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.247617 31561 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 12:26:02.374298 20628 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 32, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.380388 20628 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.259903 31337 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.346493 31484 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.018s
I20260709 12:26:02.352674 31368 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.410373 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142147us
I20260709 12:26:02.391245 31561 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Stepping down as leader of term 30
W20260709 12:26:02.391819 31579 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:02.414613 31561 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:02.324398 31541 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.358794 31522 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:02.397921 20628 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 23 to 30. New cstate: current_term: 30 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.379985 31445 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:02.419260 20628 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:02.420944 20628 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 20 to 30, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 30 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.360623 31318 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:02.350927 31351 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.352962 31480 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.054s
I20260709 12:26:02.351712 31180 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:02.423000 20628 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.378968 31482 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:02.431872 20810 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:02.438377 20628 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.436849 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3632) took 2984 ms. Trace:
I20260709 12:26:02.440011 20628 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 21 to 29. New cstate: current_term: 29 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.443584 20883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:02.444319 20883 raft_consensus.cc:606] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:02.415956 31561 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.448647 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:02.439785 20727 rpcz_store.cc:276] 0709 12:25:59.448529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.490205 (+ 41676us) service_pool.cc:224] Handling call
0709 12:25:59.502777 (+ 12572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.503145 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:59.528277 (+ 25132us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.833841 (+305564us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b180034fd80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:00.676090 (+842249us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b180034fd80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9
0709 12:26:01.653402 (+977312us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b180034fd80. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:26:02.143463 (+490061us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.160915 (+ 17452us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.161049 (+   134us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:02.420229 (+259180us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.432089 (+ 11860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.432227 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.432286 (+    59us) raft_consensus.cc:1685] finished
0709 12:26:02.432305 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.432456 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31082,"prepare.queue_time_us":24260,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":56934,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":28393,"spinlock_wait_cycles":16840576,"thread_start_us":722259,"threads_started":2,"wal-append.queue_time_us":2782094}
I20260709 12:26:02.437861 31541 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.388378 31427 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.477901 31427 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.416926 31368 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.479003 20804 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:02.479398 31482 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.480350 31368 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.432802 31449 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:02.481241 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.481880 21427 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:26:02.483667 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.483915 31180 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.486879 31505 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.486977 31368 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:26:02.492020 31580 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:02.497645 31478 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:02.498124 31478 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:02.498972 31478 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.500036 31478 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:02.453645 31577 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.052s
I20260709 12:26:02.504360 31577 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:02.504738 31577 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:02.505556 31577 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.506827 31577 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:02.456121 31538 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:02.507709 31538 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:02.508541 31538 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:02.513238 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 255064us
I20260709 12:26:02.521122 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.527097 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.442098 20628 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 20 to 23, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.468286 20879 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:02.452425 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:02.530624 20628 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 23 to 27, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 27 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:02.531082 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78883.4us
I20260709 12:26:02.534940 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.536751 20807 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 17. (term mismatch)
I20260709 12:26:02.537271 20807 pending_rounds.cc:85] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 16
I20260709 12:26:02.537539 20807 pending_rounds.cc:107] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 30.17
I20260709 12:26:02.537827 20807 raft_consensus.cc:2889] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.537864 20733 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:02.539561 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.542057 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:02.542651 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.543708 20801 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:02.545253 20801 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 12:26:02.545650 20801 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 18
I20260709 12:26:02.545920 20801 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 26.19
I20260709 12:26:02.546212 20801 raft_consensus.cc:2917] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Aborting config change with OpId 26.19: Aborted: Op aborted by new leader
I20260709 12:26:02.546693 20801 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.471884 31578 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:02.567706 20628 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 22 to 27. New cstate: current_term: 27 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.472836 31467 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:02.473778 31501 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.555826 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.557704 20731 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:26:02.540395 20726 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:02.566433 20807 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:02.566387 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.568161 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.472132 31413 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.574949 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.570531 20628 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 19 to 23, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 23 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:02.447799 31539 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.065s
I20260709 12:26:02.513476 31561 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:02.573742 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.637913 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.576534 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.581203 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.584281 20805 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:02.639356 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:02.640021 20734 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:02.585639 20727 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:26:02.566226 31572 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.578191 31157 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:02.579761 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.589000 20732 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:26:02.573838 31574 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.598328 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 302064us
I20260709 12:26:02.574196 31334 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:26:02.597802 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 265774us
I20260709 12:26:02.575345 31467 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:02.587903 31446 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.000s
I20260709 12:26:02.568667 31578 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:02.606838 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.608016 30937 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.622509 31539 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 12:26:02.611544 31480 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:02.669860 20774 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:02.612212 31570 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.615481 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:02.593490 31420 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:02.512382 31522 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.614284 31462 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.577462 20628 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:02.637017 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.698068 20730 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:26:02.644184 20728 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
W20260709 12:26:02.658246 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.621836 31465 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.636545 31177 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.647886 31318 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:02.655664 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.671490 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.672657 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.694703 20734 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:02.651731 31334 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:02.713573 31334 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:02.716161 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:02.716260 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.610949 31540 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.717211 31540 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.701066 31420 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.719570 31540 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.698998 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.720345 20730 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:02.721052 31420 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.721426 20730 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:02.722631 20730 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:02.666591 31337 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.698246 20628 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 26 to 32, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 32 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:02.724052 31568 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:02.724529 31568 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:02.723932 20628 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 33, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 33 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.725421 31568 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.725740 20628 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:02.714929 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45175.6us
W20260709 12:26:02.727562 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.728193 31368 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:02.728883 31583 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:26:02.728760 31368 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:02.731930 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95690.8us
I20260709 12:26:02.651566 31157 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.733494 20728 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:02.713685 31589 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.704759 31215 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.703917 31548 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.746959 31215 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.747056 31548 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.650539 31572 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.753652 21434 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:02.738786 20730 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:26:02.701948 31462 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:02.742066 31585 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:26:02.742264 31466 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.699265 31570 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.749251 21435 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:02.749290 21426 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:02.754343 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.654225 31574 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.765864 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.771459 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.772238 21429 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:02.773212 20730 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:02.770884 21428 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
W20260709 12:26:02.750801 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 79885.3us
I20260709 12:26:02.749305 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.757711 20809 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:02.760987 21430 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:02.761920 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.746512 20733 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:02.778826 20733 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:02.780651 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:02.782698 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.784597 31501 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.731290 31373 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 12:26:02.785615 31318 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.734798 31595 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.792384 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:02.763231 31531 consensus_meta.cc:287] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.000s
I20260709 12:26:02.752715 31420 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:02.777295 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:02.762012 31607 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.712013 31177 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.760944 31445 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:02.810853 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.786283 31318 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.813905 30840 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:02.773792 31368 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.822719 20729 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:02.794770 31413 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.825315 31318 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.798708 31505 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.835734 20727 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:02.813443 31478 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:02.856421 31215 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.764443 31467 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.828641 31480 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:02.815413 31445 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:02.754360 31589 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.807404 31540 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:26:02.832805 31373 raft_consensus.cc:2764] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:26:02.735705 31578 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.805518 31482 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:26:02.873243 21426 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:02.876328 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.881654 21435 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:02.883318 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.884405 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.884682 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:02.885283 20729 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:02.866717 31539 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:02.886745 31539 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.889343 31539 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.890918 31589 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.891517 31318 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.853279 31505 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.892787 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.893740 20729 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:02.893803 31505 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.866804 31478 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:02.894625 31474 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:02.894831 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.898047 20729 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:02.900359 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:02.910140 31157 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:02.870859 31608 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:26:02.831703 31587 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:26:02.913156 31587 consensus_queue.cc:1243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 12:26:02.915889 31599 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:26:02.916649 31368 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:02.919239 31574 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.920534 31177 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:02.895893 31351 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.909502 31478 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:02.866721 31420 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:26:02.926213 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67334.3us
I20260709 12:26:02.915937 31215 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.807679 31595 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.936851 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:02.850598 31337 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:26:02.814958 31596 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.075s
I20260709 12:26:02.926991 31413 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:02.815851 31607 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:02.927937 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:02.951432 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 126337us
I20260709 12:26:02.950328 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:02.939675 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 157118us
I20260709 12:26:02.954270 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:02.954358 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:02.954871 20729 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:26:03.000046 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:02.922950 30937 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.928146 31578 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:03.009301 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 137957us
I20260709 12:26:03.029259 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:02.938699 31531 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:03.033711 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:03.032323 21426 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:03.037249 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.057696 20808 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:03.062031 20881 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:02.947455 31593 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:02.939311 31505 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:03.046546 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 118645us
I20260709 12:26:03.066370 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:03.066542 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:02.927614 31594 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.107s
W20260709 12:26:03.049753 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 95489.8us
I20260709 12:26:02.938258 31571 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.103s
I20260709 12:26:03.030001 31596 consensus_queue.cc:1243] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 31.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:26:02.935019 31351 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.030961 31368 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:03.093871 31368 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.094329 31368 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:03.097299 31445 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.051038 31420 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:03.100026 21426 pending_rounds.cc:85] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 15
I20260709 12:26:03.100378 21426 pending_rounds.cc:107] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:26:03.100615 21426 raft_consensus.cc:2889] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.102485 31351 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:02.943466 31215 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.030422 31616 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.069795 20882 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:03.065325 31157 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:03.079339 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:03.051244 31373 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.077976 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.065922 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:03.113250 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.083613 21429 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:03.113915 20804 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
W20260709 12:26:03.087873 31602 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:26:03.089398 31607 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.055823 31318 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:03.080276 31608 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:03.029743 31474 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:03.108410 20880 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:26:03.071648 31531 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.036875 30937 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.070156 31462 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.075230 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:03.116730 20881 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:03.098769 31420 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.076164 31617 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:03.104383 31548 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.128757 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.130117 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.075394 31505 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.066092 31593 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.084514 31540 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.038s
W20260709 12:26:03.135797 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.127502 31157 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:03.131719 31474 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:03.138830 31595 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.115052 31572 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.116806 31570 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.119791 31351 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:03.147330 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72225.4us
I20260709 12:26:03.168011 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:03.148974 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.149686 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.151171 31571 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:03.122697 31318 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:03.119330 31373 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.127833 31608 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:03.160601 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.174598 20804 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 28 because replica has last-logged OpId of term: 26 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:26:03.117314 31616 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.160296 31413 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.163009 31288 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.155995 31593 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:03.162410 31474 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:03.195640 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.144897 31607 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.033s
I20260709 12:26:03.157907 31491 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.027s
I20260709 12:26:03.207585 31157 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.167410 31351 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.209995 31351 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:03.186875 31572 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
I20260709 12:26:03.212368 30937 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.183981 31318 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:03.181658 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:03.203022 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.173722 31427 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:03.205504 20879 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:03.205719 31540 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.206182 31531 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.207546 31505 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.215556 20875 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:03.187202 31608 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.176779 31480 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.040s
I20260709 12:26:03.228984 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.261384 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.262049 20809 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:03.217955 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:03.263814 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45986.8us
I20260709 12:26:03.264526 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:03.236862 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:03.266039 31589 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:26:03.223831 31575 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.781s	user 0.000s	sys 0.006s
I20260709 12:26:03.266837 31571 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:03.267326 31571 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.267755 31571 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:03.264756 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 264573us
W20260709 12:26:03.269163 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.271601 20803 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:03.276079 31589 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:03.265658 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:03.262878 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.277981 20809 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:03.278815 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.280750 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.287873 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.201721 31482 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.035s
I20260709 12:26:03.196136 31539 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:26:03.198585 31413 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.218405 31605 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:03.224486 31573 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:03.288480 20808 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
W20260709 12:26:03.259212 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93107.8us
I20260709 12:26:03.281368 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.229867 31337 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:03.302556 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.306752 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:03.303059 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.218042 31588 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.155s
I20260709 12:26:03.308820 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.204627 31288 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:26:03.309459 20804 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:03.222599 31334 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.315822 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.317230 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.317830 31334 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:03.318481 20882 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:03.289068 31538 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:03.319263 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.319746 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.320542 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.313531 20808 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:03.233170 31540 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.344583 20881 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
W20260709 12:26:03.346089 31561 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.758s	user 0.004s	sys 0.007s
I20260709 12:26:03.351158 21399 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:03.354614 20880 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:03.356411 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.357826 20881 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:03.343005 21426 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:03.341204 20803 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
W20260709 12:26:03.358596 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 192893us
I20260709 12:26:03.364220 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:03.356796 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.365904 21427 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 31 because replica has last-logged OpId of term: 30 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 17.
I20260709 12:26:03.305071 31573 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:03.367220 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.378248 20881 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:03.368415 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.362075 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:03.379570 20808 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:03.309882 31627 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:03.386464 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:03.383144 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.383847 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:03.386688 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.388648 20880 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:26:03.353838 31625 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.390259 21399 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:03.391911 20880 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:03.379997 31157 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.389374 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 82716.3us
W20260709 12:26:03.393383 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109605us
I20260709 12:26:03.387605 31561 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:03.376243 30937 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:03.397969 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:03.392287 20808 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 26: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:03.381755 31215 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.388451 30936 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:03.411677 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:03.409677 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.395951 31337 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:03.414863 31337 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.415457 31337 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:03.415745 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.417097 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:03.421046 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72606.5us
I20260709 12:26:03.401445 31548 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEARNER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:03.392735 31628 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:03.430728 31598 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.677s	user 0.001s	sys 0.017s
I20260709 12:26:03.399822 31625 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.475032 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.448779 31413 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.412653 31573 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.450120 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:03.492178 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:03.492074 31318 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.493316 31636 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:03.494007 31636 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.494702 31318 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:03.495074 20877 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:03.445283 31505 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:03.496204 31505 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:03.496596 31505 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:03.496987 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:03.497511 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.497661 21432 raft_consensus.cc:2376] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26. Re-sending same reply.
I20260709 12:26:03.492178 31573 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:03.498785 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 1968) took 1022 ms. Trace:
I20260709 12:26:03.499182 20806 rpcz_store.cc:276] 0709 12:26:02.476185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.476681 (+   496us) service_pool.cc:224] Handling call
0709 12:26:02.476996 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.477325 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.477354 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.477864 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.649807 (+171943us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:02.650760 (+   953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.650862 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.403770 (+752908us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:03.498481 (+ 94711us) raft_consensus.cc:1685] finished
0709 12:26:03.498502 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.498724 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3623,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":7527168,"thread_start_us":171053,"threads_started":1,"wal-append.queue_time_us":223066}
I20260709 12:26:03.500777 31478 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.501127 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.501298 31478 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.496894 31628 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:03.504132 31628 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.503847 31351 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.496991 31505 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:03.504587 31628 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:03.506170 31351 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.453263 31373 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.508224 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.508908 21432 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:03.510787 31571 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.510995 31616 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.514334 21428 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:26:03.513541 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.485291 31625 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.515933 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.516644 20733 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:03.517554 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:03.517861 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.518086 20733 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:03.518889 31474 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.519991 21426 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:03.520855 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.521045 31445 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:03.524501 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.525170 21430 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:03.526048 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:03.526608 21430 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:03.507704 31478 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.510622 31368 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.528245 21429 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:26:03.401072 31617 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:03.464164 31288 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.464726 31420 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:03.465832 31540 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.406188 31157 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.455976 31215 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.011s
I20260709 12:26:03.440188 31337 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.473263 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.560448 20802 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:03.561504 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.562682 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.384594 31589 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.563596 31474 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.405148 31594 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 18, Time since last communication: 0.052s
I20260709 12:26:03.480993 30936 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:03.390196 31627 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.443730 31561 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:03.539546 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:03.548329 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:03.545102 21430 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:03.532864 31636 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.570451 20733 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:03.571573 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:03.609431 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.609688 20802 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:03.555672 31288 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.609756 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:03.573698 20881 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:03.537966 31571 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:03.571719 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 160178us
I20260709 12:26:03.610149 20733 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
W20260709 12:26:03.612190 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72754.2us
I20260709 12:26:03.610538 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:03.611811 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63619.3us
I20260709 12:26:03.612442 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:03.621250 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:03.634495 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.646245 31636 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:03.616072 31474 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:03.643847 31288 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.741353 31474 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:03.741936 31474 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:03.743261 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.743840 31288 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.608803 31561 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.744567 31561 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:03.745725 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.746448 21426 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:26:03.702679 30937 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.746887 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.747483 30937 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.749706 30937 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.753768 31636 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:03.703058 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:03.754628 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140062us
I20260709 12:26:03.754813 31636 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.755704 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:03.753002 31474 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.757855 21430 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:03.704147 20805 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:03.758739 31500 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:03.759258 31500 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.759748 31500 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:03.762717 31467 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:03.705185 31616 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:03.707574 20806 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:26:03.763547 31616 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:03.763689 31571 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:03.765137 31474 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.764608 31616 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.708112 31625 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.575435 30936 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.678011 31638 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.068s
I20260709 12:26:03.769461 31625 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.713625 31593 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.715616 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.717608 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:03.717816 31337 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:03.718163 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3985) took 1082 ms. Trace:
I20260709 12:26:03.573097 31465 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.771075 30936 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.652524 31617 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.729664 31627 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.732909 31628 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.737227 31608 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:03.738646 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.739632 31413 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.550534 31478 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:26:03.810328 31413 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.741488 31368 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:03.703109 31420 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:03.772948 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.710126 31639 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.813072 31420 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:03.813452 31413 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.813227 31639 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.813694 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:03.814067 31420 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.815668 31639 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.772485 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:03.774237 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:03.808094 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.787526 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:03.819208 20726 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:03.723377 31157 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.785748 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:03.791574 31465 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.791989 31505 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.794641 31593 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.804993 21432 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:03.771689 31625 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:03.838649 20878 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:03.808183 20733 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:03.785720 20735 rpcz_store.cc:276] 0709 12:26:02.636004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.636625 (+   621us) service_pool.cc:224] Handling call
0709 12:26:02.653113 (+ 16488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:02.689282 (+ 36169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.689332 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:02.690117 (+   785us) log.cc:844] Serialized 52 byte log entry
0709 12:26:03.529123 (+839006us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b1800307960. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:03.609506 (+ 80383us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:03.609548 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.609613 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.717802 (+108189us) raft_consensus.cc:1685] finished
0709 12:26:03.717828 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.718097 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37418,"raft.queue_time_us":42343,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6579,"spinlock_wait_cycles":2919168,"thread_start_us":80158,"threads_started":1,"wal-append.queue_time_us":1006127}
I20260709 12:26:03.808292 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:03.842523 20809 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:03.820688 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.821034 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:03.846328 20730 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:03.849767 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.850651 20735 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:03.859797 31614 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:03.860445 31614 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:03.860927 31614 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:03.860918 31627 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:03.864454 31627 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.867960 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.868847 20726 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:03.869756 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:03.837601 31157 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:03.875768 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:03.878822 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:03.880143 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:03.883723 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:03.805044 20802 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:03.773319 31466 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:26:03.832085 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 74749.3us
I20260709 12:26:03.806985 31628 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:03.788976 31638 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.894809 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:03.889593 21427 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 1972) took 1406 ms (client timeout 1581 ms). Trace:
I20260709 12:26:03.887344 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:03.895839 21427 rpcz_store.cc:269] 0709 12:26:02.482821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.483325 (+   504us) service_pool.cc:224] Handling call
0709 12:26:03.889307 (+1405982us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b540013d154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:03.889500 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026721024}
I20260709 12:26:03.901379 21426 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:26:03.891724 31466 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:03.904899 20809 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:03.913786 20726 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:26:03.914984 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.915746 20726 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:03.916501 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:03.919550 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.915925 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:03.923561 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.924537 20733 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:26:03.890745 31627 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:03.892362 31157 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:03.900543 31573 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:03.925760 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:03.926105 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.891947 31337 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:03.928273 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:03.930869 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.935662 20732 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 24.
I20260709 12:26:03.936456 20882 raft_consensus.cc:2410] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 28 because replica has last-logged OpId of term: 26 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:26:03.914006 31373 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:03.937000 20726 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:03.936825 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.943045 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:03.943989 20731 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:03.947607 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.955480 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.932443 31573 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:03.906347 31638 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.938056 31373 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.937785 31337 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.947135 31628 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:03.946787 31639 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:03.971349 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.954617 31368 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.976608 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:03.965991 31589 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:03.967705 31573 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.971256 31373 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:03.971450 31337 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.993300 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:03.964408 31540 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:26:03.983167 31413 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.026803 31608 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:04.030015 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1750) took 1172 ms. Trace:
I20260709 12:26:04.030467 20728 rpcz_store.cc:276] 0709 12:26:02.857314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.029186 (+1171872us) service_pool.cc:224] Handling call
0709 12:26:04.029534 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.029713 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.029738 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:04.029780 (+    42us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:04.029808 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.029855 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.029952 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.965372 31466 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.982561 31628 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.041507 20735 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:03.991600 31589 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:03.982828 31639 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:03.985563 31368 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:04.049862 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 134109us
I20260709 12:26:04.053313 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39562 (request call id 14) took 1195 ms. Trace:
I20260709 12:26:04.053768 20733 rpcz_store.cc:276] 0709 12:26:02.857558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.998870 (+1141312us) service_pool.cc:224] Handling call
0709 12:26:04.053241 (+ 54371us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:04.056944 31652 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:04.059600 31638 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.035625 31413 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.066431 31478 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.067283 31413 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.068089 31478 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.072381 20731 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
W20260709 12:26:04.048947 31651 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:26:04.046303 31505 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.093152 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:04.094110 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3990) took 1020 ms. Trace:
I20260709 12:26:04.091537 31540 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.095614 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:04.073521 31593 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.136053 31540 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.129035 31561 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.173113 31500 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.140712 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2546) took 1062 ms. Trace:
I20260709 12:26:04.137501 31633 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.141335 31501 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:04.186514 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3689) took 1545 ms. Trace:
I20260709 12:26:04.166539 31593 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:04.181910 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 114459us
I20260709 12:26:04.183609 21429 rpcz_store.cc:276] 0709 12:26:03.078324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.132824 (+ 54500us) service_pool.cc:224] Handling call
0709 12:26:03.158810 (+ 25986us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.528689 (+369879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.140424 (+611735us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b540058d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:04.140648 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848849792}
W20260709 12:26:04.178376 31610 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.260s	user 0.006s	sys 0.006s
I20260709 12:26:04.187075 20727 rpcz_store.cc:276] 0709 12:26:02.641096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.651357 (+ 10261us) service_pool.cc:224] Handling call
0709 12:26:04.186301 (+1534944us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54003ad4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:04.186437 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282118016}
I20260709 12:26:04.160719 31478 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.138473 20731 rpcz_store.cc:276] 0709 12:26:03.073623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.093053 (+1019430us) service_pool.cc:224] Handling call
0709 12:26:04.093453 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.093674 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.093715 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:04.093775 (+    60us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:04.093821 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.093890 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.094036 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:04.204156 20883 raft_consensus.cc:2368] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:04.155633 31589 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.217069 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.218941 20882 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
W20260709 12:26:04.207378 31648 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:26:04.236274 31593 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.238308 31540 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.239682 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.239794 31368 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.240839 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:04.241895 20809 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:04.242522 31337 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.243781 20876 raft_consensus.cc:2368] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:04.246402 31337 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.232600 31559 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 12:26:04.248063 31559 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:04.248416 31559 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:04.249635 31467 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.249310 31559 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.250166 31467 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.206038 31501 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.252352 31467 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.252705 31501 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:04.253852 31603 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.385s	user 0.000s	sys 0.004s
I20260709 12:26:04.255434 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1753) took 1305 ms. Trace:
I20260709 12:26:04.255837 20728 rpcz_store.cc:276] 0709 12:26:02.949594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.039797 (+1090203us) service_pool.cc:224] Handling call
0709 12:26:04.052454 (+ 12657us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.070340 (+ 17886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.070396 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.071096 (+   700us) log.cc:844] Serialized 52 byte log entry
0709 12:26:04.206019 (+134923us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:04.206060 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.206122 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.255017 (+ 48895us) raft_consensus.cc:1685] finished
0709 12:26:04.255040 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.255366 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":60287,"threads_started":1,"wal-append.queue_time_us":158108}
I20260709 12:26:04.257628 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3863) took 1116 ms. Trace:
I20260709 12:26:04.257948 20734 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:04.250555 31559 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:04.259493 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3989) took 1186 ms. Trace:
I20260709 12:26:04.258029 20878 rpcz_store.cc:276] 0709 12:26:03.140824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.140947 (+   123us) service_pool.cc:224] Handling call
0709 12:26:04.257472 (+1116525us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400366d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:04.257573 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520462592}
I20260709 12:26:04.259876 20733 rpcz_store.cc:276] 0709 12:26:03.073196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.054634 (+981438us) service_pool.cc:224] Handling call
0709 12:26:04.055011 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.055430 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.055474 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.055976 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.195771 (+139795us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:04.195837 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.195949 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.259255 (+ 63306us) raft_consensus.cc:1685] finished
0709 12:26:04.259278 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.259434 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10765,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"thread_start_us":139318,"threads_started":1,"wal-append.queue_time_us":200053}
I20260709 12:26:04.250366 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:04.265369 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1747) took 1409 ms. Trace:
I20260709 12:26:04.202451 31633 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.269026 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:04.265873 20729 rpcz_store.cc:276] 0709 12:26:02.856399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.903416 (+ 47017us) service_pool.cc:224] Handling call
0709 12:26:02.904894 (+  1478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.955463 (+ 50569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.265088 (+1309625us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b54000c44d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:04.265299 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114164992}
W20260709 12:26:04.268227 31655 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:26:04.268656 31628 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:04.265113 31368 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.269965 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:04.270370 31483 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:04.310005 20808 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
W20260709 12:26:04.311260 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.302731 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4035) took 1063 ms. Trace:
I20260709 12:26:04.315223 20804 rpcz_store.cc:276] 0709 12:26:03.239745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.342874 (+103129us) service_pool.cc:224] Handling call
0709 12:26:03.377374 (+ 34500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.702796 (+325422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.702827 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.702870 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:03.702899 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.292144 (+589245us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540006d454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:04.302471 (+ 10327us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.302666 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546050944}
I20260709 12:26:04.321022 20807 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:04.269865 31548 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.335405 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.271096 31500 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.273257 31638 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.316529 31483 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.269559 31660 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:04.309389 31573 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.291574 31368 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.338861 31540 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:04.342806 31483 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:04.340122 31638 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.337670 31373 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.338677 31548 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.363488 31573 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:04.364488 31548 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.356212 31614 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.368624 31373 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.369184 31614 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:04.371857 31639 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:04.372402 31611 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.450s	user 0.009s	sys 0.000s
I20260709 12:26:04.373447 31633 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:04.374261 31668 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:26:04.380659 31368 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.382252 20880 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 20. (term mismatch)
I20260709 12:26:04.382730 20880 pending_rounds.cc:85] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 19
I20260709 12:26:04.383033 20880 pending_rounds.cc:107] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 26.20
I20260709 12:26:04.383438 20880 raft_consensus.cc:2889] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.384792 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.385478 20880 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:04.386413 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.387718 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.389010 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.389103 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:04.389756 20877 raft_consensus.cc:606] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:04.389874 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.355168 31561 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.392128 31638 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.392767 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.391065 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.391039 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.390232 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:04.394832 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.400780 20878 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:04.401890 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.402534 20882 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:26:04.403383 21399 leader_election.cc:400] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:04.403843 21399 leader_election.cc:403] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.404016 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.405290 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.406322 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.407487 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.408871 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.411762 31638 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:04.414276 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.418653 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3857) took 1345 ms. Trace:
I20260709 12:26:04.419287 20879 rpcz_store.cc:276] 0709 12:26:03.073518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.073764 (+   246us) service_pool.cc:224] Handling call
0709 12:26:03.074047 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.216693 (+142646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.418315 (+1201622us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540005b754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:04.418569 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301252992}
I20260709 12:26:04.421288 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3953) took 1379 ms. Trace:
I20260709 12:26:04.411779 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.421860 20875 rpcz_store.cc:276] 0709 12:26:03.042037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.044216 (+  2179us) service_pool.cc:224] Handling call
0709 12:26:03.046636 (+  2420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.226335 (+179699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.420939 (+1194604us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540022ecd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:04.421201 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388014976}
I20260709 12:26:04.425508 20878 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:04.421782 31368 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:04.426443 31368 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.417706 20877 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
W20260709 12:26:04.431041 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.355309 31540 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.434092 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:04.437021 21429 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:04.426919 31466 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:04.429278 31368 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:04.308725 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:04.441802 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.553529 21428 raft_consensus.cc:2410] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:26:04.506253 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1754) took 1228 ms. Trace:
W20260709 12:26:04.554962 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 32 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:04.555728 20726 rpcz_store.cc:276] 0709 12:26:03.277711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.249523 (+971812us) service_pool.cc:224] Handling call
0709 12:26:04.249933 (+   410us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:04.250334 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.250367 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:04.251128 (+   761us) log.cc:844] Serialized 78 byte log entry
0709 12:26:04.336829 (+ 85701us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:04.336876 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.336950 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.505133 (+168183us) raft_consensus.cc:1685] finished
0709 12:26:04.505217 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.505907 (+   690us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2596,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":78393,"threads_started":1,"wal-append.queue_time_us":111260}
I20260709 12:26:04.560108 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.561566 21429 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:04.561743 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.566890 31347 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.568300 21427 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:04.510851 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3998) took 1376 ms. Trace:
I20260709 12:26:04.569726 20732 rpcz_store.cc:276] 0709 12:26:03.134538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.247086 (+1112548us) service_pool.cc:224] Handling call
0709 12:26:04.247476 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.247869 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.247906 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.248458 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.338198 (+ 89740us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:04.338242 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.338324 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.510199 (+171875us) raft_consensus.cc:1685] finished
0709 12:26:04.510248 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.510618 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1642,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":89322,"threads_started":1,"wal-append.queue_time_us":110795}
I20260709 12:26:04.571470 21426 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:04.571777 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.569118 31347 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.572862 21428 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:04.539153 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:04.544812 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.540221 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:04.545689 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.574831 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 1910) took 1228 ms. Trace:
I20260709 12:26:04.578205 20881 rpcz_store.cc:276] 0709 12:26:03.345723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.382851 (+ 37128us) service_pool.cc:224] Handling call
0709 12:26:04.574141 (+1191290us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400059e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:04.574501 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427747712}
I20260709 12:26:04.547902 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.579843 20882 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:04.550100 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:04.577162 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:04.536232 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:04.587771 21428 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:04.587234 31446 raft_consensus.cc:2764] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
W20260709 12:26:04.588765 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53024.9us
I20260709 12:26:04.538421 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:04.546926 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:04.536058 20734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1739) took 1831 ms (client timeout 1930 ms). Trace:
W20260709 12:26:04.591512 20734 rpcz_store.cc:269] 0709 12:26:02.704891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.715994 (+ 11103us) service_pool.cc:224] Handling call
0709 12:26:04.535420 (+1819426us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54000e74d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:04.535779 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3543712512}
W20260709 12:26:04.509397 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 200594us
I20260709 12:26:04.593750 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:04.595790 31446 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:04.597834 31466 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:04.576066 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:04.598658 31466 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.569140 31177 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.586115 31445 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:04.599623 31177 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.599792 31445 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.602310 31445 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.453750 31465 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.603001 31177 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.604835 31445 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:04.605795 31445 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.606643 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:04.608722 31445 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.609457 31466 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:04.613471 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.584375 31333 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:04.450927 31480 raft_consensus.cc:1064] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:26:04.614835 31333 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:04.615619 31480 raft_consensus.cc:1068] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Unable to promote non-voter 50c1ab13351d4427b836fbb449c6d7bb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:04.615535 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.616499 31333 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.614650 21435 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:04.620716 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.626077 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:04.626425 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:04.627209 20733 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:04.627429 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.627315 20732 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:04.628944 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:04.629827 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83264.4us
I20260709 12:26:04.630173 31468 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:04.630640 31468 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.631031 31468 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:04.598053 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:04.632485 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:04.632691 20729 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:26:04.635149 21430 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 12:26:04.635624 21430 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 16
I20260709 12:26:04.635913 21430 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:26:04.636217 21430 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:04.637807 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47898.2us
I20260709 12:26:04.644135 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.457404 31522 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:04.646374 31462 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:04.647552 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.648736 20808 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:04.649020 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:04.647338 31462 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.653901 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:04.654613 31462 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.654899 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:04.655488 31482 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.656222 31482 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.657212 31462 raft_consensus.cc:2764] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:26:04.658776 31482 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.661840 20733 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:04.640750 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.663887 20804 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:04.462821 31483 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.667330 20803 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:04.668433 21432 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32.
I20260709 12:26:04.669721 31483 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.670708 31177 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.671372 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:26:04.671669 31177 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.674753 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:04.677496 31465 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:04.678449 31465 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:04.682343 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.679366 31446 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.669876 21430 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:26:04.683526 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:04.683416 31446 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.684518 21433 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:26:04.679790 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:04.685648 20810 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:04.687731 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.688789 20810 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:04.692725 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:04.694404 30936 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:04.620918 20728 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:04.695704 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:04.697634 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:04.686837 31465 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:04.699116 31177 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.701684 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:04.699939 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:04.702404 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 104711us
I20260709 12:26:04.700798 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:04.701107 31465 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.703558 20808 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:04.703823 21428 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:04.703850 31465 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:04.450058 31540 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.704975 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:04.706241 31540 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:04.658936 31462 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:04.720592 20804 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:04.700726 31446 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.444725 31615 raft_consensus.cc:993] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:26:04.728407 31420 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.729125 31420 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.730135 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:04.452776 31638 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:04.731065 31373 raft_consensus.cc:1081] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:26:04.743642 20803 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:04.731922 31538 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:04.744865 31538 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.746260 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.747592 20808 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:04.749256 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.750363 30936 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:04.751530 31445 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:04.751997 31445 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.752034 30936 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:04.752583 31445 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:04.752785 30936 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:04.733028 21399 leader_election.cc:400] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:04.734993 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:04.735373 20726 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:04.756729 31157 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:04.758262 31658 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:04.755991 21399 leader_election.cc:403] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.754874 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:04.763743 31638 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:04.765990 31420 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:04.721107 31462 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.767987 31462 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:04.769282 31633 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.576001 31347 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.770267 31633 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:26:04.453299 31654 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.302s	user 0.001s	sys 0.001s
I20260709 12:26:04.775338 31561 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.775947 31633 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.776054 31561 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.779886 21431 raft_consensus.cc:2410] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 35 because replica has last-logged OpId of term: 34 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
I20260709 12:26:04.781133 31561 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:04.781939 21431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2534) took 2034 ms (client timeout 2412 ms). Trace:
W20260709 12:26:04.444562 31600 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.548s	user 0.000s	sys 0.004s
I20260709 12:26:04.783825 31468 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.757393 31157 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.786837 31157 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:04.791815 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 89557us
I20260709 12:26:04.793089 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:04.794353 31561 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:04.800515 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69816us
I20260709 12:26:04.806388 31538 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:04.808486 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:04.809705 31636 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:04.810030 20730 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:26:04.810185 31636 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.810647 31636 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:04.810972 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:04.815423 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 48828.9us
W20260709 12:26:04.782594 21431 rpcz_store.cc:269] 0709 12:26:02.747575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.749041 (+  1466us) service_pool.cc:224] Handling call
0709 12:26:04.781648 (+2032607us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54001e5d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:04.781852 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4607203584}
I20260709 12:26:04.818964 31616 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:04.819547 31318 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.820370 31561 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:04.820204 31318 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.822185 31661 raft_consensus.cc:1064] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
W20260709 12:26:04.822700 31661 raft_consensus.cc:1068] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Unable to promote non-voter 50c1ab13351d4427b836fbb449c6d7bb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:04.822700 31318 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:04.827883 20881 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:04.830080 20881 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:26:04.754499 30936 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.832347 20881 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:26:04.834124 30936 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:04.835584 20732 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:04.836233 31446 raft_consensus.cc:2764] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:26:04.839249 31465 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.840375 31468 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.840721 20729 raft_consensus.cc:2368] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:04.845660 31465 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:04.845963 31636 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:04.847164 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 140162us
I20260709 12:26:04.886759 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.887178 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:04.890396 31595 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:04.847345 20810 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:04.795126 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:04.893009 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.893880 31522 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:04.894786 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 99869.7us
I20260709 12:26:04.896071 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:04.851722 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:04.853523 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.865537 21426 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:04.867316 31466 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.868141 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:04.868356 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 55773.5us
I20260709 12:26:04.848768 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:04.877449 31445 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:04.945916 20881 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:04.945663 31522 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:04.949856 31522 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.950311 31522 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:04.952632 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56885.3us
I20260709 12:26:04.954442 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:04.955798 31177 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:04.896785 31446 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 12:26:04.950176 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.957842 20878 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:04.958122 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.957633 31177 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:04.958849 31177 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:04.909991 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3995) took 1802 ms. Trace:
I20260709 12:26:04.960472 31561 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:04.959991 31177 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:04.963515 31177 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:04.967797 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74828.9us
I20260709 12:26:04.969609 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:04.971899 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 123266us
I20260709 12:26:04.973346 31615 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:04.959856 20727 rpcz_store.cc:276] 0709 12:26:03.081878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.205470 (+1123592us) service_pool.cc:224] Handling call
0709 12:26:04.218401 (+ 12931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.225883 (+  7482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.225926 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.226503 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.507124 (+280621us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:04.868755 (+361631us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.871430 (+  2675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.878210 (+  6780us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.878285 (+    75us) raft_consensus.cc:1685] finished
0709 12:26:04.878305 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.884266 (+  5961us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":443257,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":3883,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":5432,"thread_start_us":626229,"threads_started":2,"wal-append.queue_time_us":137147}
I20260709 12:26:04.977295 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.978101 20876 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:04.978268 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.979997 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:04.981482 31468 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:04.982295 21431 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 20. (term mismatch)
I20260709 12:26:04.969663 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:04.935545 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.986490 31561 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:04.987457 21431 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 19
I20260709 12:26:04.990634 21431 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 34.20
I20260709 12:26:04.990978 21431 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.992118 20807 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:26:04.992235 20733 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:04.992820 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.992209 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:04.990096 31561 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:04.990958 31522 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.016271 20878 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:05.014736 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.018025 20881 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:04.992540 31482 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:26:05.023799 31482 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:05.024039 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:05.017890 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:05.024282 31482 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:05.027675 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.028785 20733 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:05.031769 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:04.999625 31633 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:26:04.999133 31468 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.036947 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:05.008671 31347 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:26:05.040143 31540 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:05.040472 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:05.038786 31639 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:05.042495 31636 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:05.019675 31595 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:05.047617 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.050791 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:05.051744 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.052407 31595 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.065570 20878 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:05.056263 31446 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:05.070650 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:05.071511 20884 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:05.054625 31639 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:05.072333 31446 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.079255 31446 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:05.079869 31633 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:05.069907 31595 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:05.090976 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52584.9us
I20260709 12:26:05.067317 31157 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.060956 31638 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.027s
I20260709 12:26:05.091795 31625 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:05.107831 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.079298 31627 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.077858 31465 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:05.088321 31445 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:05.154410 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 106556us
I20260709 12:26:05.094345 31548 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:05.096779 31466 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.079257 31617 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.092s
W20260709 12:26:05.163378 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 98879.8us
I20260709 12:26:05.131471 30936 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:05.076186 31561 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.151602 31559 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:05.169301 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 142890us
W20260709 12:26:05.171247 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63627.2us
I20260709 12:26:05.160216 31347 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:05.178128 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 95620.8us
I20260709 12:26:05.177734 31607 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:05.175540 31548 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.192245 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:05.193073 20879 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:05.193409 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.193524 31548 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.194537 31636 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:05.196602 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:05.197093 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:05.197918 21426 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:05.198957 31636 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:05.199508 31636 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:05.200089 31636 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:05.176455 31466 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:05.203522 31466 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:05.204540 31466 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.205693 31466 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:05.209472 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 3997) took 2102 ms. Trace:
I20260709 12:26:05.209522 31636 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.210199 20731 rpcz_store.cc:276] 0709 12:26:03.107033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.206565 (+1099532us) service_pool.cc:224] Handling call
0709 12:26:04.207418 (+   853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.213151 (+  5733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.213199 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.213989 (+   790us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.537172 (+323183us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b180037e880. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:04.949999 (+412827us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:04.950040 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.950151 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.208892 (+258741us) raft_consensus.cc:1685] finished
0709 12:26:05.208926 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.209257 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2384,"raft.queue_time_us":5290,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4985,"spinlock_wait_cycles":7077120,"thread_start_us":380707,"threads_started":1,"wal-append.queue_time_us":918969}
I20260709 12:26:05.212460 31636 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.212833 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.213186 31636 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.214193 31636 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.208798 21426 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:05.216460 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:05.218528 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:05.225641 20884 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:05.226424 20883 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:26:05.096544 31482 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.170424 31522 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:05.233980 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 148679us
I20260709 12:26:05.235100 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:05.169963 31617 consensus_queue.cc:1243] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 31.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:05.219551 31625 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:05.174530 31462 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.234519 31468 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:05.291429 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 74869.4us
I20260709 12:26:05.178058 31627 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:05.294257 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 64473.9us
W20260709 12:26:05.302966 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 115589us
I20260709 12:26:05.169754 31639 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.306206 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:05.333233 20878 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:05.347851 31693 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:05.348806 31693 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:05.349699 31664 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 12:26:05.351630 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52758.1us
I20260709 12:26:05.352339 31692 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:05.353266 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.351787 31607 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:05.282315 31347 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.300886 31446 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.360383 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:05.367586 20849 leader_election.cc:400] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:26:05.368083 20849 leader_election.cc:403] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:26:05.357178 31615 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:05.344298 31157 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:05.346592 31482 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:05.385478 31446 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:05.401676 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46963.5us
I20260709 12:26:05.354841 31636 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.398836 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:05.424019 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 155047us
I20260709 12:26:05.414525 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:05.362550 31692 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:05.435135 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:05.457335 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.461952 31177 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:26:05.463032 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.467240 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.412915 31607 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:05.470108 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:05.431820 31483 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:05.468997 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.472494 20728 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
W20260709 12:26:05.489406 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.495041 20801 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:05.473343 31693 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:05.498272 31692 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.431095 20807 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33470
I20260709 12:26:05.500226 20807 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:05.500634 20807 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:05.502676 31607 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.503333 31483 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.506853 31462 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.511297 21426 pending_rounds.cc:85] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 18
I20260709 12:26:05.511698 21426 pending_rounds.cc:107] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 31.19
I20260709 12:26:05.511940 21426 raft_consensus.cc:2889] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.511368 31695 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:05.513563 31695 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.508241 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:05.515640 20805 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:05.473861 20731 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:05.516258 31445 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:05.495532 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.496694 31540 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:05.516669 31445 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:05.507947 31689 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:26:05.571318 31700 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:05.572786 20731 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:26:05.573144 31347 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:05.541610 31540 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.548218 20727 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:05.552708 20806 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:26:05.552850 31445 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:05.554768 31685 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:26:05.560822 31483 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:05.466125 31522 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:05.567364 31627 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.606928 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.577625 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:05.608162 20880 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:05.609742 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:05.465981 31177 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.603236 20879 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:05.611549 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:05.602699 20807 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.614362 31695 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:05.612398 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.601549 31617 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:26:05.611680 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:05.613826 31522 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.583776 31700 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:05.633402 20881 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:05.634755 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:05.637044 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 223635us
I20260709 12:26:05.642051 20801 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:05.629539 31616 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27.
I20260709 12:26:05.654561 20807 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.655082 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:05.632474 31177 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:05.648841 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:05.633733 31625 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.668814 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.679720 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:05.680760 20806 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:05.693343 20805 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:05.695453 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.654007 31704 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:05.671011 31540 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:05.705760 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.705720 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 470686us
I20260709 12:26:05.707019 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:05.708060 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:05.708640 20884 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:05.728782 20802 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:05.724870 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:05.725771 31466 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.000s
I20260709 12:26:05.676577 31625 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.719058 31702 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.745028 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:05.747382 30936 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:05.750339 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.751346 20731 raft_consensus.cc:2410] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 32 because replica has last-logged OpId of term: 31 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 16.
I20260709 12:26:05.751402 20727 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:05.752518 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.753191 20734 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:05.755319 31696 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:05.756019 31696 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.758529 31696 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.759268 31707 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:05.759701 20728 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:05.759807 31707 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.760226 31707 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:05.761210 31706 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.193s	user 0.002s	sys 0.001s
I20260709 12:26:05.763720 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.764724 20734 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:05.765288 31702 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.766382 20734 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:05.767968 31702 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:05.773432 20727 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:05.776453 31522 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:05.775064 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.782243 20728 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:26:05.776293 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:05.785703 20881 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:05.788264 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:05.794924 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:05.786850 20727 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:05.819862 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:05.822674 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.823473 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.824240 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.873238 20729 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 36: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:05.825696 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.826912 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.827531 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.874156 20730 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:05.828212 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.741060 31704 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:05.876227 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:05.836707 31595 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.737s	user 0.006s	sys 0.007s
I20260709 12:26:05.839788 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:05.895756 20875 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:05.843560 21431 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:05.843705 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:05.896108 20875 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:05.844777 31559 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:05.845129 31625 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:05.909693 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.845534 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:05.846946 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.847466 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:05.848007 31462 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:05.848198 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:05.849313 31698 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:05.851004 31694 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:26:05.855602 20850 leader_election.cc:336] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.581s (SENT)
I20260709 12:26:05.863267 31701 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:05.870046 31707 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:05.871948 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.731093 31700 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:05.898415 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:05.899171 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:05.930449 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.931173 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.937336 20806 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
W20260709 12:26:05.946679 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 99810.9us
I20260709 12:26:05.944257 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.945199 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.952975 21428 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
W20260709 12:26:05.954483 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 107075us
I20260709 12:26:05.955471 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:05.960882 20802 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:05.958140 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:05.960606 20727 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:05.961838 21427 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:05.967374 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:05.967651 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:05.970234 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 354141us
I20260709 12:26:05.954028 31696 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:05.959501 31693 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:05.952008 31701 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:05.973027 21430 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:05.897785 31595 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:06.085896 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47231.1us
I20260709 12:26:05.974970 31658 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:06.030201 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 3980) took 1016 ms. Trace:
I20260709 12:26:06.012617 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.997776 20731 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:05.998948 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:05.960248 31705 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.095579 31705 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:06.096124 31699 raft_consensus.cc:670] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } [suppressed 2 similar messages]
W20260709 12:26:06.029714 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62468.7us
I20260709 12:26:06.098793 31702 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:06.098961 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:06.002336 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:06.099979 20732 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:06.103920 31705 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.104615 31633 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:06.105605 31633 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:06.106252 31633 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:06.106585 31633 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:06.085103 31483 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:06.107456 31633 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:06.108721 31717 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:26:06.108942 31633 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:06.111939 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.112218 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 256652us
I20260709 12:26:06.112973 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:06.113708 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.755s (SENT) [suppressed 8 similar messages]
I20260709 12:26:06.114169 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:06.115504 31616 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:06.116276 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:06.117117 31482 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:06.117522 31482 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:06.117676 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:06.117964 31482 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:06.118515 31695 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:06.118937 31695 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:06.119664 31695 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:06.064154 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 133362us
I20260709 12:26:06.009912 31693 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:05.955549 31639 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.038s
I20260709 12:26:05.964695 31462 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:05.966028 31694 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:05.956555 31636 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.025s
I20260709 12:26:05.968218 31559 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:06.120904 31714 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:26:06.084841 31713 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.087531 31709 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.017273 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.065879 31177 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:06.129312 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 149915us
I20260709 12:26:06.118991 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.093801 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:06.131147 31465 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:06.092240 20876 rpcz_store.cc:276] 0709 12:26:05.013380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.013560 (+   180us) service_pool.cc:224] Handling call
0709 12:26:05.043312 (+ 29752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.069647 (+ 26335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.069678 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.084555 (+ 14877us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.294170 (+209615us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:05.294212 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.294290 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.029852 (+735562us) raft_consensus.cc:1685] finished
0709 12:26:06.029885 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.030118 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7682,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":19018,"thread_start_us":161163,"threads_started":1,"wal-append.queue_time_us":608744}
W20260709 12:26:06.151669 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62076.2us
I20260709 12:26:06.116111 31616 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.137405 20732 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:06.163966 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:06.168998 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.171408 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:06.171716 31693 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.151135 31466 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:06.172209 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.172366 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.172809 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:06.177403 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.161293 31705 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.179338 20733 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29.
I20260709 12:26:06.180032 20729 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:06.175889 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62208.5us
I20260709 12:26:06.181773 20732 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:06.169524 31465 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.184082 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:06.187225 31465 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:06.188360 31704 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.189016 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:06.189828 31616 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:06.189986 31661 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:06.190135 31704 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:06.190446 31616 raft_consensus.cc:2764] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election decision vote started in defunct term 32: won
I20260709 12:26:06.190645 31661 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:06.167335 31709 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.191289 31616 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 29.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:06.191454 31615 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:06.191717 31661 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.192983 31709 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:06.194099 31559 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.180397 31713 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:06.185174 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66261.5us
I20260709 12:26:06.196741 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:06.185427 31718 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
I20260709 12:26:06.150976 31701 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.195971 20849 leader_election.cc:400] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:06.205701 20849 leader_election.cc:403] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Cancelling election due to peer responding with higher term
I20260709 12:26:06.197901 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.212168 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.182849 31705 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.215229 31713 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.214903 31694 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.215895 31559 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:06.175603 31658 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:26:06.218106 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.219492 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.182893 31692 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:06.231830 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.232728 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.169268 31462 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.236135 20880 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:06.236475 20729 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:06.196858 31726 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:06.238957 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.236563 31540 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:06.229569 31705 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.234586 31607 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:06.244681 31726 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.245332 31726 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:06.231783 31707 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.248541 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:06.250036 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:06.251490 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:06.252115 20729 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:06.256875 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 92991.9us
I20260709 12:26:06.234535 31559 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:06.256335 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60528.1us
I20260709 12:26:06.205029 31625 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.039s
I20260709 12:26:06.261093 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:06.217274 31617 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
I20260709 12:26:06.263437 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:06.262678 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:06.265165 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:06.254676 31638 raft_consensus.cc:2764] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:26:06.255601 31709 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:06.274876 31462 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:06.280278 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.281235 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.283557 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47996us
I20260709 12:26:06.285426 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:06.265857 31707 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:06.261049 31615 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:06.289741 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:06.299464 31337 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:06.301304 31721 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:06.301796 31721 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:06.287902 30936 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:06.304652 31465 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.305965 20732 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:06.262020 31705 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:06.308197 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:06.305913 31595 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.302898 31721 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.282868 31709 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:06.315673 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:06.317390 31732 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:06.319368 31177 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.320807 31695 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.321273 20729 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:06.322242 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:06.322779 20729 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:06.323767 31615 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.324827 31482 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.293856 31638 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:06.332322 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:06.333247 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:06.331105 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:06.331559 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:06.334358 20808 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:06.325351 31639 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:06.338718 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:06.334329 31638 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:06.341640 20802 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:26:06.342031 31638 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:06.334456 31337 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:06.334892 31633 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:06.314215 31616 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:06.346033 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:06.351807 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59214.1us
I20260709 12:26:06.352952 20805 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:06.359225 31726 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.362630 31726 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:06.363461 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54326.3us
I20260709 12:26:06.335276 31695 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:06.345196 31616 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:06.353832 31337 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.366737 31465 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.345419 31661 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:26:06.343251 31639 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.491564 31639 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:06.491854 20727 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:26:06.493443 31482 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.495098 31445 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:06.344990 31721 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:06.428856 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76288.1us
I20260709 12:26:06.499596 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:06.460872 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 154324us
I20260709 12:26:06.468187 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:06.469450 21430 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:26:06.502239 21432 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:06.300048 31733 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 12:26:06.476764 20729 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:06.341780 31709 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.424139 20809 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:06.487589 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.488744 31616 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:06.488600 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:06.497323 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 171197us
I20260709 12:26:06.513113 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:06.503432 20808 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:06.518461 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:06.519233 20734 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:06.520015 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.527436 20729 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:06.528950 20729 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:06.532759 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:06.546589 20727 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:06.555598 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:06.557597 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:06.560531 21427 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:06.541855 31719 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.383s	user 0.001s	sys 0.003s
I20260709 12:26:06.561769 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:06.512428 31692 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:06.563781 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:06.566061 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:06.572039 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:06.576143 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.576994 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:06.579044 31318 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:06.580521 31318 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:06.581043 31318 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:06.583851 31729 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.343s	user 0.008s	sys 0.000s
I20260709 12:26:06.586153 31638 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.587493 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:06.588258 20808 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:06.589392 31617 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.589953 31617 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.591847 31741 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:06.592458 31617 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.592712 31741 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:06.531718 31738 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:06.542052 31482 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:06.558689 31695 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:06.605821 21432 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:26:06.594249 31741 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.615574 31695 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.546433 31739 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.096s
I20260709 12:26:06.616250 31695 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.617708 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:06.618559 20727 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:06.542732 31157 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.081s
I20260709 12:26:06.627318 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:06.628000 21430 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:06.628404 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:06.628573 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:06.629037 21431 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:06.631573 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:06.632227 21434 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:06.636497 31333 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.637115 31333 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.641367 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:06.643668 21432 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:06.644832 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:06.645570 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69390.7us
I20260709 12:26:06.645792 20728 raft_consensus.cc:2368] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:06.647383 21427 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
W20260709 12:26:06.651391 31720 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.544s	user 0.002s	sys 0.001s
I20260709 12:26:06.652228 31701 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:06.655161 21431 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:06.666177 20729 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:06.661723 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:06.659458 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:06.644966 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:06.671439 21430 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:06.691874 20734 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:06.673341 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.708640 20732 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:06.707969 20728 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:06.718396 20880 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:06.583115 31318 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.643114 31700 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.720201 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.576512 31633 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:06.618613 31695 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.706151 31446 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election won for term 35
W20260709 12:26:06.730170 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 85278.1us
I20260709 12:26:06.667315 31157 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.060s
I20260709 12:26:06.711033 31347 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.747787 21434 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:06.719920 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.734344 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:06.726030 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.722189 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:06.731320 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82097.5us
I20260709 12:26:06.663386 31445 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:06.666536 31737 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:06.728850 31633 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:06.732697 31705 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:06.727227 31333 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:06.757853 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:06.758807 20728 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:06.756639 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.754570 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:06.744019 31446 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:06.765728 20732 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:26:06.757951 31347 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.774794 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:06.776074 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.777511 31617 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.777987 31617 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.779987 31559 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:06.780244 31694 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:06.780735 31694 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.781102 31694 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:06.780046 31617 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:06.781993 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.782696 20734 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:06.759887 31639 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:06.784075 31723 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.674s	user 0.004s	sys 0.001s
I20260709 12:26:06.785423 21429 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
W20260709 12:26:06.785790 31712 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.669s	user 0.004s	sys 0.001s
I20260709 12:26:06.786598 31157 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.015s
I20260709 12:26:06.787179 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2573) took 1281 ms. Trace:
I20260709 12:26:06.787626 21426 rpcz_store.cc:276] 0709 12:26:05.505412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.510720 (+  5308us) service_pool.cc:224] Handling call
0709 12:26:05.511055 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:05.542775 (+ 31720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.542825 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:05.543519 (+   694us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.944459 (+400940us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:05.944517 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.944607 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.785183 (+840576us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:06.786857 (+  1674us) raft_consensus.cc:1685] finished
0709 12:26:06.786878 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.787103 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9855,"raft.queue_time_us":60331,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":144,"spinlock_wait_cycles":77708672,"thread_start_us":400534,"threads_started":1,"wal-append.queue_time_us":471672}
I20260709 12:26:06.766793 31733 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
W20260709 12:26:06.792588 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.793272 31333 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:06.780390 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.805732 20728 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:06.807050 20735 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: [26.20-27.21]
I20260709 12:26:06.757845 31445 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:06.758843 31466 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:06.761548 31633 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:06.809276 31695 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:06.776089 31347 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.814770 20806 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:06.833287 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:06.816177 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.818223 31318 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:06.820842 31337 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:06.821588 31616 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.822392 31607 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:06.823710 31446 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.772794 31705 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:06.792661 31658 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:06.826153 31617 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:06.826686 31747 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:06.827723 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:06.741542 31700 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:06.835479 20806 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:06.835682 20727 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 25.
I20260709 12:26:06.848322 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.809099 31737 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:06.864447 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 166172us
I20260709 12:26:06.846742 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.880923 20879 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:06.881850 20884 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:06.882347 20881 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:06.867005 31751 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:06.887100 20735 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:26:06.837635 31318 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:06.865581 31730 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.875064 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:06.876046 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48434.4us
I20260709 12:26:06.875928 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.896567 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.897372 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:06.873274 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.901216 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.902489 31333 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:06.902697 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.908718 31737 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:06.832449 31445 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:06.868674 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.963613 20880 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:06.964833 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:06.965582 20880 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:06.872473 20876 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:26:06.884982 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:06.967319 20808 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:26:06.836232 31695 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.907378 31747 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:06.870237 31658 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:06.907579 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.909111 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:06.911394 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:06.904428 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:06.916491 31616 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:06.914413 31700 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:06.864818 31177 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:06.878222 31721 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:06.885982 31466 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:06.889998 31751 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:06.946830 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 281997us
W20260709 12:26:06.947866 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 170727us
I20260709 12:26:06.899063 20878 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:26:06.954083 31615 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.008s
I20260709 12:26:06.962466 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:06.963703 20809 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:06.912922 31738 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:06.984561 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:06.986995 20884 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:06.991334 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:06.950726 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:07.068758 20884 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
W20260709 12:26:07.069152 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 118525us
I20260709 12:26:06.986076 31695 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:06.985661 31333 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:07.021705 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.035889 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:07.066923 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.072296 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:07.074149 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:07.074744 20882 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:06.912935 31705 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.18, Last appended by leader: 18, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:07.087796 20733 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 4020) took 1634 ms (client timeout 1781 ms). Trace:
I20260709 12:26:07.121351 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:06.948287 31730 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:07.092553 20733 rpcz_store.cc:269] 0709 12:26:05.453320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.453984 (+   664us) service_pool.cc:224] Handling call
0709 12:26:07.087556 (+1633572us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400249fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:07.087727 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635323264}
I20260709 12:26:07.121417 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.068321 31347 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.070335 31466 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:07.132138 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 229490us
I20260709 12:26:07.133016 20729 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:07.140777 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:07.141784 21400 leader_election.cc:400] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:07.142073 21400 leader_election.cc:403] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:07.138821 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:07.071956 31337 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:07.143962 20735 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:26:07.002792 31721 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.156023 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.121803 31559 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:07.123498 31755 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:07.063273 31177 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.164292 21430 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:26:07.165637 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.167761 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.008064 31694 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.205716 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:07.121882 31733 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 12:26:07.228699 31733 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:07.229095 31733 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:07.229918 31733 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.231732 31466 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:07.173170 31701 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.520s	user 0.006s	sys 0.006s
I20260709 12:26:07.212845 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.218181 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:07.218353 31751 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:07.163745 31559 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:07.219621 20875 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.18, Last appended by leader: 18, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.245273 20875 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:07.245199 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:07.221553 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:07.224189 31747 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:07.088099 31318 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.155467 31483 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:07.130440 31157 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.073s
I20260709 12:26:07.242981 21426 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:07.211805 21429 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 25.
I20260709 12:26:07.207039 21434 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:07.259943 21434 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:07.164294 31333 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:26:07.246495 31559 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:07.251103 20730 pending_rounds.cc:85] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 17
I20260709 12:26:07.262915 20730 pending_rounds.cc:107] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 30.18
I20260709 12:26:07.263310 20730 raft_consensus.cc:2889] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:07.263540 20881 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:07.147342 31347 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.252060 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:07.268636 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 284173us
I20260709 12:26:07.269745 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.256991 20732 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:26:07.257100 20875 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:07.240978 31733 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:07.252144 31760 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:07.279800 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3894) took 1495 ms (client timeout 1531 ms). Trace:
I20260709 12:26:07.276600 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3723) took 1327 ms. Trace:
I20260709 12:26:07.297416 20731 rpcz_store.cc:276] 0709 12:26:05.948886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.192812 (+243926us) service_pool.cc:224] Handling call
0709 12:26:07.251919 (+1059107us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:26:07.276537 (+ 24618us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13849856}
I20260709 12:26:07.298141 20729 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:26:07.286875 31701 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:07.313918 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.315850 20732 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:07.288103 31721 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.288887 31638 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:07.289463 20882 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:07.290463 20728 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
W20260709 12:26:07.291558 31758 log.cc:927] Time spent T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:26:07.294800 31730 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.295328 31318 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:07.255597 31483 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:07.286267 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.141067 31658 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:07.297312 20875 rpcz_store.cc:269] 0709 12:26:05.784097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.790772 (+  6675us) service_pool.cc:224] Handling call
0709 12:26:06.790071 (+999299us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5400259a20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1d06f 00007f9e1ec95f60 00007f9e1ec977de 00007f9e1eca327c 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2
0709 12:26:07.219221 (+429150us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b5400259a20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec1c520 00007f9e1ec95f70 00007f9e1ec977de 00007f9e1eca327c 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2
0709 12:26:07.279715 (+ 60494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92442240}
I20260709 12:26:07.319296 20804 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 15. Preceding OpId from leader: term: 32 index: 16. (index mismatch)
I20260709 12:26:07.319399 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:07.332341 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.214471 31755 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:07.357846 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71629.7us
I20260709 12:26:07.319876 31753 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.359005 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:07.317420 31766 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:07.363770 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 419193us
I20260709 12:26:07.348866 31483 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:07.349185 31638 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:07.379010 20880 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:07.361934 31462 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:07.399724 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:07.407639 20806 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:26:07.323257 31760 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.359957 31482 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:26:07.384814 31658 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:07.382642 31768 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.385995 31617 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:07.390496 31769 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.418404 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:07.390894 31730 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:07.374658 31753 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.414577 31638 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:07.453085 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 314276us
W20260709 12:26:07.432404 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 274251us
W20260709 12:26:07.439785 31759 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:26:07.423700 30936 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:07.443473 31773 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.032s
I20260709 12:26:07.406068 31767 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.038s
I20260709 12:26:07.445109 31776 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:07.445819 31760 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.446264 31739 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:07.447093 31741 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:07.473441 31741 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:07.473625 31347 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.474468 31741 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.475646 31741 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:07.470587 31739 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:07.480276 31696 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:07.425451 31462 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.037s
I20260709 12:26:07.480722 31753 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.481434 31462 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:07.481065 31739 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.22, Last appended by leader: 22, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.481851 31462 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:07.482848 31779 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.482993 31737 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.483342 31772 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:26:07.483635 31779 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.484547 31772 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.485110 31772 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.486375 20727 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 31. Ops: []
I20260709 12:26:07.424860 31617 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.487473 31617 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:07.489567 31779 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.490195 31779 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:07.490545 31779 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:07.491564 31779 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.452483 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:07.492889 31779 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:07.417924 31466 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 12:26:07.495450 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:07.459649 20801 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:07.427291 31769 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.448527 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.467757 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.469393 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.480808 31696 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:07.431861 31694 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.506567 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.522528 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:07.534476 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90102.2us
I20260709 12:26:07.537837 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:07.539150 31771 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:07.539739 31771 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:07.539781 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106337us
I20260709 12:26:07.540370 31771 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:07.541788 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.542287 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.540243 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:07.548761 20728 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:07.548864 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:07.554239 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86608.1us
I20260709 12:26:07.424125 31768 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.495522 31760 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:07.465098 30936 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:07.472365 31177 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.497143 31737 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.570351 20732 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:07.570658 20734 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:07.558596 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:07.559281 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 52855us
I20260709 12:26:07.572081 20726 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:07.451680 31730 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:07.550119 31759 log.cc:927] Time spent T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:26:07.574636 31745 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.819s	user 0.006s	sys 0.003s
I20260709 12:26:07.575954 31768 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.461256 31638 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.534420 31753 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:07.583537 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 127158us
I20260709 12:26:07.585776 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:07.567622 31778 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:07.588580 31777 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:07.589458 31775 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:07.574666 31701 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:07.591511 31559 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.594009 31483 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:07.595103 31445 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.002s
I20260709 12:26:07.600896 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:07.586545 31763 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:26:07.597069 31791 raft_consensus.cc:1064] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:26:07.588631 31772 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.601790 31337 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:07.624818 31337 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:07.625335 31337 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:07.596091 31333 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:07.627539 21431 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:07.605489 20877 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 31 because replica has last-logged OpId of term: 30 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:26:07.577790 31713 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:26:07.596765 31633 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.735s	user 0.009s	sys 0.003s
I20260709 12:26:07.598176 31760 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:07.629844 31633 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:26:07.630862 31760 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.633541 20805 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:07.618300 31482 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.634517 31773 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:07.648806 31773 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.650342 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3726) took 1037 ms. Trace:
I20260709 12:26:07.650761 20733 rpcz_store.cc:276] 0709 12:26:06.612548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.131788 (+519240us) service_pool.cc:224] Handling call
0709 12:26:07.154995 (+ 23207us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.357046 (+202051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.357077 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.385339 (+ 28262us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.515403 (+130064us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:07.544600 (+ 29197us) log.cc:844] Serialized 14 byte log entry
0709 12:26:07.601059 (+ 56459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.601128 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.650129 (+ 49001us) raft_consensus.cc:1685] finished
0709 12:26:07.650150 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.650281 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62592,"prepare.queue_time_us":37787,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":214576,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7180,"thread_start_us":333326,"threads_started":3,"wal-append.queue_time_us":245751}
I20260709 12:26:07.652244 31694 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.653285 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.652982 31694 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:07.653761 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.654165 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.654723 31772 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.594899 31783 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.100s
I20260709 12:26:07.656869 31318 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:07.629173 31713 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:07.657487 31318 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.482848 31462 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.658598 31713 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.660713 31462 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:07.622694 31445 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.623035 31791 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 18, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:07.662698 31695 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 12:26:07.663301 31695 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.637652 31769 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.611369 31777 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.611931 31778 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.668015 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:07.668439 31778 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:07.669267 31318 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.634696 31775 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:07.670644 31775 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.622193 31483 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:07.676730 31751 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:07.622334 31559 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.668107 31777 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:07.697589 31695 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.645414 31755 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.655324 31772 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.664542 20803 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:26:07.644376 31696 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.639609 30936 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.703431 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:07.713325 31694 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.703917 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:07.705300 20808 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:07.702922 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:07.708520 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.708981 31730 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.708537 31772 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.715845 21434 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:07.719848 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:07.581219 20728 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:07.726332 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.600788 31767 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.106s
I20260709 12:26:07.729581 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:07.734166 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:07.734633 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.734937 20805 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:07.735276 21426 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:07.736738 21432 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:07.634473 31633 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.738731 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.739070 21427 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
W20260709 12:26:07.730014 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 189832us
W20260709 12:26:07.742409 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.744004 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:07.732491 31694 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:07.749243 31694 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:07.749723 31694 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:07.750182 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:07.751910 31445 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.706177 31482 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:07.747512 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.731213 31638 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.747715 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.759284 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.760106 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:07.762300 20882 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:07.764478 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:07.773730 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.778151 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.779162 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.779718 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.783627 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.773018 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:07.790479 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:07.790700 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.791764 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:07.765591 31730 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:07.755110 31482 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:07.766041 31347 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:07.808794 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.815497 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:07.805006 20882 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:07.811308 21433 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:07.815872 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:07.812804 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.818325 20881 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:07.771706 31785 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:26:07.820444 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 249688us
I20260709 12:26:07.823959 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:07.824841 21430 raft_consensus.cc:2376] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32. Re-sending same reply.
I20260709 12:26:07.826891 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.827899 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:07.832584 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.808420 31769 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:07.765805 31755 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.053s
I20260709 12:26:07.806740 31465 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:07.863758 21435 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:07.866755 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:07.815425 31730 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.876693 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:07.880281 21433 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:07.881580 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:07.821357 31633 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:07.792186 31625 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.082s
I20260709 12:26:07.774546 31446 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.053s
I20260709 12:26:07.885444 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.885686 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.886600 20878 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:07.887979 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:07.888813 20733 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:07.894343 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:07.900964 20732 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:07.818286 31783 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.066s
I20260709 12:26:07.907398 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:07.908205 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 152354us
I20260709 12:26:07.865221 31347 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:07.906535 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:07.879222 31465 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.868650 31785 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:07.915244 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:07.891090 31730 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:07.890653 31638 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:07.866307 31445 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:26:07.885371 31705 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.926584 20732 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 15. Preceding OpId from leader: term: 32 index: 16. (index mismatch)
I20260709 12:26:07.924826 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:07.926916 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.943140 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.946403 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:07.947337 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.948321 21430 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29.
I20260709 12:26:07.886416 31559 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:26:07.923138 31465 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:07.919459 31692 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:07.926483 31638 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:07.925074 31446 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:07.965278 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.927891 31770 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.983635 21430 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 26.
I20260709 12:26:07.986730 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:07.987457 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:07.944921 31705 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:07.987524 21430 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:26:07.963316 31700 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:07.991921 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:07.993753 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80596.1us
I20260709 12:26:07.971979 31692 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.962889 31773 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:07.931103 31694 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:07.995921 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:07.999982 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:07.942216 31617 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.002041 20881 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:07.984483 31789 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.925040 31786 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.076s
I20260709 12:26:08.004408 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:08.005983 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 98760.1us
I20260709 12:26:08.010326 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:08.012547 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.012962 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.013648 20728 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:08.011453 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:07.999914 31692 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:08.024684 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.028261 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:08.061933 20734 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:26:08.009671 31445 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:08.081053 21430 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:26:08.004427 31347 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:08.080577 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59468.2us
W20260709 12:26:08.073226 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.075091 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.084756 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64390.2us
I20260709 12:26:08.100065 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.086210 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:08.085000 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60402.2us
I20260709 12:26:08.101150 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.101951 20881 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:08.102757 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:08.103631 20882 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:08.103422 21430 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:07.990911 31770 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.104473 21433 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:08.017771 31701 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:08.024190 31784 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:08.131423 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:08.119098 31798 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:08.024109 31772 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:08.085618 31446 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.118829 31801 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:26:08.147931 31446 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.119205 31761 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:08.149410 31770 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.111531 31696 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:08.150594 31446 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.151309 31809 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:08.137079 31798 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:08.152505 31465 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.152793 31798 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.24, Last appended by leader: 24, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.153623 20731 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:08.154006 31798 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:08.154299 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.025966 31789 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.156091 20880 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:08.155728 31337 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.156842 31337 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:08.157176 31789 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:08.018908 31638 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.158319 31638 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:08.159685 31771 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.618s	user 0.006s	sys 0.005s
I20260709 12:26:08.159955 31465 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.160141 31771 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.161474 21435 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:08.162249 31639 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.165014 20878 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 18. Preceding OpId from leader: term: 36 index: 19. (index mismatch)
I20260709 12:26:08.121692 31808 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:08.103315 31445 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.123410 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:08.123667 31694 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.024214 31741 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.087s
I20260709 12:26:08.097944 31755 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:08.128713 31705 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.130239 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.106472 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:08.107329 31700 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:08.139142 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.140834 20850 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29.
I20260709 12:26:08.172034 20850 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:08.145689 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.006903 31785 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.136240 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:08.173588 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.162956 31771 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.171403 20732 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:08.142239 31733 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:08.169360 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62992.1us
I20260709 12:26:08.179111 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.179944 31773 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:08.180337 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:08.181954 31773 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:08.182657 31773 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:08.183004 31773 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.183389 31773 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:08.187495 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:08.189682 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53582.5us
I20260709 12:26:08.190752 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:08.192931 20881 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:26:08.169129 31445 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:08.198246 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.200850 20733 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 18. Preceding OpId from leader: term: 36 index: 19. (index mismatch)
I20260709 12:26:08.201730 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:08.204348 31816 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:08.154026 31462 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:08.205389 31807 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:26:08.170807 31761 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.130180 31778 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.216723 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.165349 31730 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.172848 31639 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:08.196127 31738 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:08.175478 31337 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:08.220300 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.220968 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.212491 31705 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:26:08.189117 31806 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:26:08.207110 31694 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:08.221617 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:08.230131 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:08.230751 20732 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:08.226181 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
W20260709 12:26:08.226667 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85988.4us
W20260709 12:26:08.183349 31749 consensus_queue.cc:1175] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 28 status { last_received { term: 25 index: 19 } last_committed_idx: 19 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: [26.20-27.21]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:08.216928 31770 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:08.233707 21429 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: [30.22-30.22]
W20260709 12:26:08.219445 30936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.019s	sys 0.027s
I20260709 12:26:08.234694 20727 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:08.234764 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:08.240916 21432 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
W20260709 12:26:08.241284 30936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.020s	sys 0.028s
I20260709 12:26:08.223217 31462 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.255610 31730 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.230957 31772 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 29
W20260709 12:26:08.269335 31753 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.000s
I20260709 12:26:08.232184 31639 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:08.224679 31809 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 12:26:08.278805 31791 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:26:08.280157 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.280360 21434 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:26:08.226711 31733 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:26:08.237308 31816 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:08.294473 20726 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
W20260709 12:26:08.292872 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45404.8us
I20260709 12:26:08.298581 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:08.260722 31462 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.228127 31761 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.288973 31753 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:08.308423 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:08.307310 20805 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:08.310086 31774 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:08.310391 31789 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:08.257467 31700 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.312402 20804 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:08.312606 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.313602 31738 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:08.313962 31738 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.314356 31738 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:08.316179 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4070) took 1454 ms. Trace:
I20260709 12:26:08.316459 31639 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.316704 20803 rpcz_store.cc:276] 0709 12:26:06.861957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.862648 (+   691us) service_pool.cc:224] Handling call
0709 12:26:06.863103 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.706361 (+843258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.315808 (+609447us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b54002bff54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:08.316080 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1842384640}
I20260709 12:26:08.319604 20733 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:08.320089 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.320927 31761 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.321185 20808 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
W20260709 12:26:08.323439 31777 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.004s
I20260709 12:26:08.323917 31777 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.325186 20730 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.19-28.19]
I20260709 12:26:08.326283 31777 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.326678 31633 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:08.327236 31633 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.331243 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.333057 20803 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 31 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
I20260709 12:26:08.332965 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:08.336738 31617 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.336984 31816 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.340318 20803 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:08.334012 20808 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:08.346478 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.342473 31773 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.307263 31705 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.345626 31633 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.349965 31705 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.353025 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:08.354544 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:08.357933 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.358587 20804 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:26:08.364646 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.340332 31778 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:08.364166 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.374696 21428 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:08.376375 20730 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:26:08.342824 31692 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.381793 31741 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:08.364775 31751 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.018s
W20260709 12:26:08.385310 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.361766 31777 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.034s
I20260709 12:26:08.388365 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.380052 31789 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:08.386726 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.407723 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:08.508854 20881 raft_consensus.cc:2376] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36. Re-sending same reply.
I20260709 12:26:08.409125 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:08.418491 20808 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
W20260709 12:26:08.457782 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69510us
I20260709 12:26:08.397707 31617 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.065s
I20260709 12:26:08.464485 31773 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.520056 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.444245 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:08.463745 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:08.465277 31347 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:08.465652 31808 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:08.465049 31772 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29.
I20260709 12:26:08.447880 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:08.533195 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.533845 21434 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:08.499346 31482 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:08.392432 31705 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:08.495023 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.457432 20803 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36.
I20260709 12:26:08.393378 31753 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
W20260709 12:26:08.496798 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77688.2us
W20260709 12:26:08.455518 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.519582 20884 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:08.519232 20805 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:08.520622 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:08.524852 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:08.539841 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 153222us
I20260709 12:26:08.529347 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:08.532114 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4075) took 1641 ms. Trace:
I20260709 12:26:08.418779 31337 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:08.540799 20806 rpcz_store.cc:276] 0709 12:26:06.890498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.916959 (+ 26461us) service_pool.cc:224] Handling call
0709 12:26:06.917327 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.408329 (+491002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.531853 (+1123524us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54003a0f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:08.532047 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126453632}
I20260709 12:26:08.539963 20808 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:08.527945 20882 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:08.538954 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:08.547550 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:08.534677 31808 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.536564 31633 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:08.554759 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.557621 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.563535 31445 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.564527 31700 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.565047 31700 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.566632 20810 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:08.542259 31726 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:08.555473 31446 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.022s
I20260709 12:26:08.546285 31337 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.590251 20808 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:08.533926 31347 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.571166 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.579094 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:08.580981 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.619055 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:08.556731 31633 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.552652 31808 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:08.621855 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.599289 20727 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:08.599650 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:08.607227 21434 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:08.620275 20884 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:08.593524 31821 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 12:26:08.571064 31824 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:08.621098 20733 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:08.622566 20808 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:08.623692 21429 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: [32.20-32.20]
I20260709 12:26:08.623945 20881 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:08.620954 31633 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:08.625027 31766 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:08.627725 31800 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.643s	user 0.000s	sys 0.004s
I20260709 12:26:08.624493 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:08.633427 31157 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:26:08.635504 31823 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:26:08.618364 31826 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:08.636415 31347 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.619493 20806 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:08.567158 31700 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.641114 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:08.655017 21430 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:26:08.567324 31445 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.571596 31726 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:08.666024 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.617020 31692 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.680053 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.687872 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.673342 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:08.674427 20882 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:08.588163 31730 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.070s
I20260709 12:26:08.588970 31772 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.076s
I20260709 12:26:08.593739 31789 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.075s
I20260709 12:26:08.689625 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:08.736086 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46539.7us
I20260709 12:26:08.666782 31701 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:08.574204 31738 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.692595 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:08.694913 31839 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:08.747689 31839 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:08.748111 31839 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:08.742022 20806 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:08.686151 31776 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:08.762205 20881 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:08.692637 31617 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:08.668018 31766 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.765847 20884 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:08.768684 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4090) took 1037 ms. Trace:
I20260709 12:26:08.759075 31462 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:08.776005 31730 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:08.780337 31770 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:08.780889 31730 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.781535 31730 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:08.770182 31617 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.770864 20802 rpcz_store.cc:276] 0709 12:26:07.731676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.731813 (+   137us) service_pool.cc:224] Handling call
0709 12:26:07.741735 (+  9922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.742161 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.742203 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.745841 (+  3638us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.377776 (+631935us) spinlock_profiling.cc:243] Waited 6.53 ms on lock 0x7b18003221e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:08.480622 (+102846us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:08.480682 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.481865 (+  1183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.765430 (+283565us) raft_consensus.cc:1685] finished
0709 12:26:08.765462 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.768598 (+  3136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3428,"spinlock_wait_cycles":15025536,"thread_start_us":102573,"threads_started":1,"wal-append.queue_time_us":803145}
I20260709 12:26:08.780932 31770 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:08.793771 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 116644us
I20260709 12:26:08.784505 31462 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:08.793159 31617 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:08.802244 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 101653us
I20260709 12:26:08.812203 31766 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.813081 31738 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.815014 31633 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.817334 31633 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.799468 31726 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.818980 31462 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.846506 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.849771 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.851663 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:08.852631 20884 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:08.865372 20884 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:08.869170 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:08.873337 31738 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.895164 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:08.895597 20734 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:08.842242 31766 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:08.894189 31633 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:08.845531 31776 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:08.908372 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:08.859972 31466 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:08.930979 21432 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
W20260709 12:26:08.932123 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 3994) took 2026 ms (client timeout 1574 ms). Trace:
W20260709 12:26:08.932628 20877 rpcz_store.cc:269] 0709 12:26:06.905789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.911131 (+  5342us) service_pool.cc:224] Handling call
0709 12:26:08.931860 (+2020729us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b5400366854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:08.932038 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418556672}
I20260709 12:26:08.902240 31738 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.952142 31692 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:08.953922 31692 consensus_queue.cc:1243] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 31.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:26:08.951735 31617 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.954977 31692 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:08.956127 31692 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:08.965430 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:08.963382 31692 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:08.963856 31617 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:08.925037 31466 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:08.911386 31766 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.939504 31462 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:08.934535 31772 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:08.854375 31770 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:08.991842 31836 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:26:08.988473 31462 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:08.993498 31462 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:08.996989 31837 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 12:26:09.003762 31785 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.004565 31785 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.009770 31861 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:09.010337 31861 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:09.017087 31766 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.018334 31466 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:08.988466 31726 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:09.024151 31726 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.026486 31726 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:08.986470 31767 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.027842 31767 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.028805 20882 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:09.030611 31767 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:09.041656 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:09.054842 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 362285us
I20260709 12:26:09.057907 31844 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.058662 31844 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.079013 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.033880 31855 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
I20260709 12:26:09.051694 31465 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:09.097754 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2606) took 1221 ms. Trace:
I20260709 12:26:09.098237 21435 rpcz_store.cc:276] 0709 12:26:07.876193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.880401 (+  4208us) service_pool.cc:224] Handling call
0709 12:26:09.097556 (+1217155us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54004adb54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:09.097691 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611605888}
I20260709 12:26:09.101837 20879 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:09.104293 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2596) took 1650 ms. Trace:
I20260709 12:26:09.104745 21431 rpcz_store.cc:276] 0709 12:26:07.453633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.453822 (+   189us) service_pool.cc:224] Handling call
0709 12:26:07.454145 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.628121 (+173976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.104069 (+1475948us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54001e5d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:09.104233 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398176512}
W20260709 12:26:09.107975 31851 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:09.109076 31661 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:09.109498 31661 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:09.110837 31661 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.084635 31785 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:09.111410 31704 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:09.111856 31465 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.112655 31661 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:09.114063 31738 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:09.114821 31465 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.066962 31767 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:09.081986 31768 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:09.073166 31776 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.117847 31808 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.118801 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4046) took 1380 ms. Trace:
I20260709 12:26:09.119796 31738 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:09.124642 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 4087) took 1419 ms. Trace:
W20260709 12:26:09.125366 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.137835 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:09.149708 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.140596 21431 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:26:09.145527 20726 rpcz_store.cc:276] 0709 12:26:07.716226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.716491 (+   265us) service_pool.cc:224] Handling call
0709 12:26:07.737128 (+ 20637us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.299741 (+562613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.096331 (+796590us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54000c49d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:09.096535 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278676736}
I20260709 12:26:09.148661 20810 rpcz_store.cc:276] 0709 12:26:07.723676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.153946 (+430270us) service_pool.cc:224] Handling call
0709 12:26:09.124380 (+970434us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b540027b354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:09.124535 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946162560}
I20260709 12:26:09.147066 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.145826 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:09.162364 20808 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:09.158687 31661 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.170210 31661 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.171722 31845 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.172374 31845 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.173259 31661 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.158738 31483 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:09.174757 31845 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.166055 31749 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.010s
I20260709 12:26:09.143424 31767 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:26:09.178365 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.179361 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3856) took 1462 ms. Trace:
I20260709 12:26:09.179806 20801 rpcz_store.cc:276] 0709 12:26:07.717351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.178062 (+1460711us) service_pool.cc:224] Handling call
0709 12:26:09.179295 (+  1233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.181342 31808 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.182117 31482 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:09.182688 31482 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:09.085631 31861 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.155540 31738 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.186048 31767 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:09.193220 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.199075 20801 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:09.199926 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.196811 31861 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:09.187271 31738 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.207031 20801 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:09.207670 31739 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:09.209111 31482 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.211550 31482 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:09.200038 31730 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.218392 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:09.219776 20806 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:09.221097 31730 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:09.168004 31633 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:09.225773 31633 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.237557 31483 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:09.238654 31483 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:09.239074 31483 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:09.240170 31483 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.241850 31483 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:09.269619 20878 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 21. (term mismatch)
I20260709 12:26:09.270115 20878 pending_rounds.cc:85] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 20
I20260709 12:26:09.270409 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.270773 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.274056 20876 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:09.276082 20877 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:26:09.277693 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:09.278789 20884 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:09.270411 20878 pending_rounds.cc:107] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 30.21
I20260709 12:26:09.281481 20878 raft_consensus.cc:2889] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.285933 31844 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.290336 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2017) took 1006 ms. Trace:
I20260709 12:26:09.290988 21429 rpcz_store.cc:276] 0709 12:26:08.283476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.560981 (+277505us) service_pool.cc:224] Handling call
0709 12:26:08.623450 (+ 62469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.654007 (+ 30557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.290131 (+636124us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b5400088c50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:09.290274 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142901632}
I20260709 12:26:09.293010 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3925) took 1418 ms. Trace:
I20260709 12:26:09.293557 20880 rpcz_store.cc:276] 0709 12:26:07.874761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.158836 (+284075us) service_pool.cc:224] Handling call
0709 12:26:08.159172 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.178379 (+ 19207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.178410 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.178449 (+    39us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:08.178481 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.292665 (+1114184us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:09.292729 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.292901 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43313920}
I20260709 12:26:09.296026 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:09.303696 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.310767 31692 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:09.316509 20884 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:26:09.320533 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:09.318548 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:09.323383 20884 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:09.325083 31639 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:09.327247 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.327564 21434 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:26:09.328742 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:09.330650 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.333578 31639 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:09.318560 31709 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:09.334465 31709 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.337505 31709 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:09.338757 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:09.339610 21435 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:09.340524 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:09.337571 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.343506 31709 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:09.343971 31774 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:09.344619 31774 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:09.345757 31774 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.345108 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.350121 21432 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 24. Preceding OpId from leader: term: 35 index: 24. (term mismatch)
I20260709 12:26:09.350742 21432 pending_rounds.cc:85] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 23
I20260709 12:26:09.351049 21432 pending_rounds.cc:107] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 34.24
I20260709 12:26:09.351256 31803 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:09.356458 20728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3746) took 1156 ms. Trace:
I20260709 12:26:09.356911 20728 rpcz_store.cc:276] 0709 12:26:08.200383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.205697 (+  5314us) service_pool.cc:224] Handling call
0709 12:26:08.206006 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.206430 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:08.395978 (+189548us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.073219 (+677241us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.078715 (+  5496us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.078876 (+   161us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:09.078912 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.079031 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.353635 (+274604us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b1800427000. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:09.356054 (+  2419us) raft_consensus.cc:1685] finished
0709 12:26:09.356190 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.356388 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5839,"prepare.queue_time_us":129115,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":2457,"raft.queue_time_us":97690,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":3098,"spinlock_wait_cycles":7034240,"thread_start_us":795728,"threads_started":3,"wal-append.queue_time_us":345186}
I20260709 12:26:09.359663 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:09.360658 21429 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:09.330173 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:09.154230 31839 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.382269 31839 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.351356 21432 raft_consensus.cc:2889] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.392210 31770 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:09.395066 31337 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.403055 31774 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:09.408322 31337 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.412639 31864 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:09.413234 31864 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.423499 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:09.428081 21400 leader_election.cc:400] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36.
I20260709 12:26:09.428471 21400 leader_election.cc:403] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:09.429100 31483 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 36
I20260709 12:26:09.443840 31616 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:09.446692 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.447527 31713 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:09.447602 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.450599 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:09.454664 20806 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:09.463476 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:09.449240 31713 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 12:26:09.461910 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:09.502570 20882 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:09.504033 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:09.409245 31337 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.475741 31616 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:26:09.534943 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.019s	sys 0.009s
I20260709 12:26:09.538532 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.492350 31803 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 12:26:09.572046 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.475955 31839 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.060s
I20260709 12:26:09.539014 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.579615 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.620105 20880 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:09.619730 20806 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:09.572971 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:09.623942 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.023s	sys 0.026s
I20260709 12:26:09.624248 20882 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:09.532536 31639 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:09.615422 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.048s	sys 0.081s
I20260709 12:26:09.582741 21429 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
W20260709 12:26:09.633137 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.050s	sys 0.084s
I20260709 12:26:09.622634 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:09.633577 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 575322us
I20260709 12:26:09.574764 20879 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
W20260709 12:26:09.627744 31870 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:09.604715 31337 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:09.619390 31616 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:09.637738 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:09.634229 20879 pending_rounds.cc:85] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 17
W20260709 12:26:09.638088 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.626456 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:09.643291 20879 pending_rounds.cc:107] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:26:09.643707 20879 raft_consensus.cc:2889] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.642560 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.642807 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.647769 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:09.650682 20776 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.023s	sys 0.041s
W20260709 12:26:09.635401 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:09.652079 20809 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:26:09.574972 31779 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.607232 31709 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:09.613555 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:09.666347 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:09.682936 20729 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:09.668380 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:09.669457 20776 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.023s	sys 0.041s
W20260709 12:26:09.683895 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 648219us
W20260709 12:26:09.651372 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 494464us
I20260709 12:26:09.604758 31483 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 35. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36.
W20260709 12:26:09.722486 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 87167.8us
W20260709 12:26:09.723423 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47949.3us
W20260709 12:26:09.724647 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.725837 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:09.679894 20880 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:26:09.631175 31445 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:09.726647 20730 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:09.722352 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.631713 31864 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:09.681088 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.687403 20735 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:09.645535 31465 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:09.722352 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:09.674612 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.675504 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:09.736716 20801 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:09.645668 31616 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:09.652563 31858 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:09.662101 31337 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:09.711676 20624 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 21 to 26, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 26 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:09.576118 31700 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.112s
I20260709 12:26:09.753751 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:09.640756 31733 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.084s
I20260709 12:26:09.754050 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:09.757306 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:09.681097 31466 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:09.757953 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 25.
I20260709 12:26:09.755299 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:09.738312 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 95571.4us
I20260709 12:26:09.738592 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:09.757876 20727 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:26:09.757810 20806 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:09.651870 31861 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.087s
W20260709 12:26:09.742473 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.648054 31639 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:09.732030 31776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.016s	sys 0.021s
I20260709 12:26:09.731806 31721 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:09.755227 31789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.012s	sys 0.012s
I20260709 12:26:09.798290 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:09.685235 31779 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:09.784108 31776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.020s	sys 0.021s
I20260709 12:26:09.684942 31617 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.098s
I20260709 12:26:09.807094 20727 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:09.806886 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:09.736042 31465 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:09.761641 31636 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:09.758553 31858 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:09.813683 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:09.823537 20731 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:09.814955 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:09.728214 31709 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.762527 31466 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.754282 31445 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:09.828660 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.830371 20729 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 37. Ops: []
I20260709 12:26:09.840632 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:09.800741 31789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.013s	sys 0.012s
I20260709 12:26:09.825922 31465 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:09.827472 31858 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:09.802953 31721 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:09.861763 31318 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.011s
I20260709 12:26:09.875319 31792 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:09.879660 31739 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.671s	user 0.007s	sys 0.004s
I20260709 12:26:09.837917 31466 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:09.831736 31847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.014s	sys 0.010s
I20260709 12:26:09.895411 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:09.921306 20875 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
W20260709 12:26:09.921486 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 235923us
I20260709 12:26:09.923935 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:09.929906 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:09.868181 31840 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:09.835767 31713 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.058s
I20260709 12:26:09.931336 20878 raft_consensus.cc:2368] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28.
W20260709 12:26:09.951474 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56168.3us
I20260709 12:26:09.933336 20727 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:09.934773 20880 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:09.832891 31871 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.103s
I20260709 12:26:09.959574 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:09.891992 31483 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:09.960922 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:10.006393 31721 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:10.011790 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:10.020713 20882 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 29 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:26:10.015689 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
W20260709 12:26:10.022138 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62701.2us
W20260709 12:26:09.923796 31847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.014s	sys 0.011s
W20260709 12:26:09.919961 31633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.002s	sys 0.037s
I20260709 12:26:10.017563 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:09.924683 31733 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.025454 20883 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:09.918303 31861 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.095s
I20260709 12:26:09.978973 31776 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:10.056327 20727 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
W20260709 12:26:10.024185 31633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.003s	sys 0.038s
I20260709 12:26:10.061897 31337 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.077608 31466 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:10.095327 20803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.011s	sys 0.043s
W20260709 12:26:10.099236 20803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.012s	sys 0.044s
I20260709 12:26:10.024665 31696 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:10.106804 20804 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: [30.23-30.23]
I20260709 12:26:10.108654 31466 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.102383 20624 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 23 to 28, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 28 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:10.120211 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:09.952719 31617 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.130504 20877 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27.
I20260709 12:26:10.017467 31840 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.083s
I20260709 12:26:10.024372 31483 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.146436 20624 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 28 to 31, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 31 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:10.150429 20884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.017s	sys 0.031s
I20260709 12:26:10.055364 31733 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:10.150785 20884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.018s	sys 0.031s
W20260709 12:26:10.150234 31803 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.017s	sys 0.038s
W20260709 12:26:10.187992 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 381679us
I20260709 12:26:10.189198 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:10.189446 20624 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 23 to 24, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 24 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:10.191308 20624 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:10.136241 31617 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:10.150772 31779 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:10.190380 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:10.236313 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45997.7us
I20260709 12:26:10.200721 20801 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:10.153465 31639 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.109382 31337 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.181543 31721 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:10.194190 31739 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:10.244015 20624 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 22 to 26, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 26 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:10.241539 31779 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:10.241292 31878 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:10.241645 31445 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.249070 31617 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:10.238880 31774 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.835s	user 0.010s	sys 0.003s
I20260709 12:26:10.238134 31709 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:10.266238 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.276144 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 306) took 24555 ms (client timeout 15000 ms). Trace:
I20260709 12:26:10.268837 31633 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:10.256445 31779 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:10.322377 31347 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.008s	sys 0.012s
W20260709 12:26:10.318826 20775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.014s
W20260709 12:26:10.325867 20775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.006s	sys 0.014s
I20260709 12:26:10.276825 31739 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.328964 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.008s	sys 0.013s
W20260709 12:26:10.331101 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.008s	sys 0.013s
I20260709 12:26:10.324294 20624 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:10.334118 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:10.320956 31633 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.319293 31445 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:10.322157 31779 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:10.259554 31721 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:10.289333 31709 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:10.339064 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.341727 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.339588 20624 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 30, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 30 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:10.277518 31616 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.345779 21430 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:10.347311 21431 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:10.348279 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:10.355101 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:10.356062 21427 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:26:10.356976 21433 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:26:10.359172 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:10.363106 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.337750 31803 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.270359 31878 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:10.363936 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.010s	sys 0.017s
W20260709 12:26:10.366688 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.010s	sys 0.017s
W20260709 12:26:10.367229 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 538574us
I20260709 12:26:10.370568 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:10.336354 31739 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.371796 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:10.370579 20807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.022s
W20260709 12:26:10.379143 20807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.010s	sys 0.022s
I20260709 12:26:10.379138 21433 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:26:10.377754 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:10.366169 31483 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.367549 31883 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.396991 21427 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:10.409128 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:10.388797 20624 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:10.347383 31445 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:10.425858 21435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.011s	sys 0.026s
W20260709 12:26:10.428282 21435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.011s	sys 0.026s
I20260709 12:26:10.426450 21434 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:10.426842 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:10.446393 21427 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:10.361335 31709 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:10.428241 31694 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.015s	sys 0.027s
I20260709 12:26:10.386463 31871 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:10.450625 20624 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 19 to 24, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 24 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:10.456431 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:10.415010 31761 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:10.370028 31803 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.469630 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:10.470337 21434 raft_consensus.cc:2376] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30. Re-sending same reply.
I20260709 12:26:10.428170 31774 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
W20260709 12:26:10.459703 31694 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.016s	sys 0.027s
I20260709 12:26:10.473523 21433 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:26:10.449925 31844 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:26:10.460696 31876 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
W20260709 12:26:10.276603 20628 rpcz_store.cc:269] 0709 12:25:45.648256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.717600 (+11069344us) service_pool.cc:224] Handling call
0709 12:26:10.192903 (+13475303us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:03.174865 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:03.290251 (+115386us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:03.290306 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:26:03.300820 (+ 10514us) tablet.cc:662] Decoding operations
0709 12:26:03.347071 (+ 46251us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:03.347132 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:03.347171 (+    39us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:26:03.442205 (+ 95034us) tablet.cc:701] Row locks acquired
0709 12:26:03.442230 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:26:03.442455 (+   225us) write_op.cc:270] Start()
0709 12:26:03.442661 (+   206us) write_op.cc:276] Timestamp: P: 1783599963442387 usec, L: 0
0709 12:26:03.442690 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 12:26:03.467596 (+ 24906us) log.cc:844] Serialized 11502 byte log entry
0709 12:26:04.259722 (+792126us) op_driver.cc:464] REPLICATION: finished
0709 12:26:04.371910 (+112188us) write_op.cc:301] APPLY: starting
0709 12:26:04.372013 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:04.462823 (+ 90810us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:04.462949 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:05.661819 (+1198870us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:05.853304 (+191485us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 12:26:05.853404 (+   100us) write_op.cc:312] APPLY: finished
0709 12:26:05.854157 (+   753us) log.cc:844] Serialized 252 byte log entry
0709 12:26:06.094861 (+240704us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:06.193823 (+ 98962us) write_op.cc:454] Released schema lock
0709 12:26:06.193909 (+    86us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:06.194477 (+   568us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:06.194505 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":246967,"reported_tablets":48,"spinlock_wait_cycles":1957504,"child_traces":[["op",{"apply.queue_time_us":111894,"apply.run_cpu_time_us":77630,"apply.run_wall_time_us":1822502,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125347,"num_ops":29,"prepare.queue_time_us":260351,"prepare.run_cpu_time_us":27378,"prepare.run_wall_time_us":805876,"raft.queue_time_us":233090,"raft.run_cpu_time_us":15732,"raft.run_wall_time_us":98999,"replication_time_us":816856,"thread_start_us":1102786,"threads_started":5,"wal-append.queue_time_us":820567,"wal-append.run_cpu_time_us":23459,"wal-append.run_wall_time_us":2267170}]]}
I20260709 12:26:10.483335 20624 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 15 to 25, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 25 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:10.477378 31884 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:26:10.465090 31696 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:10.505805 21429 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:10.509821 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.429713 31883 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.481014 31844 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.476245 31633 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:10.475157 31893 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:10.538519 21430 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: [30.22-30.22]
W20260709 12:26:10.561748 21400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.028s	sys 0.042s
W20260709 12:26:10.562258 21400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.028s	sys 0.042s
I20260709 12:26:10.532356 31861 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.514238 31483 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:10.531457 31887 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.585667 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.580365 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.537075 31844 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:10.538012 31881 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.555574 31770 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.160s	user 0.016s	sys 0.000s
I20260709 12:26:10.596207 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.589372 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.625728 20883 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:10.593053 31903 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:10.586575 20876 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
W20260709 12:26:10.593390 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.631351 20801 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:10.535480 31696 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:10.552587 31885 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:10.636898 20881 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 32 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 19.
I20260709 12:26:10.637755 31903 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.638252 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:10.639485 31877 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.393s	user 0.002s	sys 0.000s
I20260709 12:26:10.639642 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:10.640198 31903 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.640753 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.641376 21432 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:10.642398 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.643090 31873 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:10.643507 31873 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:10.644356 31873 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.629864 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2017) took 1186 ms. Trace:
W20260709 12:26:10.646179 31874 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.376s	user 0.000s	sys 0.003s
I20260709 12:26:10.646339 20803 rpcz_store.cc:276] 0709 12:26:09.443611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.447425 (+  3814us) service_pool.cc:224] Handling call
0709 12:26:09.447902 (+   477us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:09.448397 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.448441 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:09.449202 (+   761us) log.cc:844] Serialized 52 byte log entry
0709 12:26:10.099923 (+650721us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:10.100000 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.100105 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.629646 (+529541us) raft_consensus.cc:1685] finished
0709 12:26:10.629664 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.629804 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35144,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":13642,"thread_start_us":650227,"threads_started":1,"wal-append.queue_time_us":878404}
W20260709 12:26:10.648135 31465 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.000s
I20260709 12:26:10.648304 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:10.648763 31465 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:10.650010 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:10.650851 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:10.651752 20802 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 32 because replica has last-logged OpId of term: 30 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 17.
I20260709 12:26:10.651690 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2010) took 2303 ms. Trace:
I20260709 12:26:10.652165 20804 rpcz_store.cc:276] 0709 12:26:08.348445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.361352 (+ 12907us) service_pool.cc:224] Handling call
0709 12:26:09.125658 (+764306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.112293 (+986635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.650801 (+538508us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b540027b354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:10.651482 (+   681us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b540027b350. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:10.651611 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4012402688}
I20260709 12:26:10.640791 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:10.592021 31483 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.598129 31445 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.651381 21428 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:26:10.657480 31721 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:10.658353 31882 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
W20260709 12:26:10.659438 31890 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:26:10.660359 31898 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.661021 31898 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.662195 31774 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:10.662604 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.662724 31774 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:10.664494 31483 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:10.594273 20884 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:10.667621 21430 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:26:10.601859 31875 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.543s	user 0.000s	sys 0.003s
I20260709 12:26:10.637322 31881 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.593010 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.597093 20882 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: [30.23-30.23]
I20260709 12:26:10.664866 31774 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.665567 31898 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:10.588485 31891 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:26:10.588881 20624 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 20 to 26. New cstate: current_term: 26 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:10.516366 31157 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:10.556346 31893 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:10.583108 31616 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:10.692440 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.699018 20726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.014s	sys 0.030s
W20260709 12:26:10.701373 20726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.014s	sys 0.030s
I20260709 12:26:10.702440 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.586979 31861 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.726559 20884 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:10.731159 20882 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:26:10.731025 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.692564 31701 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:10.732672 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 401213us
I20260709 12:26:10.727501 31739 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:10.725098 31633 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:10.612332 31887 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.701637 31157 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:10.723855 31903 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:10.691140 31885 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.735857 31465 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.758945 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:10.759567 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:10.736133 31616 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.754823 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:10.753361 31905 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.763005 31636 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.930s	user 0.011s	sys 0.002s
I20260709 12:26:10.882323 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:10.898023 20882 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:26:10.888000 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 133243us
I20260709 12:26:10.764014 31709 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.592989 31886 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:10.872375 31445 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.752730 31701 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:10.914741 31639 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.009s	sys 0.042s
I20260709 12:26:10.764916 31462 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.154s
I20260709 12:26:10.911069 31636 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:10.947150 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.767638 31633 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.956765 31337 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:10.957751 21398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.000s	sys 0.026s
W20260709 12:26:10.958093 21398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.000s	sys 0.027s
I20260709 12:26:10.958398 31633 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:10.949451 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4106) took 1300 ms. Trace:
I20260709 12:26:10.958976 20805 rpcz_store.cc:276] 0709 12:26:09.649355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.665235 (+ 15880us) service_pool.cc:224] Handling call
0709 12:26:09.665514 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.881576 (+216062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.881610 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.881646 (+    36us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:09.881676 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.949110 (+1067434us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540019a5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:10.949192 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.949369 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496450304}
I20260709 12:26:10.959851 31696 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.927301 31886 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:10.960561 20881 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4019) took 1691 ms (client timeout 1872 ms). Trace:
I20260709 12:26:10.960772 31881 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:10.960985 20881 rpcz_store.cc:269] 0709 12:26:09.269649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.270537 (+   888us) service_pool.cc:224] Handling call
0709 12:26:10.960368 (+1689831us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540019d7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:10.960506 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3139687296}
I20260709 12:26:10.962250 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:10.962420 31886 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.963117 31885 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:10.963470 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:10.921546 31721 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:10.966995 31721 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:10.959676 31733 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:10.964550 31861 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:10.912766 31905 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.948576 31898 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.017s
I20260709 12:26:10.942121 31847 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:10.965520 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2029) took 1503 ms. Trace:
I20260709 12:26:10.917564 31878 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:10.886185 31893 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.951282 20882 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:10.962924 31886 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:10.966135 20881 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:10.920686 31803 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:10.969578 20729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.005s	sys 0.026s
W20260709 12:26:11.238850 20729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.005s	sys 0.027s
I20260709 12:26:11.239688 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3767) took 1550 ms. Trace:
I20260709 12:26:10.935705 31617 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.907799 31616 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:10.994138 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.012s	sys 0.016s
I20260709 12:26:10.886701 31903 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:10.953305 31639 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.010s	sys 0.042s
W20260709 12:26:10.969249 31753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.009s	sys 0.041s
I20260709 12:26:10.998483 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.007386 20734 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:11.003566 21435 rpcz_store.cc:276] 0709 12:26:09.461939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.462060 (+   121us) service_pool.cc:224] Handling call
0709 12:26:09.484335 (+ 22275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.492982 (+  8647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.493017 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.535080 (+ 42063us) log.cc:844] Serialized 564 byte log entry
0709 12:26:10.429258 (+894178us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:10.429323 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.429445 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.965291 (+535846us) raft_consensus.cc:1685] finished
0709 12:26:10.965313 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.965459 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":893763,"threads_started":1,"wal-append.queue_time_us":1190129}
W20260709 12:26:11.077560 20730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3765) took 1389 ms (client timeout 1792 ms). Trace:
W20260709 12:26:11.250316 20730 rpcz_store.cc:269] 0709 12:26:09.688486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.727403 (+ 38917us) service_pool.cc:224] Handling call
0709 12:26:11.077349 (+1349946us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54001d8b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:11.077488 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010226944}
I20260709 12:26:11.257732 21399 leader_election.cc:400] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28.
I20260709 12:26:11.268330 21399 leader_election.cc:403] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:11.082647 31741 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:26:11.268574 31844 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.640s	user 0.010s	sys 0.000s
I20260709 12:26:11.254739 31803 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.043927 31886 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.269269 31803 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:11.045243 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 285718us
I20260709 12:26:11.271286 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:11.271564 31886 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.269016 31741 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:11.057379 31709 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.273216 31466 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:11.273705 31466 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.273480 31709 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.274153 31466 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:11.238977 31840 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:11.279280 20876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.026s	sys 0.020s
I20260709 12:26:11.279356 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:11.279904 20876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.026s	sys 0.020s
I20260709 12:26:11.245754 31895 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.099325 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3882) took 1395 ms. Trace:
I20260709 12:26:10.968055 31897 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.067298 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:11.293990 20726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.015s	sys 0.012s
I20260709 12:26:10.942833 31445 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:10.998981 31861 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.999485 31847 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.969056 31633 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.145551 20881 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:11.084431 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.300323 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.300580 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.088049 31878 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:10.988430 31894 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.996480 31883 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:11.076802 20875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3935) took 2658 ms (client timeout 1980 ms). Trace:
W20260709 12:26:11.303154 20875 rpcz_store.cc:269] 0709 12:26:08.418126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.463435 (+ 45309us) service_pool.cc:224] Handling call
0709 12:26:11.076586 (+2613151us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540019cdd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:11.076709 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3204038016}
I20260709 12:26:11.004599 31898 consensus_queue.cc:1243] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20260709 12:26:11.047863 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.025s
W20260709 12:26:11.306078 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.016s	sys 0.036s
W20260709 12:26:11.306614 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 829839us
W20260709 12:26:11.104507 31779 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.000s
I20260709 12:26:11.012790 31462 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Stepping down as leader of term 35
W20260709 12:26:11.237124 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.003s	sys 0.030s
I20260709 12:26:11.154295 31881 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.243801 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.012s	sys 0.017s
W20260709 12:26:11.319806 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.004s	sys 0.030s
W20260709 12:26:11.244176 20801 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.021s	sys 0.005s
I20260709 12:26:11.248322 31617 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:26:11.260799 20734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3761) took 1573 ms (client timeout 1505 ms). Trace:
W20260709 12:26:11.321332 20734 rpcz_store.cc:269] 0709 12:26:09.687298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.689259 (+  1961us) service_pool.cc:224] Handling call
0709 12:26:11.260587 (+1571328us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54001d8654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:11.260716 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2735684480}
I20260709 12:26:11.255833 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.269156 31844 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:11.248926 31753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.009s	sys 0.041s
W20260709 12:26:11.247109 31858 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.327s	user 0.014s	sys 0.000s
W20260709 12:26:11.321002 20801 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.021s	sys 0.006s
I20260709 12:26:11.325770 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.326105 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.326449 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:11.277292 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.328078 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.277990 31904 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28.
I20260709 12:26:11.332165 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:11.103251 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.332791 21429 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:10.941802 31157 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:10.998116 31692 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.340843 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:10.987728 31892 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:10.997684 31873 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:11.073261 31701 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:11.291231 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.011s	sys 0.026s
I20260709 12:26:11.294477 20882 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
W20260709 12:26:11.076390 31907 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.204s	user 0.000s	sys 0.000s
W20260709 12:26:11.295426 20726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.016s	sys 0.012s
I20260709 12:26:11.369535 21435 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
I20260709 12:26:11.292673 20807 rpcz_store.cc:276] 0709 12:26:09.703552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.735136 (+ 31584us) service_pool.cc:224] Handling call
0709 12:26:09.735596 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.379929 (+644333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.379977 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.380587 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.686091 (+305504us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:10.686143 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.686236 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.099084 (+412848us) raft_consensus.cc:1685] finished
0709 12:26:11.099106 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.099264 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65247,"raft.queue_time_us":155662,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":29186,"thread_start_us":807307,"threads_started":2,"wal-append.queue_time_us":547867}
I20260709 12:26:11.300884 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:11.301460 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.283224 31896 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.018s
I20260709 12:26:11.310206 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.288259 31910 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:11.296381 31709 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.299827 31847 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:11.240082 20729 rpcz_store.cc:276] 0709 12:26:09.689065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.876839 (+187774us) service_pool.cc:224] Handling call
0709 12:26:09.877167 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.877380 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:10.420882 (+543502us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.239302 (+818420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.239359 (+    57us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:11.239427 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.239494 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.239628 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":246772,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":3388,"thread_start_us":1310709,"threads_started":2}
I20260709 12:26:11.298435 31897 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.328897 20875 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:11.302874 31887 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.329432 21433 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:26:11.330595 20734 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:26:11.332468 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55260.4us
I20260709 12:26:11.334570 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.335836 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:11.337383 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.413568 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:11.338510 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.342228 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.414678 20733 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:11.342973 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.415508 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.286514 31840 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.309031 31633 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.320722 31779 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.423660 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:11.378537 31774 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:11.338263 31858 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.355978 31157 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:11.341429 21432 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
W20260709 12:26:11.356884 31915 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:11.307717 31878 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.299994 31861 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.338343 31839 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.358495 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.463466 20731 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:11.332456 31881 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.359201 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.360774 31700 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:11.360913 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.319693 31462 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:11.425172 31633 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:11.363107 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.428843 31887 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:11.314926 31721 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.365254 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.365526 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:11.366319 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.012s	sys 0.026s
I20260709 12:26:11.479861 20730 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
W20260709 12:26:11.479933 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 716507us
W20260709 12:26:11.464303 31906 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
W20260709 12:26:11.365891 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 601987us
I20260709 12:26:11.369900 21435 pending_rounds.cc:85] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 18
I20260709 12:26:11.487102 21435 pending_rounds.cc:107] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:26:11.487444 21435 raft_consensus.cc:2889] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.369920 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4072) took 1703 ms. Trace:
I20260709 12:26:11.370496 20877 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:11.298401 31895 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:11.379745 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78937us
I20260709 12:26:11.380681 31318 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.381026 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.382576 20882 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:26:11.383366 31701 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:11.387257 31337 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:11.370304 31918 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.389072 31844 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.390084 31617 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.314033 31911 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.085s
I20260709 12:26:11.390377 31445 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.377925 31739 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.401100 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.277117 31694 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:11.338613 31883 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:11.501343 31871 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.407017 20879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.017s	sys 0.021s
I20260709 12:26:11.413971 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:11.540139 20879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.017s	sys 0.021s
I20260709 12:26:11.540380 20735 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:11.416085 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.541644 20733 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:11.308621 31894 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.417062 20884 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:26:11.417645 21399 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
W20260709 12:26:11.547719 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 130260us
I20260709 12:26:11.365106 31792 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:11.418464 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.526801 31903 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.273228 31741 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.425040 20881 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:26:11.426836 20728 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:11.424897 20734 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:11.427860 21430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.005s	sys 0.017s
W20260709 12:26:11.553049 21430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.005s	sys 0.017s
W20260709 12:26:11.429253 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.012s
I20260709 12:26:11.362613 31692 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.403936 31910 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:11.411888 31905 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.565469 31878 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:11.446161 21426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.009s	sys 0.013s
I20260709 12:26:11.468762 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
W20260709 12:26:11.566212 21426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.010s	sys 0.013s
I20260709 12:26:11.510782 31318 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.567806 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:11.568611 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:11.569087 31318 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.569459 20734 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:11.516170 31337 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.571821 31337 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.589236 31779 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.296299 31347 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.024s
I20260709 12:26:11.425029 31774 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.366737 20805 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:26:11.488157 20726 rpcz_store.cc:276] 0709 12:26:09.666525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.667794 (+  1269us) service_pool.cc:224] Handling call
0709 12:26:09.668028 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.668216 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:10.131740 (+463524us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.369585 (+1237845us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.369651 (+    66us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:11.369702 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.369759 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.369868 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":282295,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"raft.queue_time_us":216047,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7241,"thread_start_us":1634022,"threads_started":3,"wal-append.queue_time_us":872024}
I20260709 12:26:11.492761 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:11.471797 31700 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:11.507838 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.509088 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.509362 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.622115 20806 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:11.506668 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.514916 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:11.521844 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140902us
I20260709 12:26:11.526278 20802 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:11.532994 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.534040 20729 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:26:11.405884 20880 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:11.533740 31739 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:11.534468 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.006s	sys 0.016s
W20260709 12:26:11.626796 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.006s	sys 0.017s
I20260709 12:26:11.542567 31871 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.624786 20882 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:11.628152 31739 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.472901 31839 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.629106 31871 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.630306 20884 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 20. (term mismatch)
I20260709 12:26:11.630707 20884 pending_rounds.cc:85] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 19
I20260709 12:26:11.630920 20884 pending_rounds.cc:107] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 32.20
I20260709 12:26:11.501114 31917 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.502986 31616 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:11.539112 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 172372us
I20260709 12:26:11.419301 20804 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:11.631162 20884 raft_consensus.cc:2889] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.524516 31886 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.537213 31761 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.065s	user 0.013s	sys 0.000s
W20260709 12:26:11.553828 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.000s	sys 0.012s
W20260709 12:26:11.637328 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.980s (SENT) [suppressed 8 similar messages]
I20260709 12:26:11.554013 21399 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27.
I20260709 12:26:11.530082 31923 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:11.541725 31920 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:11.560667 31858 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.564723 31692 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.592516 31910 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.557695 31792 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.601135 31881 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.542309 31909 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.068s
I20260709 12:26:11.534492 31803 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.672987 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2031) took 1041 ms. Trace:
I20260709 12:26:11.470489 20731 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:26:11.673599 21426 rpcz_store.cc:276] 0709 12:26:10.631679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.631788 (+   109us) service_pool.cc:224] Handling call
0709 12:26:10.653834 (+ 22046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.726635 (+ 72801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.726665 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.730555 (+  3890us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.566633 (+836078us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:11.566679 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.652289 (+ 85610us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:11.652349 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.652404 (+    55us) raft_consensus.cc:1685] finished
0709 12:26:11.652422 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.672893 (+ 20471us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35209,"raft.queue_time_us":56132,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6438,"spinlock_wait_cycles":38966528,"thread_start_us":835761,"threads_started":1,"wal-append.queue_time_us":896842}
I20260709 12:26:11.518378 31918 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.515769 31927 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.050s
I20260709 12:26:11.624061 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:11.684918 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60930.8us
I20260709 12:26:11.632812 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.553776 31776 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:11.638098 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 145427us
W20260709 12:26:11.686129 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53386.4us
I20260709 12:26:11.638128 21399 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:11.599045 31861 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.631054 31839 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:11.615414 31733 consensus_queue.cc:1175] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 27 status { last_received { term: 27 index: 20 } last_committed_idx: 19 last_received_current_leader { term: 27 index: 20 } }
I20260709 12:26:11.701320 31466 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.703377 31858 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:11.704771 31897 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.647339 31920 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.691430 31871 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:11.707325 31920 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.707419 31871 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:11.707945 31871 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:11.711791 31916 consensus_queue.cc:1175] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 27 status { last_received { term: 27 index: 20 } last_committed_idx: 19 last_received_current_leader { term: 27 index: 20 } }
I20260709 12:26:11.714285 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2068) took 1037 ms. Trace:
I20260709 12:26:11.636919 31917 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.714778 21430 rpcz_store.cc:276] 0709 12:26:10.676713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.676971 (+   258us) service_pool.cc:224] Handling call
0709 12:26:10.677250 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.753099 (+ 75849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.753134 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.753598 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.553472 (+799874us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:11.553527 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.553600 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.714061 (+160461us) raft_consensus.cc:1685] finished
0709 12:26:11.714081 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.714221 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60227,"raft.queue_time_us":73984,"raft.run_cpu_time_us":56,"raft.run_wall_time_us":55,"thread_start_us":657888,"threads_started":1,"wal-append.queue_time_us":777150}
I20260709 12:26:11.716074 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4038) took 1013 ms. Trace:
I20260709 12:26:11.639312 31761 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:11.716874 31700 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.717533 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4039) took 1014 ms. Trace:
I20260709 12:26:11.717916 20878 rpcz_store.cc:276] 0709 12:26:10.702961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.703569 (+   608us) service_pool.cc:224] Handling call
0709 12:26:10.703977 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.146884 (+442907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.146931 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.189023 (+ 42092us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.501140 (+312117us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:11.501316 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.501388 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.717328 (+215940us) raft_consensus.cc:1685] finished
0709 12:26:11.717350 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.717478 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3155,"raft.queue_time_us":183560,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":16556,"thread_start_us":693495,"threads_started":2,"wal-append.queue_time_us":402581}
I20260709 12:26:11.718945 31894 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.643669 31923 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.716554 20883 rpcz_store.cc:276] 0709 12:26:10.702648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.726279 (+ 23631us) service_pool.cc:224] Handling call
0709 12:26:10.739007 (+ 12728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.757673 (+ 18666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.757702 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.758144 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.627251 (+869107us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:11.627295 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.627363 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.715846 (+ 88483us) raft_consensus.cc:1685] finished
0709 12:26:11.715867 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.716018 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1760,"raft.queue_time_us":158111,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":7827,"thread_start_us":712915,"threads_started":1,"wal-append.queue_time_us":775544}
I20260709 12:26:11.721660 31923 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.722393 31466 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.639204 31886 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.723053 31741 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:11.641675 31911 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
W20260709 12:26:11.724749 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 170933us
I20260709 12:26:11.726055 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.688393 31721 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:11.695595 20732 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:11.695987 31792 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.696586 31918 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.697537 31895 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.670920 31840 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.722223 31864 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:11.687357 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.727506 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:11.661382 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1964) took 1067 ms. Trace:
I20260709 12:26:11.731299 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.694200 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:11.724388 31886 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:11.735743 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98442.9us
I20260709 12:26:11.724418 31927 consensus_queue.cc:1243] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 31.23 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 22
I20260709 12:26:11.741276 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.756803 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.758280 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.750825 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:11.742400 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:11.717999 31700 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:11.762439 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.732357 31839 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.745210 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:11.746891 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59611.8us
I20260709 12:26:11.744500 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:11.765368 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:11.767787 20807 raft_consensus.cc:2410] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 32 because replica has last-logged OpId of term: 30 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:26:11.763154 21428 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:11.767803 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:11.744091 20877 rpcz_store.cc:276] 0709 12:26:10.594053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.595245 (+  1192us) service_pool.cc:224] Handling call
0709 12:26:10.595619 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.498702 (+903083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.661097 (+162395us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400305554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:11.661299 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772834048}
I20260709 12:26:11.750257 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.740501 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.738225 31483 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.780093 31483 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.781625 21432 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:11.773296 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.783275 31898 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:11.783870 31898 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.786010 31898 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.787624 31886 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:11.788049 31886 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:11.788408 31886 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:11.789552 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.792176 31898 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:11.777349 20806 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 18. (term mismatch)
I20260709 12:26:11.792678 31898 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.778203 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.793112 31898 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:11.793334 20810 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:11.794490 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:11.792086 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 97993.4us
I20260709 12:26:11.792798 20806 pending_rounds.cc:85] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 17
I20260709 12:26:11.795715 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.773988 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:11.797267 31895 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:11.797679 31895 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.798026 31895 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:11.797868 31847 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.798255 31776 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:11.769034 31466 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:11.799331 31792 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:11.800190 31733 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:11.800567 31733 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:11.747761 31903 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:11.801379 31733 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 32.16, Last appended by leader: 16, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.802057 21433 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:11.734146 31639 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27.
I20260709 12:26:11.777478 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.775664 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:11.796331 20804 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:11.804116 20803 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:11.795338 31839 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.806751 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:11.811331 20875 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:26:11.813660 31881 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:11.810822 31466 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:11.814145 31881 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:11.815105 31881 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.23, Last appended by leader: 23, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.818733 31881 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:11.819528 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.820335 20804 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:11.820393 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.821378 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.821761 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.816241 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.823520 20881 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:26:11.810307 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:11.825259 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.826062 20878 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 31 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:11.795071 20810 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:11.826471 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.795831 20806 pending_rounds.cc:107] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 30.18
I20260709 12:26:11.750522 31943 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:11.827955 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.828570 20884 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:11.762519 31840 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.800563 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:11.827369 20806 raft_consensus.cc:2889] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.807843 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.829219 20875 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:26:11.815762 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:11.840044 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:11.825611 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:11.802687 31733 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:11.830125 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.855370 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.830549 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.856402 20883 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:11.808336 31792 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:11.830408 31445 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:11.832911 31462 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.802897 31927 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:11.909644 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.910436 21433 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:11.835412 31774 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.816931 31917 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:11.837872 31709 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
W20260709 12:26:11.824880 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70401.8us
I20260709 12:26:11.839658 31839 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.839372 31911 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:11.840679 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.829540 20877 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:11.845221 31883 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:11.816012 31776 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:11.845950 31943 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:11.845721 31639 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:11.924916 31897 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.840044 31894 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 12:26:11.850533 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.842840 31803 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 17, Time since last communication: 0.010s
W20260709 12:26:11.851511 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51041.8us
I20260709 12:26:11.854138 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:11.854588 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.854959 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.818608 31871 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.858099 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.830772 31909 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:11.860782 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:12.003137 20731 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:12.003212 31694 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:12.003885 31694 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:12.005019 31694 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.006704 31694 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:11.863128 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.865267 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:12.007452 31847 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.876585 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:12.008349 20726 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:12.009423 31871 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:11.877264 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.880105 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.886317 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.011373 20807 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:11.896796 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:11.894204 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 32 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:12.012328 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 118194us
I20260709 12:26:11.899525 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.900589 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.012671 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.013154 20734 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:12.013998 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.900924 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.901243 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:11.901242 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:11.902999 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:11.903293 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.015552 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:11.913442 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.920523 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:11.852049 31923 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:12.019888 20730 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:11.851693 31840 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:11.924481 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:11.948371 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:11.948890 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.026029 20732 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
W20260709 12:26:12.026296 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 123388us
I20260709 12:26:11.950565 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.026623 21431 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:26:12.017261 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:11.955662 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:11.864485 31761 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:11.969707 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:11.971524 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.970819 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:12.034802 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 110409us
I20260709 12:26:11.896860 31347 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.841807 31898 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.035874 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:11.944787 31318 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.842618 31770 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:11.815011 31466 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:11.856382 31692 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.107s
I20260709 12:26:11.909068 31927 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:11.950747 31462 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:11.986903 20875 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:11.989173 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.951529 31774 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:11.958880 31709 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:11.991714 31483 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:11.997555 31904 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:11.987978 31897 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.002311 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:12.088963 21434 raft_consensus.cc:2368] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
W20260709 12:26:12.028036 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 170064us
I20260709 12:26:11.907725 31918 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.073s
I20260709 12:26:12.015158 31639 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:26:12.047653 31858 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:12.035833 21429 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 29 because replica has last-logged OpId of term: 27 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
W20260709 12:26:12.036911 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 81389.7us
I20260709 12:26:12.032517 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 26.
I20260709 12:26:11.977458 31776 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.096215 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:12.040184 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.043143 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:11.977669 31883 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:11.981526 31943 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:12.030330 21432 raft_consensus.cc:2955] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 8 to 18, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:12.051618 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:12.047272 31905 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:12.076916 21430 raft_consensus.cc:2376] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28. Re-sending same reply.
I20260709 12:26:12.025949 31923 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.085448 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.124234 20734 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:12.033746 31761 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.043720 31909 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:12.053079 31922 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:12.053082 31893 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:12.026976 20809 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
W20260709 12:26:12.038048 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 89816.5us
I20260709 12:26:12.079133 31927 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:12.055891 31770 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.084060 31709 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:12.128072 21431 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
W20260709 12:26:12.128862 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77495.2us
I20260709 12:26:12.086284 31904 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:12.130863 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:12.133522 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.052716 31347 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.102553 31776 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:12.102975 31483 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:12.053608 31318 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.147812 20726 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:12.124668 31883 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:12.125384 31905 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.122706 31871 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:12.129727 31909 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:12.122164 31918 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:12.095474 31858 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.129577 31761 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:12.081223 31774 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.154379 31894 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:12.163884 31894 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.164263 31894 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.165164 31774 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:12.152367 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.158277 31909 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:12.166141 21429 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:12.125849 31923 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.167649 31898 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:12.169838 20810 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:12.153858 31905 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:12.172345 31897 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.135478 31847 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:12.174091 31347 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:12.174501 31923 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.175613 20729 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:12.177538 31923 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.153517 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:12.130019 31893 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:12.158906 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.185770 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:12.158375 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 172278us
I20260709 12:26:12.163120 31858 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:12.172405 31770 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:12.126859 31943 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:12.222123 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:12.227735 20809 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.229791 20807 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:12.236645 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:12.160348 31692 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.249526 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.155665 31910 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 12:26:12.249995 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:12.254537 21427 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 23. Preceding OpId from leader: term: 31 index: 23. (term mismatch)
I20260709 12:26:12.263017 21427 pending_rounds.cc:85] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 22
I20260709 12:26:12.263440 21427 pending_rounds.cc:107] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 30.23
I20260709 12:26:12.263748 21427 raft_consensus.cc:2889] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:12.197961 31893 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:12.251754 20803 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:12.268450 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:12.269778 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:12.197417 31696 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.069s
I20260709 12:26:12.229247 31916 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.275058 31916 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.278105 31847 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:12.283921 31615 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:12.181244 31918 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.242105 31904 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.293154 31904 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:12.293346 31918 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.295147 31895 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.296653 20801 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:12.296830 31887 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:12.241652 31903 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:12.297911 31895 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.298767 31895 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.298346 31903 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.259428 31692 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.302570 31916 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:12.242846 31770 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:12.308938 31318 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:12.317199 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59558.8us
I20260709 12:26:12.339955 20809 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:12.317858 31774 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:12.297287 31887 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:12.342517 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:12.350051 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.351667 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.352824 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.354408 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.362370 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:12.372185 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.372651 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.373589 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.373588 20726 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:12.379109 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:12.381253 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 280064us
I20260709 12:26:12.340075 31847 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:26:12.382756 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.390087 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.384528 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.390988 20734 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:12.343660 31958 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:12.393451 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.337256 31700 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.395315 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:12.340626 31770 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:12.398012 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 307288us
I20260709 12:26:12.397691 20727 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:12.401022 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:12.386498 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:12.402907 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:12.403810 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.404636 20809 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:12.360067 31887 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:12.405372 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:12.409385 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:12.410708 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:12.412209 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:12.336655 31895 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.419798 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.380511 31920 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
W20260709 12:26:12.414546 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 174538us
I20260709 12:26:12.424275 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.425860 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:12.428213 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.425609 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:12.433804 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:12.434361 20776 leader_election.cc:400] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:12.434693 20776 leader_election.cc:403] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:26:12.434967 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:12.392017 31840 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:12.439829 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:12.395929 31958 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:12.440062 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.448562 31922 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:12.449110 31922 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:12.448663 31958 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.449638 31922 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.450237 31958 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.413730 31839 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:12.389520 31847 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:12.451814 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:12.404204 31770 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:12.405807 31923 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:12.445917 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.473660 20727 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:26:12.446854 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.458913 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:12.423043 31616 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:12.463696 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.475392 20726 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:26:12.478413 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:12.424805 31920 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.446651 31318 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:12.452755 31916 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:12.512118 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.464632 31873 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:12.513471 20805 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:12.415118 31700 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:12.524602 20698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.060s	sys 0.107s
I20260709 12:26:12.471280 31847 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:12.524245 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.506996 31965 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:26:12.524995 20698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.061s	sys 0.108s
I20260709 12:26:12.438638 31883 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.520351 20883 raft_consensus.cc:2410] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 21.
W20260709 12:26:12.531567 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 261852us
I20260709 12:26:12.554826 31883 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.555785 21430 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:12.555687 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:12.479773 31923 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:12.556628 31923 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:12.510847 31920 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:12.508539 31616 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:12.560148 31616 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:12.565001 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.569396 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:12.567574 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:12.514014 31318 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:12.572444 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 312261us
I20260709 12:26:12.572729 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.517750 31839 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:12.518460 31873 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:12.582806 31886 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.794s	user 0.013s	sys 0.000s
W20260709 12:26:12.608469 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.581895 31963 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.513844 31709 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.550171 31965 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.523370 31774 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.080s
I20260709 12:26:12.628479 20727 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: [28.18-30.19]
I20260709 12:26:12.584815 31318 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:12.592825 31916 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:12.522652 31462 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 12:26:12.597636 31839 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:12.603614 31970 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:12.643527 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:12.653007 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 235480us
I20260709 12:26:12.656572 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:12.633021 31615 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:12.645323 31839 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:12.637341 31971 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.680114 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:12.681751 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:12.639168 31916 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:12.623339 31886 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.630424 31894 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:12.716275 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:12.681384 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.638263 31965 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:12.719885 20807 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:12.669337 31692 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.683984 31955 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:12.676251 31974 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.776261 21435 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:12.716957 31903 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.631841 31963 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:12.711237 31776 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.002s
I20260709 12:26:12.775344 31955 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.770301 31973 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:12.812134 31984 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:12.812697 31984 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.677446 31873 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.806340 31903 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:12.810479 31776 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.715505 31916 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:12.822870 20734 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:12.828217 20732 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
W20260709 12:26:12.810927 31960 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
W20260709 12:26:12.819985 31972 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:26:12.835758 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.722898 31971 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.811157 31955 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:12.847826 31971 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.815152 31873 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:12.833870 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.815302 31903 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:12.811798 31973 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:12.880852 31733 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.023s	user 0.013s	sys 0.000s
I20260709 12:26:12.882071 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:12.832231 31968 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 12:26:12.783346 31974 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.893548 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:12.897418 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 328119us
W20260709 12:26:12.840655 31909 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.003s
W20260709 12:26:12.902225 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.902552 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.902912 21428 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.840044 31898 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:12.889182 31973 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:12.935797 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.819662 31978 consensus_queue.cc:1175] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49323f66286f4798b268f216bb62b3be" responder_term: 25 status { last_received { term: 21 index: 16 } last_committed_idx: 16 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:12.863857 31883 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.888670 31927 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.740s	user 0.007s	sys 0.004s
I20260709 12:26:12.907033 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:12.886359 31903 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:12.917192 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:12.890903 20884 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:12.908937 31909 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.938933 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.947855 21434 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:12.950208 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:12.941563 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 501874us
I20260709 12:26:12.945473 21428 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:12.946158 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:12.948388 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:12.934958 31905 consensus_meta.cc:287] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.004s
I20260709 12:26:12.929809 31984 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:12.954142 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:12.955076 21434 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:12.956086 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:12.965065 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.967098 31963 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:12.946486 31761 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.783s	user 0.011s	sys 0.000s
I20260709 12:26:12.970595 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:12.973097 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.973810 21428 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:12.974996 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.930383 31898 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.977763 21434 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 29 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:26:12.974956 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.955612 31995 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:12.946794 31883 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:12.984469 31976 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:12.980993 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:12.989332 21428 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
W20260709 12:26:12.987283 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.010s	sys 0.017s
W20260709 12:26:12.990159 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.010s	sys 0.017s
I20260709 12:26:12.968829 31894 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:12.995110 31949 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.206s	user 0.005s	sys 0.000s
I20260709 12:26:12.996598 31909 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:12.951355 31927 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:12.978947 31994 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:13.016134 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:12.982300 31886 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.000249 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:12.978477 31984 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:13.067516 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:12.974210 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:13.008227 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4040) took 2303 ms. Trace:
I20260709 12:26:12.963531 31905 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:12.999822 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.071552 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 117471us
I20260709 12:26:12.976459 31761 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.011435 32002 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:13.073047 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.022362 31923 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.075984 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:13.017055 20809 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:13.000077 21428 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:13.048820 31883 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.009720 31910 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:13.004176 31861 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.101012 31923 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.100708 31861 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:13.102711 31922 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.004s
I20260709 12:26:13.103372 31922 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.048863 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:13.040544 21429 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:13.068436 31918 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:13.070685 21426 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
W20260709 12:26:13.106092 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 131949us
I20260709 12:26:13.070145 20879 rpcz_store.cc:276] 0709 12:26:10.704727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.704871 (+   144us) service_pool.cc:224] Handling call
0709 12:26:10.705260 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.540964 (+835704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.541028 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.541504 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.734308 (+192804us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:11.734379 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.734447 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.994498 (+1260051us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:13.007829 (+ 13331us) raft_consensus.cc:1685] finished
0709 12:26:13.007858 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.008146 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4907,"raft.queue_time_us":183440,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":16385,"spinlock_wait_cycles":39777664,"thread_start_us":927206,"threads_started":2,"wal-append.queue_time_us":235434}
W20260709 12:26:13.010469 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 64401.2us
I20260709 12:26:13.107434 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:13.014968 31839 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.031853 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:13.131732 20875 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:13.080806 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:13.083145 31905 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:13.084744 31893 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.747s	user 0.012s	sys 0.000s
W20260709 12:26:13.074313 31904 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.006s
I20260709 12:26:13.137140 20883 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
W20260709 12:26:13.087495 31975 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:26:13.087646 21431 raft_consensus.cc:2368] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:13.088374 31995 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:13.089154 31761 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.089979 20735 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:13.093046 31973 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:13.096189 31770 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.006s
W20260709 12:26:13.067463 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 67396.6us
I20260709 12:26:13.156237 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:13.158318 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:13.051573 31974 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.040136 31927 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.107816 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:13.108799 31861 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.159714 20879 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:13.110857 20806 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
W20260709 12:26:13.105705 31616 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.001s
W20260709 12:26:13.113371 31920 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.004s
I20260709 12:26:13.115708 31966 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:13.118144 31894 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.119226 31887 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.687s	user 0.005s	sys 0.006s
W20260709 12:26:13.121814 31958 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.671s	user 0.002s	sys 0.007s
W20260709 12:26:13.123589 31858 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.894s	user 0.002s	sys 0.010s
I20260709 12:26:13.078603 31981 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:13.124504 31955 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:13.126423 31943 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.869s	user 0.009s	sys 0.001s
I20260709 12:26:13.122694 31965 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:13.127171 31847 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 12:26:13.128103 32004 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:13.128903 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:13.129591 32002 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:13.135233 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.075227 31898 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.160920 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:13.136468 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55731.1us
I20260709 12:26:13.166427 31966 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:13.205018 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:13.133102 31839 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.209463 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1975) took 1355 ms. Trace:
I20260709 12:26:13.210017 20880 rpcz_store.cc:276] 0709 12:26:11.854087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.854252 (+   165us) service_pool.cc:224] Handling call
0709 12:26:11.854523 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.156273 (+301750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.156305 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.156347 (+    42us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:12.156377 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.209189 (+1052812us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540022fbd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:13.209245 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.209390 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693489920}
I20260709 12:26:13.170598 20875 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:13.170645 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.171485 31920 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.172303 31861 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:13.214600 20731 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:13.104146 31318 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.139894 31873 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:13.140620 31995 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:13.160413 31774 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:13.197988 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69178.1us
I20260709 12:26:13.208313 20879 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:13.206733 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:13.168536 31616 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.164367 31770 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.215168 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:13.216079 20727 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:13.216691 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46139.9us
I20260709 12:26:13.208469 31966 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:13.206574 31741 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:13.220485 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:13.251691 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.252416 20728 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:13.253301 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.198508 32002 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:13.266472 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.194093 31981 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.219542 31995 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:13.206822 32004 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.217558 31761 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:13.243363 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.280694 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.243474 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.159663 31893 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.187386 31887 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.255955 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2632) took 1168 ms. Trace:
I20260709 12:26:13.299625 21430 rpcz_store.cc:276] 0709 12:26:12.087965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.123758 (+ 35793us) service_pool.cc:224] Handling call
0709 12:26:12.136300 (+ 12542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.556273 (+419973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.255061 (+698788us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b5400750154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:13.255872 (+   811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964158848}
I20260709 12:26:13.301652 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:13.302579 21430 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:13.263957 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:13.267645 20732 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:13.192420 31858 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.177733 31894 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.270793 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.306591 20728 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:13.275339 31839 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:13.239297 31922 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.199950 31847 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.286053 20877 raft_consensus.cc:2368] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31.
I20260709 12:26:13.200219 31943 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.331339 20882 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:13.332650 31943 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:13.289579 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:13.334141 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.334830 20809 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:13.339462 20802 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
W20260709 12:26:13.290494 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.343676 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.344710 31887 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.332695 31858 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.257294 31927 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.346294 31927 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.347069 31955 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.312235 31776 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.349504 31920 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.275216 31898 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:13.349900 31973 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:13.270581 31462 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.350577 31462 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.350425 31898 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.352898 31462 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.295897 31318 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.353223 31898 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.297176 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2633) took 1203 ms. Trace:
I20260709 12:26:13.355005 20806 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:26:13.297748 31770 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:13.298939 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.269645 20726 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:13.275614 31981 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:13.309626 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:13.270573 32002 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:13.279392 31761 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:13.322288 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71710.9us
I20260709 12:26:13.322443 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.375346 20728 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:26:13.288566 31916 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.376415 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.377159 20728 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:13.334262 31847 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.275025 32005 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.019s
I20260709 12:26:13.292351 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3982) took 1025 ms. Trace:
I20260709 12:26:13.342726 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.354236 21429 rpcz_store.cc:276] 0709 12:26:12.093480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.166787 (+ 73307us) service_pool.cc:224] Handling call
0709 12:26:12.167093 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.104737 (+937644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.296829 (+192092us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:13.297079 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2008103808}
I20260709 12:26:13.365836 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:13.369138 21430 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:13.371167 20809 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:13.349465 31927 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.383813 21428 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:13.384804 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:13.385001 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.379972 20881 rpcz_store.cc:276] 0709 12:26:12.266757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.268046 (+  1289us) service_pool.cc:224] Handling call
0709 12:26:12.280749 (+ 12703us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:12.334344 (+ 53595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.334385 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:12.340387 (+  6002us) log.cc:844] Serialized 52 byte log entry
0709 12:26:12.411381 (+ 70994us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:12.551876 (+140495us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.566911 (+ 15035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.567005 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.292112 (+725107us) raft_consensus.cc:1685] finished
0709 12:26:13.292137 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.292285 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67895,"prepare.queue_time_us":192354,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":74350,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":29627,"thread_start_us":168452,"threads_started":2,"wal-append.queue_time_us":616280}
I20260709 12:26:13.383020 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.346324 31965 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.368561 31904 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:13.364609 31741 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:13.388962 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.401980 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.402230 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:13.375595 31873 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:13.377564 31761 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:13.396347 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 152979us
I20260709 12:26:13.397207 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:13.402896 20809 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:13.385216 31318 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:13.387692 31898 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:13.411118 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:13.374094 31922 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
I20260709 12:26:13.420807 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:13.411459 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:13.406625 31994 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:13.436864 31994 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.437407 31994 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:13.439929 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:13.424809 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.406021 31927 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:13.447536 31927 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.448412 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.425661 31741 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.450178 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.407013 31873 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:13.451707 31873 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:13.453493 31894 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.454695 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2638) took 1315 ms. Trace:
W20260709 12:26:13.479754 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.420748 21428 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:13.444319 31916 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:13.445381 31893 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:13.435398 31973 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:13.486070 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:13.418570 31898 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:13.455561 31847 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.455875 32014 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:13.457438 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4060) took 1053 ms. Trace:
I20260709 12:26:13.495569 20877 rpcz_store.cc:276] 0709 12:26:12.403557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.411911 (+  8354us) service_pool.cc:224] Handling call
0709 12:26:13.457235 (+1045324us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54003f6e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:13.457378 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2002161024}
I20260709 12:26:13.456274 31318 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:13.499092 20875 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:26:13.455008 31922 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:13.409962 31920 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:13.469395 32004 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:13.457165 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:13.476840 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.460992 31895 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.457013 31883 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:13.469188 32005 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:13.496846 31898 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.425021 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.542178 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.487076 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:13.490538 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:13.543527 21429 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:13.481442 21433 rpcz_store.cc:276] 0709 12:26:12.139525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.393030 (+253505us) service_pool.cc:224] Handling call
0709 12:26:12.393511 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:12.393985 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.394025 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:12.420319 (+ 26294us) log.cc:844] Serialized 52 byte log entry
0709 12:26:13.453862 (+1033543us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b180036c9a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1
0709 12:26:13.454083 (+   221us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:13.454138 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.454255 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.454319 (+    64us) raft_consensus.cc:1685] finished
0709 12:26:13.454337 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.454616 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134636,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":47415,"spinlock_wait_cycles":5250944,"thread_start_us":140522,"threads_started":1,"wal-append.queue_time_us":502229}
I20260709 12:26:13.497550 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.506670 20809 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:13.498925 31887 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:13.499112 31847 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.508351 31864 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:13.509220 31318 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.495266 32014 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:13.510759 31803 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.536142 31881 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.515587 21399 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:13.786720 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 271272us
I20260709 12:26:13.516789 31839 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:13.535349 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 99772.3us
W20260709 12:26:13.537001 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 79920.3us
I20260709 12:26:13.791548 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:13.793143 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:13.462864 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.794991 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.795917 20726 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:13.545512 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.797371 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:13.798281 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.538786 32004 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:13.656247 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.512853 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1806) took 1152 ms. Trace:
I20260709 12:26:13.782184 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.805992 20731 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:13.499552 31973 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:13.805665 20735 rpcz_store.cc:276] 0709 12:26:12.360202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.373270 (+ 13068us) service_pool.cc:224] Handling call
0709 12:26:13.512626 (+1139356us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54003ad4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:13.512790 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1645138176}
I20260709 12:26:13.547814 31916 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:13.820541 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:13.820874 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.821728 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.815783 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.825778 20809 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:13.826632 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:13.817454 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:13.517479 31893 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.266s
I20260709 12:26:13.553871 32005 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:13.533344 31920 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.532012 31922 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:13.834306 20729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 4096) took 1971 ms (client timeout 1621 ms). Trace:
W20260709 12:26:13.851644 20729 rpcz_store.cc:269] 0709 12:26:11.862570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.862826 (+   256us) service_pool.cc:224] Handling call
0709 12:26:13.834109 (+1971283us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54002e4fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:13.834235 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386526080}
I20260709 12:26:13.807421 31741 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:13.839433 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:13.844507 21429 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:13.800292 31895 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.800215 31803 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.818981 31347 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:13.547452 31616 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.800433 31864 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:13.848258 31903 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.885s	user 0.009s	sys 0.002s
I20260709 12:26:13.840678 31776 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.847368 31886 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:13.781395 31887 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.822961 31911 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:26:13.835669 31462 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.039s
I20260709 12:26:13.886623 20809 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:13.872654 31895 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:13.898146 31995 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.620s	user 0.005s	sys 0.007s
W20260709 12:26:13.874342 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 327199us
I20260709 12:26:13.900287 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:13.901336 31916 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:13.901962 31916 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:13.843693 31873 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.808928 31881 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.822715 31894 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.049s
I20260709 12:26:13.831115 32004 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:13.889519 32010 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.482s	user 0.000s	sys 0.001s
I20260709 12:26:13.870878 21429 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:13.878579 31886 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:13.898787 31895 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.822340 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:13.911702 31959 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:13.921638 32004 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:13.922812 31700 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.923488 31894 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:13.924429 31881 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:13.948204 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.932816 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3887) took 2157 ms (client timeout 2209 ms). Trace:
I20260709 12:26:13.876435 31347 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.894212 32005 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.938482 31886 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:13.953361 31347 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.947603 31959 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.956061 31959 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:13.956670 21431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2636) took 1828 ms (client timeout 1866 ms). Trace:
W20260709 12:26:13.935992 31761 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 12:26:13.957299 31887 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.958009 31887 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:13.957103 21431 rpcz_store.cc:269] 0709 12:26:12.128502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.153269 (+ 24767us) service_pool.cc:224] Handling call
0709 12:26:13.956458 (+1803189us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:13.956607 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2082476800}
I20260709 12:26:13.879343 31776 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:13.960110 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.960908 20803 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:13.869984 32014 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.963439 31776 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.876631 31903 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.964078 32014 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.936830 31709 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.898754 31995 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.966287 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:13.936117 31922 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:13.949133 20808 rpcz_store.cc:269] 0709 12:26:11.775145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.775311 (+   166us) service_pool.cc:224] Handling call
0709 12:26:13.932513 (+2157202us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54003b6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:13.932710 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3903159040}
I20260709 12:26:13.967429 31995 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:13.929361 32015 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:13.968211 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:13.957692 31761 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.968889 31803 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:13.971596 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3796) took 1085 ms. Trace:
I20260709 12:26:13.971982 20733 rpcz_store.cc:276] 0709 12:26:12.886665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.228274 (+341609us) service_pool.cc:224] Handling call
0709 12:26:13.228704 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:13.266967 (+ 38263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.266999 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:13.270903 (+  3904us) log.cc:844] Serialized 52 byte log entry
0709 12:26:13.406438 (+135535us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:13.406505 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.406608 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.971374 (+564766us) raft_consensus.cc:1685] finished
0709 12:26:13.971395 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.971539 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37772,"raft.queue_time_us":43951,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":7783,"thread_start_us":111961,"threads_started":1,"wal-append.queue_time_us":254764}
I20260709 12:26:13.976488 31761 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:13.977450 20728 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:13.967995 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:13.915333 31911 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:13.944901 20726 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:26:13.982049 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.976511 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:13.993772 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4059) took 1590 ms. Trace:
I20260709 12:26:13.994280 20882 rpcz_store.cc:276] 0709 12:26:12.403265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.403678 (+   413us) service_pool.cc:224] Handling call
0709 12:26:12.419662 (+ 15984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.331792 (+912130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.993532 (+661740us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b540026ded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:13.993709 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096139520}
I20260709 12:26:13.971031 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 1979) took 1389 ms. Trace:
I20260709 12:26:13.995816 20875 rpcz_store.cc:276] 0709 12:26:12.581374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.211936 (+630562us) service_pool.cc:224] Handling call
0709 12:26:13.212402 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.499758 (+287356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.970685 (+470927us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400305554. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:13.970942 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658465024}
I20260709 12:26:13.999250 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:13.945798 31894 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:14.021139 20699 leader_election.cc:400] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31.
I20260709 12:26:14.021587 20699 leader_election.cc:403] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:26:13.885855 31839 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:13.995524 31966 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.000s
I20260709 12:26:14.025666 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2622) took 2038 ms. Trace:
W20260709 12:26:14.025826 20805 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 2035) took 2269 ms (client timeout 2933 ms). Trace:
W20260709 12:26:14.041057 20805 rpcz_store.cc:269] 0709 12:26:11.756030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.756474 (+   444us) service_pool.cc:224] Handling call
0709 12:26:14.025669 (+2269195us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b540027bd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:14.025769 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1738234240}
I20260709 12:26:14.038661 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4063) took 1627 ms. Trace:
I20260709 12:26:13.958693 31887 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:14.042693 20883 rpcz_store.cc:276] 0709 12:26:12.410798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.553785 (+142987us) service_pool.cc:224] Handling call
0709 12:26:12.554112 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.137645 (+583533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.038410 (+900765us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:14.038600 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1340331904}
I20260709 12:26:14.041486 21427 rpcz_store.cc:276] 0709 12:26:11.987544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.987809 (+   265us) service_pool.cc:224] Handling call
0709 12:26:11.990802 (+  2993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.264583 (+273781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.025377 (+1760794us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400746b54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:14.025599 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605383936}
I20260709 12:26:13.997478 31883 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:13.919492 31916 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.025940 31700 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:14.000741 31903 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:14.039710 32002 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.655s	user 0.006s	sys 0.004s
I20260709 12:26:13.917807 31973 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.024739 31776 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:13.946606 31864 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:13.986616 31911 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.051633 31965 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:14.070047 31871 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:14.110339 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 322661us
I20260709 12:26:14.113140 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:14.116364 31903 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:14.121604 31903 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.123361 31911 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.123696 31903 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.124234 31347 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:14.124791 31347 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.116712 31871 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.048588 31966 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:14.132845 31992 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.223s	user 0.000s	sys 0.010s
I20260709 12:26:14.135407 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2629) took 2085 ms. Trace:
I20260709 12:26:14.108026 31847 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:14.135890 21435 rpcz_store.cc:276] 0709 12:26:12.050121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.104670 (+ 54549us) service_pool.cc:224] Handling call
0709 12:26:12.131649 (+ 26979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.784386 (+652737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.135130 (+1350744us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54004da8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:14.135340 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471806720}
I20260709 12:26:14.127002 31347 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.139021 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2080) took 2182 ms. Trace:
I20260709 12:26:14.101639 32002 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.139456 21432 rpcz_store.cc:276] 0709 12:26:11.956958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.957075 (+   117us) service_pool.cc:224] Handling call
0709 12:26:11.977689 (+ 20614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.978149 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:26:12.590186 (+612037us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.885421 (+295235us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.894189 (+  8768us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.903393 (+  9204us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:12.903520 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.903695 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.132224 (+1228529us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:14.138710 (+  6486us) raft_consensus.cc:1685] finished
0709 12:26:14.138814 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.138940 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34666,"raft.queue_time_us":87527,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":11348,"spinlock_wait_cycles":8385408,"thread_start_us":220358,"threads_started":2,"wal-append.queue_time_us":234268}
I20260709 12:26:14.142467 32002 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:14.109894 31965 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.145959 31965 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:14.148842 31981 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.769s	user 0.001s	sys 0.009s
I20260709 12:26:14.149451 31981 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.153183 31871 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.097970 31776 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:14.156296 20803 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:14.130970 31966 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.160218 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:14.171286 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:14.174659 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:14.175288 21431 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:14.178654 20875 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:14.175539 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:14.175630 31639 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:14.175810 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:14.153825 31927 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:14.154038 31920 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.158079 31776 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:14.184515 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:14.186002 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:14.181452 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 342084us
I20260709 12:26:14.186230 20882 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:14.187085 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.187688 20883 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 37
I20260709 12:26:14.159696 31903 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:14.188568 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.188910 20875 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:14.189436 20730 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:14.154492 32002 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:14.194797 20875 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:14.197237 20877 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:14.099655 31894 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.200155 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.183787 31873 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.202411 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.206205 31894 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:14.206419 20730 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:14.218814 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 415796us
I20260709 12:26:14.219580 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:14.220916 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.223795 20884 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:14.225509 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.226550 20879 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:14.228562 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.231865 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.225598 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.229609 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.239710 20730 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:14.213415 31893 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:14.249475 20877 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:14.247753 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.236358 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.269296 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.189810 31347 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.049s
I20260709 12:26:14.195425 31895 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.050s
W20260709 12:26:14.285166 20810 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 4130) took 2385 ms (client timeout 2082 ms). Trace:
W20260709 12:26:14.285737 20810 rpcz_store.cc:269] 0709 12:26:11.900179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.901039 (+   860us) service_pool.cc:224] Handling call
0709 12:26:14.273861 (+2372822us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54003a0f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:14.285067 (+ 11206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350696320}
W20260709 12:26:14.287338 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51076.5us
I20260709 12:26:14.215981 31903 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:14.282270 20733 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:14.292030 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.298188 21400 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:14.311498 20730 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:14.264067 32022 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.314766 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4136) took 2152 ms. Trace:
W20260709 12:26:14.289536 32006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.004s	sys 0.023s
I20260709 12:26:14.218361 32002 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.316704 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.316764 20802 rpcz_store.cc:276] 0709 12:26:12.162313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.162469 (+   156us) service_pool.cc:224] Handling call
0709 12:26:12.167726 (+  5257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.339990 (+1172264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.314527 (+974537us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540019a5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:14.314708 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2694083456}
I20260709 12:26:14.269411 31318 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:14.320152 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:14.321204 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.285899 31779 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:14.285530 31981 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:14.238545 31694 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.22, Next index: 23, Last known committed idx: 19, Time since last communication: 0.056s
I20260709 12:26:14.311419 31887 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:14.354187 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.356613 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.356932 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.357590 20730 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:14.357751 20728 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:14.364833 20732 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
W20260709 12:26:14.321540 32006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.005s	sys 0.023s
I20260709 12:26:14.272586 31893 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.374805 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.371212 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.384471 20881 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:14.384692 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.385069 20730 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:14.373154 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.387535 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.371551 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.273569 31886 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.389084 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.333416 31873 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:14.392076 20731 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
W20260709 12:26:14.393055 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 197004us
I20260709 12:26:14.351686 31696 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:14.352367 32031 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:26:14.396485 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.368450 31898 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.002s
I20260709 12:26:14.402060 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.400071 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.399849 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.333792 32032 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:14.368674 31920 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:14.403323 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83268.7us
I20260709 12:26:14.380918 31895 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.380473 31871 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.408488 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:14.411736 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.390089 31739 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:14.391896 31965 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:14.423911 20728 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:14.323390 32022 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.426568 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:14.412906 20775 connection.cc:256] RPC call timeout handler was delayed by -1.2157921879997957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 12:26:14.431358 20775 connection.cc:256] RPC call timeout handler was delayed by -1.1340435110000726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2726s for a response
W20260709 12:26:14.431867 20775 connection.cc:256] RPC call timeout handler was delayed by -1.1248921879996487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
W20260709 12:26:14.432291 20775 connection.cc:256] RPC call timeout handler was delayed by -1.0312921879999521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
I20260709 12:26:14.353018 31318 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.405625 32032 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.406224 31887 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:14.440641 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.441459 20879 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:14.442207 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.442889 20879 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:14.443545 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.444114 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.444696 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.445278 32025 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:14.408779 31873 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:14.449179 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.446813 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.389168 31741 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.035s
I20260709 12:26:14.411978 31347 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.457505 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28.
I20260709 12:26:14.468677 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.405143 31696 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.474545 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.462730 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.475332 20884 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:14.410764 31898 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.439371 31761 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:14.438933 32006 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.447497 31839 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.490022 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.447715 32032 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:14.453819 32024 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:14.463373 32037 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:14.495249 20879 raft_consensus.cc:2393] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:14.509335 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.510504 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.496660 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:14.511704 21427 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:14.420153 31895 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:14.494758 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45648.6us
I20260709 12:26:14.420228 31871 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.513208 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.514321 20732 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:14.412535 31903 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.516330 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:14.527344 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.517566 20733 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:14.520738 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:14.530313 21427 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:14.521082 20884 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:14.531865 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.532984 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.534238 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.484957 32042 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:14.430450 31739 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.472550 31887 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:14.522047 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.547551 21427 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:14.507315 31981 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:14.560702 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:14.562160 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 128130us
I20260709 12:26:14.562956 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:14.513924 31966 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.471472 31347 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:14.530652 32029 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:14.532326 31984 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:14.502971 31965 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
W20260709 12:26:14.568483 20801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 3897) took 2788 ms (client timeout 1643 ms). Trace:
I20260709 12:26:14.537346 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.573415 20883 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:14.558463 31927 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.583046 31927 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:14.585110 32036 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:26:14.585466 31927 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.586555 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3981) took 2320 ms. Trace:
I20260709 12:26:14.566543 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.587045 20878 rpcz_store.cc:276] 0709 12:26:12.266481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.267242 (+   761us) service_pool.cc:224] Handling call
0709 12:26:12.267652 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.325331 (+ 57679us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:12.325394 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.137103 (+1811709us) spinlock_profiling.cc:243] Waited 6.85 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:14.137553 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.271139 (+133586us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:14.271176 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.271275 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.586198 (+314923us) raft_consensus.cc:1685] finished
0709 12:26:14.586228 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.586481 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57188,"raft.queue_time_us":63659,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":2792,"spinlock_wait_cycles":15763328,"thread_start_us":133270,"threads_started":1,"wal-append.queue_time_us":226940}
I20260709 12:26:14.589308 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.589988 20878 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:14.590375 31922 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:14.590502 31347 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.590730 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:14.592643 31739 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.594614 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2050) took 1329 ms. Trace:
W20260709 12:26:14.595722 31700 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.002s
I20260709 12:26:14.596493 31483 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:14.596894 31700 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31.
I20260709 12:26:14.596946 31483 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:14.599689 31483 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.600907 31483 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:14.601263 31616 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:14.601706 31616 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:14.595057 20804 rpcz_store.cc:276] 0709 12:26:13.265560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.977148 (+711588us) service_pool.cc:224] Handling call
0709 12:26:13.977517 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.118730 (+141213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.118776 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.118836 (+    60us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:14.118883 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.594354 (+475471us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002b57d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:14.594411 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.594550 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324158720}
I20260709 12:26:14.579581 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:14.578464 20801 rpcz_store.cc:269] 0709 12:26:11.780508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.806626 (+ 26118us) service_pool.cc:224] Handling call
0709 12:26:14.568286 (+2761660us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:14.568411 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739296512}
W20260709 12:26:14.614724 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48958.1us
I20260709 12:26:14.511493 31839 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.615716 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.511279 32006 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.618110 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.571617 32025 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:14.619832 31347 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:14.619655 31997 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 36.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:26:14.619971 32030 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:26:14.621556 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2083) took 1701 ms. Trace:
I20260709 12:26:14.571115 31966 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.618999 20878 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:14.625483 20884 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:14.626660 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.602571 31616 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.574062 31894 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:14.524129 32037 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.555364 32038 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.062s
I20260709 12:26:14.641499 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.688651 20883 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:14.649003 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:14.650573 21426 rpcz_store.cc:276] 0709 12:26:12.920274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.119639 (+199365us) service_pool.cc:224] Handling call
0709 12:26:13.120092 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.040194 (+920102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.040229 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.040280 (+    51us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:14.040310 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.621294 (+580984us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b5400645ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:14.621345 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.621489 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115468032}
I20260709 12:26:14.515973 32024 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.608203 32023 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:14.649338 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:14.677084 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:14.692627 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66069.5us
I20260709 12:26:14.693086 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.693755 20731 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 26.
I20260709 12:26:14.693817 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.675778 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.556564 32042 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.697130 20878 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:14.692988 20884 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:14.681802 31969 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:14.699620 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50725.7us
I20260709 12:26:14.700424 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:14.701825 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:14.707015 31779 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:14.707602 31779 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:14.708410 32034 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 12:26:14.709756 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2084) took 1785 ms. Trace:
W20260709 12:26:14.710675 32039 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:26:14.713366 31966 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.714004 32048 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:14.714740 32048 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:14.715420 20734 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3782) took 2281 ms (client timeout 1866 ms). Trace:
I20260709 12:26:14.710141 21433 rpcz_store.cc:276] 0709 12:26:12.924058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.778008 (+853950us) service_pool.cc:224] Handling call
0709 12:26:13.778469 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.043730 (+265261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.043761 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.044235 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.290217 (+245982us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:14.473972 (+183755us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.507398 (+ 33426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.507481 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.707758 (+200277us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:14.709397 (+  1639us) raft_consensus.cc:1685] finished
0709 12:26:14.709521 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.709689 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33049,"prepare.queue_time_us":261196,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2884,"raft.queue_time_us":39274,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":31996,"spinlock_wait_cycles":20298112,"thread_start_us":659611,"threads_started":3,"wal-append.queue_time_us":341016}
W20260709 12:26:14.715871 20734 rpcz_store.cc:269] 0709 12:26:12.433999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.446051 (+ 12052us) service_pool.cc:224] Handling call
0709 12:26:14.715177 (+2269126us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54002495d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:14.715330 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":801113472}
I20260709 12:26:14.716403 31886 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.717443 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3980) took 2451 ms. Trace:
I20260709 12:26:14.717669 31922 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:14.572397 31873 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.717875 20876 rpcz_store.cc:276] 0709 12:26:12.266223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.266692 (+   469us) service_pool.cc:224] Handling call
0709 12:26:12.266960 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.324779 (+ 57819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.324832 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.434735 (+2109903us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:14.449572 (+ 14837us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.627683 (+178111us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:14.627720 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.627801 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.717146 (+ 89345us) raft_consensus.cc:1685] finished
0709 12:26:14.717181 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.717390 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58914,"raft.queue_time_us":59095,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":4627,"spinlock_wait_cycles":18603904,"thread_start_us":136210,"threads_started":1,"wal-append.queue_time_us":243733}
I20260709 12:26:14.625502 31970 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 35.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.076s
I20260709 12:26:14.631548 31897 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 12:26:14.719774 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.720445 31897 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:14.720780 31897 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:14.721020 31920 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:14.673444 32025 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:14.721527 31897 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 17, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.722235 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:14.722824 31897 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:14.722510 32025 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.723016 21428 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:14.726421 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77190.5us
I20260709 12:26:14.726208 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:14.756973 21428 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
W20260709 12:26:14.756961 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63269.1us
I20260709 12:26:14.742996 20877 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
I20260709 12:26:14.759136 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.630435 32017 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:14.731964 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:14.761035 21428 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:14.766743 21426 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:14.684834 31965 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:14.674486 31893 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.717396 32022 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.762707 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:14.740973 21433 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:14.628336 32052 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.146s
I20260709 12:26:14.777292 20728 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:14.761089 31989 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:14.781363 21426 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
W20260709 12:26:14.793000 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 92951.4us
I20260709 12:26:14.792325 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.761897 31616 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:14.761269 31847 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:14.765571 31920 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:26:14.783293 32027 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.006s	sys 0.006s
I20260709 12:26:14.780987 32038 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:14.767577 31917 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:14.792479 31893 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:14.857708 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.858428 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.860064 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:14.861295 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 160958us
W20260709 12:26:14.854256 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.797933 31989 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.700124 31887 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.22, Last appended by leader: 22, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:14.885226 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.884697 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.888931 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:14.889599 20728 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:14.889603 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:14.885867 20731 raft_consensus.cc:2368] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37.
I20260709 12:26:14.726126 32023 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:14.861202 32027 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.006s	sys 0.006s
I20260709 12:26:14.765713 32049 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.165s
I20260709 12:26:14.850368 31696 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.851459 31894 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:14.945832 20728 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:14.784186 32048 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.100s
I20260709 12:26:14.914999 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:14.949741 31839 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.950066 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:14.950369 31839 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:14.889163 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:14.789492 31965 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.893138 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:14.951510 31839 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:14.951740 20734 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:14.951982 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.952874 31965 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:14.953531 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 65573.1us
I20260709 12:26:14.954200 31917 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:14.892794 31847 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:14.954577 31917 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:14.955024 31917 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:14.954800 31847 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.957065 31847 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:14.957165 31922 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:26:14.941425 32041 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:26:14.942085 31895 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:14.959468 32062 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 12:26:14.960698 32061 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:14.903370 20731 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:26:14.946936 32006 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:14.953632 32054 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:14.962231 32054 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:14.962390 32037 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:14.963251 32054 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:14.853842 31920 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:14.863457 31157 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:14.887264 31989 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:14.989634 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:14.991938 20849 leader_election.cc:400] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37.
I20260709 12:26:14.992278 20849 leader_election.cc:403] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:14.952037 31839 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:14.944962 31779 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.864216 31893 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:14.994270 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:14.995066 20731 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
W20260709 12:26:14.995491 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 111595us
W20260709 12:26:14.995980 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106901us
I20260709 12:26:14.995924 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:14.996392 31922 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:14.993114 31157 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.000178 31839 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:14.898247 31871 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:14.985477 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:14.883100 32038 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.005095 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:15.006430 20875 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
W20260709 12:26:15.011441 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 147025us
I20260709 12:26:15.019002 20875 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:15.019280 20731 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:15.023577 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:15.025519 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.007606 31893 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.002962 31462 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:15.032054 31886 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:26:15.034341 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.034305 32023 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:15.035887 32038 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.036808 31779 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:15.047861 32056 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.129s	user 0.003s	sys 0.000s
W20260709 12:26:15.048442 32047 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.302s	user 0.003s	sys 0.000s
I20260709 12:26:15.054100 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.054944 20877 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:15.055613 32006 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.056133 20877 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: [33.20-33.20]
I20260709 12:26:15.056331 32037 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.056102 32006 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.056921 32037 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.058353 32006 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.023058 31847 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:15.060557 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.061079 31847 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:15.061331 20878 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:15.062232 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:15.062170 31847 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 36.19, Last appended by leader: 19, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.064356 31347 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:15.064740 31347 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.065169 31157 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.010303 32055 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 36. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37.
I20260709 12:26:15.065562 31347 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 36.19, Last appended by leader: 19, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.066741 31347 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:15.019846 31920 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.068446 31920 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:15.069501 31898 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:15.037693 31776 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.842s	user 0.008s	sys 0.012s
I20260709 12:26:15.039332 31922 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:15.040141 31909 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:15.011535 31970 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:15.040361 31466 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:15.020720 31973 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:15.108623 20878 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
W20260709 12:26:15.100250 31994 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 1.662s	user 0.007s	sys 0.009s
I20260709 12:26:15.100073 31776 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.059031 32037 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:15.090166 32032 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.001s
I20260709 12:26:15.078565 31974 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:15.141259 31973 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:15.151486 32063 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:15.140487 31994 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.175223 32027 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:15.175834 32027 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.176923 32027 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.177532 31994 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:15.178467 32027 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.181531 32042 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.163836 31973 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:15.157323 32040 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.649s	user 0.004s	sys 0.000s
I20260709 12:26:15.166697 32024 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.194296 31989 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.197451 31989 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:15.243928 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:15.247299 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 242913us
I20260709 12:26:15.252287 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3797) took 2347 ms. Trace:
I20260709 12:26:15.252939 20726 rpcz_store.cc:276] 0709 12:26:12.904620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.837796 (+933176us) service_pool.cc:224] Handling call
0709 12:26:13.940134 (+102338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.984371 (+ 44237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.251751 (+1267380us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002279d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:15.252053 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234582912}
I20260709 12:26:15.257345 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4112) took 2430 ms. Trace:
W20260709 12:26:15.257697 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.259115 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 241987us
W20260709 12:26:15.261334 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.262924 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:15.258412 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.923141 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:15.257720 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 234100us
I20260709 12:26:15.283608 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.284561 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 3995) took 1064 ms. Trace:
I20260709 12:26:15.285128 20880 rpcz_store.cc:276] 0709 12:26:14.219569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.219693 (+   124us) service_pool.cc:224] Handling call
0709 12:26:14.219980 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.402421 (+182441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.402454 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.402495 (+    41us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:14.402524 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.283929 (+881405us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54001fe854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:15.284032 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.284295 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419113216}
I20260709 12:26:15.288470 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 3992) took 1071 ms. Trace:
I20260709 12:26:15.289050 20881 rpcz_store.cc:276] 0709 12:26:14.217226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.217388 (+   162us) service_pool.cc:224] Handling call
0709 12:26:15.287948 (+1070560us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400200654. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:15.288197 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293521792}
I20260709 12:26:15.291165 21430 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 28: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:15.291323 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2643) took 1121 ms. Trace:
I20260709 12:26:15.292933 20876 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:15.293480 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2647) took 1122 ms. Trace:
I20260709 12:26:15.292047 21432 rpcz_store.cc:276] 0709 12:26:14.169495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.169671 (+   176us) service_pool.cc:224] Handling call
0709 12:26:14.170098 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.170560 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.170602 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.188957 (+ 18355us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.367210 (+178253us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:14.367272 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.367371 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.156701 (+789330us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:15.290241 (+133540us) raft_consensus.cc:1685] finished
0709 12:26:15.290303 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.290968 (+   665us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":654,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6291,"spinlock_wait_cycles":16929024,"thread_start_us":165612,"threads_started":1,"wal-append.queue_time_us":256407}
I20260709 12:26:15.295130 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.296248 21426 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:15.282917 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.298091 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.299052 21435 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:15.300192 31318 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.298290 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2043) took 2553 ms. Trace:
I20260709 12:26:15.292073 21428 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:15.320315 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.320838 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.321256 21426 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:15.257951 20727 rpcz_store.cc:276] 0709 12:26:12.826838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.842947 (+ 16109us) service_pool.cc:224] Handling call
0709 12:26:12.843310 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.246456 (+403146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.256370 (+2009914us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54000e83d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:15.257045 (+   675us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852630272}
I20260709 12:26:15.303740 21434 rpcz_store.cc:276] 0709 12:26:12.744950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.004918 (+259968us) service_pool.cc:224] Handling call
0709 12:26:13.008634 (+  3716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.297897 (+289263us) raft_consensus.cc:1534] Early marking committed up to index 24
0709 12:26:13.298299 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.404235 (+105936us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.404286 (+    51us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:13.404333 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.297704 (+1893371us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54001e5fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:15.297795 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.298087 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8182,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":56,"raft.queue_time_us":2092,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":8967,"spinlock_wait_cycles":664532736,"thread_start_us":105521,"threads_started":1,"wal-append.queue_time_us":172225,"wal-append.run_cpu_time_us":1390,"wal-append.run_wall_time_us":1001955}
I20260709 12:26:15.322113 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:15.293890 21430 rpcz_store.cc:276] 0709 12:26:14.170848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.170975 (+   127us) service_pool.cc:224] Handling call
0709 12:26:15.293197 (+1122222us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54007602d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:15.293391 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571353856}
I20260709 12:26:15.323863 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.308841 20701 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:15.309587 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:15.327385 20807 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:15.329066 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:15.330323 20727 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.333456 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.337291 20732 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
W20260709 12:26:15.330677 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.304167 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.315833 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:15.317411 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47563.1us
I20260709 12:26:15.341308 20801 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:15.313187 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50631us
I20260709 12:26:15.336890 20884 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
W20260709 12:26:15.348016 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95492.7us
I20260709 12:26:15.357733 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.359006 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.359817 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.303781 20880 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:15.360818 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.361923 20878 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:15.362636 20880 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:15.363268 20700 leader_election.cc:336] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.643s (SENT)
I20260709 12:26:15.363948 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.365389 20732 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 29 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:26:15.365710 31462 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:15.360551 21428 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:15.370296 31462 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:15.372233 21432 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:15.371968 31462 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:15.373654 31462 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:15.374115 31462 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.374197 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.099890 31466 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.376828 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:15.377238 31466 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.379014 31466 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:15.380175 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.381575 21428 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:15.158321 31910 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:15.406949 31910 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.408625 31910 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.409494 31883 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:15.410226 31910 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.411667 31903 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:15.411785 31898 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:15.412634 31903 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.317802 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:15.387637 31696 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.378247 21430 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:15.158340 31911 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:15.167234 31969 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:15.141848 31970 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:15.454167 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 136959us
I20260709 12:26:15.455581 31864 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:15.456115 31864 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.164608 31974 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.456535 31864 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:15.174368 31988 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 2.517s	user 0.011s	sys 0.013s
I20260709 12:26:15.465040 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:15.465457 32025 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:15.466054 32025 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:15.466485 32025 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:15.168305 32002 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.467327 31997 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:15.148007 32032 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:15.170104 32050 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.599s	user 0.001s	sys 0.001s
I20260709 12:26:15.385449 31462 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.302484 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:15.410411 31883 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.497826 31894 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:15.499255 20879 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:15.499939 31883 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:15.434170 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 91973.2us
I20260709 12:26:15.444101 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:15.503835 21428 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:15.437551 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:15.427814 31898 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.019s
I20260709 12:26:15.437599 31911 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.454295 31970 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.507730 31911 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 21, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.480674 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:15.480872 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.481330 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.509948 20810 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:15.509619 32081 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:15.510450 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:15.510856 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:15.511420 20807 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:15.511700 20810 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.512370 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:15.513152 31917 consensus_meta.cc:287] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.557s	user 0.005s	sys 0.004s
I20260709 12:26:15.513207 20807 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:15.513948 31917 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.514871 20807 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:15.483011 20805 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:15.483057 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.484449 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.410425 31871 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.068s
I20260709 12:26:15.494525 31974 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.478382 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:15.495600 31916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.027s	sys 0.040s
W20260709 12:26:15.533315 31916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.028s	sys 0.040s
I20260709 12:26:15.533332 32042 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:15.534679 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:15.535485 20734 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:15.536388 31776 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:15.536500 32038 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:15.538051 31462 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:15.538852 20801 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:15.539635 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.541287 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:15.542522 20809 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:15.542395 31989 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:15.544533 32064 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.545190 32064 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.547788 32089 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:15.549690 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3805) took 1175 ms. Trace:
I20260709 12:26:15.550101 20735 rpcz_store.cc:276] 0709 12:26:14.374330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.374499 (+   169us) service_pool.cc:224] Handling call
0709 12:26:15.549501 (+1175002us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002e54d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:15.549621 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316270976}
I20260709 12:26:15.554214 20810 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:15.508687 32032 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.482025 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:15.515609 20877 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:15.525100 31700 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:15.527266 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.528035 31969 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.529284 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 227121us
I20260709 12:26:15.530033 20808 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:15.531301 31871 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:15.532492 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.532673 31903 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:15.496233 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.562909 20806 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:15.565653 20727 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:15.568745 20803 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:15.449878 31696 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.566988 31973 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.582278 20808 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:26:15.583043 31696 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.583529 31973 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:15.584944 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.585847 20810 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:15.587780 32027 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:15.589049 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:15.591379 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 228257us
I20260709 12:26:15.593426 31905 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:15.594175 31905 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:15.594877 31969 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:15.595883 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.597604 31969 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:15.599550 31886 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:15.600396 20701 leader_election.cc:400] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:15.600760 20701 leader_election.cc:403] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:26:15.600317 31886 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.602759 31886 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:15.614373 20808 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:15.614163 31905 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.615914 20808 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:26:15.618088 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.603236 31917 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.603513 31873 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:15.620853 31873 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.621704 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:15.622181 31893 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.576s	user 0.002s	sys 0.012s
I20260709 12:26:15.623163 31873 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:15.623523 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.624189 20807 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
W20260709 12:26:15.599293 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 185565us
I20260709 12:26:15.591025 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:15.623028 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.625438 20803 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:15.606462 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.606673 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:15.611877 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70735.5us
I20260709 12:26:15.612597 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.613236 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:15.604738 20802 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:15.609117 20734 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:15.622625 31893 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.568156 31700 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:15.638629 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47784.7us
I20260709 12:26:15.635511 20801 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:15.642565 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.643707 20803 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:15.567792 31871 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:15.645936 21430 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:15.641104 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:15.647421 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:15.578614 32025 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.624884 32038 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:15.649570 20805 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:15.654287 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:15.652556 21428 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:15.645277 31883 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:15.633334 32081 consensus_queue.cc:1243] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 31.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:15.660579 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.658111 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.666939 20727 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:15.668392 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55380us
I20260709 12:26:15.668162 31700 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.669310 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:15.669435 31700 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:15.672595 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:15.672669 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:15.674654 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:15.675369 31989 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:15.676827 32032 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:15.677250 32032 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:15.677625 32032 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.679768 20807 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:15.680243 21399 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:15.680630 32005 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:15.681466 31871 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.682207 31886 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:15.682885 31886 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:15.684811 32090 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:26:15.684336 31886 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:15.685109 31616 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.804s	user 0.009s	sys 0.013s
I20260709 12:26:15.686614 31970 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:15.687084 31970 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.687446 31970 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:15.682904 31871 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:15.688211 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.662653 31883 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.690506 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.651585 31905 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:15.690631 31883 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:15.692582 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.683063 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.694020 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:15.709136 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:15.714546 21429 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:15.721680 20734 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:15.729573 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:15.730456 20730 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:15.666908 32064 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:15.739131 20732 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:15.740146 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:15.741449 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:15.745422 31616 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.747783 32064 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:15.748267 31847 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:15.748394 32064 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:15.748759 31847 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.749219 31847 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.749490 32064 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:15.747818 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73296.7us
I20260709 12:26:15.752015 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:15.753508 31893 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:15.756876 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:15.759320 31989 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 35.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:15.726902 31997 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:15.759594 32081 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
W20260709 12:26:15.760208 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 133422us
I20260709 12:26:15.769474 31925 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:15.769965 31925 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:15.763681 20729 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:15.774260 31911 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:15.774609 32005 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:15.775367 21428 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:15.777194 32042 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:15.777592 32042 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.778534 32042 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.779884 32042 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:15.779930 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:15.781069 31483 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:15.781780 31483 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:15.782539 31483 consensus_queue.cc:1243] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 32.23 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 22
I20260709 12:26:15.783411 31873 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:15.783766 31873 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:15.785110 32025 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:15.784778 31873 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.799273 20875 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:15.771001 31925 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.20, Last appended by leader: 20, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.800643 31925 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.800849 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.801946 32027 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:15.802408 32027 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:15.804260 21429 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:15.805783 20882 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:15.808630 31873 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:15.804289 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 133517us
I20260709 12:26:15.791276 20804 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:15.808599 20883 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
W20260709 12:26:15.797657 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 122557us
I20260709 12:26:15.813982 20727 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 27.
W20260709 12:26:15.863171 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.874402 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58058.6us
I20260709 12:26:15.793066 31692 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:15.799259 31910 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:15.875595 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:15.875530 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:15.883014 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:15.884076 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57086.9us
I20260709 12:26:15.880429 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:15.830487 31318 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:15.886489 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:15.880504 20879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4078) took 1379 ms (client timeout 1712 ms). Trace:
W20260709 12:26:15.890810 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.878976 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:15.898501 20884 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:15.794770 31864 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.900115 20881 raft_consensus.cc:2376] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 28. Re-sending same reply.
W20260709 12:26:15.896452 20879 rpcz_store.cc:269] 0709 12:26:14.500702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.527062 (+ 26360us) service_pool.cc:224] Handling call
0709 12:26:15.834644 (+1307582us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b54001fe354. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:15.880447 (+ 45803us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2169989376}
I20260709 12:26:15.881282 31692 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.907953 20882 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:15.908842 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.899513 31910 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:15.913699 31910 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.915467 32023 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.015s
I20260709 12:26:15.916138 32023 consensus_queue.cc:1243] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20260709 12:26:15.927342 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 170565us
I20260709 12:26:15.929503 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:15.931521 31920 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:15.932101 31920 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.932582 31920 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:15.939663 31739 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:15.941222 31739 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:15.943521 31996 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:15.943671 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4066) took 1466 ms. Trace:
I20260709 12:26:15.944087 31996 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:15.944279 20877 rpcz_store.cc:276] 0709 12:26:14.465080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.056986 (+591906us) service_pool.cc:224] Handling call
0709 12:26:15.057272 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:15.603665 (+546393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.906028 (+302363us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b540005a854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:15.931785 (+ 25757us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053123968}
I20260709 12:26:15.945559 31996 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:15.953311 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:15.954152 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47376.7us
I20260709 12:26:15.954748 32052 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:15.955264 32052 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:15.955447 20700 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.209s (SENT)
I20260709 12:26:15.955821 32052 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:15.955969 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:15.958230 31696 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:15.901417 31973 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:15.905418 31970 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.967751 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.968505 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:15.969782 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:15.986294 32096 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:15.943123 31739 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:15.990532 31996 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:15.992611 31692 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 28.23, Last appended by leader: 23, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.995329 31973 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.998672 31883 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:15.999439 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88759.3us
I20260709 12:26:16.001886 31616 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:16.003324 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54956.5us
I20260709 12:26:16.003545 31770 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:26:16.006138 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 176566us
I20260709 12:26:16.008293 31970 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.008564 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:15.966768 20879 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:15.969870 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:16.012019 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.047595 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:16.016146 31905 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:16.017241 20881 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:16.002774 31847 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:15.946342 20877 pending_rounds.cc:85] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 18
I20260709 12:26:16.026793 20882 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:16.027408 31739 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:16.031441 32032 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.032755 31973 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:16.033865 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.033833 31864 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:16.034564 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 79186.3us
I20260709 12:26:16.051644 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:16.065974 20883 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:16.068137 31847 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.060956 31739 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:16.069052 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.053129 20877 pending_rounds.cc:107] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 12:26:16.072089 20877 raft_consensus.cc:2889] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:16.064047 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:16.076268 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:16.079668 21427 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:16.082688 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:16.084288 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:16.085541 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:16.086227 32025 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:16.087673 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:16.089609 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.090428 20882 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:16.090000 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.094601 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:16.095866 20880 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:16.096824 20884 raft_consensus.cc:2368] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34.
I20260709 12:26:16.091113 31483 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:16.098069 20878 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:16.045993 31911 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.030s
W20260709 12:26:16.099488 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50878.9us
I20260709 12:26:16.100365 20876 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:16.103790 20878 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: [33.20-33.20]
I20260709 12:26:16.050984 31466 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.044s
I20260709 12:26:16.104351 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:16.101624 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.089040 31692 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:16.091209 32025 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:16.064965 32023 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.039s
I20260709 12:26:16.112347 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.074061 32005 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.033s
W20260709 12:26:16.128834 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.106027 31970 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:16.120389 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:16.134709 20882 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:16.133467 31847 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:16.110416 31318 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:16.157783 32032 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:16.162158 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59664.9us
I20260709 12:26:16.133386 31969 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:16.219910 31969 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:16.221057 31969 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.222337 31969 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:16.166044 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.209240 31739 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.228358 20879 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:16.230566 31739 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:16.167471 31970 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:16.169304 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:16.195358 20775 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:16.203136 31973 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.031s
I20260709 12:26:16.107468 31920 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.137401 31700 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.080s
I20260709 12:26:16.207711 31847 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:16.234089 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 187328us
I20260709 12:26:16.212064 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:16.239985 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 130892us
W20260709 12:26:16.241900 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46707.5us
I20260709 12:26:16.242139 31883 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:16.210228 31318 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:16.249511 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 100010us
I20260709 12:26:16.253750 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.251561 31318 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:16.255693 31700 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:16.256134 31700 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:16.254526 20881 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:16.257432 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:16.253249 31970 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.259083 20882 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:16.247313 31864 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election won for term 38
W20260709 12:26:16.264696 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52744.8us
I20260709 12:26:16.247481 31847 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.248921 31466 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:16.271795 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:16.265630 31873 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:16.244508 32052 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.266641 31997 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:16.257289 31925 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:16.280637 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:16.269886 31864 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.286384 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:16.284631 31997 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.286074 31864 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:16.296733 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:16.270285 32027 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.302197 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:16.302935 32052 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:16.294135 31873 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:16.294396 31989 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:16.293437 31466 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:16.300165 31997 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:16.318763 21427 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:16.306051 31779 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:16.280104 31700 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.306783 31883 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:16.317427 31973 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:16.332134 32027 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.336568 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:16.337304 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.300443 32023 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.020s
I20260709 12:26:16.339560 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.340225 21430 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:16.340559 31770 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.341321 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55058.8us
I20260709 12:26:16.338393 21432 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:16.351223 21428 raft_consensus.cc:2393] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:16.298805 31925 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.368466 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:16.321740 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:16.362632 32103 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:16.362690 31770 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.364512 31700 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:16.350811 31973 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:16.350971 31871 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.015s
I20260709 12:26:16.343695 31779 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:16.350245 31910 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:16.373821 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:16.355928 31466 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:16.397795 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 122478us
I20260709 12:26:16.411839 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:16.414127 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:16.421061 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.423079 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:16.433261 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 111582us
I20260709 12:26:16.383630 31739 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:16.434376 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:16.434787 21429 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:16.437476 20804 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:26:16.403679 31779 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:16.401081 31910 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.450176 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.403340 32038 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.455964 20729 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:16.456387 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:16.458277 31887 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.449525 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:16.462853 20803 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:26:16.424208 31996 raft_consensus.cc:2764] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:26:16.483174 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:16.499794 20803 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.435564 32042 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.490078 31965 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.490754 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:16.491807 31973 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.20, Last appended by leader: 20, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.492697 31770 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:16.495786 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 71911.4us
I20260709 12:26:16.434502 32064 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:16.498072 31925 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:16.451330 31920 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.504350 20732 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:26:16.452632 31696 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:16.463905 20699 leader_election.cc:400] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34.
I20260709 12:26:16.523741 20699 leader_election.cc:403] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Cancelling election due to peer responding with higher term
I20260709 12:26:16.455070 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:16.459329 31910 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:16.444578 31883 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.480576 20727 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:26:16.498862 32038 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.511768 20808 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
W20260709 12:26:16.536500 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81538us
I20260709 12:26:16.461735 32032 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.071s
W20260709 12:26:16.550232 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 107630us
W20260709 12:26:16.530602 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.527010 31696 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.550405 20806 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:16.452843 31873 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.527400 32042 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.553895 20735 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:16.538944 32023 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:16.560648 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:16.512555 31739 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.551414 31883 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.585678 20729 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:16.537559 31996 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.521173 31887 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.558550 31696 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:16.539973 31318 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:16.604710 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 131189us
I20260709 12:26:16.605661 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:16.586447 32005 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:16.607087 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:16.557850 31920 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.537530 31965 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.612279 31911 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:16.562058 31466 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 32.16, Last appended by leader: 16, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.581866 31616 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 12:26:16.571842 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:16.620349 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 334825us
I20260709 12:26:16.595395 31871 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.026s
I20260709 12:26:16.618304 32027 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.636204 20882 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 22. (term mismatch)
I20260709 12:26:16.636730 20882 pending_rounds.cc:85] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 21
I20260709 12:26:16.637007 20882 pending_rounds.cc:107] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 31.22
I20260709 12:26:16.637322 20882 raft_consensus.cc:2889] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:16.622120 31911 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:16.620940 31692 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:16.643955 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:16.648149 32108 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:16.655010 20733 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:16.620712 31996 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.668274 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.684257 20803 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:16.686898 21428 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34.
I20260709 12:26:16.692749 32013 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.693483 32013 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:16.695243 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88315.5us
I20260709 12:26:16.696290 20775 leader_election.cc:400] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:16.696640 20775 leader_election.cc:403] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:26:16.695262 32025 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.699532 21427 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:16.695426 32038 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.702848 31739 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.703493 31739 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:16.705323 32074 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 35.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.082s
I20260709 12:26:16.706465 32074 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:16.706832 32074 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:16.707531 32025 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.707917 32074 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.709388 32074 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:16.708140 31779 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.717515 31779 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:16.702457 31696 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.672436 32108 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:16.648976 32027 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.756778 20734 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
W20260709 12:26:16.754680 20700 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.504s (SENT)
I20260709 12:26:16.652209 31911 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.20, Last appended by leader: 20, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.782594 32110 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:16.782855 31911 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.782886 31883 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:16.817569 20624 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 277) took 36629 ms (client timeout 15000 ms). Trace:
W20260709 12:26:16.828677 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74043.6us
W20260709 12:26:16.893685 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.809629 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 391047us
I20260709 12:26:16.896049 32032 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:16.896837 32032 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:26:16.897418 31893 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:16.897728 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.898448 21427 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:16.897823 31893 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:16.796658 32027 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.816444 31696 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.832166 31886 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.785470 32042 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:16.942138 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:16.788599 32005 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:16.804569 31318 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:16.804190 31996 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:16.834666 32025 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:16.808635 32089 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:16.917008 32064 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.936462 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.947853 31969 consensus_meta.cc:287] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.725s	user 0.008s	sys 0.012s
I20260709 12:26:16.799985 32013 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.947424 20876 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:16.824826 31887 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:16.803242 31989 consensus_queue.cc:1059] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 15, Time since last communication: 0.128s
I20260709 12:26:16.947721 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:16.949275 20883 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:16.971050 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 327181us
I20260709 12:26:16.952315 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.971870 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:16.966518 20700 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.235s (SENT) [suppressed 22 similar messages]
W20260709 12:26:16.954815 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 383058us
I20260709 12:26:16.828480 32052 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.955947 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:16.933898 31700 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.032s
I20260709 12:26:16.803057 31997 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.968380 20875 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:16.968735 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.983886 20882 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34.
I20260709 12:26:16.767774 31847 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.950238 31779 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.985119 31779 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.987219 31779 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.976136 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:16.987774 20876 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:16.989033 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.976765 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:16.989919 21432 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:16.793682 31965 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:16.978019 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:16.829253 32108 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:16.955220 32089 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:16.992004 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:16.978010 20884 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:16.980521 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:16.992472 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 270242us
I20260709 12:26:16.992014 20877 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 37 index: 21. (index mismatch)
I20260709 12:26:16.993849 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:16.994664 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:17.000584 20776 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34.
I20260709 12:26:17.001152 20776 leader_election.cc:403] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Cancelling election due to peer responding with higher term
W20260709 12:26:16.994436 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.960158 32005 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:16.945083 31886 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:16.993002 31864 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.691s	user 0.006s	sys 0.007s
I20260709 12:26:17.003767 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.006645 21430 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:17.005748 20880 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: [33.22-33.22]
I20260709 12:26:16.979358 31318 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:16.997936 32013 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:17.001434 32042 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.008639 31739 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:17.004632 31925 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:17.023949 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:16.991675 32052 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:17.031373 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 83819.4us
I20260709 12:26:17.031931 20876 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:17.011443 32027 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 33. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34.
I20260709 12:26:16.999152 32064 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:16.998412 31887 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.030s
I20260709 12:26:17.018126 32013 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:17.037921 31969 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:17.053982 31873 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:17.054193 31911 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:17.077597 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.054426 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 30 committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:17.055627 31739 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:17.033504 32113 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.057507 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:17.059787 32112 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:26:17.060097 31616 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.086033 31616 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.018325 31910 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.060170 32122 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.087119 31917 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:17.097126 31917 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:16.887025 20624 rpcz_store.cc:269] 0709 12:25:40.187684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.343463 (+4155779us) service_pool.cc:224] Handling call
0709 12:26:16.817327 (+32473864us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:11.651118 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:11.719080 (+ 67962us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:11.719143 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:26:11.719161 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:11.720329 (+  1168us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:11.720423 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:11.720454 (+    31us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:26:11.721656 (+  1202us) tablet.cc:701] Row locks acquired
0709 12:26:11.721702 (+    46us) write_op.cc:260] PREPARE: finished
0709 12:26:11.721881 (+   179us) write_op.cc:270] Start()
0709 12:26:11.722037 (+   156us) write_op.cc:276] Timestamp: P: 1783599971721807 usec, L: 0
0709 12:26:11.722054 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:26:11.722777 (+   723us) log.cc:844] Serialized 4891 byte log entry
0709 12:26:11.944157 (+221380us) op_driver.cc:464] REPLICATION: finished
0709 12:26:12.175924 (+231767us) write_op.cc:301] APPLY: starting
0709 12:26:12.176029 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:12.273638 (+ 97609us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:12.273771 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:12.290328 (+ 16557us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:12.290653 (+   325us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:26:12.290705 (+    52us) write_op.cc:312] APPLY: finished
0709 12:26:12.291073 (+   368us) log.cc:844] Serialized 113 byte log entry
0709 12:26:12.291297 (+   224us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:12.291450 (+   153us) write_op.cc:454] Released schema lock
0709 12:26:12.291495 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:12.304899 (+ 13404us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:12.304931 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":283683,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":231516,"apply.run_cpu_time_us":16304,"apply.run_wall_time_us":128927,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54287,"num_ops":12,"prepare.queue_time_us":275290,"prepare.run_cpu_time_us":16662,"prepare.run_wall_time_us":202850,"raft.queue_time_us":106598,"raft.run_cpu_time_us":109449,"raft.run_wall_time_us":145489,"replication_time_us":221970,"spinlock_wait_cycles":1015552,"thread_start_us":447979,"threads_started":4,"wal-append.queue_time_us":105492,"wal-append.run_cpu_time_us":54680,"wal-append.run_wall_time_us":1459259}]]}
I20260709 12:26:17.098181 31917 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.052106 31973 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:26:17.083992 31739 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.077410 31969 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.100518 31969 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.102133 20882 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:17.103888 31692 raft_consensus.cc:3055] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:17.104090 31886 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.104370 31692 raft_consensus.cc:740] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:17.105288 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:17.104996 32122 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.105397 31692 consensus_queue.cc:260] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.065300 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.106614 20877 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
W20260709 12:26:17.065400 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 369251us
I20260709 12:26:17.108618 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:17.094717 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.109642 20881 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:17.109969 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:17.066773 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.110808 20884 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:17.110949 32027 raft_consensus.cc:2764] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:26:17.081761 32005 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.111572 31989 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:17.112102 31989 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:17.067775 20879 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:17.112563 31989 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.115566 31917 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.115849 20803 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:17.076586 32013 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:17.116464 32130 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:17.078950 32089 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.120393 32089 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:17.063912 20878 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:17.122939 31910 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.011272 31864 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.087469 31997 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.125008 31997 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:17.125703 31864 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.125537 31997 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.031324 31925 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.126587 31925 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:17.068683 32064 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.127414 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.021584 31318 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.128175 20808 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:17.081885 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.015218 31893 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.129701 20808 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:17.084049 31911 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.131511 31911 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.131798 32064 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.131067 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.133003 20808 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:17.132298 32064 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.134243 32052 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:17.055976 32114 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:17.085222 32123 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.005s
I20260709 12:26:17.106134 20883 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:17.085142 31847 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.136454 31864 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.137003 31864 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.098846 31920 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election lost for term 33. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34.
I20260709 12:26:17.139276 31864 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.139832 31864 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:17.140746 31864 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:17.141080 31864 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:17.141968 31864 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.135252 31893 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:17.079241 32025 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:17.093441 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:17.029630 32042 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.090404 20774 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34.
I20260709 12:26:17.150475 20879 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28. Ops: []
W20260709 12:26:17.149616 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59409.5us
I20260709 12:26:17.152737 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:17.144774 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51440.7us
W20260709 12:26:17.118989 32023 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.548s	user 0.007s	sys 0.000s
I20260709 12:26:17.155809 21427 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:17.157080 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:17.158890 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:17.163164 20884 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:17.163225 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:17.163867 21432 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:17.165623 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:17.174479 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:17.190943 20801 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:26:17.203096 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:17.204257 20729 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:17.202344 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.210337 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.143167 31864 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.211452 20809 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:17.092818 31770 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:17.212806 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.088363 31616 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:17.216895 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 250397us
I20260709 12:26:17.218145 20803 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:26:17.218315 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.217897 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.143726 32025 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:17.219973 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 185215us
I20260709 12:26:17.226145 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:17.217895 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:17.228289 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 184650us
I20260709 12:26:17.230247 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:17.230562 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:17.231915 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.232784 20876 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:17.234627 20805 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:17.172641 32023 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:17.236579 20735 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 26. Current term is 27. Ops: [24.17-26.18]
I20260709 12:26:17.237419 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:17.238700 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:17.180327 32032 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:17.179570 31696 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:17.203516 32074 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:17.204710 31920 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:17.186214 32038 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:17.290115 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.318998 21427 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:17.240465 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.179904 31911 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.113s
I20260709 12:26:17.300066 20807 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 19. Preceding OpId from leader: term: 37 index: 20. (index mismatch)
I20260709 12:26:17.300937 20808 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:17.301736 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.301620 20729 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:17.305145 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.325958 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:17.306536 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.227428 32130 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:17.314057 21428 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:17.290669 31965 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:17.352631 31965 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.356057 31692 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:17.359324 32081 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:17.359840 32081 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.360217 32081 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:17.361593 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.362797 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.362936 32042 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.363447 21426 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:17.364328 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:17.365362 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.365854 21426 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
W20260709 12:26:17.369364 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.316366 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.180043 32052 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.070s
I20260709 12:26:17.220007 32064 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.227617 31910 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.232139 32025 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:17.290402 20806 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:17.203152 31779 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.097s
I20260709 12:26:17.321692 21427 raft_consensus.cc:1240] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:17.298305 20804 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:17.321094 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:17.323906 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.326364 20803 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.235637 31847 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.325748 20881 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:17.333747 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:17.379223 20731 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:17.334440 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:17.327759 32130 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.321039 31883 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.335539 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.335951 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:17.334502 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.324308 32005 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:17.336992 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:17.386678 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52379.2us
I20260709 12:26:17.339493 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.342052 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:17.340399 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.392795 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:17.304065 32122 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:17.306269 31462 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.397465 20735 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:17.351863 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.375118 20733 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:17.389969 32130 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:17.241057 31886 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.130s
I20260709 12:26:17.354948 31965 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.435792 31886 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:17.436244 31886 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:17.437136 31886 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.438475 31886 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:17.383805 20700 leader_election.cc:336] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.587s (SENT)
I20260709 12:26:17.439751 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:17.395206 31883 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:17.440889 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57109.7us
I20260709 12:26:17.386461 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:17.442102 31883 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.388203 20809 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:17.389645 31847 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:17.443163 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.443387 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.444131 20809 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:17.444646 31847 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:17.414947 32123 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:17.390347 20808 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:17.390686 20727 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 35 because replica has last-logged OpId of term: 33 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 19.
I20260709 12:26:17.446347 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.446792 32116 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:17.447293 20808 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 32 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 18.
I20260709 12:26:17.092388 32113 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:26:17.322022 32074 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:17.395712 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.391476 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.397451 20882 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 16. Preceding OpId from leader: term: 34 index: 17. (index mismatch)
I20260709 12:26:17.325177 31920 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:17.377780 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:17.404063 31973 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:17.462899 20806 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.390314 31989 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.324997 32038 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.474143 31770 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.474792 31770 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.476053 32038 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.477094 31770 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:17.410207 31997 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.410890 31925 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.474134 20729 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:17.478739 31989 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.411844 32013 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.412555 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.413406 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.402928 31462 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.414372 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:17.396006 32136 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:17.389204 32064 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.420975 20805 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:17.398785 32005 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.445112 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.392139 21430 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:17.448206 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.500957 20732 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:17.501556 20808 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:17.449452 20700 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.662s (SENT)
I20260709 12:26:17.454438 20878 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:17.480075 32052 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
W20260709 12:26:17.502696 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53278.6us
I20260709 12:26:17.458034 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:17.503518 20731 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:17.503674 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.504376 20805 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.504796 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.506059 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:17.472185 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.385419 31910 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.481153 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:17.483806 20730 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:17.486003 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:17.487228 31925 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.555475 20735 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:17.500312 32123 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:26:17.495839 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.559224 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.504797 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:17.507997 32042 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.533033 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:17.564477 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.566018 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.566910 20805 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.568054 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.569422 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.570644 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.571439 20805 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:17.572247 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.572799 20735 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:17.574294 32142 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:17.541383 32013 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.539160 31969 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.578145 31969 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.543279 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.483306 31996 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:17.579697 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.580447 20809 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.581954 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:17.489230 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 147279us
I20260709 12:26:17.583880 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:17.584203 32113 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.552749 20801 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:17.560459 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55776.8us
W20260709 12:26:17.586082 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.585894 31887 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:17.537586 20803 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:17.587265 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.587426 31887 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:17.587846 20803 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:17.588721 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.589039 32137 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:17.589727 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.590461 20803 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.512925 32126 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:17.532419 31920 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.592442 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:17.560577 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:17.533716 20700 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.647s (SENT)
I20260709 12:26:17.594259 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:17.575314 31770 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.594336 32081 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.494673 32136 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.584764 32127 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:17.594688 32023 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:17.574923 32142 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.595220 31770 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.587870 31887 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:17.596112 32147 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:17.596403 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.573038 20805 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:17.598654 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:17.585826 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.558341 20726 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:17.560273 32116 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.559082 32123 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:17.508718 32064 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:17.604004 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 210369us
I20260709 12:26:17.611042 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:17.603119 20801 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
W20260709 12:26:17.629570 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 95901.4us
I20260709 12:26:17.617987 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:17.631866 21430 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:17.618953 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.613199 32074 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.620253 20808 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:17.634611 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.625937 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.633412 20882 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:17.636821 20805 raft_consensus.cc:2376] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32. Re-sending same reply.
I20260709 12:26:17.637638 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.638175 20805 raft_consensus.cc:2393] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:17.606793 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.627251 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:17.639024 20875 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:17.618839 31887 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:17.639887 20808 raft_consensus.cc:2410] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 32 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 18.
I20260709 12:26:17.640761 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.641772 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.615304 31864 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.027s
I20260709 12:26:17.642328 20808 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:17.619457 20810 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.621246 32023 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:17.622956 32147 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.616679 32038 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:17.644191 32023 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.623627 32116 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.623831 31893 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:17.647568 31893 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.647472 32146 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:17.648037 31893 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:17.613649 32136 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:17.619964 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:17.651881 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.617977 31779 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:17.652823 32124 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:17.653648 20805 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.653599 32064 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.607168 31910 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.654794 32038 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.605487 32052 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:17.657550 32052 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.657958 32038 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.625120 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.622918 20734 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:17.658507 32136 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.606391 32042 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.661422 21431 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:17.661672 32042 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.612718 31965 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:17.610986 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:17.663089 32081 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:17.663854 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52936.8us
I20260709 12:26:17.664660 31462 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.666249 31886 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:17.666785 31886 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.667256 31886 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.668478 21430 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:17.647112 32116 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.660782 32136 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.657975 32052 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:17.672019 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.672518 31318 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:17.676082 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.677089 20807 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.633306 32123 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.677960 20808 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.682253 32123 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:17.682411 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:17.683072 20700 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.735s (SENT)
I20260709 12:26:17.653314 32124 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.684325 32124 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:17.683020 20808 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:17.653281 31779 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.688304 32147 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.692261 31770 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:17.695379 20807 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:17.697904 31893 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.700318 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.683678 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:17.617224 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.701372 20808 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:17.713044 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.713478 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:17.718632 20808 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 37 index: 21. (index mismatch)
I20260709 12:26:17.723489 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.725818 20809 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:17.724601 20805 raft_consensus.cc:2376] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28. Re-sending same reply.
I20260709 12:26:17.725625 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:17.729624 20809 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:17.731597 20809 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
W20260709 12:26:17.731724 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97037.2us
I20260709 12:26:17.735126 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.736282 20805 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:17.740322 31965 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:17.717063 31692 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.738171 20809 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:26:17.744791 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 80104.1us
W20260709 12:26:17.730628 32089 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.008s
I20260709 12:26:17.744685 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:17.746644 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63622.6us
I20260709 12:26:17.752938 20734 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 37 index: 21. (index mismatch)
W20260709 12:26:17.803241 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46323.8us
I20260709 12:26:17.805542 32005 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:17.800261 31965 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:17.806474 31965 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:17.806717 31779 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.807421 31965 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.812026 20728 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:17.813036 32124 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.815533 31893 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.817512 31886 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.821298 32052 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:17.825965 32148 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.090s	user 0.001s	sys 0.001s
W20260709 12:26:17.824698 20700 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.893s (SENT)
I20260709 12:26:17.828662 20734 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:17.803143 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.890753 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.751000 31692 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.891551 20808 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: [31.18-31.18]
I20260709 12:26:17.906359 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:17.900602 20801 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.921809 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.922525 20804 raft_consensus.cc:2376] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29. Re-sending same reply.
W20260709 12:26:17.927994 32152 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:26:17.930542 20733 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:26:17.932515 31462 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.933065 31462 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:17.948187 20803 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:17.955998 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.958040 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:17.959805 32081 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:17.960412 32081 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:17.961002 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 136321us
I20260709 12:26:17.966776 20804 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:17.934796 31887 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:17.970705 32130 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.537s	user 0.012s	sys 0.000s
I20260709 12:26:17.971149 32130 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:17.972527 20806 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33.
I20260709 12:26:17.973515 32126 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.973948 32126 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:17.959857 32145 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:26:17.976969 32081 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:17.952615 32074 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:17.967948 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.935146 31462 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:17.969286 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.980823 20801 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:17.984367 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:17.985188 20810 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:17.991403 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.992121 20700 leader_election.cc:336] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 1.630s (SENT)
I20260709 12:26:17.992475 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:17.989566 32025 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
I20260709 12:26:18.001910 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.003953 20810 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.005398 31886 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:18.010125 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.011317 20801 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
W20260709 12:26:18.014564 20700 leader_election.cc:336] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Timed out: RequestConsensusVote RPC to 127.19.36.194:43085 timed out after 2.000s (SENT)
I20260709 12:26:18.015214 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:18.027398 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.028419 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.031776 20801 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:18.012832 32117 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:18.057036 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.057847 20801 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.038623 32038 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:18.013336 32025 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.065596 32025 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.066358 21434 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
W20260709 12:26:18.068320 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53803.9us
I20260709 12:26:18.035346 32142 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:18.073554 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.070591 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.075796 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.077368 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.081068 32124 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:18.083516 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.084813 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.085556 20802 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:18.086378 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:18.087504 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:18.089803 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.090925 20809 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.092841 32052 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:18.096603 20801 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:18.078963 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.086897 20802 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:18.103400 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.103869 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.104614 20729 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:18.099359 31692 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:18.110664 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.125077 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.100755 32052 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.137634 20726 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:18.156082 20727 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:18.163563 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.165503 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.166718 20733 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:18.170431 20727 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:18.172003 32167 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:26:18.192870 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.193836 20733 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:18.166297 31887 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:18.200114 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.200811 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
W20260709 12:26:18.201001 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.202009 20728 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 37 index: 21. (index mismatch)
W20260709 12:26:18.203037 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 470408us
I20260709 12:26:18.203704 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:18.206282 20733 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:18.207899 32023 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:26:18.208099 32169 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.055s	user 0.000s	sys 0.002s
I20260709 12:26:18.208362 32023 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.208770 32023 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:18.211742 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 463914us
I20260709 12:26:18.212713 32005 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:18.213275 32005 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.201894 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.215437 32161 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.215534 31925 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:18.216068 31700 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:18.216621 31700 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:18.217590 31700 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.218304 20806 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:18.219558 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.220299 20802 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.220619 32158 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:18.221882 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.162604 32052 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:18.224808 32134 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:18.225219 32134 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:18.227384 21426 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:18.228526 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.230784 31965 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:18.231443 31965 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:18.235800 32159 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.284s	user 0.002s	sys 0.000s
I20260709 12:26:18.236939 31989 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:18.216070 32161 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.216063 32005 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.226104 32134 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.244166 32134 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:18.248870 20729 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:26:18.250095 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.250765 20729 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:18.254770 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 4154) took 1051 ms. Trace:
I20260709 12:26:18.255172 20730 rpcz_store.cc:276] 0709 12:26:17.203763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.210295 (+  6532us) service_pool.cc:224] Handling call
0709 12:26:18.254492 (+1044197us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400249fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:18.254713 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441945856}
I20260709 12:26:18.242892 32161 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.271370 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:18.271764 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.272135 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70318.8us
I20260709 12:26:18.273644 20805 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.258769 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:18.275081 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.276506 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.276757 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.255815 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.256510 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.291934 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.294827 20731 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:18.294981 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:18.295053 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.294998 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.295495 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.295866 21426 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:18.294504 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.296736 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.297142 20729 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:18.297370 20733 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:18.297514 21427 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:18.296976 20810 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.278537 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.300755 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:18.302587 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:18.305897 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.306298 20732 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:18.303809 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.304831 20726 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:18.300376 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:18.304996 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.307576 20728 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:18.305627 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.307937 21430 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:18.308347 20734 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:18.305111 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.256964 31871 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:18.310801 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:18.311702 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.312965 20733 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:18.318758 20878 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:18.319370 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.321172 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:18.321270 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.321776 20883 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:18.322345 20876 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:18.277513 31692 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.328776 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.329310 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.329479 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.329504 20803 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:18.330474 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.331116 20729 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:18.323046 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:18.323402 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.323712 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.324433 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:18.277361 32146 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:18.320699 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:18.359553 21429 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:18.322944 21432 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 20. (term mismatch)
I20260709 12:26:18.333289 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:18.360548 21432 pending_rounds.cc:85] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 19
I20260709 12:26:18.360863 21432 pending_rounds.cc:107] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 27.20
I20260709 12:26:18.361179 21432 raft_consensus.cc:2889] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:18.334605 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.332383 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.328686 32136 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:18.318373 31871 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:18.328565 31887 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
I20260709 12:26:18.337376 20732 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:18.328774 32161 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.015s
I20260709 12:26:18.337872 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.390429 20728 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:18.390857 32116 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:18.391412 32116 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:18.391562 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.392455 32116 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:18.395262 32052 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.396534 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.397233 20732 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:18.398136 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.398725 20732 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:18.339005 31692 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.367795 32136 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:18.399595 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.400038 32136 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:18.400847 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.401991 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.402194 32126 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.402722 20732 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:26:18.403479 21427 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:18.383795 32146 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.404069 20732 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 19. Preceding OpId from leader: term: 37 index: 20. (index mismatch)
I20260709 12:26:18.404500 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.344643 20726 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 16. Preceding OpId from leader: term: 34 index: 17. (index mismatch)
I20260709 12:26:18.343873 20807 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:18.347319 20883 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:18.353160 20731 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:18.347811 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.356156 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.406730 20732 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:26:18.407256 20733 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:18.356648 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:18.332443 20730 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:18.363312 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.408720 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.409380 20808 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:18.310783 31996 raft_consensus.cc:2764] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:26:18.304520 32142 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election won for term 37
W20260709 12:26:18.383886 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50663.2us
I20260709 12:26:18.221186 32158 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.338618 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.392848 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.411084 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.387149 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.336802 20884 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:18.407310 20731 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:18.391479 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.304664 32038 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.080s
I20260709 12:26:18.298324 32081 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.086s
I20260709 12:26:18.407616 21430 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:18.412850 20880 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:18.411428 20803 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:18.400924 31692 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:18.412223 32137 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.007s
I20260709 12:26:18.367398 31965 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.419513 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.420375 20728 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:18.422055 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:18.425467 20877 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:18.428551 20734 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:18.429471 20726 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:18.432096 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:18.433964 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.436554 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.440516 20808 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:18.440708 20806 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.416906 31462 raft_consensus.cc:2764] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election decision vote started in defunct term 36: won
I20260709 12:26:18.471019 31692 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:18.471670 31692 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:18.472757 31692 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.478081 31692 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:18.479525 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.480436 20876 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:18.484652 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:18.488706 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54676.4us
I20260709 12:26:18.430393 31887 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:18.491639 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.431571 31965 raft_consensus.cc:2764] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:26:18.497678 20882 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:18.431933 32146 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.495966 21429 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:18.390726 31871 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.427156 32134 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.070s
I20260709 12:26:18.432497 32074 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.071s
I20260709 12:26:18.509878 31989 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:18.517704 21429 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:18.550290 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.558836 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:18.559707 20884 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:18.564527 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:18.580166 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.580878 21429 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:18.537465 32052 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.587248 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.587483 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:18.565112 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.541612 31989 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.509634 31887 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.545092 32130 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.604501 32042 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:18.606400 31887 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.599205 32176 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:18.601971 32052 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.558673 31462 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:18.600970 31989 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:18.616035 32174 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:26:18.616391 32052 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:18.625772 20730 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:18.627686 20733 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:18.631310 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:18.627578 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 269621us
I20260709 12:26:18.628763 20877 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:18.638705 20809 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:18.645334 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 130592us
I20260709 12:26:18.633950 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:18.650034 32142 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:18.661737 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:18.670543 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:18.672159 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:18.665661 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 246010us
I20260709 12:26:18.677022 32081 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:18.676551 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:18.654412 32126 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.684521 32042 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.697516 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 286488us
I20260709 12:26:18.698992 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:18.699921 32117 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:18.700289 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:18.701360 32117 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:18.702051 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:18.702852 32117 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:18.674365 32052 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:18.684837 31462 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:26:18.713574 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.726153 32179 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.666927 32130 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.030s
I20260709 12:26:18.688210 32081 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:18.693287 31893 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.708630 32158 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.744880 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.745524 20727 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:18.707463 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "626209a224524080801f3a9ad7307fbe"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:18.747269 20730 raft_consensus.cc:606] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.743876 31893 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:18.748754 32130 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:18.749476 32130 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:18.749564 32176 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:18.750001 32130 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:18.750418 31893 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:18.751004 31893 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:18.751688 31893 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:18.751364 32142 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.752714 31893 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 28.23, Last appended by leader: 23, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.728669 31462 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.755627 31462 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:18.733879 32179 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:18.758301 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.758751 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.796831 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.792387 20727 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:18.787977 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.759557 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.816440 20732 raft_consensus.cc:2393] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:18.777213 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:18.818534 20726 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:18.823213 20884 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:18.826961 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:18.818506 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:18.740001 32180 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:18.842121 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:18.760672 32178 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:26:18.783808 32175 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:26:18.854635 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 151470us
I20260709 12:26:18.709754 32042 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.855492 20728 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:26:18.859805 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 294738us
I20260709 12:26:18.760947 31700 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.759119 32179 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:18.841867 32038 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:18.863590 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:18.885965 20807 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:18.869478 32052 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.017s
I20260709 12:26:18.874133 31910 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:18.887506 31910 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.887980 31910 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:18.893509 32023 consensus_meta.cc:287] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.001s
I20260709 12:26:18.894045 32023 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.856166 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:18.885740 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:18.895793 20734 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:18.755625 32081 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:18.855825 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:18.901412 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45641.1us
I20260709 12:26:18.904292 21428 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 35: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:18.778193 31462 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:18.869153 32180 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:18.855530 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:18.860610 31925 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.040s
I20260709 12:26:18.925428 31996 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.902812 20701 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.862030 31700 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.905576 32038 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:18.931663 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76218.8us
I20260709 12:26:18.932489 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:18.943399 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 309500us
I20260709 12:26:18.946017 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:18.930516 32185 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:18.950690 31886 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:18.951143 32180 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.20, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:18.961876 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 374440us
I20260709 12:26:18.940930 32130 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.927227 31989 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:18.968348 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:18.974622 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:18.979384 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46975.7us
I20260709 12:26:18.980227 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:18.978919 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1857) took 1364 ms. Trace:
I20260709 12:26:18.929630 31996 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.987560 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:18.992820 20881 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:26:18.967964 31886 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:18.993805 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:18.983150 20735 rpcz_store.cc:276] 0709 12:26:17.614993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.615257 (+   264us) service_pool.cc:224] Handling call
0709 12:26:17.615581 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.623021 (+  7440us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:18.092368 (+469347us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b1800474d00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb25e5 00007f9e1ecb2568
0709 12:26:18.137977 (+ 45609us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.450927 (+312950us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.520600 (+ 69673us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.520803 (+   203us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:18.539179 (+ 18376us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.539900 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.540162 (+   262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.973164 (+433002us) raft_consensus.cc:1685] finished
0709 12:26:18.973201 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.978839 (+  5638us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5294,"prepare.queue_time_us":488330,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":10415,"raft.queue_time_us":3285,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":2829,"spinlock_wait_cycles":2723584,"thread_start_us":334616,"threads_started":2,"wal-append.queue_time_us":450246}
I20260709 12:26:18.996793 32197 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.997022 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:18.997349 32197 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.000495 31989 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.000849 32189 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:19.000928 31462 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.001482 32189 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:19.003211 32042 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.003877 32042 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:19.004441 32042 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.006734 32042 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:18.988327 32038 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:18.962988 32185 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.007589 32038 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.008097 32038 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:19.008718 32023 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.009338 32185 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.010257 32197 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:18.994056 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:18.997881 20804 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:19.012161 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:18.971621 32130 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:19.027024 21399 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:18.985198 32188 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:19.017910 20875 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
W20260709 12:26:18.990353 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 70145.6us
I20260709 12:26:19.019727 32142 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:19.002514 32189 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.23, Last appended by leader: 23, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.033367 32142 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.021040 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:18.979008 32116 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:19.027448 21399 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
W20260709 12:26:19.016356 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.039798 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:19.029698 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.002978 32204 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:19.001066 31989 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:19.059723 31989 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:18.989804 31925 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:19.062270 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.062582 31925 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.063277 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.063046 31925 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:19.064067 21428 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:19.064386 20876 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
I20260709 12:26:19.064723 20876 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 17
I20260709 12:26:19.064791 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.066646 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.035776 20879 raft_consensus.cc:606] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
W20260709 12:26:19.031432 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43953.7us
I20260709 12:26:19.068079 20883 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:19.051983 32052 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:19.049517 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:19.053972 32142 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.056592 31996 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.057330 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.058974 20884 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: [33.22-33.22]
I20260709 12:26:19.036015 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.065064 20876 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:26:19.032405 32197 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:19.076370 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.066148 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.078300 20877 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 28: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:26:19.077172 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 400698us
I20260709 12:26:19.042716 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:19.069197 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:19.077206 20876 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.079581 20880 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 37: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:19.077723 32197 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.081166 32197 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.081393 32218 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:26:19.082804 32195 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:26:19.083869 32200 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:19.085245 31700 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.085825 31700 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:19.086320 31700 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.078800 32081 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:19.088716 32081 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:19.089385 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.089615 32081 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.090045 20877 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:19.073405 32217 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:19.091457 31886 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.20, Last appended by leader: 20, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.091423 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:19.095842 20877 raft_consensus.cc:2393] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:19.048386 32158 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:19.072803 31692 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.000s
I20260709 12:26:19.098618 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.063433 31989 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:19.105098 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62449.9us
I20260709 12:26:19.037833 32130 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.118723 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:19.120114 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:19.123832 20876 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:19.126992 32042 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:19.127844 32042 consensus_queue.cc:1243] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 37.25 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 24
I20260709 12:26:19.128547 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.129146 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:19.132018 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.132439 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.133282 21431 raft_consensus.cc:2410] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:26:19.133968 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.134450 32130 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.112967 32214 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:19.136058 21435 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:19.069860 32202 raft_consensus.cc:2764] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 32: won
W20260709 12:26:19.137355 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.138001 32136 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.000s
I20260709 12:26:19.141942 32136 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.144101 32136 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.127255 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.149417 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:19.150318 20878 raft_consensus.cc:606] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:19.150800 20878 raft_consensus.cc:614] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:19.152590 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:19.122198 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 101871us
I20260709 12:26:19.182402 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.183730 20879 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:26:19.154663 21428 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
W20260709 12:26:19.155596 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57073.1us
I20260709 12:26:19.146389 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:19.171525 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44775.6us
I20260709 12:26:19.078536 32052 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:19.191699 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.195919 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:19.218211 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 185429us
I20260709 12:26:19.235285 32214 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:19.187914 32134 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:19.124414 32179 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:19.234778 20876 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:19.225175 32130 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.257261 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:19.245811 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50270.9us
I20260709 12:26:19.270021 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.144709 32038 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:19.132090 32183 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.188s	user 0.002s	sys 0.001s
I20260709 12:26:19.125203 31887 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.225031 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.249260 32042 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:19.257826 32182 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:19.257999 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62152us
I20260709 12:26:19.223989 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.286408 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.261636 21435 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 21. (term mismatch)
I20260709 12:26:19.189155 31965 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:19.235561 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 155995us
I20260709 12:26:19.272302 31871 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:19.235318 31973 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.037s
W20260709 12:26:19.272531 32208 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:26:19.272665 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.236341 31989 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:19.258903 32214 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.287292 21435 pending_rounds.cc:85] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 20
I20260709 12:26:19.297137 21435 pending_rounds.cc:107] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 28.21
I20260709 12:26:19.297421 21435 raft_consensus.cc:2889] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.288529 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.264848 32136 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:19.276049 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:19.292727 21433 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:26:19.251041 32217 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:26:19.272920 32179 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:19.331619 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.276971 31996 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:19.332451 31996 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:19.332221 32179 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.333402 31996 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.335906 32197 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.338198 32197 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:19.338488 32202 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:19.338915 32202 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:19.339293 32202 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:19.340294 31462 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:19.340682 31462 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:19.341435 31462 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.342772 31462 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:19.283522 31893 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:19.301242 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.343812 31893 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:19.344179 32117 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:19.344516 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:19.344679 31893 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.345754 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:19.283771 32204 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 12:26:19.347373 32213 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:19.348043 32213 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:19.348337 32206 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:19.304975 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.290980 32182 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.352393 32182 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.276916 32130 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.306792 32126 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:19.354701 20875 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:26:19.326215 32116 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:19.344696 32117 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:19.327077 31973 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
W20260709 12:26:19.358129 32207 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:26:19.358201 32117 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.359762 32117 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.299693 31989 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.364130 32116 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:19.294723 31965 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.272606 31910 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.368294 31965 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.290257 32042 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.369455 31910 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.370841 32042 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.355732 32130 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.364800 31989 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.299518 32214 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:19.383395 31700 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:19.384025 32221 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:26:19.388206 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.347174 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:19.390820 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:19.398825 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.399696 20875 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:19.400671 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:19.403172 21428 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:19.396190 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:19.397890 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50762.4us
I20260709 12:26:19.408198 20733 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:19.301440 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:19.419734 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 118350us
I20260709 12:26:19.415444 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.415879 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:19.424649 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:19.305102 32196 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:19.420295 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:19.284986 31887 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.437891 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.438627 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.439015 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:19.440832 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 249184us
I20260709 12:26:19.407480 31989 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:19.411736 32130 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:19.456825 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46401ecb9b443118647ee361081a01e"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:19.457479 20878 raft_consensus.cc:606] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.457890 20878 raft_consensus.cc:614] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:19.452461 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:19.448550 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.436738 32185 raft_consensus.cc:2764] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
I20260709 12:26:19.408061 32218 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.048s
I20260709 12:26:19.437788 31910 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:19.473922 20700 leader_election.cc:400] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 27. Current term is 28.
I20260709 12:26:19.473603 21429 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:19.435585 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.443357 32206 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:19.414180 32042 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:26:19.452641 32197 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:19.414919 32212 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 12:26:19.456892 32130 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.480394 32199 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.471573 31692 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:19.507561 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:19.494814 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 59354.9us
I20260709 12:26:19.443219 31925 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.396711 32052 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.454535 32204 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.043s
W20260709 12:26:19.532684 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 247698us
I20260709 12:26:19.480278 32189 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.059s
I20260709 12:26:19.471035 32124 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 12:26:19.490778 32220 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:19.492018 32206 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.530478 32197 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.529651 32023 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.529434 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 243048us
I20260709 12:26:19.530737 32130 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:19.495947 31700 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:19.560253 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.564777 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:19.588778 20875 raft_consensus.cc:606] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:19.589214 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 81733.9us
I20260709 12:26:19.589342 20875 raft_consensus.cc:614] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:19.590670 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.491248 32142 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.082s
I20260709 12:26:19.564164 32206 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:19.558317 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.509805 32214 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.572868 32197 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:19.563746 31692 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:19.580928 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106812us
I20260709 12:26:19.616351 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.617482 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:19.580610 32038 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.586594 32225 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:19.588039 32081 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:26:19.634086 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75891.6us
I20260709 12:26:19.581408 32218 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.046s
I20260709 12:26:19.614914 32220 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:19.641635 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:19.635349 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:19.646731 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 194314us
W20260709 12:26:19.647398 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73696.2us
W20260709 12:26:19.637072 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:19.614704 31989 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:19.646618 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.649513 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:19.641247 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.629621 32116 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.585798 32023 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.627300 31910 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:19.639384 31973 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:19.695441 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48903.9us
I20260709 12:26:19.642431 31700 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:19.631179 32202 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.698230 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:19.644802 32188 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:26:19.699330 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50151.3us
I20260709 12:26:19.652841 32081 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:19.700824 32226 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:19.703998 32238 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:19.802063 32238 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:19.652293 32227 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:19.793244 20728 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:19.725147 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75707.5us
I20260709 12:26:19.724201 31887 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.725358 32182 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.707774 32224 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:26:19.802846 32081 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.706805 32214 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.700515 31692 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.721278 32244 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:19.700351 32142 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.138s
I20260709 12:26:19.701043 31989 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.823448 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:19.904625 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81532us
I20260709 12:26:19.840981 32225 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:19.807627 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:19.909758 20733 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:19.721792 32023 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.875290 32245 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.725596 32228 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:19.850337 31887 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.701550 32222 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:26:19.791461 32116 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.842154 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.695214 32215 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.379s
I20260709 12:26:19.858268 20877 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:19.854090 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:19.851475 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:19.715888 32158 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.003s
I20260709 12:26:19.832830 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:19.850383 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 209288us
I20260709 12:26:19.851044 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:19.930503 20729 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:19.851830 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:19.852048 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:19.931540 20727 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:19.863793 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:19.858603 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:19.906188 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:19.701237 32005 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:26:19.854357 32235 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:26:19.948724 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:19.949072 31996 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:19.949425 20728 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:19.949753 31996 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.837554 32182 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.907445 31973 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:19.953450 32182 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.908795 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:19.954663 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45992.9us
I20260709 12:26:19.956238 31886 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:19.732218 32223 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.245s	user 0.006s	sys 0.000s
I20260709 12:26:19.795863 32188 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.871845 32240 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.700945 32232 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:19.877774 32243 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:19.883559 32213 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.869882 32239 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.919216 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:19.922760 20731 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:19.883991 32244 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:19.923897 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:19.925073 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83154.4us
I20260709 12:26:19.931597 20726 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:19.932264 20879 raft_consensus.cc:606] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:19.654116 32126 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.187s
I20260709 12:26:19.932639 20881 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:19.975667 20729 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:19.829121 32179 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.084s
I20260709 12:26:19.983911 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:19.991016 20879 raft_consensus.cc:614] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:19.883359 32117 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.936429 31462 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.007s
I20260709 12:26:19.938786 31925 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:19.940390 32052 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 37.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:20.001091 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:19.941593 20877 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:26:19.923166 32245 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:19.941876 32204 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:20.016356 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.017159 20810 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:19.942036 32161 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:19.942211 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:19.960644 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 127926us
I20260709 12:26:19.866807 32130 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.971962 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:19.968827 32188 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.935722 32215 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:19.976990 21435 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 23. Preceding OpId from leader: term: 30 index: 24. (index mismatch)
I20260709 12:26:19.991671 20732 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:19.915262 32228 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:19.998696 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 79591.2us
I20260709 12:26:19.967808 31996 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.015651 32196 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:19.986683 32247 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.020s
I20260709 12:26:19.998684 32126 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:26:20.008232 20806 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.20-34.20]
I20260709 12:26:20.007823 32244 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.988467 32239 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.011638 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:20.012130 32005 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:19.989585 32023 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.012895 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:19.982972 32213 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.013717 32245 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:19.988163 32243 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:20.139509 32199 consensus_meta.cc:287] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.007s
I20260709 12:26:20.023375 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:19.968501 32142 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:19.966547 32158 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31.
I20260709 12:26:19.977504 20728 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:20.026297 20803 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:20.026230 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:20.019562 20881 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:20.053295 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:20.011711 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:19.849079 32238 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.982733 31886 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:19.861997 31910 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:19.921476 31887 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.113785 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.249369 20810 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:26:20.249552 31887 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.115438 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.252800 20808 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:20.117188 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.015791 32204 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:20.122084 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.255340 20801 raft_consensus.cc:2410] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:26:20.123361 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:20.134042 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:20.057068 32190 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.008471 32180 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:20.151520 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:20.057462 32038 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.057818 32215 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:20.272406 20803 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:20.163305 20731 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:26:20.008162 32248 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.131s
I20260709 12:26:20.112640 32228 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:20.022953 32117 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.167860 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:19.990149 32240 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.026398 32214 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.121s
W20260709 12:26:20.154592 32206 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.002s
I20260709 12:26:20.122812 32196 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:20.171515 32005 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.269331 32190 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.303074 32204 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.318252 32219 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.340237 32219 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.262558 32202 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:20.303632 20802 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:20.312363 32116 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.262281 32180 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:20.316506 32255 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:26:20.342496 32219 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.286732 32038 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:20.229112 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 205808us
W20260709 12:26:20.233711 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 208272us
I20260709 12:26:20.344530 32206 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.346024 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.346661 20803 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:20.346760 32206 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.238312 20727 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
W20260709 12:26:20.239609 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87198.2us
I20260709 12:26:20.231233 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:20.257050 20881 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:26:20.258277 20806 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:20.258999 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 135751us
I20260709 12:26:20.152621 32176 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.092s
I20260709 12:26:20.268950 21431 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:26:20.277099 20728 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:20.279553 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.281780 21435 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:26:20.232753 32199 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:20.182447 32023 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.319439 32242 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:20.237215 20877 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.20-34.20]
I20260709 12:26:20.440882 32202 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:20.442404 31893 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:20.443485 32179 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:26:20.313117 32246 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 12:26:20.441948 31462 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:20.448633 31462 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:20.328097 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.348074 21427 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:20.344745 32196 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:20.349155 32158 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:20.303464 32227 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:20.327641 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.202826 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:20.424079 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:20.320796 32253 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:26:20.352630 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:20.331691 32215 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.427938 32213 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.340950 32240 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.342550 32180 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:20.481763 32158 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:20.482288 32158 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:20.484643 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:20.485219 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132664us
I20260709 12:26:20.486115 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:20.486732 32126 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:20.487160 32126 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.350696 32038 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.432132 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.488082 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:20.488240 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:20.487644 32264 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.489243 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.489187 32264 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.439380 20882 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:20.491369 31887 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:20.492421 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.491355 32264 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:20.493759 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.494402 20882 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:20.495709 20882 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 16
I20260709 12:26:20.441045 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.496475 32227 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:20.311590 32212 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:20.497715 32212 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.500658 32212 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.367723 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.243752 32244 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:20.450227 20879 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
W20260709 12:26:20.312506 32252 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.292s	user 0.001s	sys 0.001s
W20260709 12:26:20.425031 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 193886us
W20260709 12:26:20.505405 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 137801us
W20260709 12:26:20.318082 32251 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
I20260709 12:26:20.450601 31973 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.507679 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.508107 31973 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.510330 31973 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.449501 32142 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:20.449501 31462 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 37.21, Last appended by leader: 21, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.462014 20805 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:20.515347 20882 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 24.17
W20260709 12:26:20.323987 32249 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:26:20.333755 32256 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 12:26:20.441149 32250 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 12:26:20.319038 32197 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.685s	user 0.006s	sys 0.003s
W20260709 12:26:20.282778 32200 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:20.307767 31925 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.532408 31925 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.477589 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.350725 32243 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.428501 32214 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:20.534648 32214 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:20.479772 32206 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:20.535043 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.535281 31925 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.535445 32206 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:20.535912 32206 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:20.535579 32214 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.351074 32204 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:20.512835 32190 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.487082 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:20.540618 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53599.1us
I20260709 12:26:20.308187 32270 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:20.495792 21430 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:26:20.494508 32180 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.496368 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:20.316984 32254 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.297s	user 0.000s	sys 0.000s
I20260709 12:26:20.512753 20883 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:20.436589 32219 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.513870 20879 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:20.465394 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:20.515739 31462 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:20.514114 32124 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:20.516109 20882 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:20.568346 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.439651 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.320890 32117 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.478220 31996 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.071s
I20260709 12:26:20.524914 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:20.531462 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:20.542281 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:20.487672 32126 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:20.517421 32248 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:26:20.551796 32081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.006s	sys 0.014s
W20260709 12:26:20.601686 32081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.006s	sys 0.014s
I20260709 12:26:20.437466 32023 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.552250 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.514302 32038 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:20.515317 32142 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:20.570885 20881 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
W20260709 12:26:20.571362 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106064us
I20260709 12:26:20.573702 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.607199 20883 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:20.574565 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.607257 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:20.522779 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:20.530542 32158 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:20.600178 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 57980.4us
I20260709 12:26:20.613134 31462 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:20.611935 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:20.599747 32250 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:20.604441 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4129) took 1048 ms. Trace:
I20260709 12:26:20.546005 31886 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:20.608204 21435 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:20.610991 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.600380 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.623642 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:20.624506 31462 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:20.620244 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.625339 20804 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:20.626535 20883 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:20.600046 32212 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:20.629478 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:20.672582 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 44893.6us
I20260709 12:26:20.673062 20804 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
W20260709 12:26:20.674326 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50748.4us
I20260709 12:26:20.543567 32197 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.617661 32117 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:20.634341 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.623072 20878 rpcz_store.cc:276] 0709 12:26:19.555918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.556083 (+   165us) service_pool.cc:224] Handling call
0709 12:26:19.556435 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.603093 (+ 46658us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:19.603143 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.713416 (+110273us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b2c0005c6f8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:19.713823 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.887926 (+174103us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:19.887995 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.888093 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.320154 (+432061us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:20.604238 (+284084us) raft_consensus.cc:1685] finished
0709 12:26:20.604258 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.604372 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49340,"raft.queue_time_us":75561,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":8855,"spinlock_wait_cycles":64775296,"thread_start_us":219897,"threads_started":2,"wal-append.queue_time_us":309033}
W20260709 12:26:20.646132 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.602665 31973 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:20.658449 32130 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.659260 32258 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:20.664628 32228 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.615801 32038 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.553166 32190 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:26:20.567221 32185 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.058s
I20260709 12:26:20.605583 31996 consensus_queue.cc:1243] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 27.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:26:20.675151 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:20.625962 31886 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.572572 32219 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.613353 31893 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:20.759769 20880 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:20.756932 20807 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 32 because replica has last-logged OpId of term: 30 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:26:20.672588 32212 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:20.733525 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54377.9us
I20260709 12:26:20.820740 21432 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:20.736003 32188 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.000s
I20260709 12:26:20.571983 32270 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:20.783856 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 292742us
I20260709 12:26:20.827781 21399 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:20.679746 32124 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.093s
W20260709 12:26:20.832842 32267 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:20.755208 32117 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:20.818365 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 143283us
I20260709 12:26:20.832912 32206 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.833695 32117 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:20.834229 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.834749 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.839735 32270 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.817502 31886 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:20.843976 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:20.844611 32042 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.845083 32042 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.845659 21399 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:20.847101 32273 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:20.847875 32258 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:20.847797 31692 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:20.767720 32081 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:20.848773 32258 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:20.848991 32081 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.849195 32258 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.849431 32081 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:20.849596 32258 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:20.737715 32261 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.077s
I20260709 12:26:20.847401 32136 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:20.851804 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.852815 32038 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.777252 31887 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:20.854038 31887 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.23, Last appended by leader: 23, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.856640 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.857602 32206 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.863265 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:20.864523 32243 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:20.866791 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:20.867921 32081 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.868459 32197 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:20.870236 32081 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.870894 20877 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:20.871517 32228 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:20.822124 32237 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:20.872630 32237 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:20.872641 32180 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.822422 32212 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:20.876655 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:20.876932 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:20.873517 32237 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:20.883036 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 251963us
I20260709 12:26:20.885156 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:20.889780 20875 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:20.890545 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.886107 21435 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:20.892453 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:20.896801 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:20.897472 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.897634 21435 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:20.897099 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:20.912827 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 360613us
I20260709 12:26:20.893867 32190 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:20.829026 32188 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.903520 31700 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:20.902945 32261 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:20.957628 20729 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: [28.19-28.19]
I20260709 12:26:20.943305 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.958830 20804 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:20.959092 32188 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:20.946148 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.960146 20803 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:20.948679 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.961006 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.947485 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.952117 21435 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:20.962002 20805 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:20.962036 32185 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:20.964133 31910 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:20.893081 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:20.953641 20809 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.001s	sys 0.024s
I20260709 12:26:20.914640 32038 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:20.938859 32206 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.975701 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:20.895579 32196 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:20.895879 32270 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.045s
I20260709 12:26:20.939335 32081 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.944689 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:20.992324 20806 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:20.939071 32218 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:20.981395 32038 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.952828 32240 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:20.984931 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3851) took 1131 ms. Trace:
I20260709 12:26:20.986982 32285 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:26:20.987682 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 94713.8us
I20260709 12:26:20.988865 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3852) took 1106 ms. Trace:
I20260709 12:26:20.990826 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:20.992882 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:20.996325 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:21.005441 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:21.005806 21433 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
W20260709 12:26:21.004437 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.008539 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.009002 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:21.009161 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:21.010124 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:21.010462 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:21.011054 20731 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:21.011013 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.011912 20801 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:21.012025 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:21.012835 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.001227 20732 rpcz_store.cc:276] 0709 12:26:19.882279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.119250 (+236971us) service_pool.cc:224] Handling call
0709 12:26:20.150881 (+ 31631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.461363 (+310482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.461409 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.481248 (+ 19839us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.760200 (+278952us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:20.988295 (+228095us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.988563 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.988633 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.988672 (+    39us) raft_consensus.cc:1685] finished
0709 12:26:20.988691 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.988815 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37240,"raft.queue_time_us":117938,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":24207,"thread_start_us":757081,"threads_started":3,"wal-append.queue_time_us":388458}
I20260709 12:26:21.009585 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:20.998624 20733 rpcz_store.cc:276] 0709 12:26:19.853269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.944051 (+ 90782us) service_pool.cc:224] Handling call
0709 12:26:19.944365 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.260343 (+315978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.260485 (+   142us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.401702 (+141217us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.760379 (+358677us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:20.760440 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.760547 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.984711 (+224164us) raft_consensus.cc:1685] finished
0709 12:26:20.984732 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.984869 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82868,"raft.queue_time_us":176056,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":21915,"thread_start_us":511762,"threads_started":2,"wal-append.queue_time_us":523693}
I20260709 12:26:20.990662 32243 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:21.018251 20731 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.17-31.17]
I20260709 12:26:21.019624 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:21.020807 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.020651 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:20.994161 32188 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:21.022008 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:21.023343 20801 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:26:21.024869 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:21.000399 32285 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.032670 32038 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.034233 20728 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:26:21.034762 32284 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.045s
I20260709 12:26:21.040145 20735 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: [32.19-32.19]
I20260709 12:26:21.041510 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:21.042060 32042 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.042680 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:21.047609 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:21.048750 20801 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:21.048313 20733 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:21.050688 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:21.053155 31925 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:21.053498 20735 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:26:21.053578 31925 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:21.024392 20849 connection.cc:256] RPC call timeout handler was delayed by -2.1041981199996371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:26:21.057255 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:21.058336 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.058406 32284 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:21.059078 32284 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:21.059515 32284 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:21.014144 32268 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:21.060282 32268 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.060635 32268 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:21.061825 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2145) took 1655 ms. Trace:
I20260709 12:26:21.054653 31925 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.065881 32238 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:21.017225 32199 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:21.068511 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:21.069626 31925 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:21.000108 32218 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:20.993793 32206 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.073738 20807 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 21. (term mismatch)
I20260709 12:26:21.074173 20807 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 20
I20260709 12:26:21.074443 20807 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 30.21
I20260709 12:26:21.074717 20807 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.074908 20727 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:21.075992 20809 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 36 index: 25. (index mismatch)
I20260709 12:26:21.078908 20801 raft_consensus.cc:1240] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:21.082700 20809 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:21.062220 21428 rpcz_store.cc:276] 0709 12:26:19.406557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.406732 (+   175us) service_pool.cc:224] Handling call
0709 12:26:19.590579 (+183847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.629051 (+ 38472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.629089 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.851646 (+222557us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b1800086680. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a
0709 12:26:19.852591 (+   945us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:19.853174 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.024925 (+171751us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:20.024976 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.025065 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.316314 (+291249us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:20.629946 (+313632us) raft_consensus.cc:1685] finished
0709 12:26:20.629967 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.061665 (+431698us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54005df7d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:21.061769 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81480,"raft.queue_time_us":40508,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10974,"spinlock_wait_cycles":440787968,"thread_start_us":171292,"threads_started":1,"wal-append.queue_time_us":166207}
I20260709 12:26:21.080250 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:21.087585 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:20.998121 32126 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.084249 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:21.087750 20804 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:26:21.092633 20809 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38. Ops: [37.20-37.20]
I20260709 12:26:21.089427 20801 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.17-31.17]
I20260709 12:26:21.023427 32188 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.999946 32081 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.119239 20806 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 23. Preceding OpId from leader: term: 30 index: 24. (index mismatch)
I20260709 12:26:21.119362 32081 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.120870 32188 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:21.121943 32284 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.105381 32206 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.127887 20882 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:21.129165 32284 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.130326 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:21.130569 32142 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.131729 32142 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:21.132915 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47615.5us
I20260709 12:26:21.078274 32285 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:21.084584 32258 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.095407 32218 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.115693 32196 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.085306 32259 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:26:21.125882 32126 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.126536 21434 raft_consensus.cc:2368] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:26:21.126374 20805 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:21.138352 32005 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:21.110406 31886 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:21.192245 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.194626 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4073) took 1309 ms. Trace:
I20260709 12:26:21.195122 20880 rpcz_store.cc:276] 0709 12:26:19.885701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.885843 (+   142us) service_pool.cc:224] Handling call
0709 12:26:19.913767 (+ 27924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.817990 (+904223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.194253 (+376263us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b540019bed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:21.194536 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1935317376}
I20260709 12:26:21.108954 32212 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.148504 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:21.158752 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:21.198427 20877 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:21.091534 32244 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:21.156566 32241 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.145s
W20260709 12:26:21.345888 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.324702 20804 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:21.294471 32202 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:21.387440 20805 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: [32.23-32.23]
I20260709 12:26:21.294747 32218 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:21.293354 32284 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:21.372937 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 208066us
I20260709 12:26:21.304549 32220 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.442668 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:21.446404 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:21.451967 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4071) took 1595 ms. Trace:
I20260709 12:26:21.452477 20875 rpcz_store.cc:276] 0709 12:26:19.856639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.856774 (+   135us) service_pool.cc:224] Handling call
0709 12:26:19.857110 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.890395 (+1033285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.451608 (+561213us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:21.451880 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2373902592}
I20260709 12:26:21.455441 20804 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: [32.19-32.19]
I20260709 12:26:21.461897 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:21.463708 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.442670 20877 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:21.468658 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2141) took 2212 ms. Trace:
I20260709 12:26:21.468124 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 3971) took 1322 ms. Trace:
I20260709 12:26:21.469142 21431 rpcz_store.cc:276] 0709 12:26:19.256342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.256517 (+   175us) service_pool.cc:224] Handling call
0709 12:26:19.258487 (+  1970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.423324 (+1164837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.454487 (+1031163us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:21.468561 (+ 14074us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2293702528}
I20260709 12:26:21.390231 32202 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:21.471397 20802 rpcz_store.cc:276] 0709 12:26:20.121945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.122073 (+   128us) service_pool.cc:224] Handling call
0709 12:26:20.144091 (+ 22018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.342047 (+197956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.443792 (+1101745us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540006ccd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:21.444300 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332323200}
I20260709 12:26:21.472725 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:21.427618 31925 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.481449 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:21.483258 20809 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 35
W20260709 12:26:21.469656 32237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.000s	sys 0.015s
I20260709 12:26:21.388799 32244 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.087s
I20260709 12:26:21.469611 32197 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:21.468327 32185 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:26:21.488910 32237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.000s	sys 0.017s
I20260709 12:26:21.477120 32218 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.508677 32241 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:21.486999 31925 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:21.510540 32293 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:26:21.524850 32258 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.491767 32264 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:21.494728 32197 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:21.540995 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78754.1us
I20260709 12:26:21.542423 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:21.550204 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:21.552624 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:21.524894 31925 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.561228 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:21.544726 32197 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:21.470577 32220 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.465890 32284 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.556102 32212 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.567768 32281 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:21.570932 32259 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
W20260709 12:26:21.586819 32274 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.755s	user 0.000s	sys 0.012s
I20260709 12:26:21.587445 32305 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:21.587791 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1875) took 1623 ms. Trace:
W20260709 12:26:21.596573 32238 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.000s
I20260709 12:26:21.604451 32218 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.576736 31925 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:21.526957 32202 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.17, Last appended by leader: 17, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.614916 32215 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:21.629000 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
W20260709 12:26:21.611732 32117 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.777s	user 0.012s	sys 0.001s
I20260709 12:26:21.630198 20729 rpcz_store.cc:276] 0709 12:26:19.964798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.227177 (+262379us) service_pool.cc:224] Handling call
0709 12:26:20.245686 (+ 18509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.958010 (+712324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.587505 (+629495us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b540028d054. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:21.587725 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636707200}
I20260709 12:26:21.580055 31886 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.615116 32185 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:21.602918 32188 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.652580 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2107) took 1675 ms. Trace:
I20260709 12:26:21.653213 21430 rpcz_store.cc:276] 0709 12:26:19.976813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.976952 (+   139us) service_pool.cc:224] Handling call
0709 12:26:19.978016 (+  1064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.544781 (+566765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.652233 (+1107452us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b540039d854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:21.652495 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189774592}
I20260709 12:26:21.621326 32259 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:21.621888 32142 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:21.630149 32305 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.664460 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.636530 32281 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:21.671407 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 209576us
W20260709 12:26:21.671672 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.641124 32117 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.639930 31925 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.664353 32259 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:21.744799 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:21.674191 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 201542us
I20260709 12:26:21.750242 20802 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:21.752753 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:21.675169 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:21.665690 32262 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.025s	sys 0.054s
I20260709 12:26:21.674880 20731 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:21.679793 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.755774 32281 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.759341 32284 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:21.759997 32220 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.762133 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:21.762888 20807 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:21.762948 20804 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:21.765656 20803 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:21.665778 32219 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:21.722491 32268 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.661s	user 0.004s	sys 0.007s
I20260709 12:26:21.773265 20726 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:21.666262 32269 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.102s
I20260709 12:26:21.774101 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:21.791930 20877 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:21.783097 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.668253 32305 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:21.765470 32262 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.025s	sys 0.055s
I20260709 12:26:21.670858 32202 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:21.792373 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 231240us
I20260709 12:26:21.801708 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.802662 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:21.772359 32258 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:26:21.809960 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.807770 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.773260 32199 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:21.720631 32197 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.814251 20877 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:21.818135 32158 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:21.819710 32302 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:26:21.820887 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.827505 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.829641 32117 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.835604 21426 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 35 because replica has last-logged OpId of term: 34 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
I20260709 12:26:21.808054 32225 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:21.791581 32284 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.050s
I20260709 12:26:21.841311 32225 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:21.842249 32225 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.843314 32225 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:21.845454 31996 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:21.845872 31996 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:21.794865 32268 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.849215 32268 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.846642 31996 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.849884 20728 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:21.850325 31996 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:21.811151 32238 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.041s
I20260709 12:26:21.853725 32188 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.854838 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.855460 21428 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:21.856252 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:21.856629 21432 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:21.856912 21428 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:21.856967 32285 consensus_meta.cc:287] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.000s
I20260709 12:26:21.857549 32285 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.813761 32220 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Committing config change with OpId 27.20: config changed from index 12 to 20, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:21.861274 32239 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.862059 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.863426 32005 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.863960 32005 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.866271 32005 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.862985 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2695) took 1375 ms. Trace:
I20260709 12:26:21.095047 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:21.867637 21434 rpcz_store.cc:276] 0709 12:26:20.487267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.487842 (+   575us) service_pool.cc:224] Handling call
0709 12:26:21.862826 (+1374984us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b5400750154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:21.862938 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466447744}
I20260709 12:26:21.870018 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.873492 20807 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:21.878331 20802 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:21.879673 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:21.882364 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:21.889798 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:21.890478 20875 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:21.894490 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:21.901489 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:21.903457 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2697) took 1415 ms. Trace:
I20260709 12:26:21.903687 21400 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:21.904533 20882 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:21.868788 32199 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.903896 21426 rpcz_store.cc:276] 0709 12:26:20.487847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.487952 (+   105us) service_pool.cc:224] Handling call
0709 12:26:21.897268 (+1409316us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540013d154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:21.903404 (+  6136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3096858240}
I20260709 12:26:21.912786 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:21.891496 32158 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.035s
I20260709 12:26:21.927450 32285 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:21.913641 21431 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:21.891867 32188 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.891989 32269 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:21.894654 32005 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:21.939685 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:21.948204 20875 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:21.922943 32136 raft_consensus.cc:2804] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:21.951911 32136 raft_consensus.cc:697] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:21.918256 32237 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:26:21.914196 32258 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:21.948172 32005 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:21.954424 32258 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.954984 32258 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:21.955163 32005 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:21.958626 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.960345 32185 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:21.962149 32185 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 36.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:21.962477 32268 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:21.942930 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:21.939823 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:21.918293 32241 consensus_queue.cc:1059] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.025s
I20260709 12:26:21.893754 32228 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.043s
W20260709 12:26:21.945799 20625 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 296) took 110031 ms (client timeout 15000 ms). Trace:
I20260709 12:26:21.919515 32130 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.028s
I20260709 12:26:21.914077 32206 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:21.953094 32136 consensus_queue.cc:237] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.941322 32197 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:21.874593 32244 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.051s
I20260709 12:26:21.965595 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:21.968118 20877 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:21.962944 32185 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:21.971650 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:21.987810 20809 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:21.973807 20875 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:21.911026 32284 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.059s
I20260709 12:26:21.963042 32268 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.987486 32185 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:21.985401 32206 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:22.001600 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 332996us
I20260709 12:26:21.938042 32188 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:21.984009 32238 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:22.024106 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.024372 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:22.024797 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:22.027287 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:22.027281 20807 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:22.028093 20805 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:22.027580 20804 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:22.032344 20809 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:21.964937 32269 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:21.975622 32243 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.026s
W20260709 12:26:22.036550 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71172.3us
I20260709 12:26:22.035008 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:22.036504 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.993098 32158 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:22.041512 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:22.048513 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.049202 20807 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:22.050060 20805 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:22.041477 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4182) took 1031 ms. Trace:
I20260709 12:26:22.054643 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:22.060716 20879 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:22.055577 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.025839 32285 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.061868 20880 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:22.068912 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:22.069265 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.065297 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.053531 20732 rpcz_store.cc:276] 0709 12:26:21.009737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.020043 (+ 10306us) service_pool.cc:224] Handling call
0709 12:26:21.043627 (+ 23584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.043846 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:21.452556 (+408710us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.039374 (+586818us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.041099 (+  1725us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:22.041163 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.041268 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.041411 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":430570,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":32152,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3960,"thread_start_us":994536,"threads_started":3,"wal-append.queue_time_us":527274}
I20260709 12:26:22.036144 32117 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.078286 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.081374 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.078555 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:22.083019 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:21.977383 20625 rpcz_store.cc:269] 0709 12:24:31.914486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.914746 (+   260us) service_pool.cc:224] Handling call
0709 12:26:21.945660 (+110030914us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":233184,"reported_tablets":49}
I20260709 12:26:22.041303 32238 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:22.057071 32306 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:22.043053 32197 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.086038 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4081) took 1575 ms. Trace:
I20260709 12:26:22.085908 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:22.087001 20877 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:22.026757 32228 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:26:22.063056 32158 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.062769 31925 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.104709 31887 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.091288 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.092448 32199 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.094863 32311 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:22.098794 20700 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:22.056898 32185 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 37.20, Last appended by leader: 20, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.098892 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39562 (request call id 23) took 1747 ms. Trace:
W20260709 12:26:22.099687 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 55526.9us
I20260709 12:26:22.115651 20804 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:22.094578 32258 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:22.162124 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 63404.4us
I20260709 12:26:22.167084 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:22.171378 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.173060 20805 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38. Ops: [37.25-37.25]
I20260709 12:26:22.123375 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:22.060520 32241 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:22.176173 20809 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37. Ops: [36.25-36.25]
I20260709 12:26:22.101766 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2114) took 1093 ms. Trace:
I20260709 12:26:22.066802 32215 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 LEADER]: Stepping down as leader of term 34
W20260709 12:26:22.107743 32308 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:22.133790 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:22.111451 32117 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.129314 20876 rpcz_store.cc:276] 0709 12:26:20.510276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.535912 (+ 25636us) service_pool.cc:224] Handling call
0709 12:26:20.536190 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.644539 (+1108349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.644589 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.644668 (+    79us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:21.644716 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.085801 (+441085us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540021cfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:22.085844 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.085986 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2815,"raft.queue_time_us":59645,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4461,"spinlock_wait_cycles":2541212928}
I20260709 12:26:22.139660 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.090241 32268 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.100270 32142 raft_consensus.cc:3055] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:22.100090 32023 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.102108 32206 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.103955 32269 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:22.180112 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.186555 32284 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.164078 20734 rpcz_store.cc:276] 0709 12:26:20.351849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.352750 (+   901us) service_pool.cc:224] Handling call
0709 12:26:22.098565 (+1745815us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400249fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca5735 00007f9e1ec9d9fa 00007f9e23c654c9 00007f9e1d1e1c54 00007f9e1d1e1be2 00007f9e1d1e1b11 00007f9e1d1e1a8c 00007f9e1d1e0d42 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:22.098794 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3717512448}
I20260709 12:26:22.185672 32244 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:22.080039 31989 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:22.211621 32269 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:22.211850 31989 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:22.212615 32269 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:22.212893 31989 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.213639 32269 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.214286 31989 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:22.194598 20807 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: [31.18-32.19]
I20260709 12:26:22.099998 32316 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.069s
I20260709 12:26:22.146162 32317 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.128688 32281 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.048s
W20260709 12:26:22.218472 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 302030us
I20260709 12:26:22.189215 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.199741 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:22.137233 32197 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.077713 32285 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.187444 21427 rpcz_store.cc:276] 0709 12:26:21.008634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.011480 (+  2846us) service_pool.cc:224] Handling call
0709 12:26:21.011755 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.016358 (+  4603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.016407 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.024631 (+  8224us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.593756 (+569125us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b1800361e40. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:21.875011 (+281255us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:21.875048 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.875126 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.101485 (+226359us) raft_consensus.cc:1685] finished
0709 12:26:22.101506 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.101705 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4171,"raft.queue_time_us":5724,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7453,"spinlock_wait_cycles":97830272,"thread_start_us":281058,"threads_started":1,"wal-append.queue_time_us":961039}
I20260709 12:26:22.187857 32185 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:22.078121 32305 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.185957 32215 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:22.161532 32311 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:22.177778 32310 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.215514 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.141013 32306 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:22.216521 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49571.8us
I20260709 12:26:22.188001 32312 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.188226 32202 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.177488 32199 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.021s
I20260709 12:26:22.198617 32268 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.190304 32321 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:22.192131 32142 raft_consensus.cc:740] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:22.159113 31925 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.164884 31887 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.229907 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:22.271303 20876 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:22.227890 32241 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:22.290637 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.272697 20884 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:22.274457 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:22.292013 20876 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:26:22.277988 32318 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:22.278592 32117 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.280046 32258 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:22.281917 32323 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:22.201686 32023 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.217343 32244 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.210578 32284 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.222990 32316 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:22.283299 32188 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.282708 32306 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.284472 32158 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.288504 20879 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:22.294848 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.296571 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.285633 32202 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:22.303520 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 73731.7us
I20260709 12:26:22.296558 20810 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Committing config change with OpId 27.20: config changed from index 12 to 20, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:22.216162 32238 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 37.21, Last appended by leader: 21, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:22.277022 20625 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 307) took 16832 ms (client timeout 15000 ms). Trace:
I20260709 12:26:22.306320 32320 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:22.312024 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.225427 32317 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.329679 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.308121 32326 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:22.269322 32206 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 12:26:22.291519 32321 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:22.332293 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.326126 32142 consensus_queue.cc:260] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 17, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.333580 32142 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:22.333313 32321 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.324294 32212 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:22.335551 32212 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:22.336369 32212 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.337283 32244 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.305112 32215 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.21, Last appended by leader: 21, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.338742 32284 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:22.318678 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.338701 32215 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:22.339788 20731 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:22.305892 32262 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.342197 32262 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.302105 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.324035 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.343916 20730 raft_consensus.cc:2410] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 29 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
I20260709 12:26:22.325883 20727 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:22.327178 20733 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:22.345770 32206 consensus_queue.cc:1243] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 28.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:26:22.317657 32316 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:22.347177 32320 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:22.347873 32316 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.348079 32320 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.349143 32316 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:22.301616 32241 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.352169 32285 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.352329 32317 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.276520 32312 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:22.339795 32325 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:22.354113 20882 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:22.354135 32023 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.360049 32197 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:22.288827 32268 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:22.361644 32259 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.000s
I20260709 12:26:22.362085 32259 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.362766 32268 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.329031 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.363118 20883 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:22.330987 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.331666 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.302137 31925 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.364497 32241 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.364563 31887 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:22.365593 20880 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:22.316188 32311 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.366382 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.367272 32311 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.366420 31925 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.369321 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:22.315550 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 398428us
I20260709 12:26:22.271726 32310 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.373104 20801 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:22.372519 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46802.6us
I20260709 12:26:22.371573 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:22.376462 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.384665 20775 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.539s (SENT) [suppressed 5 similar messages]
I20260709 12:26:22.386402 20876 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 34 because replica has last-logged OpId of term: 33 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 17.
I20260709 12:26:22.388856 20732 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:22.381382 32241 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.391378 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.391460 32241 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.392167 20728 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:22.324512 32319 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:22.393868 32241 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.394308 32319 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.396912 32319 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.396976 32310 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.368666 32326 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.397837 32326 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:22.401098 32202 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:22.329574 20625 rpcz_store.cc:269] 0709 12:26:05.494774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.218467 (+16723693us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:22.240836 (+ 22369us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:26:22.404140 20728 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:22.404414 32262 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.355106 32312 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:22.411739 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.386261 31887 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:22.439746 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 4175) took 1432 ms. Trace:
I20260709 12:26:22.440186 20726 rpcz_store.cc:276] 0709 12:26:21.007114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.007307 (+   193us) service_pool.cc:224] Handling call
0709 12:26:22.439537 (+1432230us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54000e83d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:22.439688 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725858688}
I20260709 12:26:22.418380 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:22.443934 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.446466 20776 leader_election.cc:400] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:26:22.446754 20776 leader_election.cc:403] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:22.414291 32206 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:22.451563 20728 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:22.454365 21429 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:22.454984 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.456878 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:22.459268 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74649.9us
I20260709 12:26:22.451009 32214 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:22.461651 20732 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:26:22.462011 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.462236 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.465035 20728 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:22.462710 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.465902 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:22.445451 31989 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.019s
I20260709 12:26:22.452059 32206 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:22.470945 32206 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:22.475955 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.481805 32259 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:22.492990 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.451769 32324 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:26:22.486304 20726 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:22.488260 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.489256 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.497205 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.497619 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:22.497839 20727 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:22.498067 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.499017 20877 raft_consensus.cc:2393] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:22.499356 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.489797 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.500988 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:22.499991 20735 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:22.502008 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:22.502563 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.512852 20735 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35. Ops: [34.21-34.21]
I20260709 12:26:22.514514 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:22.512183 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:22.448761 31887 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:22.528836 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.529811 20727 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:22.532001 20733 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 27. Current term is 28. Ops: [25.17-27.18]
I20260709 12:26:22.534337 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:22.531946 32226 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.536074 32226 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.538275 32226 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:22.541678 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.517165 32142 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:22.512962 32265 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:22.521373 32306 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.539201 32117 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:22.571033 32225 raft_consensus.cc:3060] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:22.578202 32340 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:26:22.580647 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:22.524991 32268 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:22.516461 31989 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.066s
I20260709 12:26:22.610646 20810 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:22.516252 32081 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.057s
I20260709 12:26:22.613855 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:22.545239 32319 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:22.617093 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:22.572731 32196 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.619421 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.573287 32023 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:22.624256 20884 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:22.606459 32117 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.625288 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:22.589614 32265 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:22.630913 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:22.633083 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2058) took 1577 ms. Trace:
I20260709 12:26:22.633426 20881 rpcz_store.cc:276] 0709 12:26:21.055892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.056119 (+   227us) service_pool.cc:224] Handling call
0709 12:26:21.056416 (+   297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:21.056779 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.056808 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:21.057392 (+   584us) log.cc:844] Serialized 52 byte log entry
0709 12:26:21.498630 (+441238us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:21.498672 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.498740 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.524038 (+1025298us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b2400162d58. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1ba389c4 00007f9e1ba36acb 00007f9e1bbf1dc9 00007f9e1bbf1a55 00007f9e1bbf0fb0 00007f9e1e867488 00007f9e1e867be8 00007f9e1e83f398 00007f9e1e83a76a 00007f9e1e839bb6
0709 12:26:22.628074 (+104036us) raft_consensus.cc:1685] finished
0709 12:26:22.628107 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.633008 (+  4901us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1423,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":2692480,"thread_start_us":440769,"threads_started":1,"wal-append.queue_time_us":495791}
I20260709 12:26:22.637233 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.637979 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:22.640107 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:22.617949 32226 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:22.644954 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:22.644253 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.646013 20881 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:22.619083 32188 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:22.604926 32142 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.631429 31989 consensus_queue.cc:1243] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 34.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:26:22.623577 32332 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:22.635387 32117 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:22.608230 32306 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.629905 32196 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.628142 32326 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.632745 31925 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.099s
W20260709 12:26:22.713584 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.637748 32023 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.676128 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:22.789599 20884 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 32 because replica has last-logged OpId of term: 31 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 19.
I20260709 12:26:22.648607 32268 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:22.642567 32319 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:22.788003 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 157149us
I20260709 12:26:22.736268 20881 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
W20260709 12:26:22.749454 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 109730us
I20260709 12:26:22.788750 32081 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.774552 32332 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.649308 32228 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:22.814132 20701 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:22.816306 32196 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.816905 32218 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:22.659752 32226 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.837661 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:22.819660 20881 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Committing config change with OpId 27.20: config changed from index 12 to 20, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:22.853986 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:22.677107 32188 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.858999 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:22.811013 32319 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.862321 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:22.863878 20701 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:22.866719 32220 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:22.869751 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2066) took 1116 ms. Trace:
I20260709 12:26:22.879379 20883 rpcz_store.cc:276] 0709 12:26:21.753061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.753183 (+   122us) service_pool.cc:224] Handling call
0709 12:26:21.753446 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.363523 (+610077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.869518 (+505995us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540026d4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:22.869693 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1401660160}
I20260709 12:26:22.836036 32228 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:22.836499 32332 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:22.883519 20877 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
W20260709 12:26:22.866958 32338 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 12:26:22.851732 32317 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:22.861177 32219 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:22.881755 32337 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 12:26:22.868745 32319 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:22.884457 32344 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 12:26:22.885601 32228 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.756595 32265 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:22.908257 32306 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:22.927088 32349 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:26:22.910653 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.915402 32142 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:22.919656 32348 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:26:22.919746 31887 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.925648 32226 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.888751 32317 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:22.842777 32081 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.868579 32352 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:22.914512 32319 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:22.931893 32023 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:22.910504 32225 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:22.946364 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.972378 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85706.9us
W20260709 12:26:22.935586 32311 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.568s	user 0.001s	sys 0.010s
I20260709 12:26:22.934768 31925 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:22.972134 32313 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.982940 32317 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:23.004944 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.984019 32265 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:23.001847 32355 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:22.990114 32354 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:23.001178 32352 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:23.012394 32320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.004s	sys 0.020s
I20260709 12:26:22.999423 32351 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:23.015872 32355 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:23.021070 32320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.020s
I20260709 12:26:23.005628 32326 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:23.030747 32214 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:23.051702 32188 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:23.038846 32214 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:23.053066 32214 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 35.20, Last appended by leader: 20, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:23.063768 32359 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:23.010106 32313 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:23.021162 32354 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.082399 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:23.085561 20881 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:23.091778 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:23.093374 32362 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:23.093854 32362 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:23.094233 32362 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:23.077559 32305 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:23.035621 32352 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:23.084223 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 220401us
I20260709 12:26:23.105203 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:23.108711 20881 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:23.108821 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:23.109684 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:23.109758 21431 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:23.111776 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:23.111986 20619 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 8 but the committed config has opid_index of 18
I20260709 12:26:23.113026 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:23.114080 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:23.114655 20881 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:26:23.115635 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:23.121409 21431 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:23.121832 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.123663 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.128948 32212 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:23.131328 32188 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:23.133051 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.145622 20883 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:23.146235 20883 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:23.166376 20775 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:23.169083 32239 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:23.169327 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:23.169852 32239 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:23.172894 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.175046 32366 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:23.177189 32366 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:23.196617 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.200265 20883 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:23.203649 32227 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:23.204062 32227 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:23.205787 32214 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:23.207794 32324 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:23.208395 32324 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.212292 20876 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:23.213543 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 4102) took 1152 ms. Trace:
I20260709 12:26:23.094827 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:23.214067 20879 rpcz_store.cc:276] 0709 12:26:22.061007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.064962 (+  3955us) service_pool.cc:224] Handling call
0709 12:26:23.213262 (+1148300us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400306954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:23.213464 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472833408}
I20260709 12:26:23.216338 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4148) took 1140 ms. Trace:
W20260709 12:26:23.203068 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.216856 20880 rpcz_store.cc:276] 0709 12:26:22.076000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.208414 (+132414us) service_pool.cc:224] Handling call
0709 12:26:23.216057 (+1007643us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400305f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:23.216262 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235588352}
I20260709 12:26:23.219333 20877 pending_rounds.cc:85] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 17
I20260709 12:26:23.219805 20877 pending_rounds.cc:107] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:26:23.220110 20877 raft_consensus.cc:2889] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.235471 32355 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:23.238579 32355 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:23.268671 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2070) took 1202 ms. Trace:
I20260709 12:26:23.269295 20882 rpcz_store.cc:276] 0709 12:26:22.066372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.225755 (+159383us) service_pool.cc:224] Handling call
0709 12:26:22.226138 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.354498 (+128360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.268319 (+913821us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540022fbd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:23.268569 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293439872}
I20260709 12:26:23.273201 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4149) took 1197 ms. Trace:
I20260709 12:26:23.273619 20878 rpcz_store.cc:276] 0709 12:26:22.076214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.078027 (+  1813us) service_pool.cc:224] Handling call
0709 12:26:23.267767 (+1189740us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54000a3cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:23.273137 (+  5370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783255808}
W20260709 12:26:23.277612 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 162074us
I20260709 12:26:23.277783 32176 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:23.278743 32176 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:23.280151 32176 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.281455 20806 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
W20260709 12:26:23.304603 32206 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.001s
I20260709 12:26:23.305269 32206 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:23.318750 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.346122 20735 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:23.350483 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:23.350947 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:23.351810 20805 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 36 because replica has last-logged OpId of term: 35 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 34 index: 20.
I20260709 12:26:23.358034 20881 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:23.358712 20881 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:23.360999 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.372504 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93866.5us
I20260709 12:26:23.359754 20881 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.25, Last appended by leader: 25, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.373795 20881 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:23.377367 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:23.378299 20804 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:23.379024 20805 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:23.379703 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:23.354791 32347 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.511s	user 0.004s	sys 0.000s
I20260709 12:26:23.382541 20809 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
W20260709 12:26:23.385438 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.386649 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:23.423699 20810 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:23.402459 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.419441 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:23.424372 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:23.424607 21433 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:23.425175 20809 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:23.426199 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.384442 21434 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Committing config change with OpId 27.20: config changed from index 12 to 20, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:23.387174 32117 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
I20260709 12:26:23.429808 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:23.381371 32313 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:23.431391 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.431833 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:23.416653 32311 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:23.429062 32332 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.003s
I20260709 12:26:23.381779 32285 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.053s
I20260709 12:26:23.440320 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.442566 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.443897 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:23.415745 32158 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:23.421367 31886 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:23.452159 20726 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:23.453565 20733 raft_consensus.cc:2368] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:23.429626 32281 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:23.431116 32357 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:23.432425 32353 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:26:23.433068 32327 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:26:23.467653 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.498749 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:23.530825 20732 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:23.456984 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.532016 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.457231 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:23.455475 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.439991 32212 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:23.468009 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:23.553551 20728 raft_consensus.cc:2376] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35. Re-sending same reply.
I20260709 12:26:23.553572 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.554302 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:23.554766 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:23.555516 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.556002 32328 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:23.556460 32328 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.556806 32328 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:23.557672 32361 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:23.531049 20735 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:23.434991 32371 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:23.563963 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.552987 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.444433 32354 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:23.496023 32365 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:26:23.573585 20733 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:23.588244 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:23.574410 20726 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:23.491106 32206 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:23.575724 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.576108 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.553174 32245 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.433043 32117 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.552866 32355 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:23.559918 32212 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:23.604245 20729 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:23.589442 32352 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:23.586761 32176 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:23.611528 32176 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:23.612187 32228 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
I20260709 12:26:23.612658 32228 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.613777 32176 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:23.614789 32228 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:23.619838 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 266186us
I20260709 12:26:23.446795 32311 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.454257 32332 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:23.559605 32281 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.560400 31886 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.636116 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4107) took 1245 ms. Trace:
W20260709 12:26:23.629179 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 471979us
W20260709 12:26:23.628064 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 520066us
I20260709 12:26:23.637920 20731 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:23.610559 32212 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:23.467149 32158 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.436491 32317 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.636477 20884 rpcz_store.cc:276] 0709 12:26:22.390384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.886108 (+495724us) service_pool.cc:224] Handling call
0709 12:26:22.921351 (+ 35243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.921555 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:26:23.042096 (+120541us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.635793 (+593697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.635842 (+    49us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:23.635885 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.635958 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.636062 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":362756,"prepare.queue_time_us":367509,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"spinlock_wait_cycles":8879872,"thread_start_us":348365,"threads_started":2,"wal-append.queue_time_us":403486}
I20260709 12:26:23.656190 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:23.657068 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:23.627343 32382 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:23.659289 32313 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.628901 32363 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.666586 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2077) took 1037 ms. Trace:
I20260709 12:26:23.669513 20881 rpcz_store.cc:276] 0709 12:26:22.628954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.115421 (+486467us) service_pool.cc:224] Handling call
0709 12:26:23.357860 (+242439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.666155 (+308295us) raft_consensus.cc:1534] Early marking committed up to index 24
0709 12:26:23.666187 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.666228 (+    41us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:23.666264 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.666355 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.666510 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130708}
I20260709 12:26:23.559993 32357 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.577005 32371 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:23.610791 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:23.635221 32306 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.677377 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:23.680837 20884 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:23.661657 32313 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:23.679484 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 68730.9us
I20260709 12:26:23.684492 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:23.686547 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:23.687400 32352 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:23.610803 32245 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:23.690676 32245 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:23.691833 32352 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:23.695603 20775 leader_election.cc:400] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:23.696000 20775 leader_election.cc:403] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:23.697621 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:23.698830 32327 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:23.700383 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.703475 32245 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:23.706769 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:23.619709 32355 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.708348 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:23.706732 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59666.6us
I20260709 12:26:23.636736 32185 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:26:23.730324 21426 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:23.720677 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:23.734090 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:23.735411 20729 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:23.736730 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:23.677366 32142 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:23.734843 20735 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:23.660419 32307 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.075s
I20260709 12:26:23.757946 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:23.768761 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.736984 32265 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.000s
W20260709 12:26:23.784096 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.774955 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:23.683950 32268 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:23.790238 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.824402 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.827854 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 190584us
I20260709 12:26:23.831921 20881 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:23.835141 20881 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:23.823032 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:23.756850 32237 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:23.840130 32081 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:23.840955 32081 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:23.841616 32081 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.844372 32264 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:23.844874 32264 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:23.845697 32264 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.847474 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:23.708560 32005 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.110s
I20260709 12:26:23.771653 32215 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:23.791430 32206 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:23.854192 20884 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:23.681681 32363 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:23.855719 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.772109 32214 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:23.708328 32306 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.858053 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:23.862581 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:23.782936 32142 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.867048 32142 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:23.847097 32264 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:23.880575 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4108) took 1472 ms. Trace:
I20260709 12:26:23.881022 20877 rpcz_store.cc:276] 0709 12:26:22.408487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.890217 (+481730us) service_pool.cc:224] Handling call
0709 12:26:23.218981 (+328764us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:23.633341 (+414360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.633371 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:23.633964 (+   593us) log.cc:844] Serialized 52 byte log entry
0709 12:26:23.671769 (+ 37805us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:23.671827 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.671929 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.877375 (+205446us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:23.878644 (+  1269us) raft_consensus.cc:1685] finished
0709 12:26:23.878678 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.880411 (+  1733us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400305f50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:23.880519 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":484358,"raft.queue_time_us":414312,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":14731,"spinlock_wait_cycles":764866048,"thread_start_us":9474,"threads_started":1,"wal-append.queue_time_us":88312}
I20260709 12:26:23.835592 32196 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:23.790659 32238 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.076s
I20260709 12:26:23.871403 31886 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:23.871812 32376 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 12:26:23.872494 32363 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:23.873150 32391 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.874696 32392 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.874960 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:23.875133 32185 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.876729 32306 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:23.861733 32215 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.849848 32353 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:23.893399 32227 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.894876 32238 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:23.883081 32212 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.859787 32394 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:23.901576 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 173090us
I20260709 12:26:23.863790 32237 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:23.855561 31989 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.042s
W20260709 12:26:23.913806 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.902313 32117 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.012010 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:23.888366 32196 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.012183 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:23.991413 32378 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.348s	user 0.001s	sys 0.001s
I20260709 12:26:23.933360 32311 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:23.933667 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:24.022090 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.028832 21432 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:23.911072 32391 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.021739 32196 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:24.030735 21435 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:24.031360 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1891) took 1496 ms. Trace:
I20260709 12:26:24.032084 32391 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:24.034077 32158 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:24.034516 32386 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:26:24.017568 32397 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:26:24.036161 32383 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:26:24.031721 20727 rpcz_store.cc:276] 0709 12:26:22.539263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.539387 (+   124us) service_pool.cc:224] Handling call
0709 12:26:22.539745 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.578009 (+ 38264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.578044 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.222557 (+644513us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:23.222998 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.420033 (+197035us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:23.420526 (+   493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.422176 (+  1650us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.022657 (+600481us) raft_consensus.cc:1685] finished
0709 12:26:24.022681 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.031295 (+  8614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29658,"raft.queue_time_us":66290,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6502,"spinlock_wait_cycles":9177984,"thread_start_us":194452,"threads_started":1,"wal-append.queue_time_us":268317}
W20260709 12:26:24.058092 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 124512us
I20260709 12:26:24.059376 20805 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:26:24.002031 32238 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:24.084721 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.059971 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:24.085840 21432 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 29: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:24.012665 32332 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:24.034073 32388 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:26:24.015050 32227 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:23.983711 32185 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.090704 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.017257 32081 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.039359 32281 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:24.164736 20806 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 4004) took 1801 ms (client timeout 1843 ms). Trace:
I20260709 12:26:24.165431 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:24.109537 32206 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:24.175559 20802 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:24.086736 32391 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.167726 20806 rpcz_store.cc:269] 0709 12:26:22.296123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.296775 (+   652us) service_pool.cc:224] Handling call
0709 12:26:24.092305 (+1795530us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b540006d454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:24.097109 (+  4804us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156996480}
I20260709 12:26:24.097621 32397 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:24.196056 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.188139 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.197016 20727 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:24.090833 32361 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:24.088445 32117 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:24.202184 20802 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:24.110517 32382 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:24.208519 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.176872 32357 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.177098 32313 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:24.211669 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:24.208773 32362 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.114s	user 0.007s	sys 0.005s
I20260709 12:26:24.198180 32081 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.219679 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.221876 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:24.222649 20802 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:24.223754 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:24.224622 20809 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:24.176353 32206 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.231724 20727 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:24.227322 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:24.205868 32371 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.100023 32238 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.25, Last appended by leader: 25, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.207751 32319 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.196115 32391 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.245029 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:24.249707 20803 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:24.214397 32317 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.210805 32390 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:24.209216 32400 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:24.209340 32117 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.185071 32237 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.20, Last appended by leader: 20, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:24.257836 32327 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.002s
I20260709 12:26:24.235535 32081 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.274365 20802 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
W20260709 12:26:24.277087 21434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.013s	sys 0.004s
W20260709 12:26:24.277550 21434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.013s	sys 0.004s
W20260709 12:26:24.278877 21429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.009s	sys 0.008s
W20260709 12:26:24.279342 21429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.010s	sys 0.008s
I20260709 12:26:24.269589 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.280313 21435 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:24.282742 20803 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
W20260709 12:26:24.283380 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.262840 32238 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:24.264580 32215 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.293836 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.296742 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:24.297492 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:24.297665 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:24.298070 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:24.284044 32212 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.298878 20807 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:24.300163 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.303367 20810 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
W20260709 12:26:24.302513 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 202047us
I20260709 12:26:24.284399 32327 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:24.300190 32196 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.305940 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:24.304028 32214 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.334267 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.339756 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:24.341498 20809 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38. Ops: []
W20260709 12:26:24.341377 20775 leader_election.cc:336] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.539s (SENT)
I20260709 12:26:24.309203 20808 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:24.308315 20801 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:24.344147 20734 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:24.347628 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:24.316481 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.304195 32281 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:24.345933 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 166289us
I20260709 12:26:24.348533 20726 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
W20260709 12:26:24.335704 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 230814us
W20260709 12:26:24.344393 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.352335 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.335508 32406 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.372524 20877 raft_consensus.cc:1275] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:24.375921 20880 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:24.378460 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:24.345507 32185 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:24.421264 20882 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
W20260709 12:26:24.369853 32404 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 12:26:24.422081 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70411.5us
I20260709 12:26:24.352545 32317 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:24.453548 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:24.454238 20729 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:24.454594 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:24.454860 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:24.423857 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 49944.1us
W20260709 12:26:24.427762 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90247.2us
I20260709 12:26:24.456441 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:24.463594 20734 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:24.464956 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:24.372305 32327 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:24.464777 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:24.494104 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:24.455814 32264 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.005s
I20260709 12:26:24.467979 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2169) took 1084 ms. Trace:
I20260709 12:26:24.459102 20804 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
W20260709 12:26:24.452227 32388 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:24.497839 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:24.467005 32399 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 12:26:24.499017 32407 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:26:24.499563 32306 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.500735 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:24.530157 20734 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:24.501926 32081 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.469419 32262 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:26:24.512063 21434 rpcz_store.cc:276] 0709 12:26:23.383757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.383865 (+   108us) service_pool.cc:224] Handling call
0709 12:26:23.384126 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.384327 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:23.621151 (+236824us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.423762 (+802611us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.467557 (+ 43795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:24.467644 (+    87us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:24.467706 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.467773 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.467907 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43481,"raft.queue_time_us":163510,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":5861,"spinlock_wait_cycles":28027648,"thread_start_us":798399,"threads_started":3,"wal-append.queue_time_us":202726}
I20260709 12:26:24.378907 32406 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.495945 32363 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:24.231679 32362 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.519452 20882 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:24.513299 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:24.454769 32317 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.011711 32394 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.208165 32361 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.213617 32382 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.370786 32281 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:23.932623 32392 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.555860 20802 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:24.494352 32352 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.050s
I20260709 12:26:24.548077 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:24.557400 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:24.558768 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:24.352726 32214 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.461287 31887 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.074s
I20260709 12:26:24.551323 20734 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:24.518323 32371 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:24.565550 32371 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.565989 32371 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:24.560740 32264 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:24.564016 32414 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:24.564271 32413 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:26:24.564647 32357 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:24.572783 32281 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:24.565035 32408 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:26:24.574719 32382 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.576149 32324 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.577397 32371 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.545605 32363 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:24.578820 32363 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:24.580119 32371 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.580749 32371 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:24.581202 32371 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:24.582052 32371 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.583853 20883 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:24.585186 32406 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.587020 32394 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:24.589155 32373 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.102s	user 0.002s	sys 0.010s
I20260709 12:26:24.589725 21432 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
W20260709 12:26:24.595937 32328 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.039s	user 0.008s	sys 0.005s
I20260709 12:26:24.596447 32328 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.598884 32328 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.565018 32392 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:24.599692 32411 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:26:24.601382 32361 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.545110 32306 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.601907 20727 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:24.602648 32196 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.589567 32414 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:24.603925 20730 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:24.548112 32317 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:24.629508 20883 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 19. (term mismatch)
I20260709 12:26:24.637795 20883 pending_rounds.cc:85] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 18
I20260709 12:26:24.638074 20883 pending_rounds.cc:107] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 32.19
I20260709 12:26:24.638346 20883 raft_consensus.cc:2889] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.639137 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4164) took 1649 ms. Trace:
I20260709 12:26:24.639549 20883 rpcz_store.cc:276] 0709 12:26:22.989922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.145006 (+155084us) service_pool.cc:224] Handling call
0709 12:26:23.145437 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.583465 (+1438028us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b58006c92d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ebfb0aa 00007f9e1ec7d447 00007f9e1ec95fe7 00007f9e1ec977de 00007f9e1eca1b49 00007f9e1eca2416 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4
0709 12:26:24.638839 (+ 55374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.639070 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984808320}
I20260709 12:26:24.559691 32391 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:24.632731 32328 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.634817 32356 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.645128 21434 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:24.645530 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.650220 32117 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.656909 31887 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:24.657572 31887 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.659838 31887 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:24.616736 32307 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:24.667987 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 132532us
I20260709 12:26:24.669239 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.670416 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:24.671015 20733 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:24.671391 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 102372us
I20260709 12:26:24.672307 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:24.673556 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.674309 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:24.675284 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 177502us
I20260709 12:26:24.665807 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:24.676501 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 94821.2us
I20260709 12:26:24.677165 21427 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: [35.20-35.20]
I20260709 12:26:24.680328 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:24.683350 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:24.722065 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.703722 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:24.675420 32311 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:24.733134 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:24.758620 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.642794 32395 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:24.753814 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:24.776069 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90325.3us
I20260709 12:26:24.682003 32196 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:24.681586 31887 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:24.795332 20733 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:24.668351 32328 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.672045 32356 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.727522 32357 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:24.677662 32227 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.127s
I20260709 12:26:24.681275 32406 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:24.677932 32363 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:24.813602 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2715) took 2321 ms. Trace:
I20260709 12:26:24.685461 32158 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.125s
W20260709 12:26:24.836928 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49008.5us
W20260709 12:26:24.827630 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 142377us
I20260709 12:26:24.837877 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:24.677140 32307 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.816550 32311 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 37.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.012s
I20260709 12:26:24.814497 31887 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.813020 32237 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.030s
I20260709 12:26:24.838965 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:24.833806 21429 rpcz_store.cc:276] 0709 12:26:22.491910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.492074 (+   164us) service_pool.cc:224] Handling call
0709 12:26:22.501728 (+  9654us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.516942 (+ 15214us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:23.121177 (+604235us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b180046d6e0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:26:23.536733 (+415556us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.455812 (+919079us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.476329 (+ 20517us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.607688 (+131359us) log.cc:844] Serialized 26 byte log entry
0709 12:26:24.607935 (+   247us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:24.608005 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.608106 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.813339 (+205233us) raft_consensus.cc:1685] finished
0709 12:26:24.813368 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.813502 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42329,"prepare.queue_time_us":833260,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":108,"raft.queue_time_us":146478,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":37660,"spinlock_wait_cycles":4000512,"thread_start_us":1265841,"threads_started":4,"wal-append.queue_time_us":288192}
I20260709 12:26:24.812844 32431 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.046s
I20260709 12:26:24.839709 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:24.899250 32431 consensus_queue.cc:1243] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 35.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:26:24.866780 32317 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:24.899871 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 60241.9us
I20260709 12:26:24.900748 20776 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:24.901278 32317 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.854784 31887 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:24.837409 32406 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:24.856386 32438 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:26:24.927937 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 89041.7us
W20260709 12:26:24.867579 32405 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:26:24.948446 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:24.952970 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.956902 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:24.957479 32356 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:24.958094 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:24.959110 32406 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.955525 32351 raft_consensus.cc:2764] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:26:24.960315 32351 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:24.927583 32438 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.962491 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.965006 20734 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
W20260709 12:26:24.966043 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.963181 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.963651 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:24.967391 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.968171 21434 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 27.
I20260709 12:26:24.968652 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:24.968958 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:24.970494 32427 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:24.970645 20808 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 19
I20260709 12:26:24.971065 20808 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 35.20
I20260709 12:26:24.971313 20808 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.973843 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:24.974478 20809 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:24.975818 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:24.960958 32351 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:24.955127 32416 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 12:26:24.951016 32265 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:24.945804 32354 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:24.982591 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 81935.4us
I20260709 12:26:24.982672 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:24.950878 32264 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.033s
I20260709 12:26:25.001787 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:24.976944 32438 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:24.921991 32196 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:25.019737 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:24.995285 32412 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:25.026203 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:24.996464 32212 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:25.023154 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.030217 32443 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:25.042093 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:25.042886 21430 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:25.030582 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.048511 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:25.032166 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.039419 32443 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.979462 32406 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.000401 32214 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:25.026600 32238 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.720s	user 0.018s	sys 0.000s
I20260709 12:26:25.063778 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4105) took 2818 ms. Trace:
I20260709 12:26:25.017433 32440 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:25.077591 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.026247 32352 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:25.018656 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:25.018272 32351 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:25.035243 32435 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 12:26:25.038828 32432 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:26:25.090891 21430 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:24.979501 32265 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:25.074883 32328 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:24.999328 32354 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.070578 32443 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:25.078135 32363 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:25.096309 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 113695us
I20260709 12:26:25.098412 20734 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
W20260709 12:26:25.106417 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 87837.7us
I20260709 12:26:25.108345 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:25.107514 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:25.111901 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:25.114012 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.115031 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.115974 20731 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:25.118203 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:25.095388 32265 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:25.120891 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:25.096760 32237 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:25.084244 32352 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.078770 32406 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.111107 32429 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:25.112385 32264 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:25.126997 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:25.117331 32328 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:25.091293 32351 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:25.133322 32426 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:26:25.133689 32354 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:25.133973 32436 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 12:26:25.167866 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47040.5us
I20260709 12:26:25.134085 32406 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.134217 32449 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:25.120652 32439 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:25.170069 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:25.174369 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:25.175833 32452 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:25.176342 32452 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:25.176685 32452 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:25.178767 32311 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:25.131443 32328 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.134119 32444 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:25.136037 32363 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:25.110364 32196 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 21, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.170444 32264 raft_consensus.cc:2764] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:26:25.128052 32237 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.202159 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:25.211347 32430 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.245414 32430 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:25.212934 32374 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.717s	user 0.006s	sys 0.002s
I20260709 12:26:25.213549 32449 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.214125 32448 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:25.215173 32306 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.217897 32352 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.222417 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.223806 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.134975 32429 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.200390 32328 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:25.200672 32444 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.203922 32427 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:25.231650 32363 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:25.214686 32441 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.250909 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48851.4us
I20260709 12:26:25.257452 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.257689 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:25.253775 32453 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:26:25.252261 32448 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:25.293390 32377 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 1.721s	user 0.006s	sys 0.001s
I20260709 12:26:25.252434 32449 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.260071 32444 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:25.372808 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2171) took 1988 ms. Trace:
I20260709 12:26:25.373548 32437 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:25.254972 32311 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:25.373981 32441 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.374890 32430 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.375886 32351 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.275967 32442 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.262382 32363 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.378578 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 4019) took 1176 ms. Trace:
I20260709 12:26:25.274435 32456 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:25.385730 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.406991 21428 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:25.387061 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.408134 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:25.073102 20875 rpcz_store.cc:276] 0709 12:26:22.221808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.827244 (+605436us) service_pool.cc:224] Handling call
0709 12:26:23.163467 (+336223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.163788 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:26:23.370924 (+207136us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.835405 (+464481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.850240 (+ 14835us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:23.850318 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.850407 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.039638 (+1189231us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54001fed50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:25.039758 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":370602,"prepare.queue_time_us":210582,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":203795,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":25213,"spinlock_wait_cycles":772495744,"thread_start_us":332015,"threads_started":2,"wal-append.queue_time_us":232944,"wal-append.run_cpu_time_us":2668,"wal-append.run_wall_time_us":1458927}
I20260709 12:26:25.396130 21431 rpcz_store.cc:276] 0709 12:26:23.384308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.384443 (+   135us) service_pool.cc:224] Handling call
0709 12:26:23.384808 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.421225 (+ 36417us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:23.436478 (+ 15253us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.597272 (+160794us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4800280d58. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:26:24.466556 (+869284us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b1800256e40. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:26:24.616690 (+150134us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.636830 (+ 20140us) log.cc:844] Serialized 26 byte log entry
0709 12:26:24.636991 (+   161us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:24.642043 (+  5052us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.642317 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.642398 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.372597 (+730199us) raft_consensus.cc:1685] finished
0709 12:26:25.372620 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.372748 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40617,"prepare.queue_time_us":22513,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":18566,"raft.queue_time_us":38113,"raft.run_cpu_time_us":2987,"raft.run_wall_time_us":198708,"spinlock_wait_cycles":9106176,"thread_start_us":137905,"threads_started":1,"wal-append.queue_time_us":1213036}
I20260709 12:26:25.410988 32427 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:25.411484 32427 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.411849 32427 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:25.416461 32429 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.418345 32442 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.405979 20804 rpcz_store.cc:276] 0709 12:26:24.201753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.211364 (+  9611us) service_pool.cc:224] Handling call
0709 12:26:25.290903 (+1079539us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540006ccd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:25.378522 (+ 87619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303708416}
W20260709 12:26:25.426865 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 169234us
W20260709 12:26:25.428742 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 254412us
I20260709 12:26:25.439427 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.440426 21428 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:25.398531 32437 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:25.465843 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:25.469616 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.427795 32430 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:25.474007 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44546.4us
I20260709 12:26:25.471549 32317 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:25.479300 32317 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:25.479805 32317 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:25.482775 32361 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:25.473008 32081 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:25.483217 32361 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:25.491214 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:25.497255 32414 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:25.497741 32414 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.498126 32414 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:25.500170 32456 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:25.502494 32456 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 37.21, Last appended by leader: 21, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.477361 32434 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:25.510527 32448 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:25.512326 32328 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.520264 32265 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.522511 32265 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:25.523343 32237 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.524289 32427 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.532763 32427 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.533370 32400 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:25.534642 32362 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:25.534477 32400 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.537201 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2083) took 1699 ms. Trace:
W20260709 12:26:25.538743 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.539700 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.540417 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 1.19589e+06us
I20260709 12:26:25.529170 32443 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.477567 32430 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:25.530606 32438 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.004s
I20260709 12:26:25.528276 32361 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.537601 20881 rpcz_store.cc:276] 0709 12:26:23.838866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.839466 (+   600us) service_pool.cc:224] Handling call
0709 12:26:23.839750 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.839945 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:23.840252 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.284619 (+444367us) log.cc:844] Serialized 14 byte log entry
0709 12:26:24.665063 (+380444us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b180029a400. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:25.533023 (+867960us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b180029a400. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e22b7358f 00007f9e1bbb2571 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd
0709 12:26:25.536846 (+  3823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.536901 (+    55us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:25.536950 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.537029 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.537145 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1320009,"prepare.queue_time_us":1420,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":3962,"raft.queue_time_us":1359913,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":9114,"spinlock_wait_cycles":17418368,"thread_start_us":385464,"threads_started":2,"wal-append.queue_time_us":398413}
W20260709 12:26:25.546999 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.551961 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:25.552714 20734 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:25.553315 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.554210 20808 raft_consensus.cc:2410] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 34 because replica has last-logged OpId of term: 33 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 23.
I20260709 12:26:25.548839 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.563491 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.564499 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.567137 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.570365 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.573599 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.574252 20806 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:25.575605 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.576398 20731 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:25.576951 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:25.577606 20804 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:25.579742 20806 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:25.579958 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.580745 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.581074 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.582158 20733 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:25.579742 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.584358 20731 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 39: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:25.580554 20728 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:25.585114 20809 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 39: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:25.585228 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.585820 21434 raft_consensus.cc:2368] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:25.585891 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.586598 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.587304 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.588452 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.639262 20806 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:25.589680 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.512244 32452 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:25.592562 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 1.13015e+06us
I20260709 12:26:25.594447 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:25.640861 20804 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
W20260709 12:26:25.641120 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 99692.3us
I20260709 12:26:25.561395 32332 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:26:25.634878 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.638000 20735 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 30 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
W20260709 12:26:25.592759 31887 consensus_meta.cc:287] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.659s	user 0.020s	sys 0.004s
I20260709 12:26:25.569710 32438 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.540593 32434 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.637709 32394 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.637996 32313 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:25.506716 32437 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 37.21, Last appended by leader: 21, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.594813 32361 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:25.668509 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.657459 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.643051 20809 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:25.670437 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:25.644410 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.662680 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:25.658463 20883 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:26:25.665083 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:25.671926 20879 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:25.595799 32392 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.674666 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.676015 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.676517 20806 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 32 because replica has last-logged OpId of term: 30 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 16.
I20260709 12:26:25.635277 20734 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
W20260709 12:26:25.673408 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 238572us
I20260709 12:26:25.678496 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:25.677986 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.679069 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.679105 20734 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:25.680349 20733 raft_consensus.cc:2393] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:25.681924 20880 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 31 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 16.
I20260709 12:26:25.682770 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.683751 20878 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:25.684621 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:25.633867 32196 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.686537 20881 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:25.679450 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:25.689334 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:25.692201 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51216.6us
I20260709 12:26:25.636675 32401 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.642279 32316 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.651381 32393 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.700599 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:25.649403 32443 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.646377 32438 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:25.704276 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.706833 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.709806 20881 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:25.707679 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:25.708287 20849 connection.cc:256] RPC call timeout handler was delayed by -1.4035936519999268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 12:26:25.650095 32313 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.644799 31887 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.711769 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:25.708953 32393 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.712440 20884 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:25.713150 31887 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.713253 32313 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.713822 32393 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.713981 21435 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:25.715842 20882 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 25. Preceding OpId from leader: term: 37 index: 25. (term mismatch)
W20260709 12:26:25.716050 32470 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:25.716235 20882 pending_rounds.cc:85] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 24
I20260709 12:26:25.716481 20882 pending_rounds.cc:107] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 36.25
I20260709 12:26:25.716698 20882 raft_consensus.cc:2889] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:25.722312 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:25.723497 32438 consensus_queue.cc:1048] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:25.694262 32196 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.704833 32401 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.728298 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.728652 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.728760 32317 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.649559 32394 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.728327 32401 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.731668 32394 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.732744 32430 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.732815 32391 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.705708 32469 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:26:25.733667 32391 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.710779 20883 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:25.709801 20878 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:25.737274 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.737250 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.741191 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:25.700414 32354 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:25.742239 32215 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:25.746708 32215 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.735683 32397 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:25.748339 32397 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:25.748715 32397 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:25.750826 32215 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.695218 32418 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:25.692059 32392 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.708671 32316 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.753211 32446 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:25.757771 32307 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.757289 32444 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.689735 32081 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:25.771279 32446 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:25.735036 32414 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.763775 32354 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:25.778752 32390 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:25.793699 32410 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.270s	user 0.001s	sys 0.002s
I20260709 12:26:25.767975 32418 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.790225 32446 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.676998 20728 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:25.744251 20884 raft_consensus.cc:2410] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 31 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 16.
I20260709 12:26:25.916071 32361 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:25.796660 32328 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.917565 32361 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.918583 32397 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:25.869099 20735 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
I20260709 12:26:25.919998 32354 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.22, Last appended by leader: 22, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.921270 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.921944 32397 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.874289 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.923698 32397 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:25.923012 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.874971 20807 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:25.898051 20880 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:25.892138 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:25.892797 20875 pending_rounds.cc:85] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 24
I20260709 12:26:25.900216 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:25.906193 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.861413 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.897167 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.876047 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.884418 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.875763 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:25.886159 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:25.942234 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.916385 32316 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.937423 20875 pending_rounds.cc:107] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 37.25
I20260709 12:26:25.896150 20879 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:25.863873 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.898033 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:25.909119 32196 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:25.915465 32317 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.966573 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:25.867086 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:25.976250 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58424.2us
I20260709 12:26:25.977860 32005 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:25.799037 32390 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:25.981575 20808 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:25.812247 32449 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:25.889575 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 148290us
I20260709 12:26:25.973919 20875 raft_consensus.cc:2889] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:25.963639 32259 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.994390 20878 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:25.990948 20884 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:25.966146 20734 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:25.970942 32361 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:25.967818 32392 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:26.028681 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:25.998769 20805 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:26.030035 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:26.030151 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:26.030426 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.965400 32430 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:26:26.031163 21428 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:26.031774 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.034003 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.034497 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:26.035316 20735 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:26.035619 32382 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
W20260709 12:26:26.036298 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 169853us
I20260709 12:26:26.036530 32382 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:26.037132 32382 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:26.038070 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.005879 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.041805 20884 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:26.006754 20803 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:26.042240 32185 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:26.007431 32259 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.008958 20726 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 32 because replica has last-logged OpId of term: 30 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 16.
I20260709 12:26:26.042891 32185 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.044239 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.044868 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2091) took 1660 ms. Trace:
I20260709 12:26:26.045715 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:26.046403 20726 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:26.009593 20883 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:26.046838 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:26.010056 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.805840 32444 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:25.808874 32434 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:26.048422 20880 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:25.908960 32081 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:25.817163 32452 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:26.012434 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:26.012800 32391 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.804780 32441 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:25.811122 32445 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.716s	user 0.001s	sys 0.000s
I20260709 12:26:26.017330 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:25.815753 32442 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:25.813491 32414 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:26.018632 20728 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:26.019156 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:26.019415 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:25.810812 32418 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:25.971248 32468 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:26.024729 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.029042 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:26.056599 20727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1905) took 1737 ms (client timeout 1891 ms). Trace:
I20260709 12:26:26.057358 32434 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:26.059228 20730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3878) took 2628 ms (client timeout 1539 ms). Trace:
I20260709 12:26:26.074800 20808 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
W20260709 12:26:26.060571 20802 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43456 (request call id 2164) took 1763 ms (client timeout 1533 ms). Trace:
I20260709 12:26:26.038517 32382 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.054908 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.068668 20810 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:26.079394 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.080431 20733 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:26.087589 32427 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:26.089044 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:26.075603 20802 rpcz_store.cc:269] 0709 12:26:24.297318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.297448 (+   130us) service_pool.cc:224] Handling call
0709 12:26:26.060116 (+1762668us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54000a0854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.060278 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488169728}
I20260709 12:26:26.094194 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.094503 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.045311 20877 rpcz_store.cc:276] 0709 12:26:24.384771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.422693 (+ 37922us) service_pool.cc:224] Handling call
0709 12:26:24.423163 (+   470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:25.570541 (+1147378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.570575 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:25.571213 (+   638us) log.cc:844] Serialized 52 byte log entry
0709 12:26:25.700902 (+129689us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:26.041391 (+340489us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.041914 (+   523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.044424 (+  2510us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.044484 (+    60us) raft_consensus.cc:1685] finished
0709 12:26:26.044495 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.044669 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1137424,"raft.queue_time_us":1221017,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":18921,"thread_start_us":418199,"threads_started":2,"wal-append.queue_time_us":198962}
I20260709 12:26:26.096071 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:26.096614 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:26.074470 20727 rpcz_store.cc:269] 0709 12:26:24.319540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.333965 (+ 14425us) service_pool.cc:224] Handling call
0709 12:26:26.056107 (+1722142us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54001d7254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.056344 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586504448}
I20260709 12:26:26.097450 20877 raft_consensus.cc:2393] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 29: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:26.098001 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.099391 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.100515 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:26.074869 20730 rpcz_store.cc:269] 0709 12:26:23.430713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.431614 (+   901us) service_pool.cc:224] Handling call
0709 12:26:26.058726 (+2627112us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000e6ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.058989 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2593641856}
I20260709 12:26:26.102119 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.102785 20730 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:26.097484 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:26.103713 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.104682 20730 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:26.105486 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:26.106551 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.109714 20731 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:26:26.113669 21428 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 28.
I20260709 12:26:26.096984 20802 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:26.096647 20881 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:26.120039 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.117190 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:26.117552 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 98590.9us
I20260709 12:26:26.121538 20734 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.113991 32317 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.125321 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.132591 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:26.137327 32434 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.140019 32434 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:26.140597 32215 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:26.141191 32215 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.141353 32390 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.143877 32390 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:26.145006 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:26.146327 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:26.147583 32362 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:26.148129 32362 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:26.141698 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50020.2us
I20260709 12:26:26.159786 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.151800 20735 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:26.165350 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.166706 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2729) took 1165 ms. Trace:
I20260709 12:26:26.149353 32362 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.167085 21434 rpcz_store.cc:276] 0709 12:26:25.001373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.001537 (+   164us) service_pool.cc:224] Handling call
0709 12:26:26.166234 (+1164697us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54007602d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.166505 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305400576}
W20260709 12:26:26.169811 21432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 2130) took 2060 ms (client timeout 1647 ms). Trace:
I20260709 12:26:26.172582 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:26.141297 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:26.170243 21432 rpcz_store.cc:269] 0709 12:26:24.109730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.130423 (+ 20693us) service_pool.cc:224] Handling call
0709 12:26:26.169455 (+2039032us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54005302d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.169612 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817721728}
I20260709 12:26:26.151511 32391 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:26.184976 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:26.170760 32434 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:26.189260 20804 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:26.197475 32332 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:26.199350 32468 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.202817 32468 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:26.204609 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:26.211268 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.211903 32245 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:26.214182 20728 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:26.214931 32434 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:26.182687 32317 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.190577 32391 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:26.216665 32355 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:26.221321 21432 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:26.221957 32390 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:26.226321 20810 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:26:26.227427 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:26.252990 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:26.254298 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.257710 20734 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:26.260249 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:26.265532 20728 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:26.241266 32245 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:26.263538 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.272590 32215 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:26.274224 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.275630 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.276069 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:26.276486 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49525us
I20260709 12:26:26.278805 20809 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:26.279266 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.279906 32452 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:26.280553 32452 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:26.281741 20879 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:26.282917 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.283712 20879 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:26.284565 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.285257 20879 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:26.286216 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.287969 32434 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.289546 20726 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:26.291370 32390 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:26.292753 32390 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.293715 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:26.294391 21433 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.299502 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:26.300132 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:26.301661 32454 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:26.302088 32454 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.302457 32454 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:26.303717 32158 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:26.304095 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:26.304296 32158 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:26.281788 32452 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.306582 32245 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.307816 32245 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:26.309043 21430 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:26.249440 32391 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:26.308336 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:26.305667 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 140520us
I20260709 12:26:26.319476 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.320801 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.335803 32228 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:26:26.336433 32490 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:26:26.336544 32228 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:26.337738 32228 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.338376 21429 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:26:26.339455 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:26.339712 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 91755.3us
I20260709 12:26:26.341347 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:26.342048 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.344504 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:26.347236 32409 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.013s
I20260709 12:26:26.347347 32238 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.030s
I20260709 12:26:26.348134 32392 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:26.348795 32238 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:26.349220 32238 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:26.349761 32361 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:26.350230 32361 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:26.350742 32361 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:26.350278 32238 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.347733 32418 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:26.305644 32158 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.359445 32430 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:26.360284 32430 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.363314 32317 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:26.366992 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49183.7us
I20260709 12:26:26.364574 20881 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:26.372887 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.367465 20727 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:26.377825 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:26.356379 32382 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:26.383765 20734 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:26.375355 32355 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:26.375797 32418 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:26.414196 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.380715 32382 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:26.420466 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.424846 20809 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:26.427754 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:26.431644 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.433305 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.434648 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.444757 32446 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:26.447836 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 60117.8us
W20260709 12:26:26.450489 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 93315.2us
W20260709 12:26:26.452479 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50154.8us
I20260709 12:26:26.448298 20876 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
W20260709 12:26:26.456743 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87457.7us
I20260709 12:26:26.454243 20731 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:26.425794 32382 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:26.462487 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.463913 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:26.459808 32238 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:26.465613 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.466420 32362 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:26.466984 32362 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:26.422765 32355 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:26.469272 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.485069 32215 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:26.495510 20881 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:26.496714 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.497438 20881 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:26.498304 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.501634 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.504252 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.504483 32355 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 37.21, Last appended by leader: 21, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.509991 32443 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:26.511464 32419 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:26.511956 32419 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.512315 32419 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.490432 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:26.490501 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:26.499121 20881 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 34 because replica has last-logged OpId of term: 32 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 18.
I20260709 12:26:26.521359 20876 raft_consensus.cc:2410] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 34 because replica has last-logged OpId of term: 31 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 17.
I20260709 12:26:26.524214 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.533139 21433 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:26.537756 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.538686 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.523936 32185 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:26.510686 32443 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.541373 21427 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:26.525156 32319 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:26.529467 32427 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:26.547106 32427 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:26.551761 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.553710 32185 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:26.554592 32185 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:26.559392 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:26.554896 20876 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.544238 32215 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:26.565029 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.544572 32317 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:26.569082 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.567216 20879 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
W20260709 12:26:26.565595 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75160.1us
I20260709 12:26:26.584767 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:26.574481 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.468046 32362 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 36.21, Last appended by leader: 21, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.590870 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:26.593710 20808 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:26.594626 20881 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:26.570094 32215 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:26.569360 32317 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:26.604352 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:26.606555 32495 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:26:26.609206 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.609843 20735 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:26.610565 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.563951 32391 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.603641 32215 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:26.608026 32317 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.625828 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.627967 20809 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:26.631690 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.567471 32418 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.636873 20734 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:26.638692 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.641059 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.644624 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:26.645301 20727 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:26.648154 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:26.627588 32434 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.016s
I20260709 12:26:26.650700 20735 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:26.653035 32311 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.006s
W20260709 12:26:26.653661 21435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 2173) took 1652 ms (client timeout 1619 ms). Trace:
W20260709 12:26:26.654090 21435 rpcz_store.cc:269] 0709 12:26:25.001132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.019454 (+ 18322us) service_pool.cc:224] Handling call
0709 12:26:26.653251 (+1633797us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400645ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.653477 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574267648}
I20260709 12:26:26.649052 32431 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:26.655918 32431 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.656817 21431 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
W20260709 12:26:26.667229 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 56047.8us
I20260709 12:26:26.668310 20731 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39. Ops: [38.22-38.22]
I20260709 12:26:26.672878 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:26.675786 20878 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:26.679878 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:26.687729 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.689133 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:26.690462 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58940.5us
W20260709 12:26:26.692835 32496 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:26:26.695853 32478 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:26.696533 32478 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.698850 32478 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:26.701463 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:26.702759 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.703471 20804 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:26.708586 32437 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:26.713037 20801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4232) took 1161 ms. Trace:
I20260709 12:26:26.713431 20801 rpcz_store.cc:276] 0709 12:26:25.551858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.552662 (+   804us) service_pool.cc:224] Handling call
0709 12:26:26.060334 (+507672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.061202 (+   868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.061232 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.061953 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.151905 (+ 89952us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:26.244433 (+ 92528us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.301061 (+ 56628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.301169 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.301214 (+    45us) raft_consensus.cc:1685] finished
0709 12:26:26.301233 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.712732 (+411499us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54000a0850. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.712847 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92076,"prepare.queue_time_us":88016,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":84,"raft.queue_time_us":63642,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":3397,"spinlock_wait_cycles":1166566272,"thread_start_us":70892,"threads_started":1,"wal-append.queue_time_us":229255}
I20260709 12:26:26.718323 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.718658 32454 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:26.702872 32428 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.876s	user 0.006s	sys 0.000s
I20260709 12:26:26.725253 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.734442 20805 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:26.724363 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:26.723407 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.748536 20876 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:26.759709 21429 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:26.752890 20727 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:26.748343 32228 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:26.768116 20804 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:26.771503 20809 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:26.719118 32427 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.761896 32238 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:26.780956 20876 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:26.729233 32382 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.749219 32361 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.785193 20801 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.770700 32228 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:26.753559 32215 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.779388 32391 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:26.799722 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 126893us
I20260709 12:26:26.806489 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:26.805184 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 128287us
I20260709 12:26:26.810655 32500 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:26.810982 32510 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:26.811110 32361 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:26.812408 32438 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:26.815739 32228 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.817646 32228 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:26.818130 32212 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.818665 32212 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.821221 32212 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:26.829631 32215 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:26.830507 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4240) took 1267 ms. Trace:
I20260709 12:26:26.830991 20807 rpcz_store.cc:276] 0709 12:26:25.563037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.566473 (+  3436us) service_pool.cc:224] Handling call
0709 12:26:25.566732 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.963879 (+397147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.830167 (+866288us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400352ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:26.830431 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":707409664}
I20260709 12:26:26.833170 32427 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:26.834630 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.836493 32454 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:26.809898 32431 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:26.838092 32474 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.188s	user 0.001s	sys 0.002s
I20260709 12:26:26.838292 20882 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:26.855598 32431 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:26.855995 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.857234 32382 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:26.861415 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 117376us
I20260709 12:26:26.802515 32419 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.871757 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:26.876082 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69660.3us
I20260709 12:26:26.871677 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:26.884371 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3889) took 3440 ms. Trace:
I20260709 12:26:26.885003 20732 rpcz_store.cc:276] 0709 12:26:23.443910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.559223 (+115313us) service_pool.cc:224] Handling call
0709 12:26:25.284423 (+1725200us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ae9c 00007f9e1bb7aa5b 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:26:25.377025 (+ 92602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.377494 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.377523 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.377991 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.476013 (+ 98022us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:25.476058 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.476134 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.837298 (+1361164us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:26.883845 (+ 46547us) raft_consensus.cc:1685] finished
0709 12:26:26.883878 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.884130 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":472,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7342,"spinlock_wait_cycles":43534720,"thread_start_us":69451,"threads_started":1,"wal-append.queue_time_us":231950}
I20260709 12:26:26.889741 32419 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:26.899926 32438 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:26.900133 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:26.900599 32438 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:26.901424 32311 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:26.901638 32438 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.901937 32311 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:26.908524 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:26.910262 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:26.912181 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:26.915750 32392 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:26.920661 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:26.921710 32431 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:26.924124 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52564.4us
I20260709 12:26:26.925134 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:26.926235 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:26.927320 32427 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:26.927521 32434 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:26.927350 32382 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:26.928093 32434 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:26.928539 32382 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:26.928964 32382 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:26.929147 32434 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 37.20, Last appended by leader: 20, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:26.936194 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:26.937762 32427 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:26.938112 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:26.938299 32427 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:26.949105 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:26.949892 32427 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.901906 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:26.910596 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 4035) took 1043 ms. Trace:
I20260709 12:26:26.949914 20727 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:26.953792 20802 rpcz_store.cc:276] 0709 12:26:25.867306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.167105 (+299799us) service_pool.cc:224] Handling call
0709 12:26:26.829062 (+661957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.829324 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.829362 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.829438 (+    76us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:26.829482 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.829553 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.910427 (+ 80874us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b540019afd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:26.910529 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521410432}
I20260709 12:26:26.947117 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:26.948091 20881 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:26.938720 32362 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.010s
I20260709 12:26:26.955705 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 4124) took 1064 ms. Trace:
I20260709 12:26:26.957937 20801 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:26.956734 20881 rpcz_store.cc:276] 0709 12:26:25.891026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.543909 (+652883us) service_pool.cc:224] Handling call
0709 12:26:26.955650 (+411741us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:26.966712 32311 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.967442 32412 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.968258 32412 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.971426 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:26.956255 32238 consensus_queue.cc:1059] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.024s
W20260709 12:26:26.990407 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62878.8us
I20260709 12:26:26.981173 32390 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:27.000764 32441 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.001502 32441 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.006609 32392 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:27.011760 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 109942us
I20260709 12:26:27.016172 32412 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:26.979274 32317 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.018749 32317 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:27.016197 32489 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.934s	user 0.004s	sys 0.000s
I20260709 12:26:27.019891 32158 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:27.020480 32158 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.022795 32158 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:27.025597 20881 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
I20260709 12:26:27.027628 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:27.028813 20880 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 21. Preceding OpId from leader: term: 35 index: 21. (term mismatch)
I20260709 12:26:27.029348 20880 pending_rounds.cc:85] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 20
I20260709 12:26:27.029665 20880 pending_rounds.cc:107] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 34.21
I20260709 12:26:27.029989 20880 raft_consensus.cc:2889] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.989701 32431 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:26.993180 32361 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:27.041116 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:26.981770 32382 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:26.999372 32390 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:27.041800 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.077384 32390 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:27.077924 32437 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:27.078642 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4129) took 1185 ms. Trace:
I20260709 12:26:27.078717 32437 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.079344 20884 rpcz_store.cc:276] 0709 12:26:25.893305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.732219 (+838914us) service_pool.cc:224] Handling call
0709 12:26:26.732569 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.732872 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:26.861610 (+128738us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.077975 (+216365us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.078221 (+   246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:27.078272 (+    51us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:27.078318 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.078414 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.078572 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1057,"prepare.queue_time_us":150365,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":757,"thread_start_us":280635,"threads_started":2}
I20260709 12:26:27.081691 20884 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:27.081768 32437 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.083554 32185 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:27.084103 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.085042 32185 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:27.085575 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.086469 32185 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:27.036724 32238 consensus_queue.cc:1243] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 33.25 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 24
I20260709 12:26:27.089107 32472 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.089354 32215 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:27.089711 32472 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.090190 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:27.090747 20877 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:27.091859 32390 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.094835 32390 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:27.075955 32392 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.020s
I20260709 12:26:27.096261 32430 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.098030 32441 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.110957 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 2099) took 1225 ms. Trace:
I20260709 12:26:27.111047 20882 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:27.111686 20875 rpcz_store.cc:276] 0709 12:26:25.885871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.275299 (+389428us) service_pool.cc:224] Handling call
0709 12:26:27.110701 (+835402us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b5400171e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:27.110866 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662118144}
I20260709 12:26:27.112849 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4138) took 1203 ms. Trace:
I20260709 12:26:27.113330 20882 rpcz_store.cc:276] 0709 12:26:25.908947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.857220 (+948273us) service_pool.cc:224] Handling call
0709 12:26:27.110852 (+253632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.111865 (+  1013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.112614 (+   749us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400171e50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:27.112778 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582431104}
I20260709 12:26:27.117760 20877 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
W20260709 12:26:27.106932 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.140998 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.142768 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:27.143922 32259 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.144457 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:27.147855 32382 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.151214 32443 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.151386 32212 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.151968 32212 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.155409 20803 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:27.139271 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:27.073630 32361 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:27.154853 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2190) took 1642 ms. Trace:
W20260709 12:26:27.172595 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 201774us
I20260709 12:26:27.175637 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:27.182469 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.096323 32392 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:27.176239 21431 rpcz_store.cc:276] 0709 12:26:25.485512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.495518 (+ 10006us) service_pool.cc:224] Handling call
0709 12:26:26.656289 (+1160771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.657164 (+   875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.124854 (+467690us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400645ad0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:27.128286 (+  3432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2642552192}
I20260709 12:26:27.187774 20810 raft_consensus.cc:2393] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:27.151254 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:27.193210 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 106232us
W20260709 12:26:27.197687 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 259628us
I20260709 12:26:27.188967 21400 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:27.190261 32392 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:27.220840 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.215411 32441 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:27.213920 32212 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:27.226805 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 185481us
I20260709 12:26:27.215035 32382 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.239459 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88418.5us
W20260709 12:26:27.241477 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45611.9us
W20260709 12:26:27.219298 32452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.009s	sys 0.011s
I20260709 12:26:27.242429 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.244879 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:27.231943 32311 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:27.247509 20729 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:27.235950 32259 raft_consensus.cc:2764] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:26:27.258769 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:27.251322 32452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.009s	sys 0.011s
I20260709 12:26:27.266494 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:27.271116 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 82226.6us
I20260709 12:26:27.271814 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.268000 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:27.276651 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.277369 20731 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
I20260709 12:26:27.246510 32228 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:27.267711 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:27.256686 32332 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.248952 32311 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:27.287160 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.263602 32391 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:27.293762 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:27.323769 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67926.7us
W20260709 12:26:27.368462 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 81038.2us
I20260709 12:26:27.323503 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:27.259898 32382 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.372291 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:27.374079 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 173741us
I20260709 12:26:27.376199 20805 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:27.377012 21426 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:27.288821 32441 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.383040 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:27.373068 32526 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:27.286652 32434 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.384462 20728 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:26:27.316318 32311 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:27.385891 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.372026 32176 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:27.387504 32176 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.389874 32176 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:27.391817 32527 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:26:27.393947 32537 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:27.394783 32434 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.393947 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:27.398270 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73989.9us
I20260709 12:26:27.402026 20809 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:27.400416 32443 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.412973 32443 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.413559 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.261139 32361 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.325085 32332 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.298681 32228 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.417335 32332 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.419104 32228 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:27.427457 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.428910 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.430027 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.430544 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.442487 32431 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:27.436087 32332 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:27.444131 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.445577 21433 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:27.435223 32412 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.446765 32412 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.446815 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:27.447608 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64672.2us
I20260709 12:26:27.449369 32434 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.449440 32412 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.449723 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.451074 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2105) took 1101 ms. Trace:
I20260709 12:26:27.451458 20879 rpcz_store.cc:276] 0709 12:26:26.350225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.885509 (+535284us) service_pool.cc:224] Handling call
0709 12:26:26.885839 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.886232 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.886261 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.886769 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.055911 (+169142us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:27.055972 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.056078 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.450876 (+394798us) raft_consensus.cc:1685] finished
0709 12:26:27.450895 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.451020 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4422,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":168502,"threads_started":1,"wal-append.queue_time_us":324948}
I20260709 12:26:27.473101 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:27.473862 21433 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:27.290395 32392 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.396620 32441 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.434558 32472 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.494858 32441 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.498230 32441 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:27.502386 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:27.507768 32392 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:27.510062 32530 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:26:27.513633 21435 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:27.516003 32382 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.484781 32427 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 12:26:27.492527 32472 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:27.524561 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.525259 32443 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.534248 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.534655 32419 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:27.535054 21432 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:27.536172 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.538693 21432 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
I20260709 12:26:27.535045 32419 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:27.540395 32419 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.541697 32419 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:27.545271 21431 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
W20260709 12:26:27.545795 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 132329us
I20260709 12:26:27.539161 21435 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:27.567930 21432 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:27.584074 20730 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
W20260709 12:26:27.591856 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 201443us
W20260709 12:26:27.611626 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.606738 32540 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.090s	user 0.005s	sys 0.001s
I20260709 12:26:27.613029 21433 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
W20260709 12:26:27.614878 32546 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.089s	user 0.002s	sys 0.005s
W20260709 12:26:27.616358 32545 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.086s	user 0.000s	sys 0.007s
W20260709 12:26:27.610849 32543 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.099s	user 0.003s	sys 0.004s
W20260709 12:26:27.620374 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.624267 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 181433us
I20260709 12:26:27.625560 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.615041 32472 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 34.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.012s
I20260709 12:26:27.635771 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.638811 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.637712 32176 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.641701 32176 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.643903 32176 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:27.643999 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 194366us
I20260709 12:26:27.644863 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:27.647624 32542 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.126s	user 0.003s	sys 0.004s
I20260709 12:26:27.650156 32319 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:26:27.655815 32548 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.090s	user 0.009s	sys 0.003s
I20260709 12:26:27.668078 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.636919 32554 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.038s
I20260709 12:26:27.638525 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.674541 20803 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:27.676308 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.679370 32447 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:27.679667 32324 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:27.630663 32438 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:27.683525 32549 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:27.684136 32549 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:27.684625 32549 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:27.709468 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64677.6us
I20260709 12:26:27.721704 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.744467 32537 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:27.745043 32537 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:27.745697 20776 leader_election.cc:400] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:27.746058 20776 leader_election.cc:403] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:27.745862 32537 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.747159 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.753257 32557 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:27.761595 32557 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:27.761773 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.763411 32441 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:27.776891 20805 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:27.774729 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.781878 32454 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:27.787349 32437 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:27.788067 32437 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.790709 32437 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.754961 32311 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.800091 32311 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:27.755254 32472 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:27.804512 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 94301.3us
I20260709 12:26:27.806555 20776 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:27.807067 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 193155us
W20260709 12:26:27.817889 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.818419 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.822284 20809 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 37 index: 22. (index mismatch)
I20260709 12:26:27.824509 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.832185 20776 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.817852 32472 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:27.834872 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:27.832856 20802 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:27.824231 32382 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.837219 20776 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.823132 32454 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:27.826767 32362 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:27.842521 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.845340 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.846781 32324 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:27.847859 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:27.838624 32557 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:27.851234 32557 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.851624 32557 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:27.853825 32479 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:27.854213 32479 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:27.855046 32479 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.857391 20804 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:27.837850 32441 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:27.861405 32362 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.833830 32549 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.863823 32362 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:27.864984 20806 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:27.865737 32324 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:27.866102 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:27.868204 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57961.3us
I20260709 12:26:27.869103 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.870673 32441 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:27.871104 32441 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:27.871515 32441 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:27.873884 32550 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:27.874948 32549 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:27.880702 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.839175 32472 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:27.875721 32447 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:27.881946 32472 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.875080 32550 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:27.883353 32550 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:27.884271 32550 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.875870 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:27.885610 32550 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:27.880820 32564 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:26:27.888197 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.889617 20804 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 33 index: 17. (index mismatch)
I20260709 12:26:27.888266 32447 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:27.859293 32547 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:27.894672 32447 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:27.895828 32447 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.900463 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.901712 21431 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:27.903658 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:27.904338 21429 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
W20260709 12:26:27.905686 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 511791us
I20260709 12:26:27.906105 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.906840 20732 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:27.907332 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.907613 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.908143 20726 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 24. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:26:27.908267 20728 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:27.908753 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.909343 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.911371 32215 raft_consensus.cc:2764] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 32: won
I20260709 12:26:27.912115 32215 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:27.912298 32441 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.913820 20733 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:27.914695 32441 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:27.888219 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.915655 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.915900 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.916431 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.917037 20729 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:27.917481 32324 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:27.918380 32549 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:27.918762 32549 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:27.919718 32549 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:27.921221 20776 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:27.921259 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:27.921590 20878 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:26:27.921911 32479 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:27.922132 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:27.922916 20877 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:27.923353 20701 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.924218 20882 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:26:27.912820 32215 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:27.925812 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:27.891386 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:27.910914 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:27.927165 32534 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:27.933168 20875 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33. Ops: []
W20260709 12:26:27.934530 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71423.7us
I20260709 12:26:27.934669 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:27.935356 20733 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:27.935694 32418 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:27.936458 32418 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 29.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20260709 12:26:27.938405 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 90651.5us
I20260709 12:26:27.944249 20730 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:27.950300 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2108) took 1599 ms. Trace:
I20260709 12:26:27.952399 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:27.924253 32549 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:27.956879 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:27.953418 20877 raft_consensus.cc:2368] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33.
I20260709 12:26:27.950855 20876 rpcz_store.cc:276] 0709 12:26:26.351154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.903469 (+552315us) service_pool.cc:224] Handling call
0709 12:26:26.903896 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:26.919619 (+ 15723us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:27.042382 (+122763us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.375357 (+332975us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.849742 (+474385us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b1800315940. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:27.903916 (+ 54174us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b1800315940. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:26:27.904085 (+   169us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:27.904662 (+   577us) log.cc:844] Serialized 52 byte log entry
0709 12:26:27.904842 (+   180us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:27.905092 (+   250us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.945061 (+ 39969us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.949963 (+  4902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.950037 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.950077 (+    40us) raft_consensus.cc:1685] finished
0709 12:26:27.950088 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.950243 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45857,"prepare.queue_time_us":134259,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":3849,"raft.queue_time_us":569228,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":74556,"spinlock_wait_cycles":21287424,"thread_start_us":357365,"threads_started":2,"wal-append.queue_time_us":387859}
I20260709 12:26:27.922168 32324 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:27.949723 32259 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:26:27.960180 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89771us
I20260709 12:26:27.918813 32316 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:27.963078 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:27.966217 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:27.950181 32534 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:27.973153 20878 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 24. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:26:27.893150 32382 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:27.950733 32441 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:27.953860 32185 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:27.972172 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:27.961365 32324 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.962168 32454 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.014881 32324 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:28.024338 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 99543.8us
I20260709 12:26:27.962074 32259 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:28.014936 32566 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 12:26:28.001268 32534 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:28.015987 32562 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 12:26:28.016516 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 105664us
I20260709 12:26:28.020954 32557 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:28.038096 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.057581 20805 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 36 index: 25. (index mismatch)
I20260709 12:26:28.039220 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:28.074210 20806 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 19. Preceding OpId from leader: term: 38 index: 21. (index mismatch)
I20260709 12:26:28.006384 32441 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:28.072136 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:28.040818 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:28.081849 21429 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
W20260709 12:26:28.090488 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49780.1us
I20260709 12:26:28.091616 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:28.040800 32534 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:28.030414 32215 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.098137 20735 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
I20260709 12:26:28.098404 20730 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:28.098764 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:28.100253 20728 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:28.101375 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.105031 32533 raft_consensus.cc:2764] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:26:28.089007 32441 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:28.117583 20731 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:28.088593 32547 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:28.096907 32158 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:28.074584 32443 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.036s
W20260709 12:26:28.149941 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141020us
W20260709 12:26:28.152621 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58831.7us
I20260709 12:26:28.110612 32438 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:28.166429 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.167569 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:28.168954 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66398.4us
I20260709 12:26:28.128794 32158 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.146737 32537 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:28.029120 32454 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:28.118690 32215 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:28.184017 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:28.150696 32185 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:28.198369 21434 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:28.113165 32412 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 22, Time since last communication: 0.055s
I20260709 12:26:28.025102 32316 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.197530 32215 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:28.176316 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:28.200001 21432 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:26:28.202955 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.183926 32158 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:28.147665 32547 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:28.216326 20809 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:26:28.218519 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.208612 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:28.222697 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 46454.6us
I20260709 12:26:28.225805 21432 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:28.223778 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:28.229048 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.229804 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:28.230432 32227 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:26:28.232111 32227 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:28.232533 32227 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:28.232945 32227 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:28.220086 32392 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:28.150910 32259 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.225100 32215 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:28.243923 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:28.234799 32438 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:28.239542 32563 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:28.258669 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.261626 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:28.225070 32362 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 37.25, Next index: 26, Last known committed idx: 24, Time since last communication: 0.025s
I20260709 12:26:28.248832 32215 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:28.272205 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.242652 32412 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.016s
I20260709 12:26:28.253135 32438 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:28.273869 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 78822.4us
I20260709 12:26:28.280967 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:28.259670 32317 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.015s
I20260709 12:26:28.260318 32418 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:28.292155 32418 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.292518 32418 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:28.293175 32454 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.293743 32316 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.292454 32185 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:28.299224 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:28.300163 20876 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:28.273342 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:28.302189 32572 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.297s	user 0.002s	sys 0.002s
I20260709 12:26:28.303262 32438 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:28.304692 32569 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.304s	user 0.000s	sys 0.002s
I20260709 12:26:28.305099 20884 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 33 index: 17. (term mismatch)
I20260709 12:26:28.305459 20884 pending_rounds.cc:85] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 16
I20260709 12:26:28.305763 20884 pending_rounds.cc:107] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 31.17
I20260709 12:26:28.305976 20884 raft_consensus.cc:2889] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:28.262557 32391 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:28.245666 32521 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.023s
I20260709 12:26:28.281769 32556 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:28.282529 32547 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:28.282943 32571 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:28.248533 32434 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.023s
W20260709 12:26:28.283857 32573 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 12:26:28.284572 32576 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:28.288647 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:28.288946 32392 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:28.366627 32392 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:28.333040 32554 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:28.275048 32259 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:28.275621 32215 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:28.348924 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:28.376194 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:28.350841 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69957.3us
I20260709 12:26:28.380290 32454 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:28.352939 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.382354 20884 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:28.311164 32563 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:28.383119 32563 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:28.308115 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.384049 32563 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 37.25, Last appended by leader: 25, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:28.384784 20881 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:28.262785 32557 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.253432 32447 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.060s
I20260709 12:26:28.363867 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.386313 20878 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:28.386375 32414 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
W20260709 12:26:28.366791 32572 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 12:26:28.367129 32392 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:28.391294 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:28.392726 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.392863 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.361951 32583 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.091s
I20260709 12:26:28.360867 32441 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.404810 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:28.385284 32563 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:28.361882 32324 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.421131 32324 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.411721 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:28.409492 21432 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:28.428380 32438 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.430133 20801 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:28.438133 32382 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.453063 20809 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
W20260709 12:26:28.470785 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 239910us
I20260709 12:26:28.452183 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:28.481609 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76920.4us
I20260709 12:26:28.480010 20880 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:28.482573 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:28.485541 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.494158 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:28.438176 32438 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:28.501529 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 264202us
I20260709 12:26:28.451835 32419 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:28.505931 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.508180 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:28.492301 32437 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.516472 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:28.495043 32228 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:28.521730 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 114792us
I20260709 12:26:28.497160 32311 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:28.524662 20880 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:28.500726 32583 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
W20260709 12:26:28.524821 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 277657us
W20260709 12:26:28.515208 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 252733us
I20260709 12:26:28.566793 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:28.544397 20808 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: [30.20-30.20]
I20260709 12:26:28.504563 32185 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.029s
I20260709 12:26:28.512293 32579 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:28.526379 32228 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.586084 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:28.515674 32549 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.052s
I20260709 12:26:28.510116 32215 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.593055 32412 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:28.526729 32583 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:28.523595 32550 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.063s
I20260709 12:26:28.525998 32382 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:26:28.516037 32227 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.601104 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.598474 32583 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:28.524549 32419 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.586026 32418 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:28.610273 32185 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:28.594096 32533 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:28.613543 20882 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
I20260709 12:26:28.636361 20882 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 18
I20260709 12:26:28.636621 20882 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:26:28.636997 20882 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:28.615747 32549 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:28.617106 32550 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:28.629271 20880 raft_consensus.cc:2410] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 35 because replica has last-logged OpId of term: 32 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 18.
I20260709 12:26:28.592069 32228 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:28.527575 32437 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.645516 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:28.635989 32185 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:28.648842 32587 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:26:28.650911 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 130975us
I20260709 12:26:28.542332 32454 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.088s
I20260709 12:26:28.531152 32392 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.542558 32311 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.650807 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:28.654812 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 146716us
W20260709 12:26:28.667467 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 75464.8us
I20260709 12:26:28.671380 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:28.636997 32332 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:28.653898 32585 log.cc:927] Time spent T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
W20260709 12:26:28.675233 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 131994us
I20260709 12:26:28.674356 20881 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:28.680680 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:28.668497 32591 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:28.687412 32591 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.688220 32438 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:28.688681 32438 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:28.689042 32438 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:28.691577 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.692237 20882 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 39.
I20260709 12:26:28.692937 32579 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:28.693197 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:28.693748 20882 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:28.694380 32215 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:28.695541 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.696143 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.574s (SENT) [suppressed 21 similar messages]
I20260709 12:26:28.644027 32549 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:28.699572 20699 leader_election.cc:400] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33.
I20260709 12:26:28.700022 20699 leader_election.cc:403] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
I20260709 12:26:28.651912 32550 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:28.702725 20877 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:28.706166 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.704062 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.611016 32534 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.706858 20880 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:28.712405 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:28.712613 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.713236 20881 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:28.669690 32556 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:28.667053 32594 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:28.727470 32594 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:28.727965 32594 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:28.728938 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.729611 20876 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:28.730551 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.717020 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:28.675055 32533 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.042s
I20260709 12:26:28.717945 32549 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:28.718617 32534 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.718783 32437 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.707692 32332 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.719281 32215 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:28.708165 32382 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:26:28.696859 32591 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.723852 32441 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.770797 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:28.771003 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:28.771785 20804 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:28.771967 20735 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:26:28.773105 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:28.774209 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:28.775303 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:28.726256 32550 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:28.719374 32361 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:28.744385 20880 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:28.750952 20881 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:28.756343 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:28.756579 20801 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:26:28.715924 20699 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:28.712194 32537 raft_consensus.cc:3060] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:28.767834 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:28.769027 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:28.766877 20882 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:28.770084 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:28.770153 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:28.770438 32259 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.792611 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 76786.1us
I20260709 12:26:28.776677 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:28.724279 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.731164 20876 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:28.779206 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:28.781764 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:28.790827 20807 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:28.787034 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.793093 20802 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:28.796006 20881 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:28.796509 20881 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:28.793864 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:28.697258 32158 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.742606 20879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:28.743413 32430 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:28.766090 32500 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:28.797562 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.769201 32437 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:28.823731 20879 raft_consensus.cc:606] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:28.748334 32434 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:26:28.826880 20879 raft_consensus.cc:614] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:28.740342 32228 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.794450 20884 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:28.824862 20882 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:28.795588 20849 connection.cc:256] RPC call timeout handler was delayed by -1.435700289000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 12:26:28.736613 32583 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.739638 32594 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:28.835300 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 149687us
I20260709 12:26:28.838157 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:28.843617 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:28.839972 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:28.793144 32591 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:28.795251 32391 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.898365 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 202237us
W20260709 12:26:28.898524 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58662.1us
I20260709 12:26:28.909907 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:28.910813 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:28.798655 32317 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
W20260709 12:26:28.908632 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65142.4us
I20260709 12:26:28.908807 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:28.912853 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.912732 32361 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:28.835676 32550 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:28.780289 32185 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.840008 32549 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:28.897933 32500 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.908078 32176 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:26:28.792891 32561 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.239s
I20260709 12:26:28.896245 32382 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:28.941370 20734 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:28.943410 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 259106us
I20260709 12:26:28.942195 20876 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 36 index: 25. (index mismatch)
I20260709 12:26:28.925436 20880 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:28.947041 20880 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:28.947228 20804 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:28.934026 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:28.798328 32259 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:28.935317 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.951721 32176 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:28.952118 32176 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:28.899829 32591 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:28.956236 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 118164us
I20260709 12:26:28.942472 32185 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:28.835155 32437 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.918885 32534 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:28.964951 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.953094 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:28.944089 32588 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:28.926796 32418 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:28.921640 32228 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.923739 32392 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.922663 32332 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.927435 32556 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:28.964145 32597 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:28.971403 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:28.933053 32158 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.925093 32594 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.917707 32454 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.046s
I20260709 12:26:28.976946 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:28.981098 20732 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:28.937319 32500 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:28.960547 32176 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:28.918715 32391 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:28.978662 32588 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:28.974927 32604 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:26:28.966862 32311 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.968710 32227 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:29.000339 32606 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:26:29.011154 20875 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: []
W20260709 12:26:29.013669 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.013849 32437 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:28.971972 32534 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:28.970052 20879 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:29.025228 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:28.982410 32599 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:28.990082 32215 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.037164 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:28.990669 32556 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.064072 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:28.986272 32438 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.023933 32588 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:29.068135 20733 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 21. Preceding OpId from leader: term: 39 index: 22. (index mismatch)
I20260709 12:26:29.071535 20731 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:29.070093 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:29.069172 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 81544.9us
I20260709 12:26:29.072921 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:29.035777 32534 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:29.016785 32597 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.064429 32556 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:29.057677 32600 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:29.080181 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:29.076200 32324 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:29.072062 32430 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.073915 32599 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:29.097823 20731 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 34 index: 19. (index mismatch)
W20260709 12:26:29.091329 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 123016us
I20260709 12:26:29.067894 32215 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.102119 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:29.102780 21429 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:29.107285 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:29.113610 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.114179 20884 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:29.086042 32595 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:29.095834 32557 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:29.117374 21427 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 30: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:29.031569 32591 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 38.22, Last appended by leader: 22, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.090544 32332 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:29.103125 32594 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.126814 32594 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.127980 32591 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:29.131376 32332 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.132973 32609 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.195569 21429 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:29.133471 32391 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.117398 32324 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:29.135648 32601 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:26:29.125378 32617 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:29.097855 32561 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:29.119968 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:29.099772 32599 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:29.136698 32603 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:26:29.121773 32595 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:29.171420 32555 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Stepping down as leader of term 35
W20260709 12:26:29.172205 32619 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:29.172806 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.117306 32583 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.174983 32259 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.176369 32215 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.177613 32438 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.103431 32430 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.177938 32392 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.181195 32176 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.184281 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:29.184599 32227 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.185101 32598 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:29.185160 32443 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:29.144985 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:29.146363 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:29.142330 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.193843 21431 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:29.282362 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 137456us
I20260709 12:26:29.141095 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.283785 20878 raft_consensus.cc:2376] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32. Re-sending same reply.
I20260709 12:26:29.161288 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.285504 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.286497 20881 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:29.286962 20881 raft_consensus.cc:606] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:29.329000 20881 raft_consensus.cc:614] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:29.329898 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.200762 32550 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:29.202702 32419 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.184963 32611 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.331669 32611 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.120204 32557 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.332966 32557 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:29.216243 21428 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:29.228355 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:29.334506 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106269us
I20260709 12:26:29.231884 20726 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:29.244149 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.336127 20879 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:29.336176 32479 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:29.337327 32589 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:29.337416 32556 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.338670 20728 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: [39.23-39.23]
I20260709 12:26:29.246320 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:29.339471 21429 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:29.339633 32625 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:29.338685 32589 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.002s
I20260709 12:26:29.341262 32589 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:29.341620 32589 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:29.342406 32589 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 21, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.246989 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.200217 32418 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.343734 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56065.6us
I20260709 12:26:29.343978 32418 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.344640 32317 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:29.345667 32317 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:29.254768 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.351727 20880 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:29.258180 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.352847 21435 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:29.244076 32599 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:29.261121 32537 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33.
I20260709 12:26:29.243587 32561 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.224150 32617 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:29.190593 32534 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.271982 32332 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.272696 32324 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.248077 32555 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:29.149873 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.284693 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.283066 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.302734 20875 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 37 index: 22. (index mismatch)
I20260709 12:26:29.303378 32212 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:29.245577 32549 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.031s
W20260709 12:26:29.304831 32590 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.657s	user 0.000s	sys 0.010s
I20260709 12:26:29.305387 32176 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.306346 32586 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:29.307070 32430 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.225870 32609 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.389772 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.389971 32609 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.224892 32613 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.075s
I20260709 12:26:29.392117 32613 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:29.365757 32441 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 38.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.027s
I20260709 12:26:29.392484 32613 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:29.393748 32382 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:29.393410 32613 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.308305 32597 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.309695 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2118) took 1351 ms. Trace:
I20260709 12:26:29.394937 20729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:29.396020 20729 raft_consensus.cc:606] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:29.396340 20729 raft_consensus.cc:614] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:29.312141 32594 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.313419 32577 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:29.313975 32595 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.24, Last appended by leader: 24, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.395092 32555 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 21, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.275358 32563 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.285389 32215 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:29.274725 32392 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.366968 32414 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.038s
I20260709 12:26:29.415830 32414 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:29.416158 32561 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.416761 32392 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.382941 32447 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.034s
I20260709 12:26:29.417667 32447 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:29.320067 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2122) took 1012 ms. Trace:
I20260709 12:26:29.418063 32534 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.418282 20877 rpcz_store.cc:276] 0709 12:26:28.307242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.431225 (+123983us) service_pool.cc:224] Handling call
0709 12:26:28.431589 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.706614 (+275025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.319822 (+613208us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540005b754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:29.319999 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623118592}
I20260709 12:26:29.320937 32583 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:29.322104 20809 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:29.322679 21434 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:29.324098 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:29.324445 20882 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
W20260709 12:26:29.326265 20626 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 274) took 131522 ms (client timeout 15000 ms). Trace:
W20260709 12:26:29.196646 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 82925.4us
I20260709 12:26:29.346588 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.343654 21433 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:29.347332 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.347625 32550 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:29.343662 32589 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:29.349671 20884 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:29.353943 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:29.428871 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75009.9us
I20260709 12:26:29.429582 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:29.430073 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.355157 20881 raft_consensus.cc:2410] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 40 because replica has last-logged OpId of term: 38 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 21.
I20260709 12:26:29.431442 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:29.432173 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.410785 32555 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:29.433303 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:29.434047 20730 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:29.358511 20879 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:29.361668 21429 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:29.364585 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:29.347424 32588 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:29.437729 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 113696us
I20260709 12:26:29.432271 20728 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:29.439009 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.439771 20881 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:29.386178 32611 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.383013 32586 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:29.394177 32382 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:29.399446 20735 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:29.404481 32577 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:29.401217 32479 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.008s
I20260709 12:26:29.500957 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.416172 32414 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:29.372274 32617 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:29.384635 32418 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.398890 20883 rpcz_store.cc:276] 0709 12:26:27.958089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.962098 (+  4009us) service_pool.cc:224] Handling call
0709 12:26:28.001914 (+ 39816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.027544 (+ 25630us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:26:28.071543 (+ 43999us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.161679 (+ 90136us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:28.161767 (+    88us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:28.161832 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.309472 (+1147640us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b540021cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:29.309526 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.309641 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52120,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":3011,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":9891,"spinlock_wait_cycles":58246400,"thread_start_us":89638,"threads_started":1,"wal-append.queue_time_us":128703,"wal-append.run_cpu_time_us":1918,"wal-append.run_wall_time_us":1155844}
I20260709 12:26:29.414002 32563 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.284627 32227 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:29.434990 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70540.5us
I20260709 12:26:29.508459 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.484688 32534 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:29.436445 20880 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 39.
I20260709 12:26:29.376974 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:29.496029 32561 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:29.440256 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:29.442109 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.379418 20878 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:29.443367 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.489897 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:29.492551 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:29.492563 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:29.487669 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48951.4us
I20260709 12:26:29.524187 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:29.485071 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.497851 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.499377 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:29.381008 32472 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:26:29.413345 32556 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:29.407563 32613 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:29.418109 32447 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:29.422056 32583 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:29.505393 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:29.553454 20877 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:29.555613 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4212) took 1070 ms. Trace:
W20260709 12:26:29.506966 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 69946us
I20260709 12:26:29.443418 32215 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.412694 32537 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.088s
I20260709 12:26:29.487366 32609 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:29.438162 32550 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:29.510239 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:29.435231 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:29.439040 20727 raft_consensus.cc:2468] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:29.519590 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.496321 32392 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:29.521963 20875 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:29.526436 20884 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:29.534134 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44310.7us
I20260709 12:26:29.536031 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:29.544183 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.541910 32158 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:29.545996 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:29.546573 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:29.546767 32227 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:29.547649 20734 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:29.548877 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:29.549039 32586 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.549700 32622 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
W20260709 12:26:29.421838 20626 rpcz_store.cc:269] 0709 12:24:17.803394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.803693 (+   299us) service_pool.cc:224] Handling call
0709 12:26:29.326018 (+131522325us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":628472,"reported_tablets":48}
I20260709 12:26:29.586174 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.550529 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.552363 21427 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:29.556119 20878 rpcz_store.cc:276] 0709 12:26:28.485659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.554575 (+1068916us) service_pool.cc:224] Handling call
0709 12:26:29.554937 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.555151 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.555222 (+    71us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.555281 (+    59us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:29.555325 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.555394 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.555542 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:29.559940 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 183064us
I20260709 12:26:29.588331 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:29.561916 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:29.561501 20876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:29.568542 20810 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:26:29.545866 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:29.589112 20876 raft_consensus.cc:606] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:29.579589 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:29.589504 20876 raft_consensus.cc:614] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:29.579838 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 69667.5us
I20260709 12:26:29.590359 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:29.584599 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:29.588457 20878 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:29.592921 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:29.582314 32228 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:26:29.598582 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.603763 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.567232 32441 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:29.612922 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4215) took 1050 ms. Trace:
I20260709 12:26:29.613430 20878 rpcz_store.cc:276] 0709 12:26:28.543023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.587990 (+1044967us) service_pool.cc:224] Handling call
0709 12:26:29.588265 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.593196 (+  4931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.593378 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:29.618248 32631 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:26:29.619839 32600 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.620442 32600 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.621261 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.622390 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.622653 32600 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.618131 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.623865 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:29.624297 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.624941 20810 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:29.537159 32418 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.036s
I20260709 12:26:29.567565 32609 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.622613 20730 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:29.632551 20884 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:29.638263 32609 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:29.638968 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.633463 32595 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:29.566416 32215 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.633188 32259 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.015s
I20260709 12:26:29.541213 32616 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 38.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.099s
I20260709 12:26:29.592279 32392 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:29.591177 32316 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:29.652531 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:29.652666 20809 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: [30.20-30.20]
W20260709 12:26:29.670516 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.627728 32441 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.585363 32622 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:29.612571 32185 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:29.785845 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.655612 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:29.608752 32228 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.611961 32391 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:29.661331 32586 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:29.665882 21431 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:26:29.663949 32215 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:29.658314 20877 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:26:29.691576 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.646605 32158 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
W20260709 12:26:29.665266 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.636682 32454 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:29.813565 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:29.838887 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 244821us
I20260709 12:26:29.839833 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.844636 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:29.846462 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.847882 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.848731 20877 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:29.849911 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:29.783303 32441 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:29.852984 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.868717 21432 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.22-32.22]
I20260709 12:26:29.868856 21431 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 38. Current term is 39. Ops: []
W20260709 12:26:29.809983 32627 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:26:29.851470 32588 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:29.804452 32185 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.873656 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.607601 32382 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.879099 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:29.811947 32228 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:29.884307 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:29.626103 32414 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.24, Last appended by leader: 24, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.626811 32447 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 39.23, Last appended by leader: 23, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:29.887861 32391 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:29.853348 32600 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:29.846396 32215 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.645684 32583 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.964016 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:29.878563 32454 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:29.882512 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:29.678414 32550 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.611649 32577 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:29.946405 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 96588.5us
I20260709 12:26:29.894300 32185 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:29.979553 32447 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:26:29.984261 32643 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:26:29.985013 32600 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:29.988209 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:29.889889 32418 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:29.954342 20877 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:26:29.955618 32639 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:26:29.946086 32392 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:29.959867 32637 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 12:26:29.960820 32640 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:26:29.962913 32629 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.548s	user 0.007s	sys 0.004s
I20260709 12:26:29.892274 32595 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:29.991874 20849 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:29.994658 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4217) took 1371 ms. Trace:
I20260709 12:26:30.008931 20877 rpcz_store.cc:276] 0709 12:26:28.623761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.938396 (+1314635us) service_pool.cc:224] Handling call
0709 12:26:29.950417 (+ 12021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.994371 (+ 43954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.994606 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:29.965181 32589 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 12:26:30.002753 32651 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:29.936430 32622 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.003918 32563 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.012709 20877 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:30.009147 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:30.013401 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4153) took 1349 ms. Trace:
I20260709 12:26:30.013831 20877 rpcz_store.cc:276] 0709 12:26:28.664198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.010496 (+1346298us) service_pool.cc:224] Handling call
0709 12:26:30.012537 (+  2041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.013166 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.013348 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:29.998003 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 115718us
I20260709 12:26:30.017127 20877 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:30.015642 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:30.017853 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4154) took 1353 ms. Trace:
I20260709 12:26:30.018302 20877 rpcz_store.cc:276] 0709 12:26:28.664447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.015781 (+1351334us) service_pool.cc:224] Handling call
0709 12:26:30.016957 (+  1176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.017619 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.017803 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:29.954571 32382 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:30.010782 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134321us
I20260709 12:26:30.013226 32418 raft_consensus.cc:2804] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:29.997295 32412 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.022718 32454 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.016933 32651 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:30.029078 32636 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:30.029577 20803 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:30.029723 32598 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:30.030378 32653 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.001391 32550 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:30.011734 32557 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.678s	user 0.000s	sys 0.010s
I20260709 12:26:30.034229 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:30.036015 20877 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 19. (term mismatch)
I20260709 12:26:29.968559 32215 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.036414 20882 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:30.048715 20877 pending_rounds.cc:85] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 18
I20260709 12:26:30.030946 32622 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:30.038846 32554 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:30.033553 32391 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:30.038012 32534 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:30.035054 32653 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.049868 20875 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.20-32.20]
I20260709 12:26:30.007179 32600 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.022342 32316 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:30.056360 20877 pending_rounds.cc:107] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 32.19
I20260709 12:26:30.071589 20877 raft_consensus.cc:2889] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.073017 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4157) took 1407 ms. Trace:
I20260709 12:26:30.073509 20877 rpcz_store.cc:276] 0709 12:26:28.665275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.019172 (+1353897us) service_pool.cc:224] Handling call
0709 12:26:30.035691 (+ 16519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.072591 (+ 36900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.072841 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.052466 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:30.060511 32158 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:30.030925 32418 raft_consensus.cc:697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:30.086298 32158 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.086380 32418 consensus_queue.cc:237] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.076771 32316 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:30.090416 32316 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:30.079517 32634 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.315s	user 0.000s	sys 0.002s
I20260709 12:26:30.081195 32185 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:30.082691 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.083523 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 4075) took 1013 ms. Trace:
I20260709 12:26:30.056555 32419 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:30.085286 32215 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:30.095984 32215 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:30.096472 32215 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.035606 32412 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.064363 32554 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:30.050957 32557 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.071806 32534 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:30.072016 32391 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:30.117653 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.103489 32443 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:30.170135 32443 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.097836 32185 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:30.169297 32583 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:30.170605 32443 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:30.096799 32653 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:30.100373 32617 consensus_meta.cc:287] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.000s
I20260709 12:26:30.171888 20801 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:30.172281 32215 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:30.171638 32617 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.093904 20805 rpcz_store.cc:276] 0709 12:26:29.069963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.070086 (+   123us) service_pool.cc:224] Handling call
0709 12:26:29.084305 (+ 14219us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:29.367907 (+283602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.367949 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:29.382287 (+ 14338us) log.cc:844] Serialized 52 byte log entry
0709 12:26:29.551594 (+169307us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:29.551640 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.551730 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.083264 (+531534us) raft_consensus.cc:1685] finished
0709 12:26:30.083286 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.083453 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15785,"raft.queue_time_us":19872,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":2612,"thread_start_us":399253,"threads_started":2,"wal-append.queue_time_us":246870}
W20260709 12:26:30.176754 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 156205us
I20260709 12:26:30.093858 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4159) took 1428 ms. Trace:
I20260709 12:26:30.178581 20875 rpcz_store.cc:276] 0709 12:26:28.665770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.049178 (+1383408us) service_pool.cc:224] Handling call
0709 12:26:30.049571 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.093574 (+ 44003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.093801 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.168494 32391 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.180233 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.113494 32595 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.179451 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4216) took 1583 ms. Trace:
I20260709 12:26:30.193063 20876 rpcz_store.cc:276] 0709 12:26:28.595758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.635988 (+1040230us) service_pool.cc:224] Handling call
0709 12:26:29.670126 (+ 34138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.670376 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:30.050224 (+379848us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.178404 (+128180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.178466 (+    62us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:30.178515 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.179245 (+   730us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.179371 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372005,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":156,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":342730,"threads_started":2}
I20260709 12:26:30.174153 32557 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.106323 32609 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.185626 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.186169 32215 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.200095 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:30.201262 32443 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.201799 32534 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.202711 32617 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:30.187109 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.200114 32215 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.204504 32443 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.204717 32609 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.188442 32554 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 38.22, Last appended by leader: 22, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.205607 32554 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:30.206051 32215 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.065392 32228 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:30.192114 32555 consensus_meta.cc:287] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.000s
W20260709 12:26:30.153793 32599 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.754s	user 0.003s	sys 0.007s
I20260709 12:26:30.210179 20875 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:30.210572 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.211061 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4163) took 1501 ms. Trace:
I20260709 12:26:30.211272 20803 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:30.211563 20875 rpcz_store.cc:276] 0709 12:26:28.710099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.195578 (+1485479us) service_pool.cc:224] Handling call
0709 12:26:30.209926 (+ 14348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.210804 (+   878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.211010 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.215071 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:30.180661 32441 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.216876 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.207854 32391 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:30.203862 32534 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.228442 20809 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:30.217971 32259 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.229694 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:30.229717 32259 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.232849 32259 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:30.235456 32228 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.217557 32599 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.238443 32599 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:30.229866 32613 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.004s
I20260709 12:26:30.228814 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:30.241868 20877 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:30.242127 20875 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:30.244884 21429 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:30.216243 32617 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 12:26:30.247952 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:30.247511 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 195110us
W20260709 12:26:30.253351 32641 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
I20260709 12:26:30.254942 32412 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.256364 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2219) took 1049 ms. Trace:
I20260709 12:26:30.256474 32441 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.256790 21426 rpcz_store.cc:276] 0709 12:26:29.206555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.206753 (+   198us) service_pool.cc:224] Handling call
0709 12:26:29.244822 (+ 38069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.388823 (+144001us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:29.570324 (+181501us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.845678 (+275354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.845745 (+    67us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:29.845806 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.256122 (+410316us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54005df7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:30.256185 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.256312 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18432,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":51,"raft.queue_time_us":34772,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":6590,"spinlock_wait_cycles":349285376,"thread_start_us":455162,"threads_started":2,"wal-append.queue_time_us":397342}
I20260709 12:26:30.254999 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4213) took 1745 ms. Trace:
I20260709 12:26:30.259843 20882 rpcz_store.cc:276] 0709 12:26:28.509236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.547655 (+1038419us) service_pool.cc:224] Handling call
0709 12:26:30.036252 (+488597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.055801 (+ 19549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.254808 (+199007us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b5400306450. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:30.254945 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122734848}
I20260709 12:26:30.260995 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:30.261541 20882 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:30.264385 20810 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 40 because replica has last-logged OpId of term: 39 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 35 index: 22.
I20260709 12:26:30.269143 20875 raft_consensus.cc:2468] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:30.270258 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:30.270829 20875 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:30.276046 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:30.277885 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.281605 32651 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:30.282590 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.283540 20805 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:30.284263 32561 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:30.285318 32561 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.21, Last appended by leader: 21, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.287077 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:30.287395 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 729412us
I20260709 12:26:30.289696 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.290740 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:30.291986 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4224) took 1208 ms. Trace:
I20260709 12:26:30.292420 20884 rpcz_store.cc:276] 0709 12:26:29.083421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.291136 (+1207715us) service_pool.cc:224] Handling call
0709 12:26:30.291461 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.291646 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.291681 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.291729 (+    48us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:30.291756 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.291817 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.291931 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.293421 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:30.294301 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.294646 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:30.295434 32317 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
W20260709 12:26:30.295567 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 860349us
I20260709 12:26:30.295933 32317 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.296252 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.296325 32317 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:30.297761 20729 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:30.298959 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:30.299338 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 750502us
I20260709 12:26:30.299587 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.300258 20728 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
W20260709 12:26:30.300838 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.295073 32556 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:30.302282 32556 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:30.302711 32612 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:30.303210 32612 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:30.303619 32612 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:30.304291 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.296484 32549 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:30.305289 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 4241) took 1552 ms. Trace:
I20260709 12:26:30.303541 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.305657 20727 rpcz_store.cc:276] 0709 12:26:28.753107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.767911 (+ 14804us) service_pool.cc:224] Handling call
0709 12:26:30.305132 (+1537221us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54003aded4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:30.305235 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324673408}
I20260709 12:26:30.306231 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.306511 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.306857 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.307416 20730 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 33 because replica has last-logged OpId of term: 31 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 16.
I20260709 12:26:30.302857 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:30.308030 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.315294 20734 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38. Ops: [37.25-37.25]
I20260709 12:26:30.316031 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.316314 32434 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.316829 20733 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:30.316779 32434 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.302735 32556 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:30.318783 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.318594 32556 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.313289 32549 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:30.319979 32419 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:30.320230 32549 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.321290 32549 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:30.322966 32646 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:30.323324 32419 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.323516 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.323873 32419 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:30.324750 32642 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 12:26:30.323551 32646 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.326018 32419 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:30.326408 32437 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:30.326779 32437 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:30.327543 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.327651 32437 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.328807 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.329401 20732 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:30.318943 32434 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.332074 32646 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.334415 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:30.340903 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:30.345525 32617 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:30.345767 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.346962 21428 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:30.347414 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:30.348038 21426 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
W20260709 12:26:30.348353 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.349176 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54623.9us
I20260709 12:26:30.354348 20882 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:30.356176 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.357324 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:30.357328 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:30.358307 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55536.8us
W20260709 12:26:30.359226 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51253.4us
I20260709 12:26:30.360734 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.361243 32434 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:30.361523 20810 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:30.361765 32434 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:30.362893 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.362641 32434 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.363363 32563 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:30.364176 32563 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.364655 32563 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.366025 32579 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:30.366539 32579 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:30.366652 32563 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:30.366927 32650 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:30.367481 32579 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 38.22, Last appended by leader: 22, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.371487 20880 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:30.378103 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:30.376425 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:30.369829 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:30.379523 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4231) took 1053 ms. Trace:
I20260709 12:26:30.379886 20883 rpcz_store.cc:276] 0709 12:26:29.325808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.527833 (+202025us) service_pool.cc:224] Handling call
0709 12:26:30.371067 (+843234us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54001ff254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:30.379464 (+  8397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448136960}
I20260709 12:26:30.382750 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:30.383167 21430 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:30.383582 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:30.386546 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.387725 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.393235 21435 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: [28.19-28.19]
I20260709 12:26:30.394174 20728 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:30.390915 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:30.390265 21399 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:30.396317 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:30.397280 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:30.399905 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.402679 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4162) took 1692 ms. Trace:
I20260709 12:26:30.398290 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:30.403740 20877 rpcz_store.cc:276] 0709 12:26:28.709797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.074529 (+1364732us) service_pool.cc:224] Handling call
0709 12:26:30.094564 (+ 20035us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.242324 (+147760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.402456 (+160132us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400268254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:30.402623 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338309376}
I20260709 12:26:30.402653 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.410403 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:30.411758 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:30.412009 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:30.414783 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:30.414583 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:30.415517 21431 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:30.404903 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:30.416337 20877 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:30.422067 21428 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:30.422544 21435 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:30.389340 32598 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:30.426375 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.393950 32391 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:30.424180 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:30.400045 32500 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:30.428786 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:30.399804 32158 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:30.392204 32419 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.038s
I20260709 12:26:30.440116 32419 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:30.440461 32419 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:30.434513 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4225) took 1350 ms. Trace:
I20260709 12:26:30.429662 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.410799 32613 raft_consensus.cc:2804] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:30.410669 32594 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.444770 20884 rpcz_store.cc:276] 0709 12:26:29.083695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.293383 (+1209688us) service_pool.cc:224] Handling call
0709 12:26:30.293674 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.311451 (+ 17777us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:30.328080 (+ 16629us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.434094 (+106014us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.434160 (+    66us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:30.434224 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.434300 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.434453 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16204,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":4840,"thread_start_us":105638,"threads_started":1}
I20260709 12:26:30.447022 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:30.447695 20875 raft_consensus.cc:606] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:30.448416 20875 raft_consensus.cc:614] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:30.449757 20882 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:30.450490 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4228) took 1355 ms. Trace:
I20260709 12:26:30.450989 20882 rpcz_store.cc:276] 0709 12:26:29.095057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.436014 (+1340957us) service_pool.cc:224] Handling call
0709 12:26:30.449589 (+ 13575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.450248 (+   659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.450439 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.400199 32382 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:26:30.425913 32228 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:30.407666 32587 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.034s
I20260709 12:26:30.398936 32361 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.040s
I20260709 12:26:30.427423 32598 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:30.430928 32391 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:30.475091 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.438540 32158 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:30.446478 32609 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:30.495849 32418 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:30.428567 32362 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.042s
I20260709 12:26:30.449970 32500 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:30.492990 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:30.517488 32609 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:30.521597 32609 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.490887 32598 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:30.495545 32259 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.498750 32158 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:30.504276 20884 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
W20260709 12:26:30.504638 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 83554.2us
I20260709 12:26:30.451025 32593 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.049s
I20260709 12:26:30.490634 32391 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:30.527179 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:30.492578 32613 raft_consensus.cc:697] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:30.536571 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:30.561889 32645 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.559s	user 0.002s	sys 0.000s
I20260709 12:26:30.572111 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.575435 32613 consensus_queue.cc:237] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.576378 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.577206 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:30.577733 20877 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 38. Current term is 39. Ops: [38.22-38.22]
I20260709 12:26:30.578091 32653 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:30.576450 32215 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:30.578788 32185 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:30.578878 32215 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:30.579176 32651 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:30.579372 32215 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:30.523460 32418 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.579746 32651 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:30.580649 32651 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:30.581670 32635 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.584s	user 0.001s	sys 0.001s
I20260709 12:26:30.581893 32651 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:30.577620 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.582077 32418 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.583820 32418 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:30.585132 32612 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.586485 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.527999 32586 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.062s
I20260709 12:26:30.587554 32215 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:30.589462 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.589994 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.590382 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.590507 32215 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.594981 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.596251 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:30.575002 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6fe576036c04c149240cd6107285eca"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:30.523356 32259 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.597236 20878 raft_consensus.cc:606] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:30.605723 20878 raft_consensus.cc:614] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:30.548689 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4223) took 1465 ms. Trace:
I20260709 12:26:30.606581 20881 rpcz_store.cc:276] 0709 12:26:29.083114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.288178 (+1205064us) service_pool.cc:224] Handling call
0709 12:26:30.288548 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.288935 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.288967 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.289507 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.391692 (+102185us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:30.391731 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.391794 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.548404 (+156610us) raft_consensus.cc:1685] finished
0709 12:26:30.548441 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.548613 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16780,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":18963,"thread_start_us":101760,"threads_started":1,"wal-append.queue_time_us":211680}
I20260709 12:26:30.599257 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.523342 32663 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:30.546271 32660 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:30.523276 32500 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.22, Last appended by leader: 22, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.549724 20732 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:26:30.578384 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4165) took 1464 ms. Trace:
I20260709 12:26:30.555768 20882 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: [28.16-30.17]
I20260709 12:26:30.617462 20882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4229) took 1513 ms. Trace:
I20260709 12:26:30.617799 20882 rpcz_store.cc:276] 0709 12:26:29.104179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.554841 (+1450662us) service_pool.cc:224] Handling call
0709 12:26:30.555390 (+   549us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.617188 (+ 61798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.617404 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.616608 20877 rpcz_store.cc:276] 0709 12:26:29.113493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.577167 (+1463674us) service_pool.cc:224] Handling call
0709 12:26:30.577488 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.578144 (+   656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.578333 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.559028 20875 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 22. Preceding OpId from leader: term: 39 index: 22. (term mismatch)
I20260709 12:26:30.619802 20875 pending_rounds.cc:85] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 21
I20260709 12:26:30.620131 20875 pending_rounds.cc:107] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 38.22
I20260709 12:26:30.620455 20875 raft_consensus.cc:2889] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:30.604686 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 74639.7us
I20260709 12:26:30.604740 20728 raft_consensus.cc:2393] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:30.624635 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:30.625612 21429 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 37: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:30.609360 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:30.609998 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 185066us
I20260709 12:26:30.456735 32594 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.614733 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.629581 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:30.630043 32617 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:30.630744 32673 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:30.528332 32552 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:30.631332 32552 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:30.632179 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:30.632498 32552 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.633256 32673 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:30.633733 32673 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.620229 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:30.634159 32673 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:30.579077 32437 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:30.616906 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:30.640199 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4164) took 1528 ms. Trace:
I20260709 12:26:30.527258 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2140) took 1425 ms. Trace:
I20260709 12:26:30.646652 20884 rpcz_store.cc:276] 0709 12:26:29.101791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.503534 (+1401743us) service_pool.cc:224] Handling call
0709 12:26:30.504040 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.526940 (+ 22900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.527172 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.647778 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:30.648362 20884 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:30.641733 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.560424 32577 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:30.659354 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.646309 20875 rpcz_store.cc:276] 0709 12:26:29.111577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.558264 (+1446687us) service_pool.cc:224] Handling call
0709 12:26:30.558652 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.639909 (+ 81257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.640134 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.576526 32316 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:30.528884 32228 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:30.652966 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:30.655208 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.539820 32646 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:30.662200 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:30.680318 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.661334 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.477604 32419 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.25, Last appended by leader: 25, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.669559 20878 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: [35.21-35.21]
I20260709 12:26:30.661386 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:30.676625 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4220) took 1759 ms. Trace:
I20260709 12:26:30.634150 32552 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:30.702419 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49538.6us
I20260709 12:26:30.608398 32612 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:30.566383 32537 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:30.696923 20884 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:30.613471 32658 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
W20260709 12:26:30.686434 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.681515 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.698675 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4166) took 1581 ms. Trace:
I20260709 12:26:30.711715 20878 rpcz_store.cc:276] 0709 12:26:29.117117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.651894 (+1534777us) service_pool.cc:224] Handling call
0709 12:26:30.652306 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.698372 (+ 46066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.698613 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.713279 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2149) took 1050 ms. Trace:
I20260709 12:26:30.713627 20878 rpcz_store.cc:276] 0709 12:26:29.662898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.712459 (+1049561us) service_pool.cc:224] Handling call
0709 12:26:30.712779 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.712979 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.713002 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.713047 (+    45us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:30.713076 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.713126 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.713231 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.701069 20881 raft_consensus.cc:2393] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:30.635969 32663 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:30.684479 20876 rpcz_store.cc:276] 0709 12:26:28.917192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.194813 (+1277621us) service_pool.cc:224] Handling call
0709 12:26:30.200146 (+  5333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.215874 (+ 15728us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:30.216961 (+  1087us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.451599 (+234638us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.470757 (+ 19158us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.470932 (+   175us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:30.576304 (+105372us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b18003a3000. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:30.647016 (+ 70712us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.675915 (+ 28899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.676045 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.676276 (+   231us) raft_consensus.cc:1685] finished
0709 12:26:30.676315 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.676520 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10532,"prepare.queue_time_us":95608,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":73,"raft.queue_time_us":44759,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":3690,"spinlock_wait_cycles":12916096,"thread_start_us":386038,"threads_started":2,"wal-append.queue_time_us":325765}
W20260709 12:26:30.705914 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.706118 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:30.719659 20735 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:30.661420 32660 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:30.541333 32317 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.615607 32382 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:30.716130 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.661588 32650 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:30.644531 32441 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.083s
I20260709 12:26:30.642401 32447 consensus_queue.cc:1059] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.074s
I20260709 12:26:30.699635 20880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4227) took 1608 ms. Trace:
I20260709 12:26:30.678897 32228 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.617478 32622 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.094s
I20260709 12:26:30.678583 32591 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:30.716483 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 107187us
I20260709 12:26:30.743749 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.709192 32537 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:30.735606 20880 rpcz_store.cc:276] 0709 12:26:29.091561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.316477 (+1224916us) service_pool.cc:224] Handling call
0709 12:26:30.371281 (+ 54804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.372006 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.699375 (+327369us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b54001fe350. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:30.699568 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125313280}
W20260709 12:26:30.731817 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51126.8us
I20260709 12:26:30.714617 32600 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:30.731230 32583 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 12:26:30.799216 21432 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:30.726555 32419 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:30.719666 32653 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:30.719450 32479 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:30.802165 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64897.7us
I20260709 12:26:30.738126 32228 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:30.808954 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.739539 32616 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.732934 32362 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:30.806579 20880 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
W20260709 12:26:30.804601 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60923.9us
W20260709 12:26:30.811677 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 84719.6us
I20260709 12:26:30.815656 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:30.806428 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 137008us
I20260709 12:26:30.725871 32625 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.081s
I20260709 12:26:30.735970 32324 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 38.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.073s
I20260709 12:26:30.862010 20810 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:30.832834 20804 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:30.734989 32382 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.848701 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2137) took 1747 ms. Trace:
I20260709 12:26:30.848395 20809 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 38 index: 22. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
W20260709 12:26:30.876314 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60725.4us
I20260709 12:26:30.972378 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:30.969408 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.979720 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:30.970707 20883 rpcz_store.cc:276] 0709 12:26:29.101334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.439760 (+1338426us) service_pool.cc:224] Handling call
0709 12:26:30.440109 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.440472 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:26:30.440778 (+   306us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.564087 (+123309us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b18000e2000. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:30.567857 (+  3770us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b18000e2000. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f
0709 12:26:30.569577 (+  1720us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b4800240258. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:26:30.621355 (+ 51778us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.717851 (+ 96496us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.718290 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.718411 (+   121us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:30.718432 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.723713 (+  5281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.848460 (+124747us) raft_consensus.cc:1685] finished
0709 12:26:30.848478 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.848632 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6652,"prepare.queue_time_us":7176,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":13327,"raft.queue_time_us":29735,"raft.run_cpu_time_us":3818,"raft.run_wall_time_us":118647,"spinlock_wait_cycles":92918784,"thread_start_us":268373,"threads_started":2,"wal-append.queue_time_us":283211}
W20260709 12:26:30.914614 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.806939 32316 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:30.975244 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 276522us
W20260709 12:26:30.973431 21431 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.001s
I20260709 12:26:30.994951 21431 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:30.996809 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:30.869036 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52840.5us
I20260709 12:26:30.881143 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
W20260709 12:26:30.715454 32392 consensus_queue.cc:1175] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 38 status { last_received { term: 36 index: 24 } last_committed_idx: 24 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:30.845350 32549 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.524s	user 0.006s	sys 0.004s
I20260709 12:26:30.822861 32613 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.833313 32675 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:30.801527 32361 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:26:30.991199 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.808056 32600 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:30.998306 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:31.013728 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:30.867636 32595 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.148s
I20260709 12:26:30.850658 32557 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:30.819439 32454 raft_consensus.cc:3055] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:30.976632 32212 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:30.843425 32317 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.018172 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:30.822233 32589 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.187s
I20260709 12:26:30.833616 32649 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:30.967689 32646 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:31.058640 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:30.848404 32534 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:30.901248 32611 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:31.000394 32622 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:31.063870 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72961.4us
W20260709 12:26:31.076391 32643 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:26:31.016928 32644 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.048s	user 0.004s	sys 0.000s
I20260709 12:26:31.000969 32648 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:31.012858 32617 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:31.008894 32653 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:30.821606 32616 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:31.068825 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70634.5us
I20260709 12:26:30.841547 32660 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.059404 32311 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:31.081413 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 63390.6us
I20260709 12:26:30.835253 32479 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:30.825673 32362 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.094792 32534 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:31.111109 32598 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.003s
I20260709 12:26:31.091974 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:31.007268 32412 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.068s
I20260709 12:26:31.094751 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:30.823771 32434 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.197s
I20260709 12:26:31.075481 32316 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:30.851042 32537 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.080688 32324 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:31.012360 32443 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.067s
I20260709 12:26:31.072855 32454 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:31.053972 32613 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.069072 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.080054 32557 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.102240 32622 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.205279 32534 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 38.21, Last appended by leader: 21, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:31.208002 32679 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:26:31.112392 32382 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.104787 32617 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:31.087963 32651 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:31.114765 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:30.977360 32663 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.084219 32212 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:31.115478 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.126087 21434 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:31.142189 20880 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
W20260709 12:26:31.145746 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51125.3us
I20260709 12:26:31.276291 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:31.166289 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.181634 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.137565 32311 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.120168 32648 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.149555 32688 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.144755 32598 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.228668 32692 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:31.229316 32675 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.230158 32613 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.230901 32537 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:31.231571 32479 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.232569 32622 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.233979 32362 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.146436 32316 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:31.101861 32611 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:31.256693 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142001us
I20260709 12:26:31.333206 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:31.277943 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.334547 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.281842 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2150) took 1068 ms. Trace:
I20260709 12:26:31.274255 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2218) took 1846 ms. Trace:
I20260709 12:26:31.291824 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:31.291947 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:31.254952 32158 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 12:26:31.256157 32617 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:31.244153 32600 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.358911 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:31.348552 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56790.7us
I20260709 12:26:31.316864 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:31.100682 32649 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.256816 32382 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:31.292339 32589 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:31.413276 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 3964) took 1031 ms. Trace:
I20260709 12:26:31.426867 32361 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.331797 32418 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:31.346491 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.339799 20808 rpcz_store.cc:276] 0709 12:26:29.428281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.432163 (+  3882us) service_pool.cc:224] Handling call
0709 12:26:29.494246 (+ 62083us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.512610 (+ 18364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.512651 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.527520 (+ 14869us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.603926 (+ 76406us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:29.603987 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.604067 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.712618 (+1108551us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:31.253857 (+541239us) raft_consensus.cc:1685] finished
0709 12:26:31.253890 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.274152 (+ 20262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44876,"raft.queue_time_us":12296,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":31430,"spinlock_wait_cycles":10111232,"thread_start_us":34029,"threads_started":1,"wal-append.queue_time_us":256234}
I20260709 12:26:31.415707 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.296572 32651 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:31.315873 32692 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.405711 32212 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.338618 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:31.348872 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:31.447407 32651 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 21, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.448608 21431 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 32 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:26:31.447170 32212 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:31.333132 32454 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.451459 32589 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.452013 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:31.425789 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.407558 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.406145 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:31.447885 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109372us
I20260709 12:26:31.458647 20801 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:31.460049 20801 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 36 index: 22. (index mismatch)
I20260709 12:26:31.450385 32659 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:31.444486 20734 rpcz_store.cc:276] 0709 12:26:30.381880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.392254 (+ 10374us) service_pool.cc:224] Handling call
0709 12:26:30.407666 (+ 15412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.514330 (+106664us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:26:30.515842 (+  1512us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.800515 (+284673us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.805114 (+  4599us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.805583 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.805708 (+   125us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:30.805743 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.805801 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.413077 (+607276us) raft_consensus.cc:1685] finished
0709 12:26:31.413098 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.413223 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53289,"prepare.queue_time_us":26688,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":83184,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":18205,"spinlock_wait_cycles":24788608,"thread_start_us":320859,"threads_started":2,"wal-append.queue_time_us":496692}
I20260709 12:26:31.451962 32589 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.412468 20735 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 18. (term mismatch)
I20260709 12:26:31.365190 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:31.428227 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:31.365813 32622 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:31.421447 32625 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:31.451637 32212 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.445425 32361 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.454411 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:31.475518 20880 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:31.341872 20878 rpcz_store.cc:276] 0709 12:26:30.213483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.724835 (+511352us) service_pool.cc:224] Handling call
0709 12:26:30.742231 (+ 17396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.742718 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 25
0709 12:26:31.000604 (+257886us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.281358 (+280754us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.281435 (+    77us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:31.281509 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.281605 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.281768 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171142,"prepare.queue_time_us":264734,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":12530,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":16714,"thread_start_us":267827,"threads_started":1}
W20260709 12:26:31.476440 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 111375us
I20260709 12:26:31.297142 32534 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:31.461252 20801 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 22. (term mismatch)
I20260709 12:26:31.477214 20801 pending_rounds.cc:85] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 21
I20260709 12:26:31.477444 20801 pending_rounds.cc:107] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 32.22
I20260709 12:26:31.477658 20801 raft_consensus.cc:2889] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.370223 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.461863 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:31.445470 32600 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.465571 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.464834 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:31.507365 20804 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:31.474254 20735 pending_rounds.cc:85] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 17
I20260709 12:26:31.486701 32589 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.349050 32362 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:31.487857 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4235) took 1165 ms. Trace:
I20260709 12:26:31.488229 32622 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.444823 32382 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.327068 32675 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.444025 32158 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.490348 32692 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:31.491483 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.492399 32311 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:31.493443 32212 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.494961 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:31.496860 32419 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.680s	user 0.005s	sys 0.005s
I20260709 12:26:31.498015 32441 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Stepping down as leader of term 38
W20260709 12:26:31.498601 32437 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.830s	user 0.010s	sys 0.001s
I20260709 12:26:31.498957 20801 raft_consensus.cc:1275] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:26:31.406258 32557 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.500677 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.522136 32362 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:31.508283 20735 pending_rounds.cc:107] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 31.18
I20260709 12:26:31.559463 20735 raft_consensus.cc:2889] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.327303 32685 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:31.487344 32684 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:31.519678 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:31.562451 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:31.563120 20877 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:31.564365 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:31.520776 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.565490 21435 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:31.499989 32316 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:31.522389 20876 rpcz_store.cc:276] 0709 12:26:30.322282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.722800 (+400518us) service_pool.cc:224] Handling call
0709 12:26:30.724827 (+  2027us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.741790 (+ 16963us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:30.864819 (+123029us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18004f9240. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e226973c7 00007f9e1eca99a7 00007f9e1ecb90bf 00007f9e1ecb9038
0709 12:26:30.865207 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.303667 (+438460us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.321977 (+ 18310us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.322158 (+   181us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:31.322409 (+   251us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.322599 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.322681 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.487364 (+164683us) raft_consensus.cc:1685] finished
0709 12:26:31.487388 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.487786 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88870,"prepare.queue_time_us":160122,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":10159,"raft.queue_time_us":37066,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":37468,"spinlock_wait_cycles":2993024,"thread_start_us":427020,"threads_started":1,"wal-append.queue_time_us":619683}
I20260709 12:26:31.537421 20734 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:31.535782 32625 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:31.508076 32185 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:31.525889 32577 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.844s	user 0.008s	sys 0.003s
I20260709 12:26:31.544991 21429 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:31.543967 32441 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:31.464874 32418 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.669432 32441 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 38.22, Last appended by leader: 22, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.555090 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.670995 32696 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 38.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:31.308950 32688 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.529846 32646 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
W20260709 12:26:31.527932 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.612363 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:31.617587 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2142) took 2421 ms. Trace:
W20260709 12:26:31.627873 32673 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.943s	user 0.012s	sys 0.000s
I20260709 12:26:31.672964 20875 rpcz_store.cc:276] 0709 12:26:29.196471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.660503 (+1464032us) service_pool.cc:224] Handling call
0709 12:26:30.706806 (+ 46303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.081186 (+374380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.081217 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.089323 (+  8106us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.318612 (+229289us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:31.318666 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.318767 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.617337 (+298570us) raft_consensus.cc:1685] finished
0709 12:26:31.617368 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.617520 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56519,"raft.queue_time_us":20771,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7536,"thread_start_us":576721,"threads_started":2,"wal-append.queue_time_us":421674}
I20260709 12:26:31.619292 32675 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.523098 32600 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:31.640869 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74301.3us
W20260709 12:26:31.652494 32680 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:26:31.545224 32694 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.671407 20729 raft_consensus.cc:2410] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 32 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:26:31.671743 32418 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.615935 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:31.636579 20804 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:31.697324 20876 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:31.697944 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.674628 21433 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:31.609691 20877 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:31.683090 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.684795 32479 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:31.684850 20805 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:31.684793 32361 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.706828 20876 raft_consensus.cc:1240] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:31.707695 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4169) took 1384 ms. Trace:
I20260709 12:26:31.708108 20876 rpcz_store.cc:276] 0709 12:26:30.326916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.703299 (+1376383us) service_pool.cc:224] Handling call
0709 12:26:31.706604 (+  3305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.707409 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.707630 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.709470 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2157) took 1304 ms. Trace:
I20260709 12:26:31.709986 20877 rpcz_store.cc:276] 0709 12:26:30.405381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.706677 (+1301296us) service_pool.cc:224] Handling call
0709 12:26:31.707040 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.709043 (+  2003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.709085 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.709142 (+    57us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:31.709190 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.709266 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.709393 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.685601 32598 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.686605 32688 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:31.662353 32552 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.947s	user 0.010s	sys 0.005s
W20260709 12:26:31.650166 32681 log.cc:927] Time spent T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.341s	user 0.000s	sys 0.005s
W20260709 12:26:31.692099 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.687666 32594 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.665187 32649 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.712359 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:31.695293 32696 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.714447 20807 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:31.723639 20876 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:26:31.691906 32648 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:31.708025 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 247250us
I20260709 12:26:31.732546 20729 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:31.699719 32609 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.663172 32185 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.734835 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4174) took 1008 ms. Trace:
I20260709 12:26:31.730708 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:31.704846 32316 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:31.718122 20877 raft_consensus.cc:1240] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:26:31.651669 32684 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 12:26:31.739656 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.741019 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.750634 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:31.752238 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.742249 20734 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
W20260709 12:26:31.745208 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.743935 20876 rpcz_store.cc:276] 0709 12:26:30.726290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.709191 (+982901us) service_pool.cc:224] Handling call
0709 12:26:31.710243 (+  1052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.734515 (+ 24272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.734768 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.757920 20876 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:31.692879 32673 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.791862 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.783638 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.792793 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:31.793316 20734 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:31.771023 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:31.688467 32362 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 33.17, Last appended by leader: 17, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.800570 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.795432 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.742012 32611 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:31.795836 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.755447 32697 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:31.767402 32228 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.943s	user 0.009s	sys 0.003s
I20260709 12:26:31.800308 20875 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 37 index: 22. (index mismatch)
I20260709 12:26:31.744274 32648 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:31.780383 32324 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:31.730034 32418 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.037s
I20260709 12:26:31.802273 20735 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:31.714653 32479 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.764679 32595 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.703855 32694 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.813963 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.814961 20810 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:31.829988 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4175) took 1022 ms. Trace:
I20260709 12:26:31.804041 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:31.878649 20875 rpcz_store.cc:276] 0709 12:26:30.807744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.744999 (+937255us) service_pool.cc:224] Handling call
0709 12:26:31.772050 (+ 27051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.829631 (+ 57581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.829911 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:31.879721 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 75790.5us
I20260709 12:26:31.880707 20875 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
W20260709 12:26:31.864652 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.866441 20730 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:31.866619 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.742973 32696 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.756330 32651 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.063s
W20260709 12:26:31.875599 32700 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:26:31.877202 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.746001 32609 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.909528 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.816187 32646 consensus_queue.cc:1048] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.073s
W20260709 12:26:31.890401 32537 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.004s
I20260709 12:26:31.911145 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:31.919452 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.919838 20875 raft_consensus.cc:1240] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.20-32.20]
I20260709 12:26:31.795303 32675 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.083s
I20260709 12:26:31.925889 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:31.876173 32361 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.880645 32653 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:31.889086 32695 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:31.948207 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.954761 20804 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:31.951138 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:31.952283 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 185791us
W20260709 12:26:31.948022 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 177055us
I20260709 12:26:31.904434 32158 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:31.865943 32228 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.972026 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:31.979060 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.979735 20804 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:31.889851 32316 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.051s
I20260709 12:26:31.921187 32382 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:31.987357 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:31.869097 32648 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.978879 32185 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:31.985777 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:31.860862 32697 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:31.988160 20732 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:31.930708 32688 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:31.966537 32324 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:31.882154 32595 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:31.990161 20804 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 37 because replica has last-logged OpId of term: 35 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 20.
I20260709 12:26:31.938119 32692 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:31.981416 32718 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:31.958210 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:32.013795 20850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.019s	sys 0.035s
W20260709 12:26:32.018707 20850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.019s	sys 0.035s
I20260709 12:26:32.014719 21427 raft_consensus.cc:1240] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.20-32.20]
W20260709 12:26:32.020529 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48639.5us
W20260709 12:26:32.018867 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60774.3us
W20260709 12:26:32.022466 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 560640us
I20260709 12:26:32.028264 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:32.027081 20805 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:31.997686 32695 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:31.962997 32361 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.022249 32675 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:32.030820 32653 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.035159 32651 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:32.051968 32710 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:26:32.021203 32688 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:32.048250 32617 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.002s
I20260709 12:26:32.053398 32617 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.045127 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.055622 32617 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.056017 20876 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:32.056999 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.060307 32550 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:32.060891 32550 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:32.065603 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:32.036418 32718 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.038s
I20260709 12:26:32.033613 32185 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.082178 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:32.085922 20731 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:32.087699 20728 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:32.017540 32707 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.326s
I20260709 12:26:32.045178 32675 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.094719 32701 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:26:32.082710 32694 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.101701 20775 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:32.090135 32185 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.108253 32595 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.104839 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:32.104372 32324 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:32.111758 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.095355 32675 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:32.102766 20876 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:32.102293 32741 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:32.102995 32738 consensus_queue.cc:1059] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 34.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.003s
I20260709 12:26:32.103652 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.061975 32550 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.102329 32739 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.112295 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:32.103155 32727 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:32.109660 32414 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.077389 32653 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.117105 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:32.115041 32737 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.109627 32185 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:32.105341 32726 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:26:32.106961 32715 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:26:32.107479 32697 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.133693 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:32.133947 32696 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.113701 32479 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.109817 32648 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:32.124281 32702 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.291s	user 0.001s	sys 0.001s
I20260709 12:26:32.125845 32731 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:32.135267 32731 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:32.135751 32731 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.137686 32732 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
W20260709 12:26:32.139300 32717 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 12:26:32.139645 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1978) took 1513 ms. Trace:
I20260709 12:26:32.140002 20733 rpcz_store.cc:276] 0709 12:26:30.626565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.626685 (+   120us) service_pool.cc:224] Handling call
0709 12:26:32.138849 (+1512164us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:26:32.139042 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.139250 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.139287 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.139331 (+    44us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:32.139367 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.139460 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.139571 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5408128}
W20260709 12:26:32.124589 32719 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:26:32.142467 20808 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:32.143219 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:32.126236 32723 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:26:32.126297 32738 raft_consensus.cc:3055] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:32.121866 32718 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.128190 32550 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.148298 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:32.128379 32728 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:26:32.132949 32716 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:26:32.144243 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.144963 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:32.144773 32738 raft_consensus.cc:740] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:32.157478 32738 consensus_queue.cc:260] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 34.17, Last appended by leader: 17, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.158911 32738 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:32.145848 32549 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:32.165923 32549 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:32.166505 32549 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:32.107532 32695 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.168084 32600 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.169152 32609 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.169876 21426 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:32.171638 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2769) took 1766 ms. Trace:
I20260709 12:26:32.172034 21426 rpcz_store.cc:276] 0709 12:26:30.405550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.405738 (+   188us) service_pool.cc:224] Handling call
0709 12:26:30.406009 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.169703 (+1763694us) spinlock_profiling.cc:243] Waited 7.98 ms on lock 0x7b54004db0a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec16dc0 00007f9e1eca11c0 00007f9e1eca2231 00007f9e1ec9ed8f 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c
0709 12:26:32.170333 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.171383 (+  1050us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54004da8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:32.171580 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939298304}
I20260709 12:26:32.175760 32361 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.117044 32595 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:32.162424 32316 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:32.177583 32316 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:32.178834 32316 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.124753 32739 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.179947 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2775) took 1097 ms. Trace:
I20260709 12:26:32.180339 21427 rpcz_store.cc:276] 0709 12:26:31.082183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.082374 (+   191us) service_pool.cc:224] Handling call
0709 12:26:31.097708 (+ 15334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.020832 (+923124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.179706 (+158874us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b5400729a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:32.179893 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108108032}
I20260709 12:26:32.181118 32739 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:32.184758 32534 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.686s	user 0.011s	sys 0.000s
I20260709 12:26:32.188021 32556 raft_consensus.cc:3055] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:32.142627 32673 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:32.234522 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 86297us
I20260709 12:26:32.235632 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.188349 32549 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.188858 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2239) took 1565 ms. Trace:
I20260709 12:26:32.192708 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.239336 21432 rpcz_store.cc:276] 0709 12:26:30.623561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.623725 (+   164us) service_pool.cc:224] Handling call
0709 12:26:30.624115 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.805178 (+181063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.188606 (+1383428us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540049c854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:32.188773 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402121600}
I20260709 12:26:32.237999 20775 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.150274 32718 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:32.212350 32391 consensus_meta.cc:287] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.583s	user 0.015s	sys 0.001s
I20260709 12:26:32.207742 32382 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.254222 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.169920 32228 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.151846 32696 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:32.241122 32418 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.239382 32741 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:32.252538 32739 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.235278 32556 raft_consensus.cc:740] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:32.194316 32694 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:32.239179 32591 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.273648 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:32.248685 32392 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:32.254429 32732 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:32.149650 32737 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.249622 32731 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.257133 32648 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.013s
I20260709 12:26:32.263540 32646 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:32.259382 32391 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.283834 20732 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:32.276475 32732 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.262773 32382 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.282437 32648 consensus_queue.cc:1243] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 30.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:26:32.264606 32418 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.292093 20883 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:32.285688 32549 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.269523 32739 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.278779 32591 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.278960 32392 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.288614 32732 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:32.306725 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.278677 32694 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.314909 32418 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.317103 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1980) took 1690 ms. Trace:
I20260709 12:26:32.317577 20726 rpcz_store.cc:276] 0709 12:26:30.627180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.628837 (+  1657us) service_pool.cc:224] Handling call
0709 12:26:32.316267 (+1687430us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:26:32.316483 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.316678 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.316706 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.316753 (+    47us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:32.316787 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.316855 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.317031 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2800000}
I20260709 12:26:32.287302 32685 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.015s
I20260709 12:26:32.319749 32685 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:26:32.319818 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 2177) took 1937 ms (client timeout 2136 ms). Trace:
I20260709 12:26:32.320183 32685 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:32.320266 21433 rpcz_store.cc:269] 0709 12:26:30.382181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.383317 (+  1136us) service_pool.cc:224] Handling call
0709 12:26:32.319656 (+1936339us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400b6dfd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:32.319761 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2967172480}
W20260709 12:26:32.320899 32742 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:32.321182 32685 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.322525 32685 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:32.325107 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 1979) took 1698 ms. Trace:
I20260709 12:26:32.325543 20727 rpcz_store.cc:276] 0709 12:26:30.626841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.626968 (+   127us) service_pool.cc:224] Handling call
0709 12:26:30.751893 (+124925us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c78fc0 00007f9e23c63929 00007f9e23c633d2 00007f9e1d1dabd4
0709 12:26:30.799811 (+ 47918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.013652 (+213841us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:31.013978 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.291832 (+277854us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.291888 (+    56us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:31.291939 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.324830 (+1032891us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54002977d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:32.324886 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.325046 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4806,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":129829,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":44944,"spinlock_wait_cycles":496033792,"thread_start_us":277493,"threads_started":1,"wal-append.queue_time_us":551657}
I20260709 12:26:32.323058 32616 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.328176 32653 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.329619 32737 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.329941 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.330973 32718 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.331737 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.332139 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.333194 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.333775 21426 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:32.335264 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:32.336856 20731 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:32.341729 32391 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.341993 32609 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.284572 32646 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.342506 32609 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.342024 32537 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.002s
I20260709 12:26:32.344696 21432 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:32.332715 21427 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:32.347298 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:32.351701 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.352762 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.353428 20726 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:32.357920 32731 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.357991 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.358637 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.352363 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:32.359097 32735 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.003s
I20260709 12:26:32.362504 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:32.363291 21427 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:32.363471 32684 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.364003 32684 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.364526 32591 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.360615 21426 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:32.368080 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.368635 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:32.368824 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.370565 32674 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:32.371232 32684 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.371387 32722 consensus_queue.cc:1059] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:32.372269 32722 consensus_queue.cc:1243] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 50c1ab13351d4427b836fbb449c6d7bb log is divergent from this leader: its last log entry 30.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20260709 12:26:32.372511 32616 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.372742 32722 raft_consensus.cc:3055] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:32.373185 32722 raft_consensus.cc:740] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:32.373418 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.376099 21428 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:26:32.369762 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.378149 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.380070 32593 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:32.380373 32659 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.373540 32732 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.376025 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.380662 32593 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.371038 32674 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:32.384603 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:32.390774 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.390905 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.391940 20809 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:32.366685 32361 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:32.395493 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.397167 20807 raft_consensus.cc:2368] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39.
I20260709 12:26:32.399233 32646 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.400213 32732 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.401628 32259 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.389315 32534 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:32.402339 32739 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.402419 32534 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.402926 32534 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.395735 32707 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:32.399266 32587 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.007s
I20260709 12:26:32.405973 32707 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:32.406442 32707 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:32.384294 32718 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:32.398020 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:32.384025 32735 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:32.409207 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:32.382534 32731 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:32.397121 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.373023 32616 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.389123 32392 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.387722 32694 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.413972 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:32.394464 32561 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:32.410904 32735 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:32.414762 32718 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:32.416484 20883 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4237) took 1670 ms (client timeout 1914 ms). Trace:
I20260709 12:26:32.425701 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:32.426065 20883 rpcz_store.cc:269] 0709 12:26:30.745780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.982082 (+236302us) service_pool.cc:224] Handling call
0709 12:26:32.416284 (+1434202us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400267d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:32.416398 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821868800}
I20260709 12:26:32.417153 32593 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.380906 32659 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.374006 32722 consensus_queue.cc:260] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.419127 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.422194 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:32.421999 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.410807 32695 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.408259 32361 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:32.427696 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.424806 32674 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.429699 32722 raft_consensus.cc:3060] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:32.430513 32616 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.419586 32673 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:32.425494 32733 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:32.424093 32259 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:32.448042 32259 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.448480 32259 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:32.450796 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4176) took 1460 ms. Trace:
I20260709 12:26:32.434768 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.394835 32675 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.451730 20809 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:32.451978 20735 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.18-32.18]
I20260709 12:26:32.453599 32675 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.454141 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.411300 32731 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.454829 32731 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:32.455446 32684 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.455938 32684 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.458954 32673 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.459447 32673 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:32.435431 32609 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.484201 32685 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.484776 32685 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.442306 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.486797 32472 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:32.487216 32382 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.488286 32688 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.425458 32735 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:32.487262 32472 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:32.489595 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.490231 32472 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.491510 32472 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:32.491786 32622 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:32.433403 32686 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.366s	user 0.000s	sys 0.004s
I20260709 12:26:32.493221 20881 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:32.493760 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4286) took 2279 ms. Trace:
I20260709 12:26:32.493908 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2151) took 2278 ms. Trace:
I20260709 12:26:32.445220 32695 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.464046 32733 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.462845 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.466296 32698 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.468169 32259 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:32.471642 21434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 2163) took 2845 ms (client timeout 2002 ms). Trace:
I20260709 12:26:32.111383 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:32.462734 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.479334 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.487838 32732 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
W20260709 12:26:32.506940 21434 rpcz_store.cc:269] 0709 12:26:29.625717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.625993 (+   276us) service_pool.cc:224] Handling call
0709 12:26:32.471412 (+2845419us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400750154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:32.471535 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2346126208}
I20260709 12:26:32.509269 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.490432 32589 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:32.445971 21426 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 39: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:32.486843 20805 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:32.512820 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:32.494401 20881 rpcz_store.cc:276] 0709 12:26:30.215747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.492663 (+2276916us) service_pool.cc:224] Handling call
0709 12:26:32.493009 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.493666 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.493854 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.443166 20728 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:32.454800 20877 rpcz_store.cc:276] 0709 12:26:30.990276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.747351 (+757075us) service_pool.cc:224] Handling call
0709 12:26:31.747726 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.748184 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:26:31.777342 (+ 29158us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.312821 (+535479us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b18003c0400. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:26:32.405209 (+ 92388us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.405748 (+   539us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.406215 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.406341 (+   126us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:32.406371 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.406436 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.450593 (+ 44157us) raft_consensus.cc:1685] finished
0709 12:26:32.450613 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.450740 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2239,"prepare.queue_time_us":55712,"prepare.run_cpu_time_us":154,"prepare.run_wall_time_us":80,"raft.queue_time_us":8660,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":5740,"spinlock_wait_cycles":4311808,"thread_start_us":83431,"threads_started":1,"wal-append.queue_time_us":615090}
W20260709 12:26:32.499437 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.494158 20803 rpcz_store.cc:276] 0709 12:26:30.214167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.214318 (+   151us) service_pool.cc:224] Handling call
0709 12:26:30.214736 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.604150 (+389414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.604182 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.611732 (+  7550us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.802559 (+190827us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:30.802597 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.802661 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.432855 (+1630194us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:32.493446 (+ 60591us) raft_consensus.cc:1685] finished
0709 12:26:32.493469 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.493678 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":402619,"raft.queue_time_us":451641,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":23081,"spinlock_wait_cycles":9323264,"thread_start_us":165661,"threads_started":1,"wal-append.queue_time_us":268336}
I20260709 12:26:32.515215 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.516408 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:32.510896 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.517663 20876 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:32.518810 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.519624 20876 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:32.520046 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:32.520658 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.522112 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.522709 20880 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:32.523577 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.523720 20883 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 35. Ops: []
I20260709 12:26:32.524111 20880 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:32.524626 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.524755 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.525192 20881 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.505061 32733 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:32.525838 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.525218 20883 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:32.529086 20876 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:32.525453 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.531780 20882 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:32.531980 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.532562 20805 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:32.532389 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:32.533444 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.534279 20806 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:32.534001 20805 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:32.553920 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.515940 20877 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:32.554750 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.558279 20882 raft_consensus.cc:2393] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:32.557262 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.558472 20875 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:32.558683 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.559482 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.560950 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.561571 20877 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:32.558507 20805 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:32.564157 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.564440 20879 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
W20260709 12:26:32.566671 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.567698 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.573932 20881 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:32.574889 20876 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:32.513833 32316 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.535418 32317 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:32.581508 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.582444 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:32.539697 32688 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:32.529806 32259 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.508394 32698 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.507156 32447 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 12:26:32.588392 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.562346 32382 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:32.588680 20877 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 36 index: 22. (index mismatch)
I20260709 12:26:32.592044 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.592362 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.565613 32675 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:32.595252 20875 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:32.538331 32707 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:32.598788 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 129454us
I20260709 12:26:32.446657 32718 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.584213 32673 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.596024 32675 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.586737 32612 consensus_queue.cc:1059] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:26:32.612447 32589 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:32.586210 32316 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.622995 32589 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.631175 32589 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.632251 20730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 3972) took 1973 ms. Trace:
I20260709 12:26:32.632704 20730 rpcz_store.cc:276] 0709 12:26:30.659215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.661049 (+  1834us) service_pool.cc:224] Handling call
0709 12:26:32.632029 (+1970980us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54001d7c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:32.632186 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2717291776}
W20260709 12:26:32.647008 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.649691 32751 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
I20260709 12:26:32.650470 32673 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.652335 32554 consensus_queue.cc:1059] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:32.653400 32554 raft_consensus.cc:3055] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:32.653743 32554 raft_consensus.cc:740] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:32.659960 32707 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.622260 32675 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.602543 32622 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:26:32.669160 32316 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.670231 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:32.671124 21426 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:32.672197 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:32.675827 32659 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.615859 32591 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:32.677138 32472 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:32.676857 32591 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.679039 32591 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.683746 20883 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:32.688436 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:32.654690 32554 consensus_queue.cc:260] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.690047 32554 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:32.687387 32533 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:32.691877 32533 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.694051 32533 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:32.697669 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.698324 20882 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:32.698483 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.699076 32361 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.699267 20810 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:32.699110 20876 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:32.703115 21426 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:32.703488 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:32.705749 20882 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:32.707402 21427 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:32.708122 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.708433 20882 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 40 index: 23. (index mismatch)
I20260709 12:26:32.709188 21434 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:32.710080 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.702678 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.710712 20881 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:32.710794 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:32.712088 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:32.706324 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:32.714586 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.717796 20882 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:26:32.719554 20879 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:26:32.721835 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.722114 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.722510 20882 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 39.
I20260709 12:26:32.722661 20876 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:32.720474 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.726153 20875 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: [28.16-30.17]
I20260709 12:26:32.726078 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.727041 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:32.728641 32612 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:32.729789 32612 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:32.730228 32612 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:32.731379 32612 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.732411 32612 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:32.732753 32392 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.733264 32392 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.735391 32392 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:32.737978 20883 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:32.701463 32698 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.739900 20882 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:32.743819 20882 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:32.745208 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.745962 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:32.738989 32212 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:32.747951 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:32.748064 32212 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:32.748694 32212 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:32.749866 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:32.750981 20876 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
I20260709 12:26:32.753134 32737 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:32.753538 32737 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.753918 32737 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:32.755484 32549 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:32.756004 32549 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:32.756438 32549 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:32.724884 32712 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:32.760179 32712 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.760612 32712 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:32.727525 32722 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:32.687395 32472 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:32.750977 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:32.741739 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:32.797465 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:32.806386 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 55503.4us
I20260709 12:26:32.719309 32500 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:32.725282 32707 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:32.817024 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.822728 20883 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:32.824993 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:32.761384 32722 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.711676 32534 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.840152 20881 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
I20260709 12:26:32.740535 32735 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.826529 32555 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:32.838059 32591 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.849838 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:32.872041 20883 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:32.849649 32472 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.18, Last appended by leader: 18, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:32.850221 32317 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.861204 32554 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:32.826187 32731 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.820106 32673 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:32.822484 32707 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:32.866806 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.839733 32722 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:32.876447 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 134795us
I20260709 12:26:32.836130 32695 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:32.833730 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:32.887733 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:32.821396 32311 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.063s
I20260709 12:26:32.894593 32317 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:32.909392 20808 raft_consensus.cc:2368] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33.
I20260709 12:26:32.842481 32718 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.058s
I20260709 12:26:32.899782 32391 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.900064 32707 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.905743 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:32.917385 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 83765us
I20260709 12:26:32.936910 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:32.838759 32533 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.055s
I20260709 12:26:32.918291 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:32.891842 32316 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.920199 32414 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.921128 32650 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:32.923341 32721 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.908407 32593 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:32.904274 32555 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:32.910861 32692 raft_consensus.cc:2764] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election decision vote started in defunct term 38: won
I20260709 12:26:32.950035 32537 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:32.951452 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 184540us
I20260709 12:26:32.922528 32212 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.877704 32591 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:32.950392 32689 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.845s	user 0.002s	sys 0.014s
W20260709 12:26:32.997552 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.957806 32412 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.004040 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63087.1us
I20260709 12:26:33.004980 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:33.011174 20730 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 29: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:32.966089 32661 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:32.957343 32650 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.970245 32593 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:33.020443 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.953553 32361 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.053s
W20260709 12:26:33.019572 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.961787 32500 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:33.025732 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:32.962255 32316 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.019053 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:32.961292 32414 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:32.952505 32391 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.025790 32609 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:33.012217 32661 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.970427 32721 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.020232 32650 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:33.031862 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:33.033783 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:33.017298 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80229.2us
I20260709 12:26:33.034765 20730 raft_consensus.cc:1275] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:33.035920 20807 raft_consensus.cc:1240] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:33.036307 32557 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:33.038941 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:33.039731 32554 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:33.054692 32661 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:33.061435 32554 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.062110 32414 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:33.031719 20733 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:26:33.044031 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:33.053562 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.022565 32593 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:33.015550 32412 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.066550 32414 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:33.067939 32412 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.068944 32412 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:33.070835 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:33.056278 20729 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
W20260709 12:26:33.071651   300 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:26:33.072086 32412 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:33.072597 32412 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:33.073474 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.073446 32412 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:33.077834 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 108517us
I20260709 12:26:33.079563 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2144) took 3708 ms. Trace:
I20260709 12:26:33.079952 20884 rpcz_store.cc:276] 0709 12:26:29.350900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.704096 (+1353196us) service_pool.cc:224] Handling call
0709 12:26:30.704532 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.706647 (+  2115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.706693 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.732708 (+ 26015us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.831315 (+ 98607us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:30.831373 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.874021 (+ 42648us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.710122 (+1836101us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:33.048716 (+338594us) raft_consensus.cc:1685] finished
0709 12:26:33.048743 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.059108 (+ 10365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3256,"raft.queue_time_us":17186,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":8517,"spinlock_wait_cycles":4835072,"thread_start_us":21954,"threads_started":1,"wal-append.queue_time_us":282172}
I20260709 12:26:33.016093 32733 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.088856 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:33.091519 20734 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 33
I20260709 12:26:33.093605 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.048761 32609 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.095863 20884 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:33.096262 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.103726 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.104233 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:33.107671 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75885.7us
I20260709 12:26:33.108640 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:33.083974 32414 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:33.117782 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:33.049113 32673 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.057s
I20260709 12:26:33.136089 32317 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.105983 32609 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:33.114017 32766 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 12:26:33.104841   307 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.157235 32712 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.158396 20734 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
W20260709 12:26:33.159286 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73493.3us
I20260709 12:26:33.132915 32414 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:33.133642 32761 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:26:33.115511 32735 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.103724 32593 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.145732 32731 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.265352 32593 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:33.082265 32555 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.125067   303 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:33.269585   303 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.271541 32591 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:33.272066   303 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:33.274675 32762 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:26:33.274986 20805 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:33.276245   310 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.276778   310 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.278898   310 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.263450 32362 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:33.270565 32419 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:33.258874 32414 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:33.271395 32712 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.260488 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:33.286886 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.287654   313 consensus_queue.cc:1048] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:33.288453 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.290519 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.292580 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.302874 32661 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.307847   303 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.429422 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.309105 32684 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.310408 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.172415 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:33.311501 32609 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.311655 32556 consensus_queue.cc:260] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.313761 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.456534 20730 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 34 because replica has last-logged OpId of term: 32 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 19.
I20260709 12:26:33.315040 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.315531 32721 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.316067 32212 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.317246 32591 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.317684 32617 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.318300   309 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:33.319329 32733 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.327360 32692 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.328260 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.330456 32557 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:33.332018 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.478199 21426 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:33.479526 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.480574 21426 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:33.338557 32534 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.346478 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.354161 32441 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:33.381647 20880 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:33.388782 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:33.495128 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 106411us
W20260709 12:26:33.173591 32764 log.cc:927] Time spent T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 12:26:33.427825 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:33.428185 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.428690 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:33.435832 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.042s
W20260709 12:26:33.504400 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.000s	sys 0.043s
W20260709 12:26:33.505043 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 680088us
I20260709 12:26:33.506016 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:33.160272   304 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:33.257647 32317 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.441674 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:33.496896 32737 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.004s
W20260709 12:26:33.443528 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 271189us
W20260709 12:26:33.395871 32763 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 12:26:33.172679   301 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:33.449712 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.450803 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.520704 20878 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 30 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
W20260709 12:26:33.415779 32549 consensus_meta.cc:287] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.001s
W20260709 12:26:33.505673 32675 consensus_meta.cc:287] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.844s	user 0.004s	sys 0.007s
I20260709 12:26:33.426792 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:33.218145 32673 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.228s
I20260709 12:26:33.460808 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.463202 20727 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:33.464395 32316 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.464627 21427 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:33.395396 32419 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:33.468036 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:33.533032 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106378us
I20260709 12:26:33.402088 32577 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.069s
I20260709 12:26:33.475625 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.536916 20730 raft_consensus.cc:2410] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 31 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 20.
I20260709 12:26:33.471606 32692 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.484040 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.454694 32561 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:33.491549 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.465776 32609 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.497864 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:33.476145 32556 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:33.511009 21431 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:33.443106 20726 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:33.514829 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:33.514959 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.488229 32557 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:33.174786   307 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.475448   309 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.519656 32212 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:33.526647 32661 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.601997 20733 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:33.469231 32591 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.472172 32617 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.527781 32549 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.504621 32593 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:33.519826 32599 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:26:33.532827 20735 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:33.537175 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:33.616735 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 79667.7us
I20260709 12:26:33.570369 20809 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 39.
I20260709 12:26:33.168823 32500 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 21, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:33.587563 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 89813.8us
I20260709 12:26:33.623325 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:33.624571 21434 raft_consensus.cc:2410] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 31 because replica has last-logged OpId of term: 30 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:26:33.587145 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:33.539340 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:33.593178 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 78423.2us
I20260709 12:26:33.539642 32227 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:33.599305 20726 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:33.521258   301 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:33.600893 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:33.592584 32722 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.004s
I20260709 12:26:33.597575 20727 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: [35.21-35.21]
I20260709 12:26:33.646157 20804 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:33.551012   304 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:33.625697 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:33.626417 32317 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:33.625517 21430 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:33.657943 32317 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.658394 32317 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:33.658473 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:33.659884 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.660969 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.649844 32594 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.624758 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.591765   303 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.661931 20875 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 39: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:33.661638 32594 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.662966 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.663358   318 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.664005 32594 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.664021 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.665414 20875 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:33.610771 32212 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:33.601603 32441 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:33.671420 32617 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:33.615434 32419 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 35.21, Last appended by leader: 21, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.629999 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.632836 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:33.596057 32316 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:33.596341 32561 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.565842 32414 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.625937 32731 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.640349 20878 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:26:33.641481   308 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:33.677867 32561 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:33.650194 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.652985 32599 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:33.654170 21435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2179) took 3059 ms. Trace:
I20260709 12:26:33.654094 32703 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.654031 32695 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:33.649288 32227 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.654340 32688 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.656515 32718 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.657114 32479 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.615221 32593 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.628067 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:33.555692 32534 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.074s
I20260709 12:26:33.663898   318 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.671499 32447 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.612780 32612 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.880s	user 0.010s	sys 0.003s
I20260709 12:26:33.532347 32737 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.676859 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:33.727857 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 51087us
I20260709 12:26:33.536199 32675 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.678949 20875 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:33.714802 32661 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:33.651288 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.759660 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.717181 32593 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:33.684084 21435 rpcz_store.cc:276] 0709 12:26:30.595245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.598048 (+  2803us) service_pool.cc:224] Handling call
0709 12:26:30.619529 (+ 21481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.568919 (+949390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.653925 (+2085006us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b5400525154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:33.654089 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2159469440}
W20260709 12:26:33.689862 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.031s	sys 0.024s
W20260709 12:26:33.766806 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.032s	sys 0.024s
W20260709 12:26:33.767225 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 633188us
I20260709 12:26:33.656133   310 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:33.670747 32212 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.25, Last appended by leader: 25, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.674736 32714 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:33.695391 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.678892 32617 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:33.680616 32419 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:33.710824 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:33.713083 21431 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:33.688228 32599 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.712850 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:33.710306 32227 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:33.685943 32731 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.728034 32447 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:33.730700   323 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:33.813323 21433 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:33.732625 32316 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:33.736724 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.813534 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:33.814004 20878 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:33.741518 32595 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:33.744226 21434 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:33.714660 32688 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.752856 32414 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:33.715696 32554 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.661448 32722 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.760977 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:33.683943 21429 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:33.773463   318 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.776124 20699 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:33.826859 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50850.2us
I20260709 12:26:33.717681   321 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.828169   321 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.830262   321 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.698897 32695 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.692699 32703 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.779458 20727 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:33.811983 32441 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:26:33.778348 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a9c17d2fc98f481597ce7b97e8f98c43 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 30 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:33.727631   304 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:33.842856 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67349.9us
W20260709 12:26:33.764868 32650 consensus_meta.cc:287] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.003s
I20260709 12:26:33.786240 32454 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:33.787840 32587 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:33.788939   307 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.788844 32317 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.792807 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:33.794900 20733 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:33.801628 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.717547 32479 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.808014 20732 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
W20260709 12:26:33.776528   315 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 12:26:33.808504 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 95753.5us
I20260709 12:26:33.880254 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:33.717383 32718 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.809727 21431 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:33.732002 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:33.806994 32391 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.810046 32599 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:33.909374 20809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 4116) took 1493 ms (client timeout 1563 ms). Trace:
W20260709 12:26:33.909788 20809 rpcz_store.cc:269] 0709 12:26:32.416344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.452416 (+ 36072us) service_pool.cc:224] Handling call
0709 12:26:33.909220 (+1456804us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540027bd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:33.909319 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2565408512}
W20260709 12:26:33.913111 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.820808 21435 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:33.822276 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:33.915844   307 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.837610 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.916790 20876 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:33.916492   307 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.917855 20876 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:33.918602 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.919235 20810 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
W20260709 12:26:33.822914 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 194938us
I20260709 12:26:33.840418 21429 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:33.783582   310 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.816180 32625 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:33.823621   328 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:26:33.814340 32591 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:33.816604 32316 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.867033 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:33.874440 20728 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:33.818701 32692 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.842875 32441 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:33.800858 32714 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.884419 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:33.887176 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.889410 32479 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.891031 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.892462 32688 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.899559 32447 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.898453 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:33.848536 32362 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:33.901779 20730 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: [32.21-32.21]
W20260709 12:26:33.904130 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 172205us
I20260709 12:26:33.907327 32685 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:33.908054   325 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 12:26:33.915889 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 93684.8us
I20260709 12:26:33.782071 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.920425 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:33.920753 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:33.921855 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:33.922443 32732 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:33.923614 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.924672 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.926131 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.850512 32549 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.930130 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.933669 20881 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [30.17-32.18]
I20260709 12:26:33.934648 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.934525 32316 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:33.958379 20806 raft_consensus.cc:2410] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 33 because replica has last-logged OpId of term: 32 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 19.
I20260709 12:26:33.957639 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:33.836369 32554 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.949199 20727 raft_consensus.cc:2368] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:33.952092 20730 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:33.955482 20884 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 39.
I20260709 12:26:33.926362 32617 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:33.959250 20805 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:33.960378 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.960974 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.961208 20807 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 39.
I20260709 12:26:33.961352 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:33.962033 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:33.940871 20774 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:33.962744 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.964356 20801 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
W20260709 12:26:33.967417 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46731.8us
I20260709 12:26:33.969741 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:33.971050 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 49303.4us
I20260709 12:26:33.972084 20849 leader_election.cc:400] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39.
I20260709 12:26:33.937541   307 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.974319 20804 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:33.963698 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:33.980841 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:33.983111 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4197) took 1250 ms. Trace:
I20260709 12:26:33.983649 20879 rpcz_store.cc:276] 0709 12:26:32.732996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.764065 (+ 31069us) service_pool.cc:224] Handling call
0709 12:26:32.806254 (+ 42189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.810591 (+  4337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.810638 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.811242 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.285133 (+473891us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b180038b300. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:33.591044 (+305911us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:33.591084 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.591152 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.974186 (+383034us) raft_consensus.cc:1685] finished
0709 12:26:33.974207 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.983025 (+  8818us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3845,"raft.queue_time_us":3963,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7437,"spinlock_wait_cycles":12463488,"thread_start_us":305723,"threads_started":1,"wal-append.queue_time_us":881227}
W20260709 12:26:33.989086 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48697.3us
I20260709 12:26:33.990125 20877 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:33.990339 32554 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:33.990842 32731 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:33.852026 32650 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.942670   320 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:33.979112 32698 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.952822 32534 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:33.991874 32589 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:33.992609   305 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:33.990864 32595 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:33.932859 32625 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:33.994983 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.991487 20877 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:33.996392 32479 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:33.997673 32311 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.020s
I20260709 12:26:34.014761 32441 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.015141 32557 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:33.965711 32362 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:34.002019 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:33.993927 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:33.948454 32612 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.049s
I20260709 12:26:34.011238 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:33.966182   309 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.001214 32589 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.974089 32447 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.035s
I20260709 12:26:33.974526 32732 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.024704 32595 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:34.025019 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.025151 32554 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.025557 32722 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.025835 20801 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:34.020190 21430 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 40: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:34.001412   305 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.022066 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:34.005975 32737 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.027824 32391 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:34.022069 20880 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:34.033430 20805 raft_consensus.cc:2410] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 30 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:26:34.010807 32675 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.045521 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2774) took 3227 ms. Trace:
I20260709 12:26:34.137771 21428 rpcz_store.cc:276] 0709 12:26:30.817844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.818530 (+   686us) service_pool.cc:224] Handling call
0709 12:26:30.902626 (+ 84096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.376566 (+1473940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.045293 (+1668727us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400453154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:34.045464 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3387863296}
I20260709 12:26:34.039911 20727 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:34.138695 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.139631 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.124693 20876 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
W20260709 12:26:34.129555 20849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.193:34333 timed out after 1.675s (SENT) [suppressed 3 similar messages]
I20260709 12:26:34.010166   321 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:26:34.013579 32549 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.037s
I20260709 12:26:34.034186 32591 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:34.134147 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.031296 32589 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:34.030728 32212 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:34.135605 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.043673 32597 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.006137   320 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:34.162102 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 140114us
I20260709 12:26:34.162249 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.162940 20727 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:34.142668 20735 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37. Ops: []
W20260709 12:26:34.143746 20808 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 4118) took 1443 ms (client timeout 1737 ms). Trace:
W20260709 12:26:34.164079 20808 rpcz_store.cc:269] 0709 12:26:32.700390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.703249 (+  2859us) service_pool.cc:224] Handling call
0709 12:26:34.143502 (+1440253us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002a99d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:34.143668 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471769344}
I20260709 12:26:34.164393 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.165120 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.165282 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.165666 20808 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 40
I20260709 12:26:34.012868 32698 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.030648 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.150112 20803 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:34.147665 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:34.151298 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.026547 32362 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.129959 32653 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.132025   305 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:34.144492   307 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.020s
I20260709 12:26:34.161031 32714 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.176105 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.169189 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.177958 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.185047 20801 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:34.185827 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.295478 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.299875 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.361920 20727 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
W20260709 12:26:34.300118 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 127751us
I20260709 12:26:34.168838 32597 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.302049   318 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:34.302801 32731 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.303582   340 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 12:26:34.177768 32311 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:34.158238 32739 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:34.147133 32599 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:34.315862   342 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:26:34.317397 32707 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.363s	user 0.011s	sys 0.001s
I20260709 12:26:34.051147 32595 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.168510 32591 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:34.318739 32561 consensus_meta.cc:287] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.630s	user 0.005s	sys 0.002s
W20260709 12:26:34.320703   333 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:26:34.327925 32557 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:34.332283 32419 consensus_meta.cc:287] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.524s	user 0.006s	sys 0.002s
W20260709 12:26:34.173416 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.334496 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:34.339107 32593 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.004s
I20260709 12:26:34.130133 32554 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.347043 20802 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:34.144874 32695 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.353103 20803 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
W20260709 12:26:34.414498 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.423731 20875 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 36 index: 21. (term mismatch)
I20260709 12:26:34.424172 20875 pending_rounds.cc:85] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 20
I20260709 12:26:34.424284 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.366464 32732 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.367959 32650 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.378909 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.384497 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:34.434465 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:34.402558 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.407373 20809 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:34.415997 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.334542 32316 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.424451 20875 pending_rounds.cc:107] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 35.21
I20260709 12:26:34.436780 20875 raft_consensus.cc:2889] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.365788 32718 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.425776 32738 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.378000 32577 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.032s
I20260709 12:26:34.442373 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.358618 32653 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.357545 20808 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:26:34.387591   318 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.382287 32731 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.404781 32311 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:34.471671 32732 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.471911 32311 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.472492 32732 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.475489 32732 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.477878 32554 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.478624 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.479514 20803 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 40 because replica has last-logged OpId of term: 38 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 20.
I20260709 12:26:34.480353 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.482270 32731 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.484520   318 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:34.485685 32227 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
I20260709 12:26:34.486109 32227 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.488561 32227 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.449054 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.399883 32707 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.456337 32597 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.458185 32591 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:34.407784 32561 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.460386 32595 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.447576 32738 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.461990 32599 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.426723 32593 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.447890 32419 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:34.491266 20808 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:34.507915 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.491737 20803 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:34.439868 32675 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:34.511893 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.517855 20808 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:26:34.519508 20808 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:34.545219 20734 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:34.564244 32653 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.545178 32419 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:34.566123 32419 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.578819 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.579947 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.580363 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.580438 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.581050 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.581547 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.583352 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.584019 20879 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:34.591081 20805 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:34.592271 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 2001) took 1225 ms. Trace:
I20260709 12:26:34.620546 20726 rpcz_store.cc:276] 0709 12:26:33.366386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.658155 (+291769us) service_pool.cc:224] Handling call
0709 12:26:33.658440 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.852251 (+193811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.852290 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.852347 (+    57us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:33.852381 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.591960 (+739579us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54002495d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:34.592025 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.592193 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445281152}
I20260709 12:26:34.622462 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:34.594412 21429 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 2265) took 1227 ms (client timeout 1625 ms). Trace:
I20260709 12:26:34.614295 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.626032 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.627513 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:34.578977 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.614799 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.629647 20880 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:34.613384 21427 raft_consensus.cc:2393] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 40: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:34.623769 21429 rpcz_store.cc:269] 0709 12:26:33.367164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.428431 (+ 61267us) service_pool.cc:224] Handling call
0709 12:26:34.594251 (+1165820us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400645ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:34.594356 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412943232}
I20260709 12:26:34.650957 20879 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: [37.25-37.25]
I20260709 12:26:34.662508 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.663508 20877 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:34.671936 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.628201 20726 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:34.676496 20882 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:34.544633 32675 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.677567 32587 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:34.683898 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.690186 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.668715 32599 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.028s
I20260709 12:26:34.693126   301 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.694587 32675 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.696389 20727 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:34.696918 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.698731 20731 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 39 index: 26. (index mismatch)
I20260709 12:26:34.711409 20879 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:34.704739 20877 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:34.705459 20774 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:34.685379 20880 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:34.715047 20774 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:34.715421 20774 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:34.716046 32554 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:34.717087 32316 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.697664   320 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.714474   318 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
W20260709 12:26:34.724067 32589 consensus_meta.cc:287] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.004s
I20260709 12:26:34.724169 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:34.724687   347 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 12:26:34.720747   301 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:34.724257   318 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:34.744728   308 consensus_meta.cc:287] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.063s	user 0.007s	sys 0.005s
I20260709 12:26:34.744904 32593 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:34.745400   308 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:34.745798 32707 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.746533 20810 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:34.747236 32561 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.748889 32703 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.750705   310 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:34.753383 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.755726 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.756417 20727 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:34.759500 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.740185   318 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:34.760874 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:34.770704 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 967643us
I20260709 12:26:34.773422 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.776168 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:34.780997 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.787320 20700 leader_election.cc:400] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33.
I20260709 12:26:34.787685 20700 leader_election.cc:403] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:34.790480 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.793780 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.786584 32675 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:34.754866   305 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.724695 32589 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.809806 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:34.809981 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.811230 20727 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:34.811355 20808 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:34.787067   320 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:34.813779 20804 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:34.815914 20804 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:34.802652 32675 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:34.817788 32675 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:34.802757 32361 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33.
I20260709 12:26:34.827611 20729 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40.
W20260709 12:26:34.802603 20627 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 201) took 138819 ms (client timeout 15000 ms). Trace:
I20260709 12:26:34.831024 32589 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.831108   305 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.802667 32587 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 37.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.036s
I20260709 12:26:34.832719 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:34.835078 20802 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33486 (request call id 4102) took 3429 ms (client timeout 1568 ms). Trace:
W20260709 12:26:34.855468 20802 rpcz_store.cc:269] 0709 12:26:31.406034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.407350 (+  1316us) service_pool.cc:224] Handling call
0709 12:26:34.834881 (+3427531us) spinlock_profiling.cc:243] Waited 2.89 s on lock 0x7b54002bff54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:34.834971 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6643702528}
I20260709 12:26:34.837078 20808 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:34.840440 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.841565 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4278) took 1079 ms. Trace:
I20260709 12:26:34.845856 32739 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:26:34.846750 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.827800 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.856884 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.858685 20804 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:34.857088 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.856838 20875 rpcz_store.cc:276] 0709 12:26:33.761726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.765155 (+  3429us) service_pool.cc:224] Handling call
0709 12:26:33.765434 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.437286 (+671852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.841327 (+404041us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54001ff254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:34.841506 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513097984}
I20260709 12:26:34.857398 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.859593 20810 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:34.856274 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.859557 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.861732 20808 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:34.864831 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.865708 20809 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:34.867820 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.868395 20804 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:34.853854 32597 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.871826 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.852329 32554 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:34.853489 32586 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:34.876530 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:34.828826   318 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.880100 32739 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.891911 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.893647 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.894155 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:34.885969 20804 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
I20260709 12:26:34.874025 32316 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Stepping down as leader of term 36
W20260709 12:26:34.851980 20627 rpcz_store.cc:269] 0709 12:24:15.983140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.053225 (+ 70085us) service_pool.cc:224] Handling call
0709 12:26:34.802493 (+138749268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1625793,"reported_tablets":36}
I20260709 12:26:34.901546 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:34.903403 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 63667.5us
I20260709 12:26:34.903612   322 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:34.881224 32597 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.896277 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.904717 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.894789 20805 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:34.906487 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:34.887698 32419 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.907719 20802 raft_consensus.cc:2368] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36.
I20260709 12:26:34.897930 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.910164 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:34.911319 20808 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:34.889622   311 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:34.912847 20804 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 40: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:34.914363 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.915450 20810 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:34.916611 20810 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:34.903396 32667 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:34.897650 32316 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:34.890143 32586 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.924490 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.925825 32586 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:34.925902 32227 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:34.928097 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.928892 32713 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:34.929656   359 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:34.933015 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.934283 20809 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:34.919346 32361 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.018s
I20260709 12:26:34.937507 32659 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:34.937760 32472 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:34.938433 32703 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:34.910845   322 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:34.932080 32438 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:34.939985 32479 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:34.919943 32650 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:34.940421 32479 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:34.940871 32650 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:34.941318 32650 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:34.941425 32479 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.20, Last appended by leader: 20, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.942795 32479 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:34.919687   301 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.943799   301 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.946048   301 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.920928 32667 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.946384 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:34.947137 32667 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:34.947561 32713 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:34.947969 32713 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:34.948349 32713 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:34.950376 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:34.950518 32592 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:34.951805 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:34.952090 32592 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:34.952324 32720 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:34.952927 32720 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.953761 32447 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:34.954985 32317 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.957227 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2271) took 1009 ms. Trace:
I20260709 12:26:34.957684 20878 pending_rounds.cc:85] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 24
I20260709 12:26:34.958056 20878 pending_rounds.cc:107] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 37.25
I20260709 12:26:34.958544 20878 raft_consensus.cc:2889] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.957625 20807 rpcz_store.cc:276] 0709 12:26:33.947764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.976876 (+ 29112us) service_pool.cc:224] Handling call
0709 12:26:33.977156 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.133045 (+155889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.133080 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.133129 (+    49us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:34.133159 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.956995 (+823836us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540025d5d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:34.957047 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.957168 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358010240}
I20260709 12:26:34.959776 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4291) took 1813 ms. Trace:
I20260709 12:26:34.959933 20875 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:34.960227 20733 rpcz_store.cc:276] 0709 12:26:33.146448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.606906 (+460458us) service_pool.cc:224] Handling call
0709 12:26:33.607264 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.883103 (+275839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.959551 (+1076448us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54002981d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:34.959722 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431148928}
I20260709 12:26:34.961300 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:34.962280 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.963053 20880 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 40: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:34.894903 32739 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:34.965587 20876 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:34.936066 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:34.966495 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.968331 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:34.971782 20807 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 40
I20260709 12:26:34.974064 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4207) took 1199 ms. Trace:
I20260709 12:26:34.974481 20884 rpcz_store.cc:276] 0709 12:26:33.774967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.966000 (+191033us) service_pool.cc:224] Handling call
0709 12:26:33.966340 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.966694 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.966726 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.967145 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.768581 (+801436us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b1800210340. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:34.819526 (+ 50945us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:34.819576 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.819666 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.965990 (+146324us) raft_consensus.cc:1685] finished
0709 12:26:34.966012 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.973975 (+  7963us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":665,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":8169,"spinlock_wait_cycles":10605312,"thread_start_us":50665,"threads_started":1,"wal-append.queue_time_us":919482}
I20260709 12:26:34.978186 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:34.979853 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:34.980808 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.981089 20808 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 33 index: 18. (term mismatch)
I20260709 12:26:34.981465 20808 pending_rounds.cc:85] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 17
I20260709 12:26:34.981715 20808 pending_rounds.cc:107] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 32.18
I20260709 12:26:34.984359 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:34.981985 20808 raft_consensus.cc:2889] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.984789 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:34.986132 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:34.987427 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:34.987991 20882 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:34.911594 32419 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:34.991215 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.991725 21434 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 37 index: 22. (index mismatch)
I20260709 12:26:34.993571 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:35.000973 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:35.001371 20733 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:34.996117 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.002470 20731 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:35.003304 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:35.003677 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:35.004191 21427 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:35.004462 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 49323f66286f4798b268f216bb62b3be is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 34 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:35.005275 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:34.920408   311 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.009702 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:34.977326 32586 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:35.018576 20727 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:26:34.982163 32361 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:34.982335 32317 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:35.015930 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.016237 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:35.008140 32577 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.009s
I20260709 12:26:35.010993 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.000270   320 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:35.018571 32586 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:34.977735 32720 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:34.967274 32227 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.040s
I20260709 12:26:35.028595 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.004150 32472 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:35.008975 32592 raft_consensus.cc:2764] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election decision vote started in defunct term 39: won
I20260709 12:26:35.037147 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:35.017694 32362 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:35.023077 32554 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:35.043699 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.017508 32695 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:35.005539   301 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.034s
I20260709 12:26:35.008042 32316 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.22, Last appended by leader: 22, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.017254 32521 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.020s
I20260709 12:26:35.013651 32703 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.031s
I20260709 12:26:35.032644 32587 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:35.033720 32589 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:35.042761 32737 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
W20260709 12:26:35.061921 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46090.9us
I20260709 12:26:35.021057   368 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.028s
I20260709 12:26:35.044121 32720 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:35.044874 32599 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.061818 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.073104 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:35.047847   310 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:35.096361 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 112060us
I20260709 12:26:35.100024 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:35.098539 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:35.062779 32659 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:35.106196 20775 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.064379 32592 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:35.108543 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.119158 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.119282 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.069083 32597 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.120733 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.120716 20882 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:35.057107 32687 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.24, Next index: 25, Last known committed idx: 23, Time since last communication: 0.052s
I20260709 12:26:35.107856 32500 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:35.115579 32382 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:35.061074 32695 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.108050 32554 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:35.103942   318 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.103230 32316 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:35.146999 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 85717.8us
I20260709 12:26:35.124003 32589 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.095283 32587 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.155290 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:35.134476   310 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.126653 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:35.103631 32599 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.166554 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.167258 32554 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:35.150401 32597 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:35.187568 32739 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.189244   311 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:35.127750 32317 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 36.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.028s
I20260709 12:26:35.148922 32500 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:35.089102 32586 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.178007 32713 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.179715   318 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:35.181190 20881 raft_consensus.cc:2410] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 38 because replica has last-logged OpId of term: 36 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 35 index: 21.
I20260709 12:26:35.136714 32703 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.028s
I20260709 12:26:35.197132 32500 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:35.203963 32713 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:35.206596 32597 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:35.206743 21427 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:35.207721 32597 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 38.22, Last appended by leader: 22, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.155418   375 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:35.171962   360 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:26:35.152745 32311 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:35.156397 32382 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:35.219664 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 75024.8us
I20260709 12:26:35.201659 20807 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:26:35.169307 32577 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election lost for term 40. Reason: could not achieve majority
I20260709 12:26:35.172348 32554 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.190225 32227 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:35.229357 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.228938 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.260958 21400 leader_election.cc:400] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:35.257720 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.258404 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.256563 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.258093 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.266037 21431 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:35.232208 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:35.268599 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.265410 21400 leader_election.cc:403] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:26:35.265866 20807 raft_consensus.cc:2368] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:35.190299 32361 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:35.192757 32659 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:35.274968 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.286693 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:35.287807 21431 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:35.253026   366 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:26:35.259958   373 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:26:35.286809 20733 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:35.298297 20775 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:35.258697   318 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.264979 32554 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:35.306315 20733 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:35.221750 32419 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:35.264592 32227 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:35.273119 32739 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:35.286144 32703 consensus_queue.cc:1243] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 39.23 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 22
I20260709 12:26:35.223553 32675 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.232307 32589 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.073s
I20260709 12:26:35.292304   372 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:35.222358   310 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.083s
I20260709 12:26:35.304263 32311 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:35.274302 32577 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.044s
I20260709 12:26:35.298928   374 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.221338 32737 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.088s
I20260709 12:26:35.301294 32599 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.309250 32472 raft_consensus.cc:2804] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:35.345052 20731 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:35.351562 20883 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
I20260709 12:26:35.378654 21431 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:35.311146   318 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.327841   372 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:35.423511 20731 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
W20260709 12:26:35.415741   365 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
W20260709 12:26:35.428766   371 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:26:35.429286 20732 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 4281) took 3039 ms (client timeout 1675 ms). Trace:
I20260709 12:26:35.430186 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 2190) took 2136 ms. Trace:
I20260709 12:26:35.439710 32695 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.451591 21430 rpcz_store.cc:276] 0709 12:26:33.293309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.660740 (+367431us) service_pool.cc:224] Handling call
0709 12:26:35.429966 (+1769226us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54005f4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:35.430121 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816737152}
I20260709 12:26:35.393006 32659 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.441061 32587 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.418303 32667 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.442267 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.461278 20733 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:35.462345 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.462764 20733 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:35.442488 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.444620 20807 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 35 index: 19. (index mismatch)
I20260709 12:26:35.445741 32698 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.446472 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.326131 32739 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.339314   375 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.078s
I20260709 12:26:35.446630 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 4316) took 1410 ms. Trace:
I20260709 12:26:35.447834 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 4019) took 1149 ms. Trace:
I20260709 12:26:35.467726 20803 rpcz_store.cc:276] 0709 12:26:34.037410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.480110 (+442700us) service_pool.cc:224] Handling call
0709 12:26:35.446401 (+966291us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54003a0f54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:35.446547 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24948608}
I20260709 12:26:35.344813   374 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.447460 20805 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
W20260709 12:26:35.451427 20732 rpcz_store.cc:269] 0709 12:26:32.390363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.390518 (+   155us) service_pool.cc:224] Handling call
0709 12:26:35.429054 (+3038536us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540024a4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:35.429199 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3257510656}
I20260709 12:26:35.468385 20729 rpcz_store.cc:276] 0709 12:26:34.298918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.334241 (+ 35323us) service_pool.cc:224] Handling call
0709 12:26:35.447623 (+1113382us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54000e6ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:35.447767 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943583104}
W20260709 12:26:35.450342 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 151176us
I20260709 12:26:35.417680 32227 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.24, Last appended by leader: 24, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.457476   372 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 36.22, Last appended by leader: 22, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.463744 20726 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:35.424249 32472 raft_consensus.cc:697] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:35.448093 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:35.470090 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.472271 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.469173 32419 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:35.345854 32361 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.473053 21429 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:35.342965 32382 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.481084 32739 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.471401 32316 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:35.481935 20808 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:35.482149 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4292) took 2327 ms. Trace:
I20260709 12:26:35.482273 32316 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.482515 20735 rpcz_store.cc:276] 0709 12:26:33.154366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.615939 (+461573us) service_pool.cc:224] Handling call
0709 12:26:33.640315 (+ 24376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.163489 (+523174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.481919 (+1318430us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54003b5454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:35.482091 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1154791168}
I20260709 12:26:35.484495 32316 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:35.484752 32650 consensus_meta.cc:287] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.001s
I20260709 12:26:35.485044 32685 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.485183 32650 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:35.487121 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.487249 32419 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.487839   318 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.489333 21428 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:35.489457 32419 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:35.490638 32675 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:35.490912   344 log.cc:927] Time spent T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.348s	user 0.005s	sys 0.000s
W20260709 12:26:35.491456 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.491861   318 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:35.492444   318 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.493577 32586 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:35.494468   318 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.495827   374 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.496672 32667 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.499519 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:35.500571 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.500586 32685 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.501230 32685 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.500257 32419 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.503434 32685 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:35.505136 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:35.505901 32685 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:35.506444 32685 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 40 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:35.507393 32685 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.511049 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:35.511818 32419 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:35.512188 32419 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.512509 32419 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:35.516207 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.516894 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.510943 32472 consensus_queue.cc:237] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.519728 32419 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.520058 32561 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:35.520582 32561 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:35.521749 32419 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:35.521667 32561 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.487357   372 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:35.524075 32316 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:35.524497 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.525913 32316 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:35.526858 32739 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:35.527251 32739 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:35.528182 32739 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.528690 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.531414 32650 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.503171   322 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:35.538534 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.537647 20775 leader_election.cc:400] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40.
I20260709 12:26:35.546059 20775 leader_election.cc:403] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:35.545149 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:35.547559 32591 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:26:35.548151 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68814.8us
I20260709 12:26:35.548842   318 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election lost for term 39. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40.
I20260709 12:26:35.551430   376 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:35.551827 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.552222 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.553020 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:35.553063   306 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:35.553793   306 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.557217 20807 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:35.551661 20726 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:35.560313 20726 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:35.562068 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.563706 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:35.489490 32479 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.001s
I20260709 12:26:35.523088 32561 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:35.571782 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.574388 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:35.572118 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:35.570377 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 2009) took 1754 ms. Trace:
I20260709 12:26:35.576710 20810 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:35.576295 20734 rpcz_store.cc:276] 0709 12:26:33.815780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.444552 (+628772us) service_pool.cc:224] Handling call
0709 12:26:34.448912 (+  4360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.545814 (+ 96902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.570155 (+1024341us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b54000e92d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:35.570320 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220735360}
I20260709 12:26:35.561342 20726 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.25, Last appended by leader: 25, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.577987 20726 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:35.547317 20808 raft_consensus.cc:2368] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 39. Current term is 40.
I20260709 12:26:35.595309 21431 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: [37.25-37.25]
I20260709 12:26:35.583802 20801 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 34 index: 20. (term mismatch)
I20260709 12:26:35.598048 20810 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:35.598912 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:35.599687 20801 pending_rounds.cc:85] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 19
W20260709 12:26:35.559088   320 consensus_meta.cc:287] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.002s
I20260709 12:26:35.602372 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:35.639796 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 101338us
I20260709 12:26:35.545782   322 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:35.671253 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73448.3us
I20260709 12:26:35.593357 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.688309 20726 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 38 index: 25. (term mismatch)
I20260709 12:26:35.688935 20726 pending_rounds.cc:85] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 24
I20260709 12:26:35.689281 20726 pending_rounds.cc:107] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 37.25
I20260709 12:26:35.689630 20726 raft_consensus.cc:2889] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.641019 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:35.600777 20801 pending_rounds.cc:107] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 33.20
I20260709 12:26:35.703881 20801 raft_consensus.cc:2889] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.598359 32391 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.675951 32591 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:35.581888 32586 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:35.581184 32587 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:35.570071 32500 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.723338 20878 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:35.675631   320 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:35.751883 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 190139us
I20260709 12:26:35.754098 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:35.709939 32591 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:35.760332 20878 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:35.761114 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:35.792227 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 151312us
I20260709 12:26:35.793140 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.793408 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.721254 32586 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.779441 32591 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:35.709925 32391 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.803948 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:35.804563 20809 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 29.
I20260709 12:26:35.804816 32391 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.810722 32591 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.814702 32419 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:35.814814 32391 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.815362 32419 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.817612 32419 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.818184   306 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.786732 32685 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:35.824239 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.813162   377 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:35.744710 32587 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.830318 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.835740 21399 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:35.839389 20730 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:35.839982 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:35.841357 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.841531 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:35.842227 21431 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 40: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:35.842640 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.842757 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.842586 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.843955 21426 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:35.842242 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.845745 20734 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:26:35.846355 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:35.794996 32667 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.036s
I20260709 12:26:35.862114   376 consensus_queue.cc:1059] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.024s
I20260709 12:26:35.837644 32685 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:35.828848   386 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.042s
W20260709 12:26:35.899650 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.901880 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.904265 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:35.904677 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.904808 20733 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 38 index: 22. Preceding OpId from leader: term: 40 index: 23. (index mismatch)
W20260709 12:26:35.903406 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 110357us
I20260709 12:26:35.905587 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:35.873308   377 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.906993 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:35.893589 20835 heartbeater.cc:646] Failed to heartbeat to 127.19.36.254:46039 (9 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.19.36.254:46039 timed out after 15.000s (SENT)
I20260709 12:26:35.911564 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:35.913873 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:35.911839 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:35.915446 20881 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:35.916885 20880 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
W20260709 12:26:35.918535 20728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 1989) took 2898 ms (client timeout 2157 ms). Trace:
I20260709 12:26:35.920974 20878 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:35.873728 32659 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.031s
I20260709 12:26:35.923974 20878 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 22. Preceding OpId from leader: term: 37 index: 22. (term mismatch)
I20260709 12:26:35.926513 20878 pending_rounds.cc:85] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 21
I20260709 12:26:35.924542 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:35.927772 20878 pending_rounds.cc:107] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 36.22
I20260709 12:26:35.927627 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:35.928210 20878 raft_consensus.cc:2889] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.918474 32391 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:35.929414 32391 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:35.932173 32738 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:35.924144 32563 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:35.933120 20884 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: [37.25-37.25]
I20260709 12:26:35.933715 32361 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:35.935760 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:35.936952 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.761093   322 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:35.918990 32554 consensus_meta.cc:287] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.001s
I20260709 12:26:35.925305 20880 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:26:35.926390 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:35.946336 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:35.918975 20728 rpcz_store.cc:269] 0709 12:26:33.020062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.020186 (+   124us) service_pool.cc:224] Handling call
0709 12:26:35.918292 (+2898106us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b54001d6d54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:35.918467 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956087168}
I20260709 12:26:35.947407 20882 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:35.913168   375 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.908783 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:35.904354 32419 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:35.909484   306 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:35.911978   372 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:35.921602 32695 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.953279 20881 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:26:35.967330 32419 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:35.972668 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.993459 32419 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:35.993602   372 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:35.994412 32722 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 38 current term: 38 is higher.
I20260709 12:26:35.994841   372 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.956848 20806 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41. Ops: [40.23-40.23]
W20260709 12:26:35.963603 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 54926.1us
I20260709 12:26:35.985545 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53814 (request call id 2199) took 1027 ms. Trace:
I20260709 12:26:35.997483 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:35.997967 20879 rpcz_store.cc:276] 0709 12:26:34.958298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.958420 (+   122us) service_pool.cc:224] Handling call
0709 12:26:34.958727 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.002624 (+ 43897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.002654 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.003055 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 12:26:35.109186 (+106131us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:35.109225 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.985071 (+875846us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b54000a41d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:35.985128 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.985185 (+    57us) raft_consensus.cc:1685] finished
0709 12:26:35.985202 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.985473 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":629,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":7714,"spinlock_wait_cycles":100203776,"thread_start_us":83997,"threads_started":1,"wal-append.queue_time_us":152997}
I20260709 12:26:35.944694 32563 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:35.989634 32227 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:35.968537 32500 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.967228   398 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:36.016592 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:35.976256   306 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:36.006064 32625 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.036314 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:36.117280 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.031337 20883 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:35.954912 32554 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:35.948405 32685 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.009279 32316 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.005s
I20260709 12:26:36.006954 32419 raft_consensus.cc:2764] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election decision vote started in defunct term 34: won
W20260709 12:26:36.029747 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 194102us
I20260709 12:26:36.014462 32622 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:36.015136 32587 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:35.990835 32695 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.179298 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:36.182991 20809 raft_consensus.cc:2368] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34.
I20260709 12:26:36.186703 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:36.187685 20881 raft_consensus.cc:2368] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40.
I20260709 12:26:36.138873   398 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:36.155910 32622 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:36.199992 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51546.8us
W20260709 12:26:36.206547 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 361053us
I20260709 12:26:36.245273 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:36.246254 20876 raft_consensus.cc:3060] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:36.258193 20882 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 39 index: 26. (index mismatch)
I20260709 12:26:36.138018   306 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.182076 32391 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:36.307595 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.315693 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 307791us
W20260709 12:26:36.319602 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.209182   398 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:36.373517 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.393074 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 467792us
I20260709 12:26:36.312184 32622 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 37.22, Last appended by leader: 22, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.261708 32563 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 34, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.437096 20880 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:36.451134 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:36.438967 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:36.440040 32391 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.440164 20882 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:26:36.442118 32722 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:36.453279 32391 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.453877   306 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:36.455570 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:36.455749 32391 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:36.457177 32316 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:36.457753 32316 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:36.458547 32316 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:36.460764 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:36.461568   370 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:36.462004   370 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:36.462814   370 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.463745   370 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:36.463832 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.466082 21429 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:26:36.467661 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.468289 21429 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:36.469286 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.470549 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.472241 32555 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:36.472852 32555 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:36.473277 21426 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38. Ops: [37.22-37.22]
I20260709 12:26:36.472214 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:36.473830 32555 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.475586 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.461589 21427 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:36.445351 32561 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.008s
W20260709 12:26:36.450217 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 207454us
I20260709 12:26:36.479839 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:36.481407   306 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:36.482090 32625 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.480930 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:36.479162 32391 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:36.474918 32555 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:36.484169 32391 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.487010 32391 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:36.488505 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:36.489943 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:36.490217   396 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:36.492034 21428 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2202) took 1238 ms. Trace:
I20260709 12:26:36.492429 21428 rpcz_store.cc:276] 0709 12:26:35.253606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.277746 (+ 24140us) service_pool.cc:224] Handling call
0709 12:26:35.278125 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.489760 (+211635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.491770 (+1002010us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b540042ed54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:36.491970 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485265664}
I20260709 12:26:36.494030 32586 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.483326 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:36.481817 20883 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:36.479264 32738 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.482631 32625 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.484848 32591 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:36.499008 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.491022   396 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:36.480937 20876 raft_consensus.cc:2468] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:36.504410 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:36.501235 20849 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:36.592902   416 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.581435 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.583519 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.526983 32552 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Stepping down as leader of term 37
W20260709 12:26:36.584448 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 106772us
I20260709 12:26:36.614151 32552 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:36.616317 32552 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 37.22, Last appended by leader: 22, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.618120 32552 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:36.623478 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.629796 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:36.632391 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 2013) took 1637 ms. Trace:
I20260709 12:26:36.632977 20727 rpcz_store.cc:276] 0709 12:26:34.995124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.042387 (+ 47263us) service_pool.cc:224] Handling call
0709 12:26:35.059593 (+ 17206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.197683 (+138090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.197715 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:35.197759 (+    44us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:35.197791 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.631695 (+1433904us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400226fd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:36.631792 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:36.632109 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317113728}
W20260709 12:26:36.636438   414 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:26:36.585482 32500 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:36.665431 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:36.665663 32500 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.512434 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.505235 32391 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.015s
I20260709 12:26:36.502254 32738 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.673337 32591 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:36.673931 32591 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:36.675745 32738 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:36.502353 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.507400   380 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:36.678471 32739 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:36.508399   401 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:36.497550 32419 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:36.680066 32591 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.20, Last appended by leader: 20, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.680929 20884 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
W20260709 12:26:36.547844   403 log.cc:927] Time spent T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:36.682590 32591 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:36.495766 32622 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:36.668977 32500 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.508634 32625 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:36.694331 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:36.521042 32695 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.530742   369 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:36.586264 20776 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 29. Current term is 30.
I20260709 12:26:36.816043   369 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:36.817303 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 231706us
I20260709 12:26:36.818296 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:36.817831   369 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:36.595068 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 93781.1us
I20260709 12:26:36.639933   398 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:36.585784   374 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.024s	sys 0.054s
I20260709 12:26:36.648742 32554 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.594321   416 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.653606 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:36.654147 32586 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:36.655390 32362 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:36.832406 20731 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: [29.22-29.22]
I20260709 12:26:36.655727 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:36.834754 20729 raft_consensus.cc:2376] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40. Re-sending same reply.
I20260709 12:26:36.834856 32586 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.657444 32722 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.664083 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:36.664458 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:36.682773   396 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:36.667017 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:36.756891 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.697933 32725 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:36.759334   412 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:36.761863 32391 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:36.767737 20728 raft_consensus.cc:3060] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 36
W20260709 12:26:36.807456 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77139.4us
W20260709 12:26:36.808233 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 89627us
I20260709 12:26:36.809816 21435 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 38 index: 25. Preceding OpId from leader: term: 40 index: 26. (index mismatch)
I20260709 12:26:36.810632 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.812368 32561 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:36.808450 20633 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:36.830406   374 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.025s	sys 0.054s
I20260709 12:26:36.850384   412 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.850849 20883 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:36.843560 32725 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:36.851387 32391 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.851817 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:36.852052   416 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:36.847801 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68421.3us
I20260709 12:26:36.860492 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:36.861150   378 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.018s
I20260709 12:26:36.861871 32625 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:36.850852 20880 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:36.843431   396 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:36.846923 32561 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:36.839349 32722 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.861889 32586 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.889772 32587 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:36.880328   369 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:36.891952 32227 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.832s	user 0.006s	sys 0.013s
I20260709 12:26:36.891745 32685 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:36.895689 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.896296 32739 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:36.896193 32561 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.897418 32739 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.899480 20882 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:36.891479 20633 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:36.898074 32561 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:36.879364   396 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:36.900929 32722 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.923898 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:36.924651 32391 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:36.929644 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 276250us
I20260709 12:26:36.931524 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:36.932492 20727 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:36.933212 32739 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:36.933811 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 4141) took 2028 ms. Trace:
I20260709 12:26:36.934264 20805 rpcz_store.cc:276] 0709 12:26:34.856194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.908266 (+ 52072us) service_pool.cc:224] Handling call
0709 12:26:35.447097 (+538831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.468843 (+ 21746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.854521 (+1385678us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54003a0f50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:36.884356 (+ 29835us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238374656}
I20260709 12:26:36.913336   431 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:36.936094   431 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:36.936748 32725 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 40.23, Last appended by leader: 23, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.937278   431 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 30.24, Last appended by leader: 24, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:36.937937 32725 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:36.938791   431 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:36.939536 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:36.925557 20633 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 15 to 21, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 21 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:36.940423 21435 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:36.941682 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:36.944243 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:36.945053 32586 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:36.945487   416 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:36.945711 32586 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:36.946173 20700 leader_election.cc:400] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:36.946499 20700 leader_election.cc:403] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:36.947477   320 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:36.947733 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 2022) took 1381 ms. Trace:
I20260709 12:26:36.947914   412 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:36.948333 32362 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:26:36.949481 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:36.951731 20774 leader_election.cc:400] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40.
I20260709 12:26:36.953320 32625 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:36.954277 32622 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:36.956235 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:36.948930 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62474.3us
I20260709 12:26:36.961086 20728 raft_consensus.cc:2468] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:36.900871   398 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:36.942062 20633 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:36.976433 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.969733   398 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.946997 32586 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 38.21, Last appended by leader: 21, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:36.989439   372 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:26:36.990541 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:36.990748 20633 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 20 to 21, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 21 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:36.992760   416 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:26:37.008356 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.009738 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 149447us
I20260709 12:26:37.009919 32625 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:37.010793 20809 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:26:37.012529 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.013511 32738 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:36.993181   376 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.022s
I20260709 12:26:36.971709   320 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:37.014461   320 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.014889   320 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:37.015723 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.016372 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.017123 20878 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
W20260709 12:26:37.018823 32316 consensus_meta.cc:287] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.560s	user 0.016s	sys 0.008s
I20260709 12:26:37.019524 32316 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:37.023351   419 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.383s	user 0.006s	sys 0.000s
W20260709 12:26:37.024179   370 consensus_meta.cc:287] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.011s
I20260709 12:26:37.024524   322 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:36.974180   302 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:26:36.948138 20732 rpcz_store.cc:276] 0709 12:26:35.566050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.566210 (+   160us) service_pool.cc:224] Handling call
0709 12:26:35.566571 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.579073 (+ 12502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.579102 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.579698 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.664536 (+1084838us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b1800476260. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:36.881886 (+217350us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:36.881948 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.885401 (+  3453us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:36.947257 (+ 61856us) raft_consensus.cc:1685] finished
0709 12:26:36.947277 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:36.947560 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85902,"raft.queue_time_us":12479,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":16570,"spinlock_wait_cycles":3142400,"thread_start_us":182879,"threads_started":1,"wal-append.queue_time_us":1363004}
I20260709 12:26:37.009758 20801 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 38 index: 22. Preceding OpId from leader: term: 40 index: 23. (index mismatch)
I20260709 12:26:37.008234   416 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:37.013881 21434 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
W20260709 12:26:36.995916 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.014956 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:37.030827 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.001750   398 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:37.031637   416 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.055639   408 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.003s
I20260709 12:26:37.056944   408 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:37.057379   408 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:37.032346   372 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.010715 32625 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.065349 32362 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:37.064023   416 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:37.091696 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.094108 20729 raft_consensus.cc:1275] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:26:37.094384 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:37.065472   412 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.095039 32625 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.058614   408 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.097330   372 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.101348   372 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.108688 20883 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:37.110954 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.113082 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.114037 20729 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
W20260709 12:26:37.130743 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 65565.2us
I20260709 12:26:37.132899 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.136950 20732 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:37.142858 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:37.148252 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.114802 32622 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:26:37.147866 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 296250us
I20260709 12:26:37.149060 32725 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:37.155827 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.153874 20728 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:37.120218 32625 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:37.152714 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:37.162789 20729 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:37.163626 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.150902 32316 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.174516 20633 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 23 to 26, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 26 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:37.169661 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 194626us
W20260709 12:26:37.170874   433 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:26:37.173669 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.175470 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:37.176204 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.176057 21434 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 35 index: 23. (term mismatch)
I20260709 12:26:37.176637 32447 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.155539 32738 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:37.152766 32622 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.190892 21434 pending_rounds.cc:85] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 22
W20260709 12:26:37.190822 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 94393.7us
I20260709 12:26:37.157843 32554 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.142102 32586 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.028s
I20260709 12:26:37.197089 20775 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:37.199028 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.198729 32447 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.199517 20876 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:37.149864   372 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.052s
W20260709 12:26:37.203311 32555 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.658s	user 0.006s	sys 0.016s
I20260709 12:26:37.194837 32622 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.208889 32554 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.150463   412 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.193375 21434 pending_rounds.cc:107] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 34.23
W20260709 12:26:37.215764 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 74955us
W20260709 12:26:37.218971 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.229959 21434 raft_consensus.cc:2889] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.237249 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:37.237938 20633 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:37.220347   408 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:37.244777 21431 raft_consensus.cc:2410] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 34 because replica has last-logged OpId of term: 32 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 20.
I20260709 12:26:37.236377 32586 raft_consensus.cc:2764] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
W20260709 12:26:37.257531 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 110824us
I20260709 12:26:37.236181 32725 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.259908 32725 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.267004 32316 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:37.267547 32316 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:37.268579 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.267967 32316 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:37.272701   443 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:37.273244   437 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.273531   443 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.273994   437 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.276649 21428 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:37.276862   437 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.262384 32625 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.280283 32625 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.280870 32625 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.281549 32625 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.283766   443 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:37.287909   438 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:26:37.288316   398 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:37.289001   441 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:26:37.290535   398 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:37.293663 20801 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:37.302968 21432 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:37.304836 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 2206) took 1464 ms. Trace:
I20260709 12:26:37.305248   443 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:26:37.306690   412 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.307152 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:37.307361 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52398.2us
I20260709 12:26:37.308269 21433 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:37.308732 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2282) took 1757 ms. Trace:
I20260709 12:26:37.309324 20804 rpcz_store.cc:276] 0709 12:26:35.550955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.553079 (+  2124us) service_pool.cc:224] Handling call
0709 12:26:35.553332 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.611457 (+1058125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:36.611526 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:36.611620 (+    94us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:36.611707 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.308206 (+696499us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002b5cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:37.308273 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.308516 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379647744}
I20260709 12:26:37.302958 20879 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: [39.26-39.26]
I20260709 12:26:37.308969 20700 leader_election.cc:400] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 39. Current term is 40.
I20260709 12:26:37.315773 20700 leader_election.cc:403] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Cancelling election due to peer responding with higher term
I20260709 12:26:37.305296 21432 rpcz_store.cc:276] 0709 12:26:35.840610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.840728 (+   118us) service_pool.cc:224] Handling call
0709 12:26:37.304544 (+1463816us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540029ca54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:37.304720 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3347583232}
I20260709 12:26:37.331867 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:37.332731 20729 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:37.268455 32725 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.338161 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.342621 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.307207   396 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:37.357245 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 162986us
I20260709 12:26:37.349377   372 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election lost for term 39. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 39. Current term is 40.
W20260709 12:26:37.360868 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 138085us
I20260709 12:26:37.366616 20875 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:37.359746 32625 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.342867   320 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.362181 32725 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:37.378340   412 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:37.381386 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4229) took 1192 ms. Trace:
I20260709 12:26:37.382542   448 raft_consensus.cc:2804] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:37.383879 32447 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:37.384541 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 67345.2us
I20260709 12:26:37.386657 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.381814 20881 rpcz_store.cc:276] 0709 12:26:36.188792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.210843 (+ 22051us) service_pool.cc:224] Handling call
0709 12:26:36.213498 (+  2655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:36.213992 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:36.214023 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.214560 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.129615 (+915055us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b18002add20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:37.241696 (+112081us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:37.244204 (+  2508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.255085 (+ 10881us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.380902 (+125817us) raft_consensus.cc:1685] finished
0709 12:26:37.380921 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.381171 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4443,"raft.queue_time_us":105,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":15818,"spinlock_wait_cycles":75063040,"thread_start_us":85572,"threads_started":1,"wal-append.queue_time_us":1142310}
W20260709 12:26:37.396107 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 87355.4us
I20260709 12:26:37.379146   412 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.399868 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.392647 32447 consensus_queue.cc:1048] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 12:26:37.406028   396 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.407581 32554 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.416785 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:37.420318 32561 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:37.421048 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:37.407778 32622 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.422052 32725 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:37.422530 32725 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:37.422593 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.422943 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.423847 32622 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.409176   380 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.424291 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:37.424651   380 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.426746 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.409654   320 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.410557 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.410730 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:37.428094 21428 raft_consensus.cc:3060] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:37.428558 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.389087   448 raft_consensus.cc:697] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:37.410946 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.414362 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:37.422982 32725 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:37.400425   429 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.161s	user 0.001s	sys 0.001s
I20260709 12:26:37.426914   380 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:37.431485 21429 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:37.397917 32555 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.434379 21435 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:37.440237 32625 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.437662 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.437667 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:37.441684 21432 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:37.449239 32472 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:37.449373 21431 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:26:37.440922 32625 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.449878 32472 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.438655   320 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:37.447139 32622 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
W20260709 12:26:37.467329 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44861.6us
I20260709 12:26:37.468362   453 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:37.468724   446 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.469230   453 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.470914   445 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:37.471424   445 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:37.471724 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78043.9us
I20260709 12:26:37.472921 32447 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.473527 32447 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.473789 32713 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:37.471297 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.469312   446 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.471859   445 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:37.468271 32176 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.018s
I20260709 12:26:37.476091 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.480680 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:37.476969 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:37.453069 32739 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:37.481611 20878 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.446574   408 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.040s
I20260709 12:26:37.485193 32176 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:37.485599 32176 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:37.486763 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.487473 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.471694   453 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.486433 32176 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.445293 32561 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.033s
I20260709 12:26:37.491782 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.492580 20732 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:37.494254 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.494676   404 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:37.495083 20809 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:37.496603 32176 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:37.466755 32622 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.505756 20878 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:37.506572 21428 raft_consensus.cc:2468] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:37.511864 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:37.512528 21428 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:37.522691   456 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.024s
I20260709 12:26:37.523552 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.524489 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:37.523159   448 consensus_queue.cc:237] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 34.17, Last appended by leader: 17, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.526803 32739 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:37.527237 32739 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:37.527614 32739 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.528075   412 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.528654   408 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:37.529095   408 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:37.529450   408 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:37.535651 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:37.528098 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 47851.5us
W20260709 12:26:37.528219   430 log.cc:927] Time spent T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:26:37.451331 32555 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.549081 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.549075 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.550002 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.509316 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.486369   446 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.564867 20732 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:37.558349 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.568027 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:37.568423 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:37.568612 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.569162 20728 raft_consensus.cc:2393] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:37.570504 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.572968 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.505412 32622 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:37.569149 21434 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:37.570539 32447 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.578510 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.579084 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:37.579921 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.583741 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.585513 20804 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:37.585879 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.581771 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.588718 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:37.505395   453 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:37.603528   453 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.605944   453 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.583237   454 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.607420 32472 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:37.607736   454 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.609725 32738 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:26:37.579697   412 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.032s
I20260709 12:26:37.592985 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:37.595745 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.599766 32625 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:37.600237 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 106947us
I20260709 12:26:37.609951   454 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.595470 21433 raft_consensus.cc:2393] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 34: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:37.618046 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:37.621110 20801 raft_consensus.cc:2393] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:37.627698 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.630375 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149985us
I20260709 12:26:37.629777 20879 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
W20260709 12:26:37.640112 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52025.5us
W20260709 12:26:37.658753 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 116967us
I20260709 12:26:37.674010 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.678630 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:37.678894 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.639473 32472 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:37.683470 21428 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:37.679661 20809 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
W20260709 12:26:37.692548   451 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 12:26:37.693351 20879 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:37.693548 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.694209 20775 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:37.695318 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2807) took 1125 ms. Trace:
I20260709 12:26:37.695703 21426 rpcz_store.cc:276] 0709 12:26:36.506570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.506807 (+   237us) service_pool.cc:224] Handling call
0709 12:26:36.507264 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:36.507600 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:36.507628 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.833055 (+325427us) spinlock_profiling.cc:243] Waited 9.53 ms on lock 0x7b2c0014b4c8. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:36.833673 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.884255 (+ 50582us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:36.935598 (+ 51343us) log.cc:844] Serialized 14 byte log entry
0709 12:26:36.967002 (+ 31404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.974391 (+  7389us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.615697 (+641306us) raft_consensus.cc:1685] finished
0709 12:26:37.622039 (+  6342us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.631584 (+  9545us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184515,"prepare.queue_time_us":27015,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":41,"raft.queue_time_us":31490,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":5827,"spinlock_wait_cycles":21926784,"thread_start_us":50094,"threads_started":1,"wal-append.queue_time_us":400273}
I20260709 12:26:37.698786 20803 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: [33.20-33.20]
I20260709 12:26:37.701793 32419 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:37.687285   449 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 12:26:37.711004 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:37.711414 20881 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:37.694793 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:37.714692 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.702972 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:37.718045 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98772.2us
I20260709 12:26:37.719133 20699 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.714148 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:37.725523 20810 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:37.707018 32472 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.710453   454 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:37.672372   445 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.744374 20803 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 38 index: 25. Preceding OpId from leader: term: 40 index: 26. (index mismatch)
I20260709 12:26:37.680066 32554 consensus_queue.cc:1059] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: INVALID_TERM, Last received: 34.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:26:37.749111 20881 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 37 index: 20. Preceding OpId from leader: term: 40 index: 22. (index mismatch)
I20260709 12:26:37.750792 20875 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 34 index: 20. (term mismatch)
I20260709 12:26:37.767473 20875 pending_rounds.cc:85] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 19
I20260709 12:26:37.767807 20875 pending_rounds.cc:107] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 33.20
I20260709 12:26:37.768100 20875 raft_consensus.cc:2889] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.683096 32362 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.050s
W20260709 12:26:37.786288 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77989.3us
I20260709 12:26:37.735802   431 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:37.732781 32472 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:37.787942 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:37.746389   454 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:37.725633 32739 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:37.797861 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71703.6us
W20260709 12:26:37.798048 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83979.2us
W20260709 12:26:37.792057 32316 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.519s	user 0.006s	sys 0.005s
I20260709 12:26:37.808221 20881 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 39 because replica has last-logged OpId of term: 37 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 36 index: 21.
W20260709 12:26:37.806766 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 87710.5us
I20260709 12:26:37.790603   431 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:37.800544 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.798758 32625 raft_consensus.cc:3055] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:37.799853   454 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:37.803839 32586 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:37.824853   431 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:37.794675   462 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:37.831904   462 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:37.833312   462 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.24, Last appended by leader: 24, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.792773   446 consensus_queue.cc:1059] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 37.25, Next index: 26, Last known committed idx: 24, Time since last communication: 0.029s
I20260709 12:26:37.851250 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.855857 20700 leader_election.cc:400] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34.
I20260709 12:26:37.856200 20700 leader_election.cc:403] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Cancelling election due to peer responding with higher term
I20260709 12:26:37.859009 32563 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:37.859798   461 raft_consensus.cc:2804] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:37.861047   311 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.853518 20810 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:26:37.861653   311 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:37.862447 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.862967 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.827039 32625 raft_consensus.cc:740] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:37.863210   445 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:37.863839 20808 raft_consensus.cc:2410] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 37 because replica has last-logged OpId of term: 36 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 34 index: 20.
I20260709 12:26:37.864189 32625 consensus_queue.cc:260] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:37.865149   458 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:26:37.865751 32625 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:37.867410 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:37.861049   459 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:37.868075 20806 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
W20260709 12:26:37.913096 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 220773us
I20260709 12:26:37.925976 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:37.939258 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.940315 32739 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:37.945298 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.946067 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:37.946849 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.817152 32316 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:37.948441 21432 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
W20260709 12:26:37.951711 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 105309us
I20260709 12:26:37.968030   461 raft_consensus.cc:697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:37.882535   372 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.978917 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.973210 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:37.976161 32176 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.047763 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.059461   372 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:38.065505 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 247443us
I20260709 12:26:38.066039 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.053994 20808 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:38.067396 20803 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 41: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:37.962057 32739 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.980600 32362 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election lost for term 33. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34.
W20260709 12:26:38.098774 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 271261us
W20260709 12:26:38.099882 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 241004us
I20260709 12:26:38.117579 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:38.109923 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 55311.1us
I20260709 12:26:38.123397 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:38.126986 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:38.128926 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 242119us
I20260709 12:26:38.132491 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:38.132525 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 324930us
I20260709 12:26:38.091852 32563 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election won for term 35
W20260709 12:26:38.102612   416 consensus_meta.cc:287] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.980s	user 0.008s	sys 0.014s
I20260709 12:26:38.116187 32419 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.134292 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:38.069634 32176 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.157687 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:38.134078   374 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:38.162482 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.163333 20726 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:38.100696 32739 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.103329   474 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:38.168563 20810 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:38.180879 20776 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:38.163417 32362 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:26:38.192572 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64212.7us
I20260709 12:26:38.202404 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:38.146063   416 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.204097 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.235697 20810 raft_consensus.cc:2410] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 39 because replica has last-logged OpId of term: 37 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 36 index: 21.
I20260709 12:26:38.199658   452 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:26:38.244925 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 44112.2us
I20260709 12:26:38.203725 32362 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.147902 32419 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:38.248003 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86257.2us
I20260709 12:26:38.248790 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:38.256728 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:38.271674 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.247349   462 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:38.245498   477 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.251061   453 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:38.254227 32362 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:38.280330 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:38.263355   495 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:38.269524   452 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:38.289387 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:38.287025 32622 consensus_meta.cc:287] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.666s	user 0.006s	sys 0.005s
I20260709 12:26:38.281078 32316 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.280795   453 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.285794 32738 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:38.283269   380 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.250609   461 consensus_queue.cc:237] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.292484   495 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.319327   483 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:26:38.342391   463 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.440s	user 0.001s	sys 0.000s
I20260709 12:26:38.343246   372 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.343920   372 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:38.348138   416 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.348497 32176 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.329766   462 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:38.349466   466 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.348137   452 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.350131   466 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.328079   453 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:38.352741   481 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:38.353356   481 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.342423   498 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.356177   498 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.358320   498 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.343689   499 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:38.287708 32563 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:38.347066   408 consensus_meta.cc:287] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.817s	user 0.009s	sys 0.002s
I20260709 12:26:38.393811 20806 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:38.336702   380 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.361106   483 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.339825   495 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:38.328452 32622 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.318230   477 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.390098   499 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:38.401503 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 129987us
W20260709 12:26:38.406905 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 135247us
I20260709 12:26:38.420857 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.343300 32316 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.071s
W20260709 12:26:38.424273 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 267458us
I20260709 12:26:38.412137 32739 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.412127   483 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:38.434705 20804 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: [35.19-35.19]
I20260709 12:26:38.436780 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.401094   408 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.420001   481 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.439302 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.424005   499 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:38.442745 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:38.453032 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:38.453303   490 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:38.453508 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.453598   466 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.454279 20807 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:38.453917   490 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.456197   490 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.456068 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.456796   494 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.456863 20882 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:38.457746   408 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.458160   311 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.458372   504 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.458896   504 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.451314   476 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:38.460151   476 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.460603   476 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.461012 20734 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:38.461150   504 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.462764   476 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:38.468406   489 log.cc:927] Time spent T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:26:38.468494 32316 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:38.469491 32316 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:38.457412   494 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.473300 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.441804   412 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:38.474396   475 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.480979 20880 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:38.442333   445 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:38.429857   498 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.022s
W20260709 12:26:38.486739   482 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 12:26:38.486749 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.491662   508 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:38.495303   500 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.071s
I20260709 12:26:38.495512 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.495978   491 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:38.521692   491 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.523978   491 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.526389 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:38.528170   502 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 12:26:38.532644 32472 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.724s	user 0.018s	sys 0.006s
I20260709 12:26:38.533151 32472 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.533828 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:38.534453 20802 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:38.535902   475 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.539551   475 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.514158 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.540508 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:38.541491 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.542522 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.496127 32419 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.544898 20801 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
W20260709 12:26:38.497094 32725 consensus_meta.cc:287] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 1.046s	user 0.014s	sys 0.001s
W20260709 12:26:38.497551   472 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.229s	user 0.002s	sys 0.002s
I20260709 12:26:38.499620 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.500299 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.501472 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.502559   494 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:38.503994   471 log.cc:927] Time spent T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:26:38.446552   479 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.557897   479 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.558386 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 2222) took 1006 ms. Trace:
I20260709 12:26:38.558786 21432 rpcz_store.cc:276] 0709 12:26:37.552047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.552187 (+   140us) service_pool.cc:224] Handling call
0709 12:26:38.558230 (+1006043us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54005df7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:38.558332 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856562432}
W20260709 12:26:38.559338   431 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.009s
I20260709 12:26:38.559762   431 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.560367   479 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.561486 20878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4318) took 1001 ms. Trace:
I20260709 12:26:38.513069 20803 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: [33.18-33.18]
I20260709 12:26:38.513294 20877 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:38.514130   380 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.514462 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.543000 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:38.544602 21431 raft_consensus.cc:2368] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:26:38.545526 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.546938 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.547794 20802 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:38.549319 21430 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
W20260709 12:26:38.548246 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59938.6us
I20260709 12:26:38.571903 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.572211 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.574290 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.577572 20850 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.510986   508 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.561858 20878 rpcz_store.cc:276] 0709 12:26:37.560108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.560464 (+   356us) service_pool.cc:224] Handling call
0709 12:26:37.569332 (+  8868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.637684 (+ 68352us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:37.684572 (+ 46888us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.788545 (+103973us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:37.803493 (+ 14948us) log.cc:844] Serialized 26 byte log entry
0709 12:26:37.803628 (+   135us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:37.803663 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.803736 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.466698 (+662962us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:38.561217 (+ 94519us) raft_consensus.cc:1685] finished
0709 12:26:38.561308 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.561430 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12687,"prepare.queue_time_us":14912,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":19662,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":15843,"spinlock_wait_cycles":23451392,"thread_start_us":43164,"threads_started":1,"wal-append.queue_time_us":443632}
I20260709 12:26:38.573443 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.585505 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.588483 20729 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:38.588666 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.589414 20801 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:38.586212 20732 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:38.590723 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.591355 20729 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:38.595546 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.595880 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:38.596361 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:38.596536 20803 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:38.598745 20807 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: [34.20-34.20]
I20260709 12:26:38.595362 32739 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:38.599835 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:38.600386 20805 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:26:38.601557 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.601909   477 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.604677 32586 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:38.606122   401 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:38.606539   401 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.606878   401 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:38.613088 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.613581   372 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.614185 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 68726us
I20260709 12:26:38.614295 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.614794   500 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.615002   484 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:38.615365   491 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:38.615486   484 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.615662   511 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.615842   491 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:38.615943   484 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:38.616281   491 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:38.617093 32586 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:38.618242 32472 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.564697 32725 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.615317   500 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.620425 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:38.620837 32725 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.621116   500 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.588166   378 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:38.621618 32739 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:38.621786   378 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.622246   378 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:38.622442 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.614102   372 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.622673 32739 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.624248 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.624677   372 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:38.625731 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.627511 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.628118 20878 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:38.628532 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2224) took 1008 ms. Trace:
I20260709 12:26:38.628911 21434 rpcz_store.cc:276] 0709 12:26:37.619761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.619883 (+   122us) service_pool.cc:224] Handling call
0709 12:26:37.632771 (+ 12888us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:37.674207 (+ 41436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.674240 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:37.735220 (+ 60980us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.125663 (+390443us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:38.125721 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.125813 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.628289 (+502476us) raft_consensus.cc:1685] finished
0709 12:26:38.628311 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.628461 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25368,"raft.queue_time_us":37284,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":23563,"thread_start_us":309707,"threads_started":1,"wal-append.queue_time_us":851552}
I20260709 12:26:38.563908   445 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.631991 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.633773   445 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.633813 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.634567   412 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.635156   412 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.635708 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.635980 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:38.636303 20877 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:38.637372   412 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.637759 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.638777   370 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:26:38.639302   454 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.000s
I20260709 12:26:38.639544 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:38.640410 20809 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:38.639752   454 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.628899   496 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.590555   508 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:38.641317   496 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.616228   511 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.506225 32563 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.643673   511 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.643815 32362 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.645010 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.590328 32419 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:38.649140 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.640679 20776 leader_election.cc:400] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:26:38.652670 20776 leader_election.cc:403] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:38.641209 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.634752 20882 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:38.648262 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.561941   431 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.664592 20877 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:38.665038 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:38.666051 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.667923 32622 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.668063 20807 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:26:38.668392 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.669488 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:38.666651 20732 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:38.670037 20875 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:38.672837 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.673959 32622 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:38.674701 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.675321 20727 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:38.675722   454 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.676324 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.677671 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:38.678258 20733 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:38.679347   412 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:38.674540 32622 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.680194 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.682458 32622 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.683947 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.689208 20730 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:38.690521 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.691999 20805 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:38.692334 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.693017 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.694216 20805 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:38.696033 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.691349 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:38.698668 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:38.702271 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.689558 20733 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:38.703961 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.730254 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.753397 20875 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:38.696086 20735 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:38.760583 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.761561 20878 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:38.762286 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.693852 32362 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.768986   456 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:38.771931   456 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.774236   456 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.778455 20803 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:38.779431   496 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.779740 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.780985   496 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.781477   496 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.781564 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.783401   484 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.694600   445 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35.
I20260709 12:26:38.786353 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.792062   456 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:38.792779   456 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.794154   453 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.790160   448 raft_consensus.cc:3055] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:38.784668   499 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:38.792177 32659 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:38.702244   431 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 33.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.087s
I20260709 12:26:38.803408 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.796959 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.770136 20810 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:38.785710 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:38.802628 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:38.804437 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:38.799971 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:38.809327 20878 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: [39.26-39.26]
I20260709 12:26:38.761472 32472 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.756608 32622 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:38.823527 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.822850 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:38.824992 21431 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:38.820515 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:38.826262 20726 raft_consensus.cc:2368] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:38.795363   448 raft_consensus.cc:740] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:38.795151   456 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.829648 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:38.832086 20732 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:38.803232   454 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.839468 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.838053 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:38.819072 32561 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:38.846254 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.847048 20875 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:38.795732   508 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.848716   494 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:38.849159   494 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:38.849568   494 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:38.850698   508 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.830955 32555 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:38.845593   454 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.854357   454 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:38.860857 20732 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 40 because replica has last-logged OpId of term: 39 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 25.
I20260709 12:26:38.868185 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.870383 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:38.886567 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 249772us
I20260709 12:26:38.887475 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.814559   374 consensus_queue.cc:1059] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
I20260709 12:26:38.822188 32713 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 29.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.040s
I20260709 12:26:38.845074   404 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:38.822252 32362 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.040s
I20260709 12:26:38.847110   496 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:38.856685 32561 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.931022 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.831504 32622 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.938064 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:38.940057 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52696.5us
I20260709 12:26:38.942713 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:38.944044 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:38.918358   404 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:38.933843   396 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.003s	sys 0.024s
I20260709 12:26:38.855897 32472 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:38.932085 32739 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:38.952831 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.925896 32561 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:38.939028   492 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.950269   396 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.004s	sys 0.025s
I20260709 12:26:38.961818 20732 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 35 index: 19. (index mismatch)
I20260709 12:26:38.945572 32419 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:38.964295 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:38.933981   412 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:38.947989   404 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:38.989364 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:38.956701 32739 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:38.995867 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:38.974095   484 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:38.993611 32316 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.519s	user 0.001s	sys 0.009s
I20260709 12:26:39.003798 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:38.988377   322 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:39.013830   516 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.091s	user 0.001s	sys 0.001s
I20260709 12:26:39.015913   527 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:39.016860   527 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:39.017452   527 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.015377   322 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.023172   412 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.025581 32561 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.027719 20732 raft_consensus.cc:2368] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:38.984275   492 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.031332 32739 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.19, Last appended by leader: 19, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.032517 32739 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:38.947836   483 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:39.035732   519 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:26:39.037648 20802 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:38.961575   378 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.011317   528 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:39.045728   494 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.048883 32419 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.051218 20803 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:39.009842   401 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.055418 20801 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:39.057941 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:39.009045 32316 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:39.060309   509 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:39.060731   509 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:39.061121   509 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:39.063580 20883 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:26:39.066767   491 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.068782 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.069937 20809 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:39.070278 21434 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:39.003041   495 consensus_meta.cc:287] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.579s	user 0.005s	sys 0.005s
I20260709 12:26:39.062685   374 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:39.079684   495 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.085201 32316 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:39.088634 20806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 4349) took 1146 ms. Trace:
I20260709 12:26:39.089434   448 consensus_queue.cc:260] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 39.26, Last appended by leader: 26, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.090541 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.098053 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:39.097848 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.096580 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.095504 20806 rpcz_store.cc:276] 0709 12:26:37.941868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.945720 (+  3852us) service_pool.cc:224] Handling call
0709 12:26:39.088437 (+1142717us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400126e54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:39.088553 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346041728}
I20260709 12:26:39.102869 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.103722 20881 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.104110 20882 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:39.106287 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.106567 20875 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:39.106854 20805 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.108139 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.120131 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:39.120214 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 176259us
W20260709 12:26:39.124058 20849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.130033 20735 raft_consensus.cc:2368] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:39.127302 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.130903 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.131865 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.134083 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.131430 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.133776 20809 raft_consensus.cc:2393] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:39.097787   496 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:39.141757 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.137269 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.137559 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.144572 20733 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:39.144853 20730 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 35 because replica has last-logged OpId of term: 32 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 19.
I20260709 12:26:39.092829   374 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.147797 20884 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 40: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:39.136037 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.139369 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.150172 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.129354   448 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:39.150995 20801 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:39.156044 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.158986 20809 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.162884 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.163689 20806 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:39.161439   533 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.014s
I20260709 12:26:39.175868   475 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:39.178362   495 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.179328   494 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.179351   535 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:39.180605   535 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.183208 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.119647 32316 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.187111 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.192891 20806 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:39.191771 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52519.5us
I20260709 12:26:39.194542 20849 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.195823 20805 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:39.204270   494 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.204787   494 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.206724   494 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.211556   475 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:39.211993   475 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.211314   374 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.213183   374 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.212858   475 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.214491   475 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.220527 20881 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:39.223745 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.224774 20805 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 38 index: 21. Preceding OpId from leader: term: 40 index: 22. (index mismatch)
I20260709 12:26:39.225142 20809 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
W20260709 12:26:39.222150 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58025.5us
I20260709 12:26:39.186136   474 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:39.231807 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.201412   527 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.232853   534 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.233079 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.233767 20735 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:39.233474   534 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.234637 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.238783 20728 raft_consensus.cc:1240] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:39.202983   492 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.237267 20735 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:39.237895 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.242144 20731 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:39.229562 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:39.222863   396 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:39.246762 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.247689 20728 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 31: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:39.249889 20805 raft_consensus.cc:1240] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 33. Current term is 34. Ops: [33.18-33.18]
W20260709 12:26:39.250779 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56332.5us
I20260709 12:26:39.251590 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.236374   495 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:39.257120 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.258508 20731 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:39.263665 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.264351 20731 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:39.267513 20806 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:39.247179   492 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:39.238183   498 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.025s
I20260709 12:26:39.274748 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.276028 20735 raft_consensus.cc:2410] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 42 because replica has last-logged OpId of term: 40 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 39 index: 23.
I20260709 12:26:39.256327   527 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:39.272697 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.282601 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.283393 20728 raft_consensus.cc:3060] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:39.285991 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:39.300432 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:39.303630 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.306670 20735 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:39.295872   492 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.309890   534 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.298183   527 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:39.315812   527 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:39.316198   527 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:39.316440 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:39.318292 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.321065   535 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:39.321580 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.321691   535 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:39.322150   535 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:39.322510   535 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.323283 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.323580   396 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:39.324759 32316 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.307307   495 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:39.325490   495 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:39.325901   495 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:39.325964 32563 raft_consensus.cc:3055] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:39.326499 32563 raft_consensus.cc:740] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.327404 32563 consensus_queue.cc:260] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.328467 32563 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:39.332482 32472 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.334348 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.335280 20802 raft_consensus.cc:2410] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 41 because replica has last-logged OpId of term: 38 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 20.
I20260709 12:26:39.340308   491 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.343269   528 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:39.348708 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 241492us
I20260709 12:26:39.356128   474 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:39.356621   474 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.357512   474 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.358573   474 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:39.359812   492 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.365065 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.366523 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:39.367079 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.384013 20728 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:39.386951 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.387969 20805 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 35 because replica has last-logged OpId of term: 32 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 19.
I20260709 12:26:39.388172 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.391284   496 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election lost for term 40. Reason: could not achieve majority
I20260709 12:26:39.391635   404 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.392467 20732 raft_consensus.cc:1240] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:39.393501 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.393746   404 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.398651 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.410741 20849 leader_election.cc:400] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:39.410794 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.416457 20735 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 17. Preceding OpId from leader: term: 36 index: 18. (index mismatch)
I20260709 12:26:39.419638 20849 leader_election.cc:403] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.420176 20805 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:39.396402   503 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:39.412954 20775 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:39.395753   456 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:39.400501   496 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:39.429517 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.432276 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.396250   485 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 36.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.029s
I20260709 12:26:39.415613   445 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:39.399864 32472 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 34.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.027s
I20260709 12:26:39.424166   503 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:39.429330   534 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 38
I20260709 12:26:39.441147   503 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:39.427026 32554 raft_consensus.cc:2804] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:39.445577 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.445998 32554 raft_consensus.cc:697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:39.446342 20730 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:39.447027 32554 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.448334   431 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:39.448937   431 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.449469   535 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.415794   509 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.427091   456 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.409605   491 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 29.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.020s
I20260709 12:26:39.455363 20801 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:26:39.470124 20732 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 35 because replica has last-logged OpId of term: 34 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
I20260709 12:26:39.463328 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.452564 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.478642 20849 leader_election.cc:400] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:39.454780 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 38 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.467092 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.478969 20849 leader_election.cc:403] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.512822 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:39.515563 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:39.517598 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 91798us
I20260709 12:26:39.522872 20849 leader_election.cc:400] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:39.523412 20849 leader_election.cc:403] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.459517   456 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:39.531481 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.451894   485 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:39.532348 20733 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 34 because replica has last-logged OpId of term: 33 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 17.
I20260709 12:26:39.532337 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.533231 20735 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:39.453137   376 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:26:39.528461 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.451164   431 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.537478 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:39.538599 21432 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:39.463784 32622 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.464247   535 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.530946 32587 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:39.455370   453 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.452613   483 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.457737 32561 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.565203 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.529232 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2820) took 1057 ms. Trace:
I20260709 12:26:39.566500 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.566589 21426 rpcz_store.cc:276] 0709 12:26:38.472040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.472211 (+   171us) service_pool.cc:224] Handling call
0709 12:26:38.486660 (+ 14449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.561366 (+ 74706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.561406 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.561947 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.802683 (+240736us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:38.802738 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.528861 (+726123us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b540047ead4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:39.528914 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.528958 (+    44us) raft_consensus.cc:1685] finished
0709 12:26:39.528970 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.529177 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73419,"raft.queue_time_us":53641,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7330,"spinlock_wait_cycles":32682240,"thread_start_us":240332,"threads_started":1,"wal-append.queue_time_us":386240}
W20260709 12:26:39.542079 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.571568 32561 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:39.571933 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.572072   509 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.530524   372 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.541913 21428 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:26:39.573625   372 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.576251   372 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.539922 20849 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.548233 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:39.578121 32739 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:39.510978   496 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:39.579547   496 raft_consensus.cc:2804] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:39.572182 32561 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.583658 32561 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.540865 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:39.584882 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44154.1us
I20260709 12:26:39.585983 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.522480 32316 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:39.586094 20733 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:39.587467 32316 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:39.588009 32316 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:39.591873   320 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:39.592518   320 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.558908 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:39.557907 20730 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:39.593891 21431 raft_consensus.cc:2368] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
I20260709 12:26:39.562391 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.594488 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.522406   396 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:39.454125   445 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.539707   376 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:39.461144   503 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.589772 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.598423 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:39.598543 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.518218   533 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:39.536535   485 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:39.606669 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:39.551978   524 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.613196 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.613832 20878 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:39.605041 20733 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:39.615489 20700 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:39.580114   496 raft_consensus.cc:697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:39.616436 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.621138   456 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.597186 32362 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:39.599464 32561 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:39.542969   527 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.629348 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:39.630054   320 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 29.23, Last appended by leader: 23, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.631596 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.632730 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.643263 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.617483   372 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:39.632088   374 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:39.665403 20875 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:39.627123   492 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.006s
I20260709 12:26:39.672236 32362 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:39.672273 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.674482   372 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.674968   372 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:39.675148 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.675843 20726 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.594197 32587 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.679615   322 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.680786   503 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.624269   517 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:39.687311 20878 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:39.688455 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.632424   491 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:39.632544   485 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 36.25, Last appended by leader: 25, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.690915   491 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.691530   485 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:39.691869   534 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 37. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:39.693894   491 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.694151   527 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.695837 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.705634 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.706573 20804 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:39.696630 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.698758 20734 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 35 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 22.
I20260709 12:26:39.700593 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.708431 20728 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.702080 32587 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:39.702291   514 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.746s	user 0.002s	sys 0.003s
I20260709 12:26:39.702905   445 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:39.703760 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.709805 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.715142 20732 raft_consensus.cc:2368] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
I20260709 12:26:39.715866 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.685087   496 consensus_queue.cc:237] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 35, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.710525 20802 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:39.717593 20802 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:39.719528 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.720252 20884 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:26:39.618173   495 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.728942   495 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.711735 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.729408 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.729979 20733 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:39.730614 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.731151 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.731805 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:39.732734 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:39.733641 21431 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:39.729949   453 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:39.689831   517 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.737339   517 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:39.737749   445 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:39.739284 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.740723 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:39.745896 20735 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:39.749760 20884 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:39.747174 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.755120 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.778450 21431 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
W20260709 12:26:39.736553   542 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:26:39.788547 20810 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:39.789573 20733 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:39.791311 20728 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
W20260709 12:26:39.780405 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 201354us
I20260709 12:26:39.781162 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:39.793468 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.752668   503 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:39.794898 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.803306 20726 raft_consensus.cc:2468] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:39.805461 20729 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:39.817070 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.804517 32362 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.821130 20726 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:26:39.794979   456 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.816821   499 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.801172   495 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:39.838514 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:39.838394 32316 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.840147 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.841744 32316 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.842160   370 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:39.843111   370 raft_consensus.cc:3060] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.850309 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:39.851500 32587 raft_consensus.cc:2764] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election decision vote started in defunct term 39: won
I20260709 12:26:39.852320 20733 raft_consensus.cc:1240] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:39.852833   448 raft_consensus.cc:3055] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:39.853376   448 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:39.855850 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:39.859815   473 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:39.860255   473 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.860630   473 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.863587   401 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:39.868585 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:39.868703   376 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 29.23, Last appended by leader: 23, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:39.869601   503 raft_consensus.cc:2764] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election decision vote started in defunct term 37: won
I20260709 12:26:39.870013   376 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:39.872942   378 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:39.876418 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:39.877393 21399 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:39.879118   453 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:39.880482 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:39.881867   509 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:39.882709   509 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:39.883733   509 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 30.24, Last appended by leader: 24, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:39.885390   552 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:26:39.888234 20850 leader_election.cc:400] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
I20260709 12:26:39.888759 20850 leader_election.cc:403] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.889074   370 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:39.889552   548 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 12:26:39.890944 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:39.891572 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:39.892359 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:39.832551 32362 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 36.22, Last appended by leader: 22, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:39.893898   546 log.cc:927] Time spent T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:26:39.895056   495 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
I20260709 12:26:39.833659   551 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:39.897388 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:39.899150 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:39.899842 21434 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:39.900483 21434 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 34.
I20260709 12:26:39.888311 20804 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:39.902577 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:39.903275 21434 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:39.892549   380 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:39.909050 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:39.831542 20730 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:39.910817 20805 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:39.902886 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:39.800208   555 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.111s
I20260709 12:26:39.892432 20807 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:39.804294   374 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:39.916600 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.916702 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.917790 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.934315 20883 raft_consensus.cc:1813] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Handling vote request from an unknown peer 657bc62d20374e539b23641715f22745
I20260709 12:26:39.918095 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.914640 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:39.919605 20876 raft_consensus.cc:1275] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 17. Preceding OpId from leader: term: 36 index: 18. (index mismatch)
I20260709 12:26:39.920502 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:39.923311 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.935657 20882 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:39.943301 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.946465 20884 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:26:39.947347 21428 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:39.950073 20879 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:39.956133 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:39.956413 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54623.3us
I20260709 12:26:39.956727 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.956976 20883 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:39.854456   448 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.957527 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:39.963289 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:39.965075 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50501.8us
I20260709 12:26:39.910140   453 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:39.979597 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.982522 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:39.986091 20880 raft_consensus.cc:2368] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:39.971725 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:39.972478 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.970090 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.973706 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.975472 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:39.976240 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.990836 20883 raft_consensus.cc:2410] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 36 because replica has last-logged OpId of term: 35 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 18.
I20260709 12:26:39.992249 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.993165 21433 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:39.993681 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:39.919994   445 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:39.982913 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:39.985585 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:39.995734 20804 raft_consensus.cc:2368] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39.
I20260709 12:26:39.920248   484 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.016047 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:40.016717 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:39.999729 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:40.025806 20726 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:39.955682   370 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:39.958434   380 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:40.026823 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59925.9us
I20260709 12:26:39.997514 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:40.055644 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:40.059444 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:40.059859 20884 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:39.997776 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:40.061460 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:39.978313   534 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:39.996259   468 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:40.072100 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.085597 20730 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
W20260709 12:26:40.081205 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83505.5us
I20260709 12:26:40.081723 20883 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:40.082261 20807 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:26:40.087030 20849 leader_election.cc:400] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31.
I20260709 12:26:40.056497   491 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:40.064013 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 63940.9us
I20260709 12:26:40.088034 20849 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:40.017119   508 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:40.015508   461 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:39.993026 32316 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:40.089771 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:40.092599 20883 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:26:40.015167   528 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:40.091040 20878 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:39.973042   527 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.110s
I20260709 12:26:39.972754   456 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.110s
W20260709 12:26:40.092777 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 117393us
I20260709 12:26:40.094228 20849 leader_election.cc:400] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39.
I20260709 12:26:40.106328 20849 leader_election.cc:403] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Cancelling election due to peer responding with higher term
I20260709 12:26:40.096467   508 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:40.107300   508 raft_consensus.cc:3060] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:40.107959 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:40.111141 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:40.112447 20805 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 41: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:40.113485 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:40.115682 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:40.101574 20726 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:40.102995 20735 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:39.989714   372 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:39.995709   454 consensus_queue.cc:1059] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 34.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.106s
I20260709 12:26:40.082846   551 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:40.106926   461 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:40.093508   496 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:40.100220   431 raft_consensus.cc:2804] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:40.058034   484 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.164454 20732 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 39: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:40.128458 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:40.100267   534 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:40.171938   534 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 37.22, Last appended by leader: 22, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.123737 32554 consensus_queue.cc:1059] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 34.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.049s
I20260709 12:26:40.124022   374 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 33.17, Last appended by leader: 17, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.093698   370 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:40.087690   491 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.167670   380 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:40.176734 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 48369.4us
W20260709 12:26:40.177754 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 199213us
I20260709 12:26:40.186995 21434 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 35 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 22.
I20260709 12:26:40.155601   483 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:40.183668   412 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:40.163012   551 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.155783   496 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.190537   473 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.191565   508 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.191884   517 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.193958   484 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.194334   431 raft_consensus.cc:697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:40.199054   524 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 12:26:40.197932   461 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:40.104041 32316 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.203478 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.155279   499 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.046s
I20260709 12:26:40.187436 20882 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:40.195808   480 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:40.198411   453 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:40.239941 20731 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:40.185514 32622 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:40.240715 32622 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:40.241125 32622 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:40.241732   551 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:40.241887   499 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:40.228047   524 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:26:40.242267   499 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:40.243130   499 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.243853   491 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:40.243924   370 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.245081 21426 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:40.246616   372 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.249372   528 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:40.252446   508 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:40.253468 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.238577   527 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election lost for term 38. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39.
I20260709 12:26:40.238528   494 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:26:40.245153 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.207048   496 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:40.244567   499 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:40.255304   401 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:40.228688 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.262308 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:40.263610 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:40.284502 20809 raft_consensus.cc:2393] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:40.263042 20877 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 34
I20260709 12:26:40.263996 32316 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:40.264837   473 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.268831 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:40.295225 20882 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:40.268894   485 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
W20260709 12:26:40.199014 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 216167us
I20260709 12:26:40.257392   454 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:40.280882   480 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:40.270866 20734 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:40.272498   517 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.273113   431 consensus_queue.cc:237] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.225896   461 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:40.280297   557 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.484s	user 0.002s	sys 0.001s
I20260709 12:26:40.310729 32316 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:40.300242 20776 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 34.
I20260709 12:26:40.311833 20776 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:40.312875 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.313830 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:40.297947 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 208799us
I20260709 12:26:40.283982   535 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.314997 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:40.315601   535 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:40.305371 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 76735.3us
I20260709 12:26:40.316468   453 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:40.317751   535 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:40.315083   565 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 34
I20260709 12:26:40.305431   322 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:40.306900 32739 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:40.319947 32739 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:40.320812 32739 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.321993 32739 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:40.322819 32561 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:40.307729   461 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:40.326361 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:40.318028 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.318181   448 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:40.328006 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.330302 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:40.331911 20806 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:40.332880 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:40.333619 20801 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 39: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:40.334339   473 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:40.335410   562 log.cc:927] Time spent T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.229s	user 0.000s	sys 0.003s
I20260709 12:26:40.335604 20734 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:26:40.337389 20807 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
W20260709 12:26:40.338827 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.301669   480 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.340631 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:40.341065   567 raft_consensus.cc:3055] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:40.341512   567 raft_consensus.cc:740] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:40.342276   567 consensus_queue.cc:260] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.19, Last appended by leader: 19, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.304638 21428 raft_consensus.cc:1275] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:26:40.300895 21427 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 23. Preceding OpId from leader: term: 30 index: 24. (index mismatch)
I20260709 12:26:40.213660   445 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:40.279007   496 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.341205 20807 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:40.359876 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:40.350244 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.360930 20734 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:40.361552 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.362908 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.363590 20730 raft_consensus.cc:2368] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34.
I20260709 12:26:40.351711 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.310302 20882 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:40.355674 20877 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:40.316300 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.352850 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:40.369129 20730 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
W20260709 12:26:40.370338 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 254862us
I20260709 12:26:40.373787 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:40.391474 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75242.1us
I20260709 12:26:40.391343 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.392298 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:40.392369 20726 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:40.386206 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:40.396215 20849 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34.
I20260709 12:26:40.396533 20849 leader_election.cc:403] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:40.370039 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:40.398605 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48623.3us
I20260709 12:26:40.400429 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.289983 20881 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:40.406039 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:40.406651 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:40.426352 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.444303 20728 raft_consensus.cc:2468] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:40.390332 32316 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election lost for term 40. Reason: could not achieve majority
I20260709 12:26:40.447088 21434 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:40.405737   528 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:40.349578 32554 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.082s
I20260709 12:26:40.405730 32587 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:40.425843   567 raft_consensus.cc:3060] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:40.437191   524 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 33. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34.
I20260709 12:26:40.425230   571 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:40.368634   453 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.078s
I20260709 12:26:40.435984 32659 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:40.316326   483 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.120s
I20260709 12:26:40.507218 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:40.550632 20878 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 39: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:40.509096 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:40.503933 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:40.398075   448 consensus_queue.cc:1048] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.064s
I20260709 12:26:40.517979 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:40.359232   461 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.399782   508 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.055s
I20260709 12:26:40.520691   572 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:40.521557   569 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:40.522749   495 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:40.530867 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 54285.6us
I20260709 12:26:40.532229 20801 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:26:40.405579   475 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.065s
I20260709 12:26:40.539110 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:40.424853   555 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.081s
I20260709 12:26:40.436394 32563 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.080s
I20260709 12:26:40.516355   528 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:40.560241 21426 raft_consensus.cc:1275] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:26:40.572803   468 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:40.585278 20806 raft_consensus.cc:1240] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:40.583895 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:40.563231   575 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:40.562494 32659 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:40.506611   445 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.563738 32554 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.058s
I20260709 12:26:40.616978   370 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:40.585289   524 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.064s
I20260709 12:26:40.580310   495 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.626600   431 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:40.624131   412 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.073s
W20260709 12:26:40.690487 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.691857 32659 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.24, Last appended by leader: 24, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.692865   492 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:40.631680   572 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:40.693956   496 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:40.693585   528 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 40.23, Last appended by leader: 23, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.617690   575 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:40.634498   565 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 31. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 34.
I20260709 12:26:40.636343   302 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:26:40.703470 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:40.697997   551 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:40.701717 20801 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 22. Preceding OpId from leader: term: 38 index: 23. (index mismatch)
W20260709 12:26:40.699663 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 264343us
I20260709 12:26:40.718494 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:40.709846   572 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.723943   496 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:40.724709   567 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:40.716398   551 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.732376   576 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 12:26:40.735762   570 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:26:40.738375 20807 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:40.737490   551 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:40.741288 32622 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.742218   461 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.732491 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:40.742851 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:40.776057 20774 leader_election.cc:400] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:40.698465   370 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.760720 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:40.778839 20774 leader_election.cc:400] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:40.782977 20774 leader_election.cc:403] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:40.780215 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:40.786175 20849 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:40.784102 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:40.795957 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.796088   551 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:40.796607 20729 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:40.762946 32739 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:40.798146   572 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:40.797874 32739 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.787290   576 consensus_queue.cc:1048] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 12:26:40.801307   551 raft_consensus.cc:2804] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:40.801899   551 raft_consensus.cc:697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 34 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:40.803133   551 consensus_queue.cc:237] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.807440   524 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:40.807854   524 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.807809 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:40.808391   524 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:40.718629   575 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.25, Last appended by leader: 25, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.810174 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:40.840359 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.783846   370 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:40.784267   567 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:40.810446   483 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:26:40.848824   468 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.852003   495 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:40.828692   453 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:40.854486 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:40.880905   494 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.881736   494 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:40.840333 32316 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:40.891574   453 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:40.923372   560 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.751s	user 0.005s	sys 0.000s
I20260709 12:26:40.925441 20808 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4365) took 1297 ms. Trace:
I20260709 12:26:40.925858 20808 rpcz_store.cc:276] 0709 12:26:39.627661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.627992 (+   331us) service_pool.cc:224] Handling call
0709 12:26:39.633158 (+  5166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.633630 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:39.672470 (+ 38840us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.680100 (+  7630us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b18005301a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:39.880829 (+200729us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b18005301a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:26:40.210094 (+329265us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.226480 (+ 16386us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.226702 (+   222us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:40.238900 (+ 12198us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.239081 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.239158 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.338273 (+ 99115us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b18005301a0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:40.922782 (+584509us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:40.925167 (+  2385us) raft_consensus.cc:1685] finished
0709 12:26:40.925196 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.925372 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47808,"prepare.queue_time_us":51686,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":147934,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":41105,"spinlock_wait_cycles":109997440,"thread_start_us":284363,"threads_started":2,"wal-append.queue_time_us":499320}
I20260709 12:26:40.921939   494 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.950060 32622 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.951905 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:40.954883   372 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.955533   372 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.959736 20801 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 17. Preceding OpId from leader: term: 35 index: 18. (index mismatch)
I20260709 12:26:40.960523 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:40.961122 20805 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:40.961359 21432 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:40.964020   554 consensus_queue.cc:1059] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 34.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:40.938114   453 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 37.22, Last appended by leader: 22, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.964881   554 raft_consensus.cc:3055] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:40.965238   554 raft_consensus.cc:740] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:40.965975   554 consensus_queue.cc:260] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:40.966711 32561 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 40.22, Next index: 23, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:40.959540   454 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.973554 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:40.977447 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:40.977849 20810 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:40.979463 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:40.972738   372 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:40.980448   302 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:40.980991   302 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.967118   554 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:40.967705   555 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 12:26:40.974185   454 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:40.976450   558 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.878s	user 0.002s	sys 0.001s
I20260709 12:26:40.978127 20801 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:40.981398   302 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:40.999241   454 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:41.001655 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.974409 32739 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:40.987514 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:40.981500 32472 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:40.992479 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2315) took 2343 ms. Trace:
I20260709 12:26:41.005828 32472 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:41.007357 32739 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:41.007812 32739 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:41.008185 32739 raft_consensus.cc:3060] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:41.008385 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:41.009330 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:41.010090 20728 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:41.010834   527 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:41.007169 32472 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.014366 32472 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:41.014879 20729 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:26:41.018347   503 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:41.018857   503 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.020999   503 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:40.998579   575 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:41.026749 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:41.033221 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:41.029255 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:41.033926 21428 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:41.035887 21427 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: [34.25-34.25]
I20260709 12:26:41.036953   528 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:41.006518 20803 rpcz_store.cc:276] 0709 12:26:38.648590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.052022 (+403432us) service_pool.cc:224] Handling call
0709 12:26:39.052396 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.052777 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.052805 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.710659 (+657854us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b2c0005bf68. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:39.736095 (+ 25436us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.914472 (+178377us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:39.964517 (+ 50045us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.313054 (+348537us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b18002d5920. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1e8474bc 00007f9e1bb9548b 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287
0709 12:26:40.313172 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.313248 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.975863 (+662615us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:40.992258 (+ 16395us) raft_consensus.cc:1685] finished
0709 12:26:40.992278 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.992406 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28959,"prepare.queue_time_us":50116,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":120456,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":4300,"spinlock_wait_cycles":38559104,"thread_start_us":219179,"threads_started":2,"wal-append.queue_time_us":190237}
W20260709 12:26:41.040428   561 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.868s	user 0.005s	sys 0.000s
I20260709 12:26:41.042714 20877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 2268) took 1058 ms. Trace:
I20260709 12:26:41.043100 20877 rpcz_store.cc:276] 0709 12:26:39.984478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.061186 (+ 76708us) service_pool.cc:224] Handling call
0709 12:26:41.042562 (+981376us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400306454. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:41.042657 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461873920}
I20260709 12:26:41.043982 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:41.045943   566 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:41.046298   566 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.046612   566 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:41.051025 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:41.052480   524 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.054565   524 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:41.058538 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:41.050875   564 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.879s	user 0.001s	sys 0.001s
I20260709 12:26:41.059971 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4338) took 1062 ms. Trace:
I20260709 12:26:41.060492 20731 rpcz_store.cc:276] 0709 12:26:39.997730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.999267 (+  1537us) service_pool.cc:224] Handling call
0709 12:26:40.060895 (+ 61628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.240443 (+179548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.059760 (+819317us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54001d8154. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:41.059916 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411330304}
I20260709 12:26:41.063066 20877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:41.064030   498 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:41.064342 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:41.063879 20877 raft_consensus.cc:606] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:41.065527 20877 raft_consensus.cc:614] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:41.065865 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:41.066731 32563 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:41.068405 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:41.068526 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:41.069041 20878 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:26:41.071735   483 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 31. Current term is 32.
I20260709 12:26:41.075304 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:41.075879 20875 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:41.077919 20881 raft_consensus.cc:1240] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: [33.23-33.23]
I20260709 12:26:41.083053 20875 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41.
I20260709 12:26:41.083847 20877 raft_consensus.cc:1240] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:41.084998 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:41.085340 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:41.087935 20878 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:41.093886 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:41.094763 20881 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:41.096311 20875 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 22. Preceding OpId from leader: term: 38 index: 23. (index mismatch)
I20260709 12:26:41.099320 20877 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 17. Preceding OpId from leader: term: 35 index: 18. (index mismatch)
I20260709 12:26:41.099260 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:41.101655 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:41.102380 20730 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:41.123337 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.154075 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:41.147204 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49054.6us
I20260709 12:26:41.153677 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:41.161499 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:41.163265 21433 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 38 index: 23. (index mismatch)
I20260709 12:26:41.169744 20732 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:26:41.172789 20734 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:41.166428 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:41.156406   535 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:26:41.131414   551 consensus_queue.cc:1048] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.062s
I20260709 12:26:41.201120   322 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:41.146508   503 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:41.190136 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:41.195766 20809 raft_consensus.cc:1275] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:26:41.202935 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:41.204056 20805 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 22. Preceding OpId from leader: term: 38 index: 23. (index mismatch)
I20260709 12:26:41.198354 20730 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:41.046604   370 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.210017   576 consensus_queue.cc:1059] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.006s
I20260709 12:26:41.211925 20875 raft_consensus.cc:606] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:41.146680   607 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:41.215981 20875 raft_consensus.cc:614] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:41.216157 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:41.186642   534 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.221601 20727 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:41.228183   534 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.207301   565 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:41.186472   454 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.055s
I20260709 12:26:41.207510   503 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:41.235224 21434 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: [32.23-32.23]
I20260709 12:26:41.238505 20875 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:41.212570   572 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:41.214761   499 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.026s
I20260709 12:26:41.277904 20875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:41.278599 20875 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:41.204536   498 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 40.22, Next index: 23, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:26:41.278939 20875 raft_consensus.cc:614] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:41.231178   534 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:41.287799 32739 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.244526   503 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:41.290076 32739 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:41.291026 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 74969.2us
I20260709 12:26:41.293874   524 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:41.293926   551 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:41.294531   524 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:41.294646   551 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.295423   524 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.19, Last appended by leader: 19, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.296279 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:41.297084 20801 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 41 because replica has last-logged OpId of term: 39 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 25.
I20260709 12:26:41.297784 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:41.299504 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:41.300621 20728 raft_consensus.cc:2410] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 41 because replica has last-logged OpId of term: 39 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 25.
I20260709 12:26:41.302207 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:41.302970 20801 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:41.268091   565 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:41.325721 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:41.231608   554 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 12:26:41.342321 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:41.351475 20727 raft_consensus.cc:2393] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:41.321899   551 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:41.358364 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 191988us
W20260709 12:26:41.359907 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 141400us
I20260709 12:26:41.282684   474 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.067s
W20260709 12:26:41.359350 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 135819us
I20260709 12:26:41.369853 20805 raft_consensus.cc:2468] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:41.347304   431 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:41.323982   535 consensus_queue.cc:1048] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.038s
I20260709 12:26:41.371006 20849 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:41.220722   302 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.372062   498 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:41.378832 32739 raft_consensus.cc:2804] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:41.399842   498 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.402077   498 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:41.402304   503 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.404557   503 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:41.405270   370 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:41.419703   613 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:41.396777   565 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 40.22, Last appended by leader: 22, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.430406   535 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:41.431020   535 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:41.435910 20809 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 23. Preceding OpId from leader: term: 41 index: 24. (index mismatch)
I20260709 12:26:41.435178 20801 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 34 index: 19. (index mismatch)
W20260709 12:26:41.443576   616 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:41.448385   302 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:41.448314   627 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:41.440739 32739 raft_consensus.cc:697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:41.449360   627 consensus_queue.cc:1243] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 39.23 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 22
I20260709 12:26:41.449898 32739 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.454845 20809 pending_rounds.cc:85] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 22
I20260709 12:26:41.455226 20809 pending_rounds.cc:107] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 39.23
I20260709 12:26:41.455485 20809 raft_consensus.cc:2889] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:41.431957   535 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:41.464809 20776 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.726s (SENT) [suppressed 6 similar messages]
I20260709 12:26:41.417591   566 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.473670   566 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:41.475251 20802 raft_consensus.cc:1240] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:41.480063   555 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:41.480564   555 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:41.481160   456 raft_consensus.cc:3055] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:41.481700   456 raft_consensus.cc:740] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:41.481446   555 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 40.22, Last appended by leader: 22, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.482966   555 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Advancing to term 41
W20260709 12:26:41.495613 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 251538us
I20260709 12:26:41.496407 32659 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:41.497915   485 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.498592   485 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.501075 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:41.502122 20733 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:41.503814 20726 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 22. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
I20260709 12:26:41.504761   378 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:41.509511 20875 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 24. Preceding OpId from leader: term: 32 index: 25. (index mismatch)
I20260709 12:26:41.509554 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:41.510726 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:41.507440 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:41.482693   456 consensus_queue.cc:260] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.516263 20877 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:41.517294 21427 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 33: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:41.517596 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:41.525058 20806 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:41.528268 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:41.528520 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 63738.4us
I20260709 12:26:41.528674 20735 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:41.528939 20881 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
W20260709 12:26:41.532341 21431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2835) took 1576 ms (client timeout 1726 ms). Trace:
W20260709 12:26:41.532806 21431 rpcz_store.cc:269] 0709 12:26:39.956045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.971493 (+ 15448us) service_pool.cc:224] Handling call
0709 12:26:41.532172 (+1560679us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540058d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:41.532289 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2684361088}
I20260709 12:26:41.532212   503 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:41.537850 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:41.536557 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 314087us
I20260709 12:26:41.501441   485 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:41.531054 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:41.542017 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 182886us
I20260709 12:26:41.558085 20806 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:41.560132 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:41.561007 21430 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:41.564982 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:41.565937 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:41.566556 20882 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:41.567325 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53331.4us
I20260709 12:26:41.514489   453 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:41.533916   431 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:41.566587 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:41.539650   544 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.571658 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:41.543745   456 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:41.575425 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:41.602623 20807 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:41.583786 20878 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 40: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:41.579749   590 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.015s
I20260709 12:26:41.604214   527 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:41.568143   612 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
I20260709 12:26:41.597276   544 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.580438   453 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.610795   544 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:41.612231   453 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:41.614856   528 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.015s
I20260709 12:26:41.550649   607 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:41.619827   607 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:41.620788   607 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:41.628355 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59461.4us
I20260709 12:26:41.551285 32316 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.073s
I20260709 12:26:41.534364   569 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.049s
I20260709 12:26:41.639719 20876 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:41.648656 20882 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
W20260709 12:26:41.640806   625 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:26:41.650197 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85331.5us
I20260709 12:26:41.571081 20806 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 34 index: 25. Preceding OpId from leader: term: 35 index: 25. (term mismatch)
I20260709 12:26:41.659541 20806 pending_rounds.cc:85] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 24
I20260709 12:26:41.659850 20806 pending_rounds.cc:107] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 34.25
I20260709 12:26:41.660110 20806 raft_consensus.cc:2889] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:41.661375 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 101576us
I20260709 12:26:41.661810   376 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:41.663331   480 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:41.624294   431 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.687367   473 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:41.687805 21400 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:41.688730   378 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:41.689139   378 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.688958   491 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:41.689508 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:41.689515   378 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:41.690778   644 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:41.692080 20806 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 24. Preceding OpId from leader: term: 32 index: 25. (index mismatch)
I20260709 12:26:41.695065 20802 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:26:41.696233 20884 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:41.699590 20883 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:26:41.704030 20882 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 35 index: 25. (index mismatch)
W20260709 12:26:41.739547 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.670128   503 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 39.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.026s
W20260709 12:26:41.823496 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 128726us
I20260709 12:26:41.824621 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:41.824715 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:41.691241   644 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:41.844883 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:41.871098 20776 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:41.831344   607 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.834911   644 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:41.863750 32472 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.892181 20884 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:41.834610   590 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.050s
I20260709 12:26:41.876550   412 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.882763   483 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:26:41.916110 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.901639   640 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.017s
I20260709 12:26:41.876708 32659 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.025s
W20260709 12:26:42.027493 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97314us
I20260709 12:26:41.903074   534 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.903316   494 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.129s
I20260709 12:26:41.896961 32472 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:41.884341   607 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:42.088572 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45401.2us
I20260709 12:26:42.017959   491 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:42.134538   627 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.135336   627 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.043605   569 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:42.158285   569 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.094388   527 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.171303   527 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.174029   527 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:42.174927   491 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:41.910631   412 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.187420 32472 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.052393   534 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.204743 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.183059   412 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:42.235436   378 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.003s
I20260709 12:26:42.236207   378 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.240686   586 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.241344   586 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.208777   431 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.209789   569 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.248399   586 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:42.253394   378 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.268209 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:42.269928 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.270351   607 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.270588 20880 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:42.271914 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:42.286412 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.287670 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.289134 20882 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 40 index: 22. Preceding OpId from leader: term: 41 index: 23. (index mismatch)
I20260709 12:26:42.290643 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.291404 20882 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:42.303854 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:42.276997   607 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:42.275718   491 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:42.304551   555 consensus_meta.cc:287] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.820s	user 0.012s	sys 0.001s
I20260709 12:26:42.312177 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:42.320183 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:42.309832 32316 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:42.335695   535 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:42.323369   607 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:42.325562   494 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:42.332850   627 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:42.347992 32472 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:42.373871   565 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.374778   607 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:42.378271   567 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:42.400116   658 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:42.413669 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.437644 20807 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:42.345135 32316 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.419783 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.421012 20804 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:42.439575 20804 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:42.424676 21427 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:42.350359   494 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:42.436400 20810 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:26:42.426957 32472 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.441999 20804 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 36.19, Last appended by leader: 19, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.448132 20804 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:42.458720 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:42.447028   456 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.823s	user 0.004s	sys 0.009s
I20260709 12:26:42.440965   567 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.465152 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2243) took 1307 ms. Trace:
I20260709 12:26:42.465309   627 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:42.465188   567 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.465636 21434 rpcz_store.cc:276] 0709 12:26:41.157396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.157908 (+   512us) service_pool.cc:224] Handling call
0709 12:26:41.158206 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.241665 (+ 83459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.464905 (+1223240us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b54005df7d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:42.465091 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176321408}
I20260709 12:26:42.470849   590 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.471380   590 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.474592   590 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.474879   567 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:42.475431   567 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.477385   567 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.484328   483 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:42.484776   644 consensus_meta.cc:287] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
I20260709 12:26:42.485131   483 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.485459   644 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.486766 20879 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:42.490976 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.491884 21431 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:42.492394 20809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4403) took 1038 ms. Trace:
I20260709 12:26:42.492784 20809 rpcz_store.cc:276] 0709 12:26:41.453915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.454269 (+   354us) service_pool.cc:224] Handling call
0709 12:26:41.454643 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:41.470078 (+ 15435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.470124 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:41.470825 (+   701us) log.cc:844] Serialized 52 byte log entry
0709 12:26:41.637404 (+166579us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:41.668156 (+ 30752us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.673485 (+  5329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.673589 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.265414 (+591825us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b1800530b00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:42.490090 (+224676us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b1800530b00. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:42.492180 (+  2090us) raft_consensus.cc:1685] finished
0709 12:26:42.492203 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.492342 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6738,"prepare.queue_time_us":14671,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":1086,"raft.queue_time_us":16580,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":3183,"spinlock_wait_cycles":20216576,"thread_start_us":153167,"threads_started":1,"wal-append.queue_time_us":176395}
I20260709 12:26:42.346650   652 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 19, Time since last communication: 0.109s
I20260709 12:26:42.445990   494 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:42.495137 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.500133 20875 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:42.497223 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.499910 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:42.449307 32472 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
W20260709 12:26:42.503129 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.506392 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.438701   607 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.438925   565 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.449040 32622 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.057s
I20260709 12:26:42.495952   534 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:42.515738 20879 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
W20260709 12:26:42.515254 20806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.022s	sys 0.049s
W20260709 12:26:42.516469 20806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.023s	sys 0.049s
W20260709 12:26:42.511508 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 191421us
I20260709 12:26:42.509969   652 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:26:42.519999 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.507155   509 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.013s
I20260709 12:26:42.532188 20875 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:42.533360 20877 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 22. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
I20260709 12:26:42.536711 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.537477 20877 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:42.520903   652 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.544436 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:42.548250 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.555447   565 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.550315   576 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:42.557279   576 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:42.557685   576 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:42.558840   660 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.235s	user 0.000s	sys 0.002s
I20260709 12:26:42.562180 32316 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.568023   453 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:42.568462   453 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:42.571715   607 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.541266 32659 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:42.577059 32659 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.549690   468 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.579576   468 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.581598   468 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:42.551879   461 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 39.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.031s
I20260709 12:26:42.540175   483 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.546000   566 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.046s
I20260709 12:26:42.548885   652 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:42.591739   630 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.592309   630 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.594571   642 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.538637   456 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.595120   642 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.595528   456 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.597296   642 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.597576   456 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.598398   630 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.610821 32659 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.634847   494 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.637089 20804 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 19. Preceding OpId from leader: term: 37 index: 20. (index mismatch)
I20260709 12:26:42.637465   494 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.638621   644 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.574572 32472 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.639773   630 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.645220 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:42.645407 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 1.0737e+06us
I20260709 12:26:42.647567 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.649470 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.650331 20734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:42.668195 20734 raft_consensus.cc:606] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:42.656927 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.665692 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.667043 20806 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:42.670383 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.669483 21426 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:42.671383 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.684494 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.700732 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:42.711275 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.718070 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:42.719014 21431 raft_consensus.cc:2368] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:42.730455 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:42.772116 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:42.772584 20850 leader_election.cc:400] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:42.790277 20850 leader_election.cc:403] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Cancelling election due to peer responding with higher term
I20260709 12:26:42.659612   524 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.659512 32659 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.777858 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:42.796788 20806 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:42.797111 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 129923us
I20260709 12:26:42.779496   511 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:42.674721   302 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.799486 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:42.781584 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 131648us
I20260709 12:26:42.809473 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.781767 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:42.784690 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.810364 20805 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:42.675734   565 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:42.671437   474 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:42.728789 32554 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 37 current term: 37 is higher.
I20260709 12:26:42.774385 32561 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:42.672140 32316 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.776135   572 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.772889   640 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:42.810354 20699 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:42.692685   630 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.816363 32472 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.809521 32659 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.864705   453 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 39.22, Last appended by leader: 22, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.865955   453 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:42.866353 21432 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
W20260709 12:26:42.886701   664 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
W20260709 12:26:42.900801 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90530.3us
I20260709 12:26:42.822853 20806 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:42.708259   642 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.103s
I20260709 12:26:42.786760   475 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.808924   524 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.824172   534 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election lost for term 37. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:42.853979 20884 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:42.943903 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:42.944617 20806 raft_consensus.cc:2368] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:42.825790   302 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.827605   454 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:42.858466   630 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:42.813536   511 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.831387   474 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.831053   509 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:42.830816   565 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.951076 32739 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.025s
I20260709 12:26:42.847040 32316 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.975919 32739 consensus_queue.cc:1243] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 36.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:42.859442 32561 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:42.977464   565 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.817008   631 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.978320 32561 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.978746   631 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.981356   631 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.983346 20878 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:42.907557 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:42.894498 20633 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 204) took 78726 ms (client timeout 15000 ms). Trace:
I20260709 12:26:42.984808   652 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.905854 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.985982   511 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.905167 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.986706   511 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:42.900183 20882 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:42.987460 20801 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:42.988013 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.988996 20882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:42.989490 20882 raft_consensus.cc:606] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:42.989733   474 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.967561   566 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:42.991127   566 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.991439 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:42.992036 20808 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:42.846719   572 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.993232   566 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.994100 20803 raft_consensus.cc:2393] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:42.895697   640 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:42.925371 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:42.994360   302 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.970227 32622 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.995313 32622 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.996510   640 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:42.996620   576 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:42.994066   572 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.997371 32622 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:42.948470 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.968560 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57551.5us
W20260709 12:26:43.001502 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57917.1us
I20260709 12:26:42.971436 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.003000 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.004897 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.005685 20884 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:42.818145 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:43.007516 20876 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:42.947846 20849 leader_election.cc:400] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38.
I20260709 12:26:43.006711 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.010265 20884 raft_consensus.cc:2410] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 35 because replica has last-logged OpId of term: 33 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 22.
W20260709 12:26:43.008706 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 61113.4us
I20260709 12:26:43.018412 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.021270 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.022022 20884 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
W20260709 12:26:43.023386 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.026348 20810 pending_rounds.cc:85] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 18
I20260709 12:26:43.026752 20810 pending_rounds.cc:107] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 36.19
I20260709 12:26:43.027042 20810 raft_consensus.cc:2889] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:43.024315 20881 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 29.
I20260709 12:26:43.033447 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:42.987056   652 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:42.998467   555 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:43.042539 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.033973 32554 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:43.058802 20884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:42.966138   475 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.062652 20884 raft_consensus.cc:606] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:43.058739 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:43.030588   454 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.063064 20884 raft_consensus.cc:614] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:43.034958   509 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:43.066829   630 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:43.040490   572 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:43.038735   445 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:42.984632 20633 rpcz_store.cc:269] 0709 12:25:24.189017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.037063 (+8848046us) service_pool.cc:224] Handling call
0709 12:26:42.894382 (+69857319us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:37.748991 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:38.098220 (+349229us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:38.118128 (+ 19908us) write_op.cc:435] Acquired schema lock
0709 12:26:38.133347 (+ 15219us) tablet.cc:662] Decoding operations
0709 12:26:38.330568 (+197221us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:38.381090 (+ 50522us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:38.395287 (+ 14197us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:26:38.433251 (+ 37964us) tablet.cc:701] Row locks acquired
0709 12:26:38.433304 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:26:38.433487 (+   183us) write_op.cc:270] Start()
0709 12:26:38.433648 (+   161us) write_op.cc:276] Timestamp: P: 1783599998433410 usec, L: 0
0709 12:26:38.433698 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:26:38.508314 (+ 74616us) log.cc:844] Serialized 2835 byte log entry
0709 12:26:39.534058 (+1025744us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.111736 (+577678us) write_op.cc:301] APPLY: starting
0709 12:26:40.111841 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:40.275568 (+163727us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:40.275690 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:40.445354 (+169664us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:40.528867 (+ 83513us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:26:40.528956 (+    89us) write_op.cc:312] APPLY: finished
0709 12:26:40.529381 (+   425us) log.cc:844] Serialized 73 byte log entry
0709 12:26:40.764336 (+234955us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.784296 (+ 19960us) write_op.cc:454] Released schema lock
0709 12:26:40.830554 (+ 46258us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:40.850017 (+ 19463us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.850054 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7872,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":553858,"apply.run_cpu_time_us":21489,"apply.run_wall_time_us":738168,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33387,"num_ops":7,"prepare.queue_time_us":210407,"prepare.run_cpu_time_us":29754,"prepare.run_wall_time_us":1201801,"raft.queue_time_us":420123,"raft.run_cpu_time_us":126303,"raft.run_wall_time_us":480517,"replication_time_us":1002692,"spinlock_wait_cycles":59264,"thread_start_us":1015991,"threads_started":5,"wal-append.queue_time_us":544346,"wal-append.run_cpu_time_us":49491,"wal-append.run_wall_time_us":2505727}]]}
I20260709 12:26:43.045835   404 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:43.024130   511 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.084990   511 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.085794 20624 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 26 to 30, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 30 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:43.080979   404 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.088786   404 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:43.089238 20624 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 22 to 29. New cstate: current_term: 29 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:43.097872 32659 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.098906 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.101851   320 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:43.104818   496 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:43.105327   496 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:43.110036   456 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:43.110621   456 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.062476 32554 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.079820 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.060614   372 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.073935 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:43.117621   372 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.118125 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:43.060534   474 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:43.060488   565 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.063829   607 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.071327   454 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.067229 32176 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:43.112824   445 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.076144   630 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.074784   572 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.111079   456 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:43.119168 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:43.159775 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:43.160446 21429 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:43.120122   372 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.120990   474 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.167972   474 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.168830 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:43.121272   565 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.169744   642 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.170404   642 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.172286 32739 raft_consensus.cc:993] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:26:43.172870 32739 raft_consensus.cc:1081] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:26:43.173039   642 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:43.122516   475 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.174330   495 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:43.123636   576 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.174933   495 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.175546   495 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:43.123952   607 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.177927   607 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:43.124089   664 log.cc:927] Time spent T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:43.125376   524 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.126283   454 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:43.127104   668 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:43.134792   445 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.138403   509 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 36.25, Last appended by leader: 25, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.146917   484 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.187693   484 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.188942 32554 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:43.148075   630 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:43.148413   572 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:43.149783 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:43.195478 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45810.4us
I20260709 12:26:43.152369 32316 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:43.153894 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.154386 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:43.156271   631 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:43.202598   631 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:43.139734   669 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.290s	user 0.003s	sys 0.000s
I20260709 12:26:43.146823 32176 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.205233 32176 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.207597   454 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:43.208236   454 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.208794   454 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:43.102612   320 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.106359   496 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.222699   496 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:43.224910 32176 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:43.227907   565 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.232213   631 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.233079 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.243744 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.247849   320 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.248899 21429 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44462
I20260709 12:26:43.275039 21429 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:43.275418 21429 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:43.249475 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.278438 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.279031 20880 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:43.279040 20803 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:43.281422 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.091030 20624 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 28 to 31, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 31 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:43.263278 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.263549 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.283483 20624 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 28 to 31, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 31 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:43.272908 20881 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:26:43.274291 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.293921 20774 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.300750 20809 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 35 because replica has last-logged OpId of term: 33 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 22.
I20260709 12:26:43.302462 20624 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 26 to 29, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 29 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:43.238767 20876 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41.
I20260709 12:26:43.307142 20774 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.305925 20624 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 31 to 35, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:26:43.308444 20700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.013s
W20260709 12:26:43.308863 20700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.005s	sys 0.013s
W20260709 12:26:43.313028 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.321128 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60558.9us
W20260709 12:26:43.309186 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 511147us
I20260709 12:26:43.320645 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.355136 20882 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
W20260709 12:26:43.356261 20804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.040s	sys 0.082s
I20260709 12:26:43.304366   565 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:43.365067 32316 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.361527 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:43.357764 20879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.033s	sys 0.052s
W20260709 12:26:43.366648 20879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.033s	sys 0.052s
W20260709 12:26:43.358664 20804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.040s	sys 0.082s
I20260709 12:26:43.306835   607 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:43.362871 20624 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 24 to 28, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 28 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.365607 32316 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.321043   618 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.354538   576 raft_consensus.cc:2804] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election won for term 41
W20260709 12:26:43.387501 20729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.024s	sys 0.025s
I20260709 12:26:43.358457 32659 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:43.384909 20624 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 24 to 26, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 26 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:43.390019 20729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.024s	sys 0.026s
W20260709 12:26:43.395501 20802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.024s	sys 0.032s
W20260709 12:26:43.396502 20802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.024s	sys 0.032s
I20260709 12:26:43.390184 21429 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.363622   565 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.373078   607 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.403210 20624 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 22 to 30, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 30 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:43.358140   320 consensus_queue.cc:1059] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.040s
I20260709 12:26:43.382339   517 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.392974   453 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:43.395213   555 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:43.392813 32659 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.427956 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:43.437023 32316 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:43.437184 20809 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:43.427964 21429 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.400161   576 raft_consensus.cc:697] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:43.415469 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 4097) took 1914 ms. Trace:
I20260709 12:26:43.419725   565 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:43.395521   618 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.445191 21432 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:43.419605   511 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:43.422389   607 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:43.450556 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.453331 20803 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:43.436707   475 raft_consensus.cc:3055] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:43.461311 32316 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 38.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:43.452886 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.467159 20808 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:43.466185 20624 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 26 to 29, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 29 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:43.468230 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.441691 32659 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:43.469980 32316 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:43.510903 32316 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:43.511807 32316 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 38.23, Last appended by leader: 23, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.513264 32316 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:43.450254   678 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.473271   495 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.450788   677 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:43.515640   678 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.515878   677 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.517114   495 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.518347   678 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.518554 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:43.476408 20809 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 39.
I20260709 12:26:43.519150 21429 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:43.487496 32622 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.519769 32622 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.519708 20624 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be reported cstate change: config changed from index 8 to 18, VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194) evicted. New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.522017 32622 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.521744 20624 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 30 to 32, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 32 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.523074 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.442719   320 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:43.507990 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.453764   484 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:43.539496 32563 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.452944   555 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.543426   618 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:43.563683 21399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.020s	sys 0.027s
W20260709 12:26:43.525683 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.546070 20882 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 37 because replica has last-logged OpId of term: 34 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
I20260709 12:26:43.545367 20624 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 23 to 26, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 26 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:43.547344 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.466045   475 raft_consensus.cc:740] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:43.551160 21429 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:43.428083   517 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.558248 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.560561 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.559033 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.566080 21400 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.576247 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.577244 20882 raft_consensus.cc:2393] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 41: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:43.572988 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.578377 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:43.579381 32316 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:43.579811 32316 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.580212 32316 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:43.462713   453 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:43.544361   679 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:43.567334 21399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.020s	sys 0.027s
W20260709 12:26:43.551608 21427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.016s	sys 0.016s
I20260709 12:26:43.573932 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:43.597671 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 593832us
W20260709 12:26:43.588712 20810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.024s	sys 0.026s
I20260709 12:26:43.595293 20803 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39. Ops: []
W20260709 12:26:43.554574 32561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.029s	sys 0.037s
I20260709 12:26:43.615401 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:43.573592   376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.014s	sys 0.050s
I20260709 12:26:43.536684   572 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.600057 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:43.600576 20810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.024s	sys 0.026s
W20260709 12:26:43.597191 21427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.017s	sys 0.017s
I20260709 12:26:43.601460 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.590610 32739 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:43.603842 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.574206   474 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:43.628130 20878 raft_consensus.cc:2410] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 31 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 20.
I20260709 12:26:43.607177 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:43.603053 32316 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.551819   631 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:43.462422   511 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.553093   678 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:43.599879   679 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.622293 32554 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.574752   456 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.591162 32176 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:43.621954 20624 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 26 to 28, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 28 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:43.587069   468 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:43.556015   320 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:43.618059 32561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.029s	sys 0.038s
I20260709 12:26:43.464450   630 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.563110 32563 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:43.682053 20775 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.018s
W20260709 12:26:43.682407 20775 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.005s	sys 0.018s
W20260709 12:26:43.682711 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 519193us
W20260709 12:26:43.602737 20698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.009s	sys 0.015s
I20260709 12:26:43.578320 32659 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.629215 20876 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 38. Current term is 39. Ops: []
W20260709 12:26:43.679908 20701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.005s	sys 0.041s
I20260709 12:26:43.681546 32316 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:43.684258 20698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.009s	sys 0.015s
I20260709 12:26:43.576807   555 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:43.688406 20701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.005s	sys 0.041s
W20260709 12:26:43.624951   376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.014s	sys 0.051s
W20260709 12:26:43.692930 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.635879   474 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.700943   320 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.704294   474 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.704644 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:43.692772 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 629489us
I20260709 12:26:43.714607 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.714493 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.715397 20734 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 40 index: 23. Preceding OpId from leader: term: 41 index: 24. (index mismatch)
W20260709 12:26:43.679756   302 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.014s	sys 0.031s
I20260709 12:26:43.639251   401 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:43.724895 20728 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:43.725108 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.726080 20735 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:43.724925 20624 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 30 to 33, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 33 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.596145   576 consensus_queue.cc:237] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 39.26, Last appended by leader: 26, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.727507 20806 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:43.728766 20883 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
W20260709 12:26:43.747560 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.749439 20730 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:43.637131 32739 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:43.749800 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.759979 20732 raft_consensus.cc:2410] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 35 because replica has last-logged OpId of term: 34 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 19.
I20260709 12:26:43.750530 20731 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 31. Current term is 32. Ops: [31.21-31.21]
I20260709 12:26:43.751844 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.752429 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.688238   544 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.758108 20728 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:26:43.758215 20727 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 37 index: 20. (index mismatch)
I20260709 12:26:43.765735 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:43.728261   404 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.639s	user 0.013s	sys 0.001s
I20260709 12:26:43.636504   678 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:43.727540   302 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.014s	sys 0.032s
I20260709 12:26:43.790395 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:43.763140 20735 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:26:43.760761 20726 raft_consensus.cc:1275] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 34 index: 19. (index mismatch)
I20260709 12:26:43.724718 32561 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:43.767004 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.760826 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.762841 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.769397 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.692553   677 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.789400 20624 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 32 to 35. New cstate: current_term: 35 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.638301   631 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.724709 32176 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:43.727100   474 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.687740   468 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:43.799996 20624 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 26 to 30, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 30 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:43.760097   509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.005s	sys 0.026s
I20260709 12:26:43.788298 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.831111 20731 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:43.792348 20805 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:43.743207   685 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.795323 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.637989   475 consensus_queue.cc:260] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 30.24, Last appended by leader: 24, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.825745 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:43.798748   683 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:26:43.723897   453 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.827126   468 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:43.788813   544 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.861402 20624 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 20 to 27, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:43.862701 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.854939   511 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:43.891775 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:43.876592 20732 raft_consensus.cc:2410] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 35 because replica has last-logged OpId of term: 33 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 22.
I20260709 12:26:43.798239 32561 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.794010   404 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.882349 20805 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:43.903363 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.884030 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.905304 20624 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 24 to 29. New cstate: current_term: 29 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.442777 20729 rpcz_store.cc:276] 0709 12:26:41.501000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.505963 (+  4963us) service_pool.cc:224] Handling call
0709 12:26:41.506239 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.506419 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:26:41.550477 (+ 44058us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.600049 (+1049572us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b1800096b20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:26:42.832326 (+232277us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.411810 (+579484us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:43.411858 (+    48us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:43.411904 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.412079 (+   175us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.412192 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119859,"prepare.queue_time_us":463099,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":12053,"raft.queue_time_us":1117939,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":30430,"spinlock_wait_cycles":100814720,"thread_start_us":790612,"threads_started":3}
I20260709 12:26:43.830318   474 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:43.874405 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:43.872359   509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.006s	sys 0.027s
I20260709 12:26:43.900701 32176 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 38.23, Last appended by leader: 23, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:43.874750   517 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:43.883582 20808 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:43.907150 20624 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.908295 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.904809 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:43.911753 20806 raft_consensus.cc:2368] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:43.927222 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:43.938803 20883 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 36
I20260709 12:26:43.939986 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.944386 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.945545 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.946013 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:43.946842 20875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.006s	sys 0.019s
I20260709 12:26:43.948593 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:43.947211 20875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.007s	sys 0.019s
I20260709 12:26:43.950920 20624 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 29, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 29 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:43.972179 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.973389 20801 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:43.969978 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:43.962587 20806 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:44.009423 20624 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 25 to 27, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.926891   572 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.016091 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:43.883141   685 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.013926 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:43.948570   508 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:44.017503 20876 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:44.015383 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:44.019358 20624 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:43.876142 32739 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 37.22, Last appended by leader: 22, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.038808 20878 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:44.039274 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:43.977831 32554 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.006950   511 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.007252   320 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.017253 32659 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:44.016883   456 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.054309 20624 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 25 to 31. New cstate: current_term: 31 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:44.047856 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 420678us
I20260709 12:26:44.057242 20876 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:44.061272 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.064606 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:44.042704   508 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:44.067255 20624 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 24 to 26, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 26 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:26:44.068434 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.052474   679 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.047844   572 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.066433 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.059847 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.069322 20624 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:44.082046 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:44.083283 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 314743us
I20260709 12:26:44.087484 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.088595 20730 raft_consensus.cc:2410] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 35 because replica has last-logged OpId of term: 33 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 17.
I20260709 12:26:44.091665 20733 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 36. Current term is 37. Ops: [36.19-36.19]
I20260709 12:26:44.093060 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.095885 20728 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:44.097325 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.097940 20735 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:44.099063 20731 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 40 index: 22. Preceding OpId from leader: term: 41 index: 23. (index mismatch)
I20260709 12:26:44.093627 20733 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:44.081276 20624 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 22 to 27, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 27 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:44.100579 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.101284 20731 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:44.100932 20624 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 28 to 33. New cstate: current_term: 33 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:44.099304 20806 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:44.119545 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.116171 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.123288 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.124090 20729 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:44.071401   456 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:44.128505 20731 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:44.125485 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.072353   631 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.131817 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.135450 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.058797   320 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.058916   511 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.076633   679 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.143478 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.162374 20731 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:44.144595   376 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:44.146342   404 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.147341 32561 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.147959   544 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.111763   401 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:44.155678   508 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 38.23, Last appended by leader: 23, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:44.164985 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.073369   517 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 38.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.062s
I20260709 12:26:44.166347 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:44.163930   689 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 12:26:44.184623 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 278242us
I20260709 12:26:44.173415 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.185561 20806 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:44.125075   555 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.075155   572 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:44.176101   694 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 12:26:44.163820   695 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 12:26:44.163671   682 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
W20260709 12:26:44.164002   680 log.cc:927] Time spent T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
W20260709 12:26:44.163443   681 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
W20260709 12:26:44.193714   454 consensus_meta.cc:287] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.984s	user 0.010s	sys 0.004s
W20260709 12:26:44.163568   676 log.cc:927] Time spent T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 12:26:44.184178 20731 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:26:44.187510 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.143801   456 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:44.189221 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 171239us
I20260709 12:26:44.193002 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.167378 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:44.197633 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 252013us
I20260709 12:26:44.197973 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.201351   468 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.232285 20728 raft_consensus.cc:3060] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:44.177127 32472 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.245659 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.203539 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.246367 20882 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:44.246699 20730 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:44.163156   687 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:44.216001 20883 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:44.219614 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.221165 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.169951   679 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.233860 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.176642   495 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.255815 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.230320 20733 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:44.257012 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.242100 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.242972 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.242967 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.258903 20876 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:44.244094 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.248099 20882 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:44.262192 20881 raft_consensus.cc:2410] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for term 33 because replica has last-logged OpId of term: 32 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 21.
I20260709 12:26:44.262878 20880 raft_consensus.cc:2393] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:44.264204 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.266570 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.268486 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.269097 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:44.269760 20806 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:44.270349 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.270934 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4429) took 1245 ms. Trace:
I20260709 12:26:44.271894   698 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:44.280573 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.273386 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.281239 20730 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:44.281869 20731 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
W20260709 12:26:44.271771 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 146363us
I20260709 12:26:44.271369 20810 rpcz_store.cc:276] 0709 12:26:43.025518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.025651 (+   133us) service_pool.cc:224] Handling call
0709 12:26:43.026093 (+   442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:43.035836 (+  9743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.035868 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:43.036435 (+   567us) log.cc:844] Serialized 52 byte log entry
0709 12:26:43.628311 (+591876us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:43.628350 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.628421 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.270716 (+642295us) raft_consensus.cc:1685] finished
0709 12:26:44.270737 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.270876 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15347,"raft.queue_time_us":9529,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4616,"thread_start_us":591572,"threads_started":1,"wal-append.queue_time_us":762758}
I20260709 12:26:44.245258   535 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:44.291734 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2352) took 1647 ms. Trace:
I20260709 12:26:44.292174 20802 rpcz_store.cc:276] 0709 12:26:42.643947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.646790 (+  2843us) service_pool.cc:224] Handling call
0709 12:26:42.647088 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.647414 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.647450 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.650309 (+  2859us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.408575 (+758266us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:43.408614 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.408683 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.291537 (+882854us) raft_consensus.cc:1685] finished
0709 12:26:44.291559 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.291679 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2086,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":757899,"threads_started":1,"wal-append.queue_time_us":954320}
I20260709 12:26:44.283926 20700 leader_election.cc:400] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:44.284103 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:44.223268   454 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.309218 20807 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 30: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:44.310712   454 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.311281   535 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.313393   535 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:44.313884   535 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.314893   404 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:44.315434   404 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.317541   404 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.255429   555 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 35.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.043s
I20260709 12:26:44.276829   401 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.321964   630 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:44.324574   607 consensus_meta.cc:287] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.817s	user 0.009s	sys 0.001s
I20260709 12:26:44.325023   607 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.326093 20801 raft_consensus.cc:2368] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:44.256183 32472 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.368718 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.334543 20879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4311) took 1658 ms. Trace:
I20260709 12:26:44.330050 20804 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2353) took 1685 ms. Trace:
I20260709 12:26:44.453788 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.297346 20700 leader_election.cc:403] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:44.471517 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:44.398425 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.418999 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.401299 20728 raft_consensus.cc:2468] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41.
I20260709 12:26:44.475502 20727 raft_consensus.cc:3060] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:44.426682 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.449326 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.380559 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2859) took 1713 ms. Trace:
I20260709 12:26:44.398998 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:44.486227 21432 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:44.480472 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:44.490706 21429 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:44.461875 20879 rpcz_store.cc:276] 0709 12:26:42.676410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.676710 (+   300us) service_pool.cc:224] Handling call
0709 12:26:42.677126 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.709096 (+ 31970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.709129 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.772400 (+ 63271us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.367144 (+594744us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:43.368941 (+  1797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.369037 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.334201 (+965164us) raft_consensus.cc:1685] finished
0709 12:26:44.334237 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.334461 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31533,"raft.queue_time_us":32055,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":44231,"thread_start_us":554119,"threads_started":1,"wal-append.queue_time_us":797328}
W20260709 12:26:44.493543 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 209736us
W20260709 12:26:44.440943   706 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 12:26:44.330091   696 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:26:44.485463 21426 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:44.446391   496 consensus_meta.cc:287] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 1.174s	user 0.013s	sys 0.000s
I20260709 12:26:44.494416 20700 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:44.412079   687 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:26:44.327103   607 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.314334   535 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.463888   630 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.477061 20804 rpcz_store.cc:276] 0709 12:26:42.644283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.647596 (+  3313us) service_pool.cc:224] Handling call
0709 12:26:42.647859 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.648142 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.648167 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.683448 (+ 35281us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.367711 (+684263us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:43.367764 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.367855 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.329687 (+961832us) raft_consensus.cc:1685] finished
0709 12:26:44.329737 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.329950 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1539,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":51,"thread_start_us":588650,"threads_started":1,"wal-append.queue_time_us":851643}
I20260709 12:26:44.282984   677 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:44.284390 32561 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:44.284124   544 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.229s
I20260709 12:26:44.283811   631 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:44.328830   565 consensus_meta.cc:287] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.866s	user 0.011s	sys 0.000s
I20260709 12:26:44.330653   704 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.275522   495 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.485647 21427 rpcz_store.cc:276] 0709 12:26:42.667599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.667740 (+   141us) service_pool.cc:224] Handling call
0709 12:26:42.668108 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.668560 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.668600 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.695181 (+ 26581us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.625056 (+929875us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:43.625107 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.625198 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.380356 (+755158us) raft_consensus.cc:1685] finished
0709 12:26:44.380376 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.380498 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":636,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":36797,"thread_start_us":837935,"threads_started":1,"wal-append.queue_time_us":988660}
I20260709 12:26:44.328949   709 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.390751   509 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:44.496403 20806 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:44.463366   524 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.518241 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:44.523515   700 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:44.520879 21432 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:44.526830 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.531337 20875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4308) took 1987 ms. Trace:
I20260709 12:26:44.531507   572 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.531852 21429 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
W20260709 12:26:44.532404 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72687.4us
I20260709 12:26:44.501719   705 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:44.517396 32739 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.027s
I20260709 12:26:44.546407 20879 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:44.519603 20877 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:44.554176 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:44.542382   677 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:44.286127   456 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:44.552167 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 69274.8us
I20260709 12:26:44.560844 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.532508   719 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:44.551147 20875 rpcz_store.cc:276] 0709 12:26:42.543929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.544740 (+   811us) service_pool.cc:224] Handling call
0709 12:26:42.545134 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.545627 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:43.031228 (+485601us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.951506 (+920278us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.968218 (+ 16712us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:44.006361 (+ 38143us) log.cc:844] Serialized 26 byte log entry
0709 12:26:44.018411 (+ 12050us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:44.018466 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.018561 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.531084 (+512523us) raft_consensus.cc:1685] finished
0709 12:26:44.531110 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.531270 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11395,"prepare.run_cpu_time_us":5652,"prepare.run_wall_time_us":24244,"raft.queue_time_us":26159,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":4419,"thread_start_us":1390804,"threads_started":2,"wal-append.queue_time_us":1106454}
I20260709 12:26:44.567499 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.568662   719 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:44.527549   630 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.565924 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:44.542187   565 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.572006 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:44.573396 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.576422 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:44.553469   524 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.579440 20809 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:44.569136   718 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:44.547948   509 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.550372   709 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.573361 32554 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:44.559842   700 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.551296   704 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.563688   705 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:44.590204   713 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:26:44.585891   697 log.cc:927] Time spent T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.412s	user 0.003s	sys 0.000s
I20260709 12:26:44.578238   714 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:44.608995   714 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:44.609429   714 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:44.571233   677 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 38.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.020s
I20260709 12:26:44.601917   535 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.602173   495 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.602713 32472 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.595225   721 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:44.589109   678 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.594256 32739 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:44.580773   723 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:44.637250 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69230.4us
I20260709 12:26:44.638434   728 consensus_queue.cc:1059] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:44.640839 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:44.642186   728 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:44.644201 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.595544   534 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:26:44.649722 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:44.650497 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.662128 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.666764 20882 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 39.
I20260709 12:26:44.670257 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.671139 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.672683 20879 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:44.663949 20734 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:44.680034 20883 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 36 index: 25. Preceding OpId from leader: term: 38 index: 26. (index mismatch)
I20260709 12:26:44.642694   728 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.684810 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.686488 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:44.687866 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.688478 20729 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:26:44.677707   678 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:44.655551   685 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.683247   719 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.009s
I20260709 12:26:44.693538   509 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:44.714951 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.715657 20734 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:44.716320   700 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:44.717406   709 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.686599   728 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:44.752934 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 172101us
I20260709 12:26:44.754647 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:44.755560 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.701157   576 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:44.712796 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.713135 20729 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:44.700723 20727 raft_consensus.cc:2468] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:44.738322   524 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.739580   704 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.740502   630 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.741031   565 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.761004 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.757465 20732 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
W20260709 12:26:44.759725 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 164957us
I20260709 12:26:44.763890   508 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:44.764338   508 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.764674   508 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:44.721041   705 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.778887 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.771268   302 raft_consensus.cc:993] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:26:44.783432 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:44.784085 20876 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:44.784962 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:44.788120 20883 raft_consensus.cc:3060] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:44.789533 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.790277 20774 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:44.791445 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.792915   302 raft_consensus.cc:1081] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Signalling peer 657bc62d20374e539b23641715f22745 to start an election
I20260709 12:26:44.793465   565 raft_consensus.cc:2804] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:44.794001   565 raft_consensus.cc:697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:44.794102   302 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:44.794555   302 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:44.759986 32739 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.028s
I20260709 12:26:44.760293   721 consensus_queue.cc:1048] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:26:44.805956 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.806803 20735 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 35: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:44.806972 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.808060 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.808620 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.809693 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.811704 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:44.812364 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.813082 20733 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:44.757048   576 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:44.816799   576 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.815526 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:44.822856 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.823472 21427 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:44.827374 21430 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:44.832844 20876 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:44.834550 20883 raft_consensus.cc:2468] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:44.836030 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:44.833648 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72849.1us
I20260709 12:26:44.840737 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:44.838995   721 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:44.850183 20849 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:44.852576   474 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:44.853029   730 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.857430 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.858177 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.858883 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.859933 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.861493 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.863044 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:44.863893 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.794888   565 consensus_queue.cc:237] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.20, Last appended by leader: 20, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.865379 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.876644 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:44.865993 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.866171 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.839764 32739 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.884508 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:44.885682 20802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46"
dest_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:33486
I20260709 12:26:44.886438 20802 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:44.886874 20802 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:26:44.888118 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100292us
I20260709 12:26:44.888336 21427 raft_consensus.cc:2368] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
W20260709 12:26:44.871927 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.873020 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.885648 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.902788 32554 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:44.896688 32739 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:44.905972 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.890038   630 raft_consensus.cc:2764] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election decision vote started in defunct term 34: won
I20260709 12:26:44.907871   453 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:44.908859 32739 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:44.910300 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.911124 21428 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 33: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:44.890110   721 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:44.921509 21427 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:44.924535 20727 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:44.962488 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.883878   730 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.954665 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.950490 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.952677 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:44.907226   524 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:44.950095 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:44.976758 20733 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:44.911687 32554 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:44.920619   454 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.964475 20802 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.835999   302 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 38.23, Last appended by leader: 23, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:44.925992   721 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:44.901451   712 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.062s
I20260709 12:26:44.966235   535 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.989609 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:44.979770 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:44.987515 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:45.051362 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71733.4us
I20260709 12:26:45.052016 20802 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:45.052414 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.054143 20701 leader_election.cc:400] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35.
I20260709 12:26:45.055157 20701 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:44.985324   677 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:44.945168   474 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.043s
W20260709 12:26:45.044804   727 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 12:26:45.044377   721 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:44.961910   725 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 37.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.090s
I20260709 12:26:45.050561   302 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:45.090297 32554 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.092500   725 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:45.092909   725 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:45.093700   725 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 37.22, Last appended by leader: 22, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.094839   725 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:45.089510   730 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:44.989020   454 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.053393   535 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.061559   718 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:45.069478   690 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:45.064874   719 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:45.195842 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140770us
I20260709 12:26:45.215562 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:45.236716 20880 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:45.237131 20880 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:45.237993 20880 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 39.24, Last appended by leader: 24, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:45.238842   454 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:45.230127 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.240031 20880 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:45.240718   535 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.242542 20875 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:45.244469 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.102927   714 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:45.247641 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.247646 20881 raft_consensus.cc:1240] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:45.248472   690 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:45.248864   714 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:45.248965 20701 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.250286 20701 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:45.251199   719 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:45.224799   740 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:45.254350   740 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.255127 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.256724   740 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.257503 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:45.290870 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 394622us
I20260709 12:26:45.311486 20880 raft_consensus.cc:2410] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 40 because replica has last-logged OpId of term: 39 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 38 index: 23.
W20260709 12:26:45.344992 20734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.003s	sys 0.010s
I20260709 12:26:45.221662   741 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:45.376103 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.385761   565 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
W20260709 12:26:45.346930 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 91907.5us
I20260709 12:26:45.347496   690 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:45.297569   508 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.001s
I20260709 12:26:45.348130   719 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:45.395287 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.423784   718 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:45.424351   475 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:45.418601   742 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.311s	user 0.002s	sys 0.001s
I20260709 12:26:45.390116 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:45.427227 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:45.428275 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:45.428755   743 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:26:45.428977   474 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:45.390183   714 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.429533   474 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.429939   714 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:45.431665   572 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:45.432432   714 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:45.432667   572 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:45.433229   572 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:45.433703   572 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:45.433934   565 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
W20260709 12:26:45.430974   728 consensus_meta.cc:287] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.672s	user 0.007s	sys 0.000s
I20260709 12:26:45.439671   728 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.441596 20730 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 36: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:45.461278 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:45.468278 20729 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.019s	sys 0.024s
W20260709 12:26:45.468680 20729 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.019s	sys 0.024s
I20260709 12:26:45.466245 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:45.446442   728 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.443459   474 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.384403   741 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:45.480219 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 92127.2us
I20260709 12:26:45.468412   454 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:45.479735 20775 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:45.501394 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:45.486425 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.503054 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.506129 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.507099 20734 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 22. Preceding OpId from leader: term: 38 index: 23. (index mismatch)
I20260709 12:26:45.477146   607 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
W20260709 12:26:45.510447 20877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54476 (request call id 4391) took 1260 ms (client timeout 1533 ms). Trace:
W20260709 12:26:45.510870 20877 rpcz_store.cc:269] 0709 12:26:44.242320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.243741 (+  1421us) service_pool.cc:224] Handling call
0709 12:26:45.502754 (+1259013us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540026cad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:45.502853 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597251328}
I20260709 12:26:45.465370   721 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.528680 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.521265 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:45.530642 32316 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:45.531028 32316 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:45.531386 32316 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:45.532574 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.533398 20733 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:45.534984 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 55433us
I20260709 12:26:45.536547 20775 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:45.538854   565 raft_consensus.cc:2804] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:45.518810   496 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:45.504483   576 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.539446 32622 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:45.540055 32622 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.539898   576 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.540575 32622 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:45.542111   576 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.542680   576 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:45.543685   730 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:45.539489   565 raft_consensus.cc:697] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:45.544379   721 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.475572   508 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:45.545219   565 consensus_queue.cc:237] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:45.546684   508 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:45.546768 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.547453 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:45.548903   740 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.549861   740 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:45.553128 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.555486   607 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:45.556212   607 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.557036 20732 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:26:45.558175 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.560060   691 raft_consensus.cc:3055] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:45.560597   691 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:45.561276 20882 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 41: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:45.561688   691 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:45.564113   750 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:45.499969   454 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:45.565204   454 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:45.565773 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.566360 20727 raft_consensus.cc:2376] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35. Re-sending same reply.
I20260709 12:26:45.567145 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.568120 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.569106 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.569708   468 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:45.572270   748 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:26:45.573761 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.577939 20875 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:45.579106 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.579412 20803 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:45.580193 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:45.579113 20807 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 4431) took 2316 ms (client timeout 1810 ms). Trace:
I20260709 12:26:45.581573 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.582659 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.583922 20876 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 38. Current term is 40. Ops: []
I20260709 12:26:45.585048 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.558923 20732 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:45.588361 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.589326 20730 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:45.590188 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:45.580735 20807 rpcz_store.cc:269] 0709 12:26:43.262644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.263279 (+   635us) service_pool.cc:224] Handling call
0709 12:26:45.578897 (+2315618us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540019afd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:45.579016 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2360043392}
I20260709 12:26:45.593912 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.597092 20879 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 41 index: 27. (index mismatch)
I20260709 12:26:45.558868   607 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.600288 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.563082   691 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:45.608937 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.610700 20875 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:45.612068 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.622859 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.623775 20734 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:45.627216 32316 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:45.632447 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.631438   474 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:45.602751   725 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:45.640529   725 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.640900   453 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:45.641404   453 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.646551 21432 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:45.647516 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:45.648418 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:45.649143 21434 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:45.652630 20733 raft_consensus.cc:2410] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 33 because replica has last-logged OpId of term: 32 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 22.
I20260709 12:26:45.655856 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.656824 20734 raft_consensus.cc:2376] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41. Re-sending same reply.
I20260709 12:26:45.652143 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:45.661288 20727 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:45.661772 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.662843 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.663748 32316 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:45.665650 21434 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:45.668674   495 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:45.669221   495 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.669274 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.669826   495 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:45.672008   740 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.675132 20879 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:45.676151 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.677152 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:45.677597   741 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.677827 20879 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:45.678362 20809 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33470 (request call id 4442) took 2104 ms (client timeout 1538 ms). Trace:
I20260709 12:26:45.678786 20879 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 29. Current term is 30. Ops: []
W20260709 12:26:45.678869 20809 rpcz_store.cc:269] 0709 12:26:43.573521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.573669 (+   148us) service_pool.cc:224] Handling call
0709 12:26:45.678146 (+2104477us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54002b70d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:45.678292 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257607040}
W20260709 12:26:45.679739   699 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 1.496s	user 0.006s	sys 0.011s
W20260709 12:26:45.677624   745 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.598s	user 0.008s	sys 0.000s
I20260709 12:26:45.679780 20882 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:45.680755 21435 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:45.680111 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.681448 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.663383   728 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.682358   728 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.689639 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.643029   725 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.691761 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:45.678465 20805 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:45.691596   453 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:45.693246 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.761978 20877 raft_consensus.cc:2393] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:45.629760   690 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:45.703598 20726 raft_consensus.cc:2376] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41. Re-sending same reply.
I20260709 12:26:45.773141 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:45.723650 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56014.9us
I20260709 12:26:45.760222 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:45.692627 20733 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:26:45.825342 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47553.9us
I20260709 12:26:45.827924 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:45.828751 21434 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:45.828794 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:45.829739 20726 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Advancing to term 41
W20260709 12:26:45.839253 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 58759.9us
I20260709 12:26:45.896519 20734 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:45.774827   376 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:45.772526 32176 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:45.772559   475 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.155s
W20260709 12:26:45.930980 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 413424us
I20260709 12:26:45.940971 20726 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:45.838827   372 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.091s
W20260709 12:26:45.974699 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 486749us
W20260709 12:26:45.976248 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.892081   725 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.037s
I20260709 12:26:45.982084   725 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:45.982587   725 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:45.983678   725 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.985183   725 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:45.985574 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.920887   712 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:45.988613 20733 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
W20260709 12:26:45.980837   572 consensus_meta.cc:287] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 12:26:45.993829   572 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.995246 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:45.996569   572 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.007570 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:45.929745 32176 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:45.926072   376 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.014048 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:46.010643   757 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:26:46.000649   454 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.026540   454 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.028434   453 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:46.029222   453 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.032009 20733 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:26:46.037992 20734 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 42.
I20260709 12:26:46.038468   453 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.009232   302 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:45.992888   712 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.017604   320 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.050333 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.052999 20806 raft_consensus.cc:1240] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:46.055258 20806 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:46.056164 20807 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:46.057667 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.058101 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.058436 20807 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:46.039341   762 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:46.049245   454 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:46.051482   576 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.060416 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.079285   454 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.062073   764 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:46.070218   762 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.082643 20730 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:46.085265 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.085660   712 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.069890 32561 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:46.089818 20807 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:46.072785 32622 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.002s
I20260709 12:26:46.086103 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.100317 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.097293 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.101140 20875 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:46.102677 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.100345 20801 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:46.110093 20809 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:46.111799 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.099114 32561 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:46.118038 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.118672 20801 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:46.116899 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.120831 20809 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:46.122246 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.090523   576 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.121726   372 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.008s
I20260709 12:26:46.132875   762 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.094444   764 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.136621 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.143411 20809 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:46.104893 32622 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.146112   770 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.146692   770 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.135649 20807 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:46.144418 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.117969   768 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:46.151791 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.116320   691 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.027s
I20260709 12:26:46.154440 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:46.155406 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.157989 20775 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.143534 32176 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.162380 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.150707   768 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.160627 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.163163 20809 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 41: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:46.151824   679 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.151330   770 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.168540 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.159587   746 raft_consensus.cc:2764] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election decision vote started in defunct term 38: won
I20260709 12:26:46.162402   453 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:46.161952   691 consensus_queue.cc:1243] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 657bc62d20374e539b23641715f22745 log is divergent from this leader: its last log entry 37.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
W20260709 12:26:46.173286 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58904.1us
I20260709 12:26:46.176111 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.175017   607 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:46.177968   511 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.170492   761 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:46.181185   764 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.181999   454 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:46.182363   495 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 12:26:46.182874   576 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.183961 32622 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.184331 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.184832 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.185851 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.186591 20801 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:46.187088 20810 raft_consensus.cc:1240] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:46.187532 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.187750 20801 raft_consensus.cc:1240] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:46.188297 20807 raft_consensus.cc:2393] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:46.188445 20810 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:26:46.165647   376 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.189163 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.181945   677 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:46.190846   677 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:46.191620   677 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.173996 20775 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.192771 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.193104 20880 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:46.194562 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.195020 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.195779 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.196635 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.201721 20880 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:46.203136 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.192391   768 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.199791 32622 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:46.182852   495 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.205576 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.207085 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.199029   691 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:46.200793 32176 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:46.219527 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:46.241303 20875 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 41: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:46.209404   677 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:46.247995 20878 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 37 index: 22. (index mismatch)
I20260709 12:26:46.204824   372 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:46.248840   372 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.249274   372 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:46.238830 32622 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:46.206020   725 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:46.254566 32176 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:46.204581   679 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
W20260709 12:26:46.256099   769 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.057s	user 0.004s	sys 0.000s
W20260709 12:26:46.253963   771 log.cc:927] Time spent T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.060s	user 0.000s	sys 0.004s
I20260709 12:26:46.257810   496 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.258384   496 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.258831 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:46.259510 20730 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:46.259986   730 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.260509   730 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.260730   496 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.262396   730 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.258877 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.267028 32316 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:46.267500 20730 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:26:46.268656 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.280936 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.262755   746 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.282191 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.256645   495 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:46.284912 32316 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:46.287871 32316 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.288412 32316 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:46.286322 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.269755 32176 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.23, Last appended by leader: 23, Current term: 36, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.279969 32563 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.296283 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.282935   534 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:46.290685   730 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
W20260709 12:26:46.304364 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 98289.9us
I20260709 12:26:46.304697 32316 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.305769 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.298969   495 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:46.310930 32316 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.313503 32316 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election lost for term 40. Reason: could not achieve majority
I20260709 12:26:46.320086 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:46.321110 20735 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33906 (request call id 2122) took 1514 ms (client timeout 1966 ms). Trace:
W20260709 12:26:46.321535 20735 rpcz_store.cc:269] 0709 12:26:44.806369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.807327 (+   958us) service_pool.cc:224] Handling call
0709 12:26:46.319875 (+1512548us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b5800005198. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7aa22 00007f9e24a938d9 00007f9e23bedee0 00007f9e23cb70ab 00007f9e23cb7131 00007f9e23c650a0 00007f9e23c64984 00007f9e1d1df6d4 00007f9e1d1df662
0709 12:26:46.321011 (+  1136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19315584}
I20260709 12:26:46.322595   453 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:46.323346 20803 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 4224) took 2559 ms. Trace:
I20260709 12:26:46.323880 20803 rpcz_store.cc:276] 0709 12:26:43.764016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.764141 (+   125us) service_pool.cc:224] Handling call
0709 12:26:45.579214 (+1815073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:45.579774 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.323134 (+743360us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b540019afd0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:46.323266 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4173464320}
I20260709 12:26:46.325256 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.304771   730 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:46.328620 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.290515   746 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:46.331598 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.334764 20884 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:46.336520 20730 raft_consensus.cc:1275] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 39 index: 26. Preceding OpId from leader: term: 41 index: 27. (index mismatch)
I20260709 12:26:46.337096 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.312486   495 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.338269 20730 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [31.21-32.22]
I20260709 12:26:46.338335 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.342942 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.338701 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.339672 20727 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:46.344708 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.344744 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:46.346298 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.346540 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.348157 32563 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:46.347864 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.349793 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:46.350335 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.352056 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.352674 20803 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:46.353030   723 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:46.353703 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.355188   453 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:46.356086   453 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:46.356751 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.350332 20727 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:46.336790   778 consensus_queue.cc:1059] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 40.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.008s
I20260709 12:26:46.343178 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.308799   496 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:46.347909 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.363621 20730 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 41: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:46.367050 20729 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:46.376717 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.332424   677 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.375741 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.369731 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.380507 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:46.381067 20730 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:46.383720 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.384991 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.369160 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:46.372826 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.356410   453 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:46.344883   495 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:46.334440   730 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:46.396914 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.392212 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.396850 20727 raft_consensus.cc:2468] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:46.342442   728 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.374871   721 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:46.333935 32472 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.046s
I20260709 12:26:46.425971 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.426440   728 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:46.427691 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.311236 32622 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 38.23, Last appended by leader: 23, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.429862   782 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.366566   534 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:46.374186   496 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.431569 20727 raft_consensus.cc:1275] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:46.374281   565 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.058s
I20260709 12:26:46.391607   484 consensus_queue.cc:1059] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.041s
I20260709 12:26:46.397240   679 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:46.399367 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.425518 20805 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 4235) took 2533 ms. Trace:
I20260709 12:26:46.449827 20805 rpcz_store.cc:276] 0709 12:26:43.891835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.969288 (+ 77453us) service_pool.cc:224] Handling call
0709 12:26:45.678290 (+1709002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:45.703614 (+ 25324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.425284 (+721670us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54002b70d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:46.425434 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3929506048}
I20260709 12:26:46.379731 32563 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:46.440358   534 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:46.433712 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.397576   468 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:46.425235   302 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:46.451697 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.396154   762 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:46.424734   454 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:46.461812 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 62548.2us
I20260709 12:26:46.432551   607 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:46.462499 20730 raft_consensus.cc:2468] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41.
W20260709 12:26:46.479668 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 46246.7us
I20260709 12:26:46.480813 20729 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:46.430476   761 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.430757   764 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:46.478945 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 123752us
I20260709 12:26:46.434918   475 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.484108 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.391844   723 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.071s
I20260709 12:26:46.424175   691 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.057s
I20260709 12:26:46.397807   576 consensus_queue.cc:1048] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 24, Time since last communication: 0.083s
W20260709 12:26:46.491297 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 130940us
I20260709 12:26:46.512554 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.513811 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.513923 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.514353 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.515129 21426 raft_consensus.cc:2410] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 32 because replica has last-logged OpId of term: 30 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 23.
I20260709 12:26:46.516000 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.515002 21432 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:46.516666 21429 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:46.442018   474 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:46.462935   762 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:46.491856 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 59844.4us
I20260709 12:26:46.462960   534 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.441746   782 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.534790   534 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:46.528520   725 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:46.447010   496 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:46.525377 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:46.536337 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 84594.2us
I20260709 12:26:46.536412   782 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:46.540819 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.540779 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:46.542454 21426 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:46.544492 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.396883   705 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.093s
I20260709 12:26:46.396085   677 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.489181 32563 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:46.488749   302 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:46.509140   607 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.515839   770 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:46.570128 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.556151   762 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:46.583050 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:46.590250 20850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.196:44339 timed out after 1.550s (SENT) [suppressed 21 similar messages]
I20260709 12:26:46.491360   475 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.488381   764 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.558764   775 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:46.633178 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:46.567611   555 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.048s
I20260709 12:26:46.635890 20727 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:46.651651   764 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:46.655716   781 log.cc:927] Time spent T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:26:46.656750   730 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:46.658017   738 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 1.579s	user 0.010s	sys 0.031s
I20260709 12:26:46.561303   725 raft_consensus.cc:2764] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 32: won
I20260709 12:26:46.516814   679 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.062s
W20260709 12:26:46.611575 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 120734us
I20260709 12:26:46.528496   763 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.061s
I20260709 12:26:46.614200 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:46.488497   761 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.658012 20883 raft_consensus.cc:1240] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:46.512570   576 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.059s
I20260709 12:26:46.674927 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.696221 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.647665 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.698355 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.694317 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.679864 20877 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:46.681368 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.682401 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.699582 20879 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:46.682942 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.661603 20881 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4349) took 1768 ms. Trace:
I20260709 12:26:46.678217 20878 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 38 index: 23. Preceding OpId from leader: term: 39 index: 24. (index mismatch)
I20260709 12:26:46.698983 21431 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:46.700846 20881 rpcz_store.cc:276] 0709 12:26:44.892913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.893274 (+   361us) service_pool.cc:224] Handling call
0709 12:26:44.893546 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:45.248326 (+354780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.661365 (+1413039us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54000a37d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:46.661537 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":813856000}
I20260709 12:26:46.589982   607 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.714206 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:46.704900 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:46.660810   725 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Stepping down as leader of term 35
W20260709 12:26:46.724843 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 77305.5us
I20260709 12:26:46.660581   780 raft_consensus.cc:3055] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:46.730304 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.659117   730 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.698141   782 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:46.733644 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 149078us
I20260709 12:26:46.731462 20775 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.727348   484 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:46.660494   372 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.748046 21434 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:26:46.730537   725 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:46.753491 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 271642us
I20260709 12:26:46.735262   764 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:46.755715 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.756429 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.732846   780 raft_consensus.cc:740] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:46.752372 20775 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:46.762840 20735 raft_consensus.cc:2393] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:46.763408 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.750208   565 raft_consensus.cc:2804] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:46.767145 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.744840   794 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.761951 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:46.756520   730 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:46.743654   746 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.771268 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.773820 21434 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
W20260709 12:26:46.774106 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 294123us
I20260709 12:26:46.774814 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.780268 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:46.758080   764 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.743232   782 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.786562 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:46.764211 32176 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:46.765151   763 raft_consensus.cc:2804] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:46.767328 32472 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:46.818013 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.778256 32554 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:46.784879   677 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.760478   789 consensus_queue.cc:1048] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.064s
I20260709 12:26:46.777205   565 raft_consensus.cc:697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:46.761431   468 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.054s
I20260709 12:26:46.823073   746 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
I20260709 12:26:46.826843 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.845794 20732 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
W20260709 12:26:46.847604 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142793us
I20260709 12:26:46.792387   725 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.18, Last appended by leader: 18, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.777526   794 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:46.834589   795 log.cc:927] Time spent T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:46.792997   730 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:46.793311   764 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:46.843043 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:46.784610   454 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.045s
I20260709 12:26:46.812947   799 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:46.818638   788 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:46.791289   511 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:46.824072   376 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
W20260709 12:26:46.851634 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65184.2us
I20260709 12:26:46.829936   763 raft_consensus.cc:697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:46.821915   723 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.029s
I20260709 12:26:46.861596 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:46.874524 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:46.877503 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.875272 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:46.878328 20803 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:46.846092   690 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:46.841001   475 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.880477 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.865804   794 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.885695 21434 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:46.888155 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.881335 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:46.879809 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.862628   778 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:46.863827   372 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:46.893810 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 111934us
I20260709 12:26:46.893527 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.895668 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.877107   782 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:46.901463 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:46.885243   690 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:46.891403   800 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:46.879709   511 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.921819   800 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:46.921748   763 consensus_queue.cc:237] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 40.23, Last appended by leader: 23, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.923131   511 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:46.878697   788 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:46.908521   784 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.337s	user 0.003s	sys 0.000s
W20260709 12:26:46.909260   803 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:46.908627   690 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:46.909920   802 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:46.911904   607 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.863011   565 consensus_queue.cc:237] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.914227   495 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.926808   491 raft_consensus.cc:3055] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:46.934473   491 raft_consensus.cc:740] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:46.923092   800 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.935586   495 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.935371   491 consensus_queue.cc:260] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.936767   491 raft_consensus.cc:3060] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:46.910933   534 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.930907   762 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.940066   534 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.862628   730 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 39.22, Last appended by leader: 22, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.824990   780 consensus_queue.cc:260] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 40.23, Last appended by leader: 23, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:46.926782   788 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.894415   778 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:46.897728   761 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.921104   721 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:47.019603 20877 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:47.020872   721 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.021391 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.021920   721 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.023653 21434 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 22. Preceding OpId from leader: term: 38 index: 23. (index mismatch)
I20260709 12:26:46.847926 20732 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [31.21-32.22]
I20260709 12:26:47.024236 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:46.911682 32561 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.945190   762 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:46.942312   764 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.027575 32561 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:46.948501   780 raft_consensus.cc:3060] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Advancing to term 41
W20260709 12:26:47.013897 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 130414us
I20260709 12:26:47.039536 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:47.041802   495 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:47.042331   495 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.042768   495 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:46.933411   453 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:46.914868   496 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.064182   453 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:47.026387 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:46.996170 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 212381us
I20260709 12:26:46.985606 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:46.947305   778 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:47.084977 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 99882.8us
I20260709 12:26:47.087734   804 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:47.088268   804 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:47.088773   804 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:47.089454   484 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:47.090026   484 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:47.011927 20805 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:46.937796   690 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:47.009454 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.019677   511 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.141321   511 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.030546 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.031298   764 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.035547 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.054942   496 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:47.112026   786 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.126572 20803 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
W20260709 12:26:47.135125 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.159054   786 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.160169   511 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.162083   786 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:47.163645 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.164918 20735 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 37 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 22.
I20260709 12:26:47.168215 20734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37276 (request call id 4406) took 1503 ms. Trace:
I20260709 12:26:47.169739 20726 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 41. Current term is 42. Ops: [41.23-41.23]
I20260709 12:26:47.169994 20734 rpcz_store.cc:276] 0709 12:26:45.664607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.681189 (+ 16582us) service_pool.cc:224] Handling call
0709 12:26:47.167752 (+1486563us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54000e6ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:47.167986 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477039872}
I20260709 12:26:47.170416   764 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 41.23, Next index: 24, Last known committed idx: 21, Time since last communication: 0.006s
W20260709 12:26:47.179876 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.180640   763 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Stepping down as leader of term 41
I20260709 12:26:47.181169   763 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:47.179647 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.182693   763 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 41.23, Last appended by leader: 23, Current term: 41, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.187124   763 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:47.187315 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.195395 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.201900 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 4153) took 1566 ms. Trace:
I20260709 12:26:47.208012 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.210965 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.211830 20734 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:47.202404 20733 rpcz_store.cc:276] 0709 12:26:45.635075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.034789 (+399714us) service_pool.cc:224] Handling call
0709 12:26:46.035144 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.057406 (+ 22262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.057451 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.066580 (+  9129us) log.cc:844] Serialized 26 byte log entry
0709 12:26:46.143562 (+ 76982us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:46.161500 (+ 17938us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.161728 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.091596 (+929868us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b18004e3940. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:47.201178 (+109582us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54003ad4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:47.201251 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.201355 (+   104us) raft_consensus.cc:1685] finished
0709 12:26:47.201369 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.201641 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3559,"prepare.queue_time_us":16586,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":3798,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":4679,"spinlock_wait_cycles":47724416,"thread_start_us":46231,"threads_started":1,"wal-append.queue_time_us":109499}
I20260709 12:26:47.214912   511 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.174907   453 consensus_queue.cc:1059] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 39.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.032s
I20260709 12:26:47.224617 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:47.220293 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.216001   511 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.228886   787 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:47.229638   787 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.230736   511 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.231505 20875 raft_consensus.cc:2410] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 39 because replica has last-logged OpId of term: 38 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 36 index: 25.
I20260709 12:26:47.233525   456 raft_consensus.cc:3055] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:47.234050   456 raft_consensus.cc:740] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:47.235646   456 consensus_queue.cc:260] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 39.23, Last appended by leader: 23, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:47.236828 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 292925us
I20260709 12:26:47.234124 20883 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 32. Current term is 33. Ops: [32.22-32.22]
I20260709 12:26:47.236958   456 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:47.241411 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.209887   786 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:47.253273 20728 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 37 index: 22. (index mismatch)
W20260709 12:26:47.256037 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 256720us
I20260709 12:26:47.259696 32561 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:47.259829 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.260831 20732 raft_consensus.cc:2410] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 37 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 22.
I20260709 12:26:47.260484 32561 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.263262 32561 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:47.265807   819 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.155s	user 0.002s	sys 0.000s
I20260709 12:26:47.267386   728 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.268079   728 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.270536   728 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.280159 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.283118   725 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:47.292232   534 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.293119   534 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:47.295392 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.296088   534 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:47.284228   787 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.297351   786 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:47.284925 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.300704   786 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.300323 20700 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.312651 20727 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 23. Preceding OpId from leader: term: 40 index: 24. (index mismatch)
I20260709 12:26:47.313136 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.318699 32561 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:47.319315 32561 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:47.320479 32561 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 30.24, Last appended by leader: 24, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.309010   728 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.325495   728 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.327677   721 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.329500 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.330377 20808 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 26.
I20260709 12:26:47.331694 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.332401 20803 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:47.335374 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:47.336010 20727 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:47.336450   484 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.345594 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.346450 20883 raft_consensus.cc:1275] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:47.346418 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.346436 20881 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:47.347755 20884 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:47.346621   786 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.351614   786 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.353840   786 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.366766 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.367169 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.366005 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.371073 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.366458 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.378031 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.380892 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.381496   721 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.384404 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.385058 20732 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:26:47.385937 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:47.382539   721 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.386634 20808 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:47.387341 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:47.387964 20884 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:47.388370 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.350781   495 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.391587 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:47.392313 20805 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:47.392385   495 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.398278   484 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:47.369287 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.399624   511 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:47.400012   511 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.400372   511 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:47.405133 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.405751 20808 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:47.406741 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.411773 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.363080   728 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:47.413589 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 98997.3us
I20260709 12:26:47.415904 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.416765   810 consensus_queue.cc:1059] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 31.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:47.417666 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.417797   810 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:47.418834   705 raft_consensus.cc:3055] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:47.419253   705 raft_consensus.cc:740] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:47.420279   705 consensus_queue.cc:260] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.421715   705 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:47.424701   721 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.428943 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.431622 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.433627 20883 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 39 index: 24. Preceding OpId from leader: term: 40 index: 24. (term mismatch)
I20260709 12:26:47.434198 20883 pending_rounds.cc:85] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 23
I20260709 12:26:47.434459 20883 pending_rounds.cc:107] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 39.24
I20260709 12:26:47.434731 20883 raft_consensus.cc:2889] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.433586 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.436117   484 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 35.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:26:47.437481   484 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:47.437871   484 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:47.438769 20883 raft_consensus.cc:1240] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: [36.21-36.21]
W20260709 12:26:47.440387 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 54318.7us
I20260709 12:26:47.441330 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.442078 20879 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:47.442268 20876 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:47.450014 20805 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:47.452278 20727 raft_consensus.cc:2468] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:47.453450 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:47.462234 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.463562 20879 raft_consensus.cc:2393] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 32: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:47.465483 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.436125   780 consensus_queue.cc:1059] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.029s
I20260709 12:26:47.470571 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.475582 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.472909 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.473779 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:47.474547 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.468869 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.418278   511 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.479544 20879 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:47.460510   764 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:47.485358 20884 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:47.488137 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.488605 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.488821 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.489992 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.490823   456 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:47.491315   456 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.491235   484 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.25, Last appended by leader: 25, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.491866   456 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:47.491819   565 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:47.492286 21428 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:47.494428 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.494867 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:47.495512 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.497951 20776 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:47.499437   453 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:47.500365   453 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:47.505534   804 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.474623 32622 consensus_queue.cc:1048] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.010s
I20260709 12:26:47.512663 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:47.513036   511 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.513528 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:47.513759   511 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.514120   728 consensus_queue.cc:1059] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 34.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:47.514372   511 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.515467   690 raft_consensus.cc:3055] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:47.515888   690 raft_consensus.cc:740] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:47.516996   690 consensus_queue.cc:260] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.515882   534 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:47.519469 20726 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:47.520148   728 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:47.509788 20801 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [33.23-34.24]
I20260709 12:26:47.520576   728 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.521005   728 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:47.522047 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.522984 20803 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:47.523630   484 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:47.525158 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.507776   778 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.487500   764 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.518452   690 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:47.514772   719 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:47.510128 20802 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:47.526881 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.536857 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:47.542820 21426 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:47.540493 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:47.537395 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70430.9us
I20260709 12:26:47.532799   719 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.547150 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 49419.2us
I20260709 12:26:47.531791   764 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:47.552485 32316 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:47.552811   320 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:47.553251 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:47.554484 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:47.555310   833 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:26:47.555440 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.555959   723 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.556535   723 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.557859   791 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:47.558627   791 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:47.560110   804 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.547963   725 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:47.566021   456 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.566486   799 raft_consensus.cc:3055] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:47.573428 32316 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:47.558952   723 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:47.573699 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.583631 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.550329   719 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:47.589222 20728 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:47.558979   791 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.598522 20734 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: [35.22-35.22]
I20260709 12:26:47.596979 21428 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:47.601112 32316 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.574368   799 raft_consensus.cc:740] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:47.617542 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:47.618736 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.619109   454 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.037s
I20260709 12:26:47.619539 20729 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:47.625000   725 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:47.627295 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:47.635425   763 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:47.636250   456 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.636989   723 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:47.637540   723 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:47.592381   787 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:47.639770   787 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:47.642601 20733 raft_consensus.cc:1240] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:47.644356 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.645004 20884 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:47.646178 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.591892   804 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:47.603811 32561 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:47.605790   791 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:47.638599   723 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 38.23, Last appended by leader: 23, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.655879 20729 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 29.
I20260709 12:26:47.654371 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:47.666114 20877 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:47.674978 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.668562 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.685145   495 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.685842   495 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.688235   495 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.687848   787 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.25, Last appended by leader: 25, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:47.693802 21400 connection.cc:256] RPC call timeout handler was delayed by -1.0302536940002938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:26:47.695297 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.696161 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 113057us
I20260709 12:26:47.697999 20730 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:47.702275   719 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.703421 32554 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:47.703897 32554 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:47.704250 32554 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:47.707710   778 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:47.709200 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:47.710713 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:47.711777   799 consensus_queue.cc:260] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.713235   799 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:47.716485 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:47.717231 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.697991 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:47.718005   534 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election won for term 42
I20260709 12:26:47.718569   454 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.721869 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.721989 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:47.722194   454 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.723067 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:47.719710 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 52245.4us
I20260709 12:26:47.723179 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:47.725478 20775 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.718616   534 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:47.727537 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.727278 32561 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.728847   572 raft_consensus.cc:2804] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:47.729329 20803 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:47.729668 32561 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.727159   534 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 41.23, Last appended by leader: 23, Current term: 42, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.733368 20728 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 22. Preceding OpId from leader: term: 41 index: 23. (index mismatch)
I20260709 12:26:47.721614 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:47.736294 21426 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:47.724833 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.723542 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.738427 20734 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:47.743144 21428 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
W20260709 12:26:47.743448 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 80713us
I20260709 12:26:47.753599 20849 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:47.758806 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.729377   572 raft_consensus.cc:697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:47.759977 20877 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:47.760944 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:47.654711   804 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.762269 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.723901   511 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:47.772714 20728 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:47.772581 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:47.724411   454 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:47.775358 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:47.774610 20729 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:47.783614 20801 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:47.786916 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:47.785772 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:47.776907 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:47.787968 20809 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 22. Preceding OpId from leader: term: 41 index: 23. (index mismatch)
I20260709 12:26:47.795799 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:47.796890 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98990.8us
I20260709 12:26:47.798309 20726 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:47.773164   456 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:47.813732   456 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.814203   456 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:47.814589 32472 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:47.815176 32472 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:47.776710 32622 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.025s
I20260709 12:26:47.790555   495 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:47.790524   572 consensus_queue.cc:237] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.22, Last appended by leader: 22, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:47.803912   830 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:47.807050   775 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:26:47.807446 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 75067.2us
I20260709 12:26:47.819010 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.793834 20729 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:47.820065 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:47.783784   565 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:47.830861 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.833521 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:47.793689   511 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:47.856909 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:47.815809   782 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:47.816447   495 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:47.911033 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:47.913765 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 161667us
I20260709 12:26:47.921541 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:47.816275 32472 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 39.23, Last appended by leader: 23, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:47.923301 21433 raft_consensus.cc:1275] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:47.926043   791 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:47.927642 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 66434.1us
I20260709 12:26:47.928557 20776 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:47.929925   320 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.930478   320 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:47.865727   782 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:47.889027   495 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:47.886149   511 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:47.918694   721 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election won for term 41
W20260709 12:26:47.980279 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.951475   782 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:47.984615 20876 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:47.970872   810 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:47.916706   453 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:48.000722 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:48.005817   705 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:48.013628   320 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:47.988704   778 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.027s
I20260709 12:26:48.033138 20734 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:48.041389 20883 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 39. Current term is 40. Ops: [39.24-39.24]
W20260709 12:26:48.057722 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 304169us
I20260709 12:26:47.957185   728 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.065608 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:48.066286 21435 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:48.083149 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 310772us
I20260709 12:26:48.088712 32316 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:48.096282 32316 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.097024 32316 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.099810 32316 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:48.106032 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.059055   719 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:48.136317 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 78675.9us
I20260709 12:26:48.147529 20883 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:48.041064   810 consensus_queue.cc:1059] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.087s
I20260709 12:26:48.149302 20735 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:48.041131   453 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.065680   780 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:48.157305 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:48.101516   804 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.165292 20809 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 22. (term mismatch)
I20260709 12:26:48.139153 32316 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.007s
I20260709 12:26:48.129992 32554 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.169961 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:48.168215 20809 pending_rounds.cc:85] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 21
I20260709 12:26:48.165321 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.025867   511 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.22, Last appended by leader: 22, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:48.172685   839 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:48.177805 21435 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:48.178740   841 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.181229 20883 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:48.194206 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.181273   456 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:48.183934   690 consensus_meta.cc:287] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 12:26:48.085654   764 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.186363   780 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.200033 20801 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:48.188691 20809 pending_rounds.cc:107] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 32.22
I20260709 12:26:48.200619 20809 raft_consensus.cc:2889] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:48.160233 32472 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:48.205757 20731 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 37 because replica has last-logged OpId of term: 36 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 34 index: 20.
I20260709 12:26:48.198936 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.204754 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:48.200170 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.193604 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.206986 20879 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:48.202522 20700 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.211114 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:48.211755 20807 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:48.211979 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.210373 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:48.212678 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.192646   804 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:48.218690 20808 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:48.210966 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.224295 32316 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.002s
I20260709 12:26:48.227331   741 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.225625 20883 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:48.207401 32622 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:48.231523 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.233003   741 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.233521 20805 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 29 index: 23. Preceding OpId from leader: term: 32 index: 25. (index mismatch)
I20260709 12:26:48.209468 32472 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:48.236770   730 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:48.252391 20727 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:48.227486   800 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.042s
I20260709 12:26:48.197238   841 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.228406   804 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.255250 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.253762 20803 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:48.260353   857 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:48.261415   453 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.252796   565 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:48.264453 32554 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.233971   725 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
W20260709 12:26:48.267521   863 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.078s	user 0.004s	sys 0.000s
I20260709 12:26:48.277199 32472 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
W20260709 12:26:48.271118 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 287424us
I20260709 12:26:48.274166   782 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.275900   847 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:48.282331 20807 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
W20260709 12:26:48.283497 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 131029us
I20260709 12:26:48.230997   787 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:48.264257 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.269563   484 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:48.264288   721 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:48.260375   780 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.284111   719 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:48.293588   725 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.303617 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.309854 20802 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:48.311800   846 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:48.312610   741 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.312827   495 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.314332   841 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.314759   856 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:48.295063   690 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.315564   728 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:48.316412 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64469.5us
I20260709 12:26:48.317114   764 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.291486   565 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:48.328595 32316 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:48.299554   859 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:48.323726   721 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:48.299772   453 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:26:48.346552   864 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:48.335671   846 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.342813   690 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.348678   782 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:48.349232   846 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.349840   690 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.346874   565 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.22, Last appended by leader: 22, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.353183 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.331682   725 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:48.337006   856 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:48.343643   859 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.367913   859 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:48.368916 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.369886   856 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:48.371353   764 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:48.372074   764 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:48.372720   804 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.373495   764 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:48.373276   453 consensus_queue.cc:1175] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 35 status { last_received { term: 33 index: 21 } last_committed_idx: 20 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:48.324327   787 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.380908   495 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.384146 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:48.384773 20878 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:48.385633   858 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:48.386956 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:48.387557 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.388511 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44466 (request call id 2374) took 1507 ms. Trace:
I20260709 12:26:48.388893 21426 rpcz_store.cc:276] 0709 12:26:46.880768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.881027 (+   259us) service_pool.cc:224] Handling call
0709 12:26:48.388296 (+1507269us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54005f3a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:48.388413 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481543936}
I20260709 12:26:48.390297 21428 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 24. Preceding OpId from leader: term: 32 index: 25. (index mismatch)
I20260709 12:26:48.387006 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:48.392976   836 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.585s	user 0.000s	sys 0.002s
I20260709 12:26:48.394068 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.394868 32622 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:48.357548 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 105194us
W20260709 12:26:48.359938 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 103754us
W20260709 12:26:48.362010 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 58501.7us
I20260709 12:26:48.325796   847 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.364701   456 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.365185   565 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:48.365700   852 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:26:48.366562 20850 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.367322 20701 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:48.412474 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 45214.2us
I20260709 12:26:48.413491 20878 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:48.414526 20699 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:48.350160   730 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.415132 20728 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:48.415709 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:48.416340 20733 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:48.408010 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.421895 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.426558 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 39 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.428500 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:48.424728 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:48.431160 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.439535 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.440318 20802 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 32.
I20260709 12:26:48.427173 20806 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 41.
I20260709 12:26:48.454473   847 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.459249 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:48.465842 20732 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 40 index: 23. Preceding OpId from leader: term: 41 index: 24. (index mismatch)
W20260709 12:26:48.468271 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.420162   730 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.471292   730 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:48.471200   859 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.472002 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:48.471379 21428 raft_consensus.cc:1275] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 40 index: 23. Preceding OpId from leader: term: 41 index: 24. (index mismatch)
I20260709 12:26:48.473452   859 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.473052   721 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.395037   725 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.476301 20805 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:26:48.477186   725 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.477245   607 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:48.481805 20734 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:48.481999 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51567.5us
I20260709 12:26:48.484017 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.419111   804 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:48.420392   690 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:48.422519   511 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:48.477949   607 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.495776 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:48.495934 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.491680 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:26:48.489804 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 68708.9us
I20260709 12:26:48.492470 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:48.496492 20884 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:48.424134   495 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:48.494831 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.495635 20808 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:48.499943 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:48.500418 20878 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:48.501783 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:48.502400 20876 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:48.495422   775 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:48.503288 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:48.505422   872 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.006s
W20260709 12:26:48.506565 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 140103us
I20260709 12:26:48.499481   859 consensus_queue.cc:1048] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.009s
I20260709 12:26:48.508689 20850 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.510056   730 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:48.424337   728 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.072s
I20260709 12:26:48.514087 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:48.515807 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:48.516850 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:48.517884 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:48.518839 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:48.498446   804 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:48.521600   804 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.519438 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:48.519680 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:48.519958 20880 raft_consensus.cc:1240] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:48.519985 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.510007 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.526156 20733 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:48.529018 20876 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:48.529999 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:48.534251 20726 raft_consensus.cc:606] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:48.534597 20726 raft_consensus.cc:614] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:48.546777 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 119870us
I20260709 12:26:48.550971 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:48.546373   495 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
W20260709 12:26:48.560899 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 139054us
I20260709 12:26:48.561764 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.562161   791 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.499416   721 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:48.499581   690 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
W20260709 12:26:48.564821 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 67926.3us
I20260709 12:26:48.569140 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.575569 20878 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:48.576911 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.579790 20699 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:48.563007   847 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:48.581418 20700 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:48.582728 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:48.587891 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:48.585753 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.568487   799 raft_consensus.cc:3055] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:48.582410   719 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:48.593398 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:48.593919 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 83988.2us
I20260709 12:26:48.580978   511 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.582116   873 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:48.609565 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:48.607060   723 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:48.609673 20802 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 41 index: 23. Preceding OpId from leader: term: 42 index: 24. (index mismatch)
I20260709 12:26:48.582602   804 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:48.591557   791 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:48.591496   799 raft_consensus.cc:740] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:48.618714 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:48.621238 20727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "761ea542665e419c8a63392e3014efa7"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:48.623845 20727 raft_consensus.cc:606] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:48.587069   728 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.026s
I20260709 12:26:48.624348 20727 raft_consensus.cc:614] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:48.591986 32472 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.634488 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:48.615566   791 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.612596   723 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:48.632247 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46569.6us
I20260709 12:26:48.612411   864 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:48.609563   800 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:48.612834   576 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.613202   511 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.623943   804 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:48.631242   780 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.640216   791 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:48.643637   723 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:48.641008   719 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:48.607915   866 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.093s
I20260709 12:26:48.696012   780 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.746981   780 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:48.747754   780 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:48.749078 20810 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:48.749370   607 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.751348 21426 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:48.751744 21426 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:48.752517 21426 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 39.24, Last appended by leader: 24, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.753502 21426 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:48.693183   576 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.696439   775 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.761368 20806 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:26:48.761550   775 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.763722   775 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.763813   320 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.764303   320 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.766047   778 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:48.766160 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:48.766507   778 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:48.767506   778 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.768664 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:48.769299   880 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:26:48.769361 20878 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:48.770736 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:48.771476 32561 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:48.771847 32561 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.687105   846 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.088s
I20260709 12:26:48.776615 20775 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:48.776932 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 152085us
I20260709 12:26:48.772137   576 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.803984 21426 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 39 index: 24. Preceding OpId from leader: term: 40 index: 24. (term mismatch)
I20260709 12:26:48.804468 21426 pending_rounds.cc:85] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 23
I20260709 12:26:48.804860 21426 pending_rounds.cc:107] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 39.24
I20260709 12:26:48.805284 21426 raft_consensus.cc:2889] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:48.772203 32561 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:48.811578 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.812252 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:48.690043   873 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.118s
I20260709 12:26:48.766525   320 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.815467   607 consensus_queue.cc:1048] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.081s
I20260709 12:26:48.758821   856 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.802397   857 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:48.866590 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 122834us
I20260709 12:26:48.899580 20878 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:48.857615   576 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:48.856057   786 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.045s
I20260709 12:26:48.881811   870 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.801524   719 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.816630 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:48.904147 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 259247us
I20260709 12:26:48.802556   778 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:48.637490   690 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 33.25, Last appended by leader: 25, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.690199   800 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:48.912281 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.926803 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.776969   810 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 39.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.130s
W20260709 12:26:48.920328 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 103761us
I20260709 12:26:48.928610 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:48.929924   868 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:48.929689 20625 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:48.917580   870 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.931699   868 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.931468 20625 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 29 to 32. New cstate: current_term: 32 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:48.932204   868 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:48.932567   870 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.923995 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.944032 20878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:48.944692 20878 raft_consensus.cc:606] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:48.945043 20878 raft_consensus.cc:614] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:48.947338 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:48.903474   721 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.023s
I20260709 12:26:48.949719 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 40 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:48.951411 32316 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
I20260709 12:26:48.951833 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:48.915761   511 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.940218   846 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:48.944885   786 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:49.010805 32316 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:49.013325 32316 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.013718 32316 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:48.915149   782 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.035s
I20260709 12:26:48.951020   868 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:48.914119   857 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:49.019663 20624 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 278) took 46275 ms (client timeout 15000 ms). Trace:
I20260709 12:26:48.933033 20625 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 26 to 32, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:48.940574   866 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.069s
W20260709 12:26:49.033902 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 130169us
I20260709 12:26:48.995908   725 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:49.050150 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 126329us
W20260709 12:26:49.055027 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 103262us
I20260709 12:26:49.018100 32472 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:48.952797   495 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.082s
I20260709 12:26:49.033174   870 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.022289   607 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:49.065831 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:48.947053   804 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:49.095978   894 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:49.117597   875 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.194s
I20260709 12:26:49.060309   725 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:49.088007   886 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:26:49.122063   888 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:49.083353   870 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:49.120219   882 log.cc:927] Time spent T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
W20260709 12:26:49.089040   897 log.cc:927] Time spent T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:49.090011   895 log.cc:927] Time spent T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:26:49.119347   884 log.cc:927] Time spent T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 12:26:49.026307 20624 rpcz_store.cc:269] 0709 12:26:02.719497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.315717 (+14596220us) service_pool.cc:224] Handling call
0709 12:26:17.833152 (+517435us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400002490. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7b0b2 00007f9e24af4759 00007f9e24a8ca76 00007f9e24a5dd58 00007f9e24b5c04e 00007f9e2301d2a4 00007f9e2301d232 00007f9e2301d161 00007f9e2301d0dc
0709 12:26:48.994849 (+31161697us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:44.608876 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:44.609448 (+   572us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:44.609487 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:26:44.609513 (+    26us) tablet.cc:662] Decoding operations
0709 12:26:44.718594 (+109081us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:44.718685 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:44.718711 (+    26us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:26:44.903312 (+184601us) tablet.cc:701] Row locks acquired
0709 12:26:44.903334 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:26:44.903489 (+   155us) write_op.cc:270] Start()
0709 12:26:44.903595 (+   106us) write_op.cc:276] Timestamp: P: 1783600004903441 usec, L: 0
0709 12:26:44.903628 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:26:44.904351 (+   723us) log.cc:844] Serialized 11132 byte log entry
0709 12:26:45.668514 (+764163us) op_driver.cc:464] REPLICATION: finished
0709 12:26:46.075855 (+407341us) write_op.cc:301] APPLY: starting
0709 12:26:46.075994 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:46.669983 (+593989us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:46.670151 (+   168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:46.914970 (+244819us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b18002d3160. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2557 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:47.164352 (+249382us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:47.164983 (+   631us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=0,mrs_lookups=0
0709 12:26:47.165059 (+    76us) write_op.cc:312] APPLY: finished
0709 12:26:47.165887 (+   828us) log.cc:844] Serialized 244 byte log entry
0709 12:26:47.217458 (+ 51571us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:47.285135 (+ 67677us) write_op.cc:454] Released schema lock
0709 12:26:47.285234 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:47.287806 (+  2572us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:47.287844 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":236024,"reported_tablets":48,"spinlock_wait_cycles":298760448,"child_traces":[["op",{"apply.queue_time_us":392792,"apply.run_cpu_time_us":51972,"apply.run_wall_time_us":1215070,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121167,"num_ops":28,"prepare.queue_time_us":133848,"prepare.run_cpu_time_us":24670,"prepare.run_wall_time_us":772638,"raft.queue_time_us":242645,"raft.run_cpu_time_us":12733,"raft.run_wall_time_us":339160,"replication_time_us":764745,"spinlock_wait_cycles":7106688,"thread_start_us":724579,"threads_started":5,"wal-append.queue_time_us":293980,"wal-append.run_cpu_time_us":18503,"wal-append.run_wall_time_us":2647560}]]}
I20260709 12:26:49.099776   791 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.105372 32561 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.116735   856 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.109865   799 consensus_queue.cc:260] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.22, Last appended by leader: 22, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.057085   851 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 33.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.304s
W20260709 12:26:49.122108 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59915.7us
I20260709 12:26:49.233700 20804 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: [33.22-33.22]
I20260709 12:26:49.233803 32561 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.234486 32561 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:49.234543 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:49.088445   787 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.091825   484 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:49.235543 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.235688   484 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:49.236275 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.236591   867 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.237090 20802 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:49.237151   867 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.107131   690 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.238325   690 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.236641   484 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.24, Last appended by leader: 24, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.239992   484 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:49.240233 20624 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 308) took 16017 ms (client timeout 15000 ms). Trace:
W20260709 12:26:49.240614 20624 rpcz_store.cc:269] 0709 12:26:33.222439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.240010 (+16017571us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:49.240155 (+   145us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:26:49.241014   690 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.241747   690 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.089334   800 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.087035 20625 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 33 to 35, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 35 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:49.143919   496 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:49.148005   889 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:49.243448   496 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:49.155731 20810 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33470 (request call id 4494) took 1025 ms. Trace:
I20260709 12:26:49.163678   893 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.244300 20810 rpcz_store.cc:276] 0709 12:26:48.130059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.130314 (+   255us) service_pool.cc:224] Handling call
0709 12:26:48.158953 (+ 28639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.749590 (+590637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.155419 (+405829us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540027c254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:49.155651 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105461504}
I20260709 12:26:49.194888 32316 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:49.195804 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 113577us
W20260709 12:26:49.244845 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.195713   894 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:49.207736   320 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.242333   690 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.222502   728 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.272249 32316 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:49.253118   782 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:49.196254   607 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:49.339977 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 400167us
W20260709 12:26:49.358592 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 104284us
I20260709 12:26:49.235057 32561 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.531245 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.269999   320 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:49.287235   728 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.261651   893 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.378361   719 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.268396   496 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.556420   496 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:49.556679 20883 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 23. Preceding OpId from leader: term: 40 index: 24. (index mismatch)
I20260709 12:26:49.558473 20884 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:49.559474   890 consensus_queue.cc:1059] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 37.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.124s
I20260709 12:26:49.360042   857 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.560563   728 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.543845   866 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:49.562573   891 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:49.562436   866 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.563007   891 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:49.564123   891 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.565560   891 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:49.566113 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.566702 20734 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:49.544422 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2903) took 1017 ms. Trace:
I20260709 12:26:49.567890 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.569871 32561 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.519618   741 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.569116   848 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:49.571283   741 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.568488   850 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:49.571847   848 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:49.572268   850 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:49.573098   850 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.24, Last appended by leader: 24, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.573756   741 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.574460   848 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.568537 20734 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 42.
I20260709 12:26:49.578047   870 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.244002 20625 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 27 to 30, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 30 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:49.579618   791 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:49.512192 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.580349 20625 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 28 to 36, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 36 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:49.512437 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.261471   780 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.271716   856 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.378058   730 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:49.522279 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:49.545126 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.271203   904 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:49.530860 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.278100   867 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.532946 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2904) took 1005 ms. Trace:
I20260709 12:26:49.532096 20879 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 39 index: 23. Preceding OpId from leader: term: 40 index: 24. (index mismatch)
I20260709 12:26:49.536266 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:49.606045 20729 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:49.538233 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.542122 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:49.608017 20726 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 41
I20260709 12:26:49.545053 32316 consensus_queue.cc:1059] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.002s
I20260709 12:26:49.547024 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:49.525439 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.553939 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.554108   725 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.554723 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:49.554899 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.555431 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.586951   873 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.611874 20730 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:49.588106 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:49.612437 20881 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:49.589636   775 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:49.612840 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 90683.1us
I20260709 12:26:49.611905 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.613705 20728 raft_consensus.cc:2393] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:49.590178   778 consensus_meta.cc:287] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.000s
I20260709 12:26:49.503742   841 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:49.621423 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 40 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.566823   864 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:49.414556   894 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 39.23, Last appended by leader: 23, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:49.531468   892 log.cc:927] Time spent T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:26:49.605314 21432 rpcz_store.cc:276] 0709 12:26:48.527854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.527959 (+   105us) service_pool.cc:224] Handling call
0709 12:26:48.534162 (+  6203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.534539 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.534566 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.535040 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.631677 (+ 96637us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:48.631736 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.631831 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.532745 (+900914us) raft_consensus.cc:1685] finished
0709 12:26:49.532767 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.532891 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1761,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":96190,"threads_started":1,"wal-append.queue_time_us":280863}
I20260709 12:26:49.628809 20625 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 33 to 35. New cstate: current_term: 35 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:49.536331 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.605039 20880 raft_consensus.cc:2393] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 41: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:49.606570 20805 raft_consensus.cc:3060] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:49.538763 20802 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:49.610982 20801 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:49.630649 20810 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:49.611338 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:49.611449 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.587438 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.610128 21427 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:49.615280 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.633262 20803 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:49.634104 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.568382   782 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 40.26, Next index: 27, Last known committed idx: 26, Time since last communication: 0.046s
I20260709 12:26:49.617604 20727 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:49.613543 21433 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:49.621681 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:49.642453 20879 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:49.621800 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.634845 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:49.643095 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.591027 21430 rpcz_store.cc:276] 0709 12:26:48.527554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.527686 (+   132us) service_pool.cc:224] Handling call
0709 12:26:48.535797 (+  8111us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.536169 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.536197 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.536605 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.806564 (+269959us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:48.806622 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.806716 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.544216 (+737500us) raft_consensus.cc:1685] finished
0709 12:26:49.544235 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.544368 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":389,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":65,"thread_start_us":269488,"threads_started":1,"wal-append.queue_time_us":397899}
I20260709 12:26:49.635454 20883 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:49.625612 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.644892 20730 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:49.645812 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.642315 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.649469 20728 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:49.650178 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.651078 21428 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:49.663757 20729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "116023b4c0c94bfea737fc238bda3838"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:49.664598 20729 raft_consensus.cc:606] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Received request to transfer leadership to TS 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:49.665052 20729 raft_consensus.cc:614] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:49.621692   725 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:49.654841 20625 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 24 to 30, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 30 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:49.672443 21433 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:49.639343   763 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.603912   780 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.607227   856 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.682862 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:49.692248 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 631196us
I20260709 12:26:49.662150   894 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:49.679323 20625 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 31 to 34, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 34 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:49.609135   730 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:49.678021   900 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:49.700860   902 log.cc:927] Time spent T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 12:26:49.662791   888 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.649278   778 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 38.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.043s
I20260709 12:26:49.622220   873 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:49.754637 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 116749us
W20260709 12:26:49.755353 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 53916.7us
I20260709 12:26:49.686633 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:49.702607 20733 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:49.704071 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.757974 21435 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:49.713992   893 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.719473   320 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.683317   725 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:49.760838   725 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.762655 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.762990 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:49.763784 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.764266 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:49.765285 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.766347 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:49.766857 21428 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:49.765040 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.690753   910 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:49.639439   775 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.641399   864 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.697072   607 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.18, Last appended by leader: 18, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:49.793021 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 106496us
I20260709 12:26:49.750134 20729 raft_consensus.cc:1240] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 35. Current term is 36. Ops: [35.22-35.22]
I20260709 12:26:49.766693 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.767786 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:49.795300 21434 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:49.621716   904 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.769045 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.744004   866 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.700964   900 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.774406   856 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.775355   725 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:49.756644   888 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.732245 20625 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 21 to 27, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 27 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:49.779951 20881 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33.
I20260709 12:26:49.780223 20879 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
W20260709 12:26:49.776561 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 154828us
I20260709 12:26:49.807600 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.779953 20726 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
W20260709 12:26:49.790299 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 201778us
I20260709 12:26:49.749383 20805 raft_consensus.cc:2468] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:49.794854 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.765062 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.809625 21435 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:49.769200   841 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:49.815375 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58898us
W20260709 12:26:49.796523 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 242072us
I20260709 12:26:49.796016 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.808514 21433 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
W20260709 12:26:49.843499 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 48944.6us
I20260709 12:26:49.792593   910 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:49.844959   868 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:49.845705 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.845571   868 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:49.847729 20810 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:49.813105 20625 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 21 to 25, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 25 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:49.809648 20880 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:49.749430   763 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.851069 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.810192   565 raft_consensus.cc:2764] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election decision vote started in defunct term 40: won
I20260709 12:26:49.855360 20625 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 26 to 28, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 28 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:49.857704   730 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:49.858955   867 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:49.859277   763 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.859488   867 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.859951   867 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:49.860929   864 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.792939   320 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 20, Time since last communication: 0.053s
I20260709 12:26:49.862732 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:49.864369 21430 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:26:49.864951   775 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.868985 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.870059 20802 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:26:49.870188 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.871423 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.871634   872 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.871737   868 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.872161   872 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.873531   873 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:49.874568   872 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.872186 20875 raft_consensus.cc:2368] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41.
I20260709 12:26:49.841100   841 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:49.878687 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:49.879807 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:49.880439 20802 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 34
I20260709 12:26:49.881302 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.882555 21428 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:49.876518 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.890116 20727 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:49.890364 20849 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.891659 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:49.892511 21430 raft_consensus.cc:3055] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:49.893815 21430 raft_consensus.cc:740] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:49.893751 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.892771 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:49.894614 20805 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:49.898764 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.899725 20727 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:49.887287 20625 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:49.895766 21400 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.903210 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.903913 20805 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:49.905268 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:49.906256 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:49.912523 20875 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 35: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
W20260709 12:26:49.912660 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.885742   905 raft_consensus.cc:2764] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
W20260709 12:26:49.916091 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 159985us
I20260709 12:26:49.886920   690 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.917924 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.883347   875 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.026s
I20260709 12:26:49.908921 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:49.911703 21430 consensus_queue.cc:260] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 33.25, Last appended by leader: 25, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:49.938107 20625 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 21 to 26, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 26 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:49.940258 20625 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:49.943100 20877 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 25. Preceding OpId from leader: term: 34 index: 26. (index mismatch)
I20260709 12:26:49.943351 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:49.946349 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.904072   913 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader pre-election won for term 43
W20260709 12:26:49.928946   723 consensus_meta.cc:287] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 1.025s	user 0.013s	sys 0.000s
I20260709 12:26:49.950502 21430 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:49.886320   320 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.042s
I20260709 12:26:49.951835 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:49.952227 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:49.944228 20727 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:49.927345   741 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:49.907819   511 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
W20260709 12:26:49.966029 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.917179   775 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:49.915988   922 raft_consensus.cc:2804] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:49.908964   920 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
I20260709 12:26:49.910676   904 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.972462 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:49.973166 32316 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:49.974694   810 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:49.982255 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:49.950078   900 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:49.956045   690 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:49.928089   856 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.039s
I20260709 12:26:49.961067   719 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:49.966625   728 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:50.005141 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:50.007077 21434 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:49.980914   775 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.009539 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:49.966306   511 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:49.967595   780 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.044836 20625 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 27 to 31. New cstate: current_term: 31 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:50.012037   719 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:50.048522   864 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.050202 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.050834 21434 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:49.965400   723 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.052868 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:50.012610   922 raft_consensus.cc:697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:50.058765 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.061929 21400 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.010453   728 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.056749 20727 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41.
I20260709 12:26:50.011698   775 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:50.064626 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:50.069151 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.012310   730 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
W20260709 12:26:50.069187 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 96850.8us
I20260709 12:26:50.071172 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.010483   873 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.043838   877 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:50.074368 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:50.049248   864 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.055251   920 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:50.048954   719 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.054757   511 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:50.056058   782 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.075899 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:50.007589 32561 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.060s
I20260709 12:26:50.066653   495 raft_consensus.cc:3055] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:50.069588   728 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:50.086170 20727 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:50.081985 20625 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 26 to 27, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 27 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:26:50.079921 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.081499   800 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:50.073002   875 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.077160   764 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:50.068508   778 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.108495   778 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:50.084717   929 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:26:50.091027 20850 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.099797   875 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.100885   922 consensus_queue.cc:237] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 39.23, Last appended by leader: 23, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.101892   921 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:50.087589   782 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.118379 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:50.119586 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:50.119624 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:50.119957   782 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.120481 20804 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
W20260709 12:26:50.121162 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.121948   800 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
W20260709 12:26:50.125732   891 consensus_meta.cc:287] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 12:26:50.086357   495 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:50.112735 20625 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 25 to 26, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 26 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:50.117311   864 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:50.088035   920 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:50.131220 21430 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 25. Preceding OpId from leader: term: 34 index: 26. (index mismatch)
I20260709 12:26:50.098205   730 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:50.097796   764 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:50.138168 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.132946 20625 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 22 to 33. New cstate: current_term: 33 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:50.130074 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:50.143163   764 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:50.080571   877 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.151511   877 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.157878   764 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.158146   841 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.161688   875 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.161871 20802 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:50.007653   910 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.24, Last appended by leader: 24, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.163852   841 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.164606 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:50.081514   873 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.148062 20625 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 29 to 32, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:50.175889   778 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.169346   730 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:50.165912   856 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.019s
I20260709 12:26:50.192349 21432 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 24. Preceding OpId from leader: term: 35 index: 24. (term mismatch)
I20260709 12:26:50.192898 21432 pending_rounds.cc:85] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Aborting all ops after (but not including) 23
I20260709 12:26:50.193171 21432 pending_rounds.cc:107] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Aborting uncommitted NO_OP operation due to leader change: 34.24
I20260709 12:26:50.193454 21432 raft_consensus.cc:2889] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.188899 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:50.187961   723 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.168669   495 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.190194 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:50.199364   723 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:50.199803   723 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:50.199491   920 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.200208   723 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:50.208662   894 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.001s
I20260709 12:26:50.194747   511 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.209389   913 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.209522   728 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.194904 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.188410 20625 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 20 to 26, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 26 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:50.212385 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:50.230608 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.229686 21400 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:50.234103   891 consensus_queue.cc:1059] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:50.234743 21431 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 39 index: 23. Preceding OpId from leader: term: 40 index: 24. (index mismatch)
I20260709 12:26:50.235112   780 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:50.235522   728 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.235749   877 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:50.236209   877 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.236584   877 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:50.236975   873 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.238644   867 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.235600   780 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.240046   780 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:50.183465   905 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 37.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.059s
I20260709 12:26:50.240943   867 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.236667 20880 raft_consensus.cc:1240] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:50.207739   775 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.245373 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.228219   913 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:50.249614 20625 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 27 to 28, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 28 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:50.242897   905 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:50.258724 20877 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:50.259066 21434 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:50.261751 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:50.262813 20729 raft_consensus.cc:2393] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 27: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:50.264554 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.264717 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.272219 20875 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:50.272842 20880 raft_consensus.cc:2393] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 34: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:50.274470 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.275182 21432 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:50.276247 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.277423 20875 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:50.246917   778 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 38.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.019s
I20260709 12:26:50.275261   775 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.263391   905 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:50.281893   905 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.24, Last appended by leader: 24, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.283087   780 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.284904 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:50.291303   780 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.276252   764 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.293272 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with delete_type TABLET_DATA_TOMBSTONED (TS 657bc62d20374e539b23641715f22745 not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:34158
I20260709 12:26:50.296893 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:50.298118 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:50.298978   946 log.cc:927] Time spent T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:50.301050 21427 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:50.302341 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:50.307307 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.309484 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.277356 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:50.317435 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 77274us
I20260709 12:26:50.318169 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:50.283110   905 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:50.326539 20878 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:50.318982 21432 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:50.329624 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.307147   914 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:50.334839 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.322542   728 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:26:50.360993 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.328341   920 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:50.325565   856 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.383375   920 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.383579   856 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.384021   920 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:50.386586 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.382027   841 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.387588   856 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.387815   841 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.390072   841 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.339622   728 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:50.391523   728 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:50.400530 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.401327 20880 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:50.401885 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:50.402294 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:50.402448 20881 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:50.402829 20880 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:50.405023 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.406150 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.406720 20876 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:50.333873   914 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:50.408720 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.409785 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.418371 20881 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:50.425331 20880 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:50.431881 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.431778 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.408034   877 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.433853   873 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.005s
I20260709 12:26:50.427261   914 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 40.26, Last appended by leader: 26, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.440614   914 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:50.424439   841 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.460173   841 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.461907   910 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:50.462491   910 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:50.450739 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:50.464471   778 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:50.465042 21399 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:50.435091   955 tablet_replica.cc:333] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:26:50.467118   705 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:50.456881   894 consensus_queue.cc:1059] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:50.459506 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.476541 21426 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 27.
I20260709 12:26:50.466109   778 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:50.456285   850 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:50.473246 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:50.458552   728 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.469296 21399 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:50.483876 21399 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.475159 20803 raft_consensus.cc:1240] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 33. Current term is 34. Ops: [33.22-33.22]
I20260709 12:26:50.436525   899 raft_consensus.cc:3055] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:50.500646   899 raft_consensus.cc:740] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:50.501521   899 consensus_queue.cc:260] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 37.22, Last appended by leader: 22, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.449749   891 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 34.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.053s
I20260709 12:26:50.478619   721 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.503585   891 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:50.503746   721 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.504022   891 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.504699   891 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:50.463749   910 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.494930   870 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.497076 20733 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 40 index: 22. Preceding OpId from leader: term: 42 index: 24. (index mismatch)
I20260709 12:26:50.437723   723 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.499791   893 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:50.502712   899 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:50.504050   873 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:50.462333   913 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.462935   841 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.506664 20734 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 18. Preceding OpId from leader: term: 37 index: 19. (index mismatch)
I20260709 12:26:50.507767   721 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:50.508033   728 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.514025   870 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.515910   780 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:50.541328 21399 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:50.544273   893 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:50.545212   873 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:50.559631 21399 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:50.560869   894 raft_consensus.cc:2804] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:50.564100   864 raft_consensus.cc:2804] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:50.564196 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.564611 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:50.565272 20734 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:50.565459 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:50.567243   894 raft_consensus.cc:697] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:50.567404   908 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:50.567698   955 raft_consensus.cc:2243] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:50.568516   955 raft_consensus.cc:2272] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:50.568691   913 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.568244   894 consensus_queue.cc:237] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.569633   496 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.570142   496 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.573659   955 ts_tablet_manager.cc:1916] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:26:50.573637   893 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.574980   893 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:50.564641   864 raft_consensus.cc:697] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:50.580788 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:50.585983   888 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.580854   864 consensus_queue.cc:237] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.588335   496 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:50.588904 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.589663   905 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:50.589424   873 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.590874   873 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:50.592247   918 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.592943   856 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.592875   918 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.593468   856 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.594187 21427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2913) took 1001 ms. Trace:
I20260709 12:26:50.592353   728 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.594571 21427 rpcz_store.cc:276] 0709 12:26:49.593216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.641796 (+ 48580us) service_pool.cc:224] Handling call
0709 12:26:49.674194 (+ 32398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.311217 (+637023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.593953 (+282736us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b540047f754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:50.594124 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439254912}
I20260709 12:26:50.600612 20733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:50.604609   728 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.605023 20733 raft_consensus.cc:606] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:50.605070   877 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.605425 20733 raft_consensus.cc:614] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:50.596864   856 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.606773 21427 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:26:50.606969   728 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.595057   918 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.607714 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:50.596427   841 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.608479   841 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.609537 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:50.610492 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:50.610693   841 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.504374   920 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.597162   495 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:50.597615 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:50.603606   870 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.621039 20734 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 42.
I20260709 12:26:50.614508   910 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:50.638794 20729 raft_consensus.cc:1240] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:50.643551   901 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:50.677834 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.681195 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:50.681981 21432 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:50.698314 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:50.700862 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:50.707943 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.712725 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:50.717032 20776 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:50.638861   841 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:50.702956   780 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:50.731037 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 186362us
I20260709 12:26:50.713349   778 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.715878   899 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
W20260709 12:26:50.741496 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.828579 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.730712   841 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.724174   719 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:50.732730   913 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
W20260709 12:26:50.834373 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.839380 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72069.3us
I20260709 12:26:50.841413   800 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.841897   800 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.765570   899 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:50.848830 20807 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 42. Current term is 43. Ops: []
I20260709 12:26:50.852075 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:50.852413 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:50.834833   841 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:50.880641 20726 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:50.766001   780 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.880649 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:50.831827   778 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.858356   899 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:50.889346 20728 raft_consensus.cc:2393] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:50.892472   968 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.896049   920 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:50.896318   968 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.894025 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:50.900207 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.914481 20801 raft_consensus.cc:2410] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for term 38 because replica has last-logged OpId of term: 36 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 34 index: 20.
W20260709 12:26:50.897637   966 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:26:50.844666   719 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.922528 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:50.933007 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 90817.4us
W20260709 12:26:50.942628   916 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.185s	user 0.004s	sys 0.000s
I20260709 12:26:50.944132 20731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4434) took 2431 ms. Trace:
I20260709 12:26:50.944480 20731 rpcz_store.cc:276] 0709 12:26:48.512413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.524582 (+ 12169us) service_pool.cc:224] Handling call
0709 12:26:48.548995 (+ 24413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.566938 (+ 17943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.566968 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.578253 (+ 11285us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.135083 (+556830us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b18003c7ba0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:49.487358 (+352275us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:49.487443 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.487685 (+   242us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.942081 (+1454396us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:50.943885 (+  1804us) raft_consensus.cc:1685] finished
0709 12:26:50.943906 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.944053 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140,"raft.queue_time_us":3674,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3591,"spinlock_wait_cycles":19655808,"thread_start_us":351989,"threads_started":1,"wal-append.queue_time_us":1109618}
I20260709 12:26:50.948271   955 ts_tablet_manager.cc:1929] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.17
I20260709 12:26:50.949210   955 log.cc:1199] T f15b5f1ac1444b63bad60a89fa9f8a29 P 657bc62d20374e539b23641715f22745: Deleting WAL directory at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/wals/f15b5f1ac1444b63bad60a89fa9f8a29
I20260709 12:26:50.950492 20699 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.951323   320 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:50.952410   799 raft_consensus.cc:3060] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:50.953433 20699 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:50.954596   962 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:50.955824   320 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:50.956226   320 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:50.956573   320 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:50.957623   780 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:50.965173   320 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:50.919801   741 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:50.969594 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.920689   787 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:50.970767 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:50.972802 20734 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 23. Preceding OpId from leader: term: 41 index: 24. (index mismatch)
I20260709 12:26:50.967670 32316 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:50.979526 32316 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:50.981124 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:50.935408   847 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:50.987725 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:50.992039 20877 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
I20260709 12:26:50.982013 21427 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 27.
I20260709 12:26:50.940668   800 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:50.979923   741 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:51.000784 20699 leader_election.cc:400] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41.
I20260709 12:26:51.005311 20699 leader_election.cc:403] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.975723   914 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.985620   920 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:50.995441   850 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.510s	user 0.005s	sys 0.005s
I20260709 12:26:51.008379   918 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 40. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41.
I20260709 12:26:50.988960   963 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 42.24, Next index: 25, Last known committed idx: 21, Time since last communication: 0.025s
I20260709 12:26:51.009004 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:50.993623   787 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:50.993467   320 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.021960 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.035136 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:51.025640   921 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 41.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.026s
I20260709 12:26:51.011845   914 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.034978   870 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:51.039299   730 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:51.063737   891 consensus_meta.cc:287] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.539s	user 0.006s	sys 0.006s
I20260709 12:26:51.092756 20726 raft_consensus.cc:1240] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:51.098901 21434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44460 (request call id 2916) took 1207 ms. Trace:
I20260709 12:26:51.091542 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:51.096163 21431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2909) took 1537 ms. Trace:
I20260709 12:26:51.101490 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:51.039028   920 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.032s
I20260709 12:26:51.101557 21431 rpcz_store.cc:276] 0709 12:26:49.541834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.541948 (+   114us) service_pool.cc:224] Handling call
0709 12:26:49.543767 (+  1819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.235125 (+691358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.078599 (+843474us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54004da8d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:51.078805 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569494400}
W20260709 12:26:51.074263   948 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.751s	user 0.002s	sys 0.000s
I20260709 12:26:51.099493 21434 rpcz_store.cc:276] 0709 12:26:49.891373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.068877 (+177504us) service_pool.cc:224] Handling call
0709 12:26:51.098647 (+1029770us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54001e69d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:51.098827 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432694144}
I20260709 12:26:51.102255 20733 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:51.104972 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet f15b5f1ac1444b63bad60a89fa9f8a29 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:26:51.111613 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:51.112385 20735 raft_consensus.cc:3060] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:51.105749   964 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:26:51.039748   496 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.030s
I20260709 12:26:51.117223 20728 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:51.125308 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:51.125051 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:51.128774 20733 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:26:51.106721 32622 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:51.138808 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.140909 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.141568 20805 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 31.
I20260709 12:26:51.119169   778 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:51.096850   891 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.147701 20806 raft_consensus.cc:1275] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 23. Preceding OpId from leader: term: 41 index: 24. (index mismatch)
W20260709 12:26:51.152937   973 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:26:51.133157   921 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.154659   921 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.156632   921 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:51.157863   893 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.003s
I20260709 12:26:51.135480   786 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.167840   607 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:51.168047   786 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.169726   914 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.170179   786 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:51.170517   841 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:51.171233   495 consensus_meta.cc:287] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.533s	user 0.012s	sys 0.000s
I20260709 12:26:51.172705   841 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.168421   607 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:51.173171 21429 raft_consensus.cc:1275] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:51.173396 20849 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:51.173839   607 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.174728   787 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:51.174799 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2910) took 1632 ms. Trace:
I20260709 12:26:51.144955 32622 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:51.178866   873 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.001s
I20260709 12:26:51.176348   782 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Stepping down as leader of term 42
W20260709 12:26:51.184079 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.190105 20805 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: [36.23-36.23]
I20260709 12:26:51.126744   968 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.187467 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.184787 20735 raft_consensus.cc:2468] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:26:51.191044 20802 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:51.192016 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.192408 20730 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:51.192638 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:51.186808 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:51.193547 20810 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:51.180971   891 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.198213 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.177918   725 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.199339   725 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.201177   719 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:51.231241 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.222635 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.236106 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.237252 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.222374 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.204197 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:51.213151 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.192407 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.177779 21429 rpcz_store.cc:276] 0709 12:26:49.542153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.542279 (+   126us) service_pool.cc:224] Handling call
0709 12:26:49.562006 (+ 19727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.173574 (+1611568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.174581 (+  1007us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540058d9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:51.174747 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3704152832}
W20260709 12:26:51.238605 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 113648us
I20260709 12:26:51.238809 20808 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:51.240387 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.241312 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:51.241497 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.241875 20878 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 37 index: 21. (index mismatch)
I20260709 12:26:51.242125 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.242551 20776 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:51.242822 20877 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:51.242878 20776 leader_election.cc:403] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.243824 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.242564 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.241887 20809 raft_consensus.cc:3060] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 38
I20260709 12:26:51.251611 20882 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 30: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:51.252832 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.245518 20883 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:51.265492 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.175203   787 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.275525 20810 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:51.180135 32622 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.201727   725 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.282145 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:51.272725 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.189761   782 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:51.293136 20802 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 37 index: 21. (index mismatch)
I20260709 12:26:51.294323 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.284565 20805 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:51.317281 20808 raft_consensus.cc:2393] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:51.331385 20809 raft_consensus.cc:2468] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:51.179114   899 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.327689 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.327980 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.290733 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.330452 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:26:51.337169 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46576.4us
I20260709 12:26:51.338042 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.339612 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.256171   454 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.065s
I20260709 12:26:51.344420 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.344718 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.345489 20878 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:51.345693   800 consensus_queue.cc:1048] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:51.346678 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.325088   868 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.023s
I20260709 12:26:51.348975   705 raft_consensus.cc:3055] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:51.349370   705 raft_consensus.cc:740] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:51.349954   782 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 42.24, Last appended by leader: 24, Current term: 42, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.350234   705 consensus_queue.cc:260] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.341889   496 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:51.347748 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.321738   730 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:51.338909 32622 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 40.24, Next index: 25, Last known committed idx: 22, Time since last communication: 0.018s
I20260709 12:26:51.293478   870 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:51.351342 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.360291 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:51.354935 21433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44462 (request call id 2303) took 1171 ms. Trace:
I20260709 12:26:51.283772   787 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:51.353279 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.367199 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.367435 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.354312 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.370903 20803 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:51.283324   456 consensus_queue.cc:1048] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.059s
I20260709 12:26:51.373977 20875 raft_consensus.cc:2368] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:51.352627 20810 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:26:51.376029 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:51.334427   848 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 37. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:51.405910 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.410527 20804 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:51.377667 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.407505 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.351167   782 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:51.388371 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.413259 20876 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:51.351450   982 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.364746 21433 rpcz_store.cc:276] 0709 12:26:50.183366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.197516 (+ 14150us) service_pool.cc:224] Handling call
0709 12:26:50.299699 (+102183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:50.300081 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.300109 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:50.304236 (+  4127us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.420068 (+115832us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:50.640971 (+220903us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.643351 (+  2380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.644566 (+  1215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.644625 (+    59us) raft_consensus.cc:1685] finished
0709 12:26:50.644639 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.354749 (+710110us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400088c50. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:51.354873 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":122805,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":1541,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":18526,"spinlock_wait_cycles":234062720,"thread_start_us":324642,"threads_started":2,"wal-append.queue_time_us":149696}
I20260709 12:26:51.435761 20875 raft_consensus.cc:2393] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:51.358317 32622 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:51.366206   786 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.357358   799 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.415783 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.404059 32561 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.439069 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:51.440440 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 116582us
I20260709 12:26:51.442626 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.448462 20774 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:51.357494   893 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:51.377898   496 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.439050   495 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:51.459803 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.460865   456 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:51.451289 20774 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.451653 20809 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:51.452720 20875 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 18. Preceding OpId from leader: term: 37 index: 19. (index mismatch)
I20260709 12:26:51.441653 32622 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
W20260709 12:26:51.445030   981 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:51.405714   730 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.049s
W20260709 12:26:51.445395   910 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.802s	user 0.003s	sys 0.012s
I20260709 12:26:51.356024   705 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:51.451810 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.372339   891 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:51.461447   456 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.455581   850 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:51.441633   982 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.459969   721 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:51.454038   496 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.472396 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.454342 32561 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.484768 32561 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.486078 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 4235) took 1417 ms. Trace:
I20260709 12:26:51.486510 20727 rpcz_store.cc:276] 0709 12:26:50.068909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.087537 (+ 18628us) service_pool.cc:224] Handling call
0709 12:26:50.095364 (+  7827us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:50.115754 (+ 20390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.115828 (+    74us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:51.144772 (+1028944us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b2c00025e88. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1d598a11 00007f9e1d5a8372 00007f9e1ecdaa01 00007f9e1ec9f43a 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2
0709 12:26:51.145195 (+   423us) log.cc:844] Serialized 52 byte log entry
0709 12:26:51.336638 (+191443us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:51.337193 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.337287 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:51.485758 (+148471us) raft_consensus.cc:1685] finished
0709 12:26:51.485782 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.486011 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1957,"raft.queue_time_us":19904,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"spinlock_wait_cycles":5201152,"thread_start_us":96608,"threads_started":1,"wal-append.queue_time_us":227253}
I20260709 12:26:51.489108 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.490125   454 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
I20260709 12:26:51.495535 20881 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:26:51.496855   741 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.498252 20807 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 36. Current term is 37. Ops: []
W20260709 12:26:51.500023   925 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 1.433s	user 0.000s	sys 0.004s
I20260709 12:26:51.500238   990 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:51.500860   990 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.502398   982 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:51.502861   990 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.477581   495 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:51.447873   799 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.506461 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.446605   786 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.509956 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.510504 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.513136 20732 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 2148) took 2316 ms. Trace:
I20260709 12:26:51.513511 20850 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.460196   913 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.020s
I20260709 12:26:51.517405 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.518355 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:51.518855 20699 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.518659   870 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:51.520040   728 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:51.517942 20877 raft_consensus.cc:2376] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 33. Re-sending same reply.
I20260709 12:26:51.513542 20732 rpcz_store.cc:276] 0709 12:26:49.196780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.207381 (+ 10601us) service_pool.cc:224] Handling call
0709 12:26:49.245533 (+ 38152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.272565 (+ 27032us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:26:49.272617 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.342290 (+ 69673us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.813792 (+471502us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:49.813846 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.813916 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:51.462085 (+1648169us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:51.507252 (+ 45167us) raft_consensus.cc:1685] finished
0709 12:26:51.507274 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.513072 (+  5798us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113277,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":16482,"spinlock_wait_cycles":7200000,"thread_start_us":204647,"threads_started":1,"wal-append.queue_time_us":415172}
I20260709 12:26:51.524394 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2918) took 1555 ms. Trace:
I20260709 12:26:51.524835 21430 rpcz_store.cc:276] 0709 12:26:49.969147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.140714 (+171567us) service_pool.cc:224] Handling call
0709 12:26:50.141060 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.186497 (+ 45437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.186532 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:50.186574 (+    42us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:50.186614 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.524200 (+1337586us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b54007a80d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:51.524241 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.524344 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104002304}
I20260709 12:26:51.528680 20875 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
I20260709 12:26:51.527633 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:51.515290 32561 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:51.515290   495 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:51.533700 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.536821   786 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.523130   721 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:51.544227 20884 raft_consensus.cc:1275] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:26:51.540006 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.525175   922 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:26:51.544412 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60929.8us
I20260709 12:26:51.463007   893 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.525032   741 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.552642 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.553534 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.555223 20807 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:51.543607   870 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:51.555610 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.557148 20774 leader_election.cc:400] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:51.531646   914 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.023s
I20260709 12:26:51.542436   982 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.569577 20699 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:51.572229 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:51.515408 32622 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 40.24, Last appended by leader: 24, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.545130   922 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.561153 32561 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:51.584308   920 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:51.585110   920 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:51.586269   920 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.557406   870 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:51.558281   990 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:51.589709 21399 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.592270 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:51.543877   778 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.028s
I20260709 12:26:51.558696   891 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.604498   841 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.605274   841 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.607532   456 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.608542   799 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:51.609297 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.609652   496 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:51.592015   870 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:51.611361   841 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.613333 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.589183   922 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:51.568663   974 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.034s
I20260709 12:26:51.591821   986 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:51.576140   719 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.045s
I20260709 12:26:51.598436   741 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:51.583531   982 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:51.644519 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 73716.2us
W20260709 12:26:51.659245 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.631475   841 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.664456 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:51.666383 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 64443.9us
I20260709 12:26:51.590799   921 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 41.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.047s
I20260709 12:26:51.617106   891 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:51.635600 32622 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:51.672700 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.625252   850 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.049s
I20260709 12:26:51.674626 20875 raft_consensus.cc:2393] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:51.616895   787 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.668393   730 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 36.26, Next index: 27, Last known committed idx: 25, Time since last communication: 0.008s
I20260709 12:26:51.676462 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.680217 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.681392 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:51.678164   979 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:51.704386   982 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.706656   787 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:51.713081 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.717998   723 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.722173 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:51.724855 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.727689 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:51.722729   725 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.728814 20732 raft_consensus.cc:2368] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:51.719285   963 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Stepping down as leader of term 41
I20260709 12:26:51.729890   963 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:51.732789   963 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 41.24, Last appended by leader: 24, Current term: 41, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:51.733306   983 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.399s	user 0.002s	sys 0.002s
I20260709 12:26:51.734012   963 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:51.673348   841 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.728888   998 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:51.766830 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:51.760206   998 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:51.775923   841 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:51.781380 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56573.7us
I20260709 12:26:51.782998 20700 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.759692   719 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader election lost for term 42. Reason: could not achieve majority
I20260709 12:26:51.737776   725 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.793955 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.798151 20882 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:51.758558   922 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.783439   511 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:51.805531 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.807003 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:51.814824 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:51.820037 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.812165   841 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:51.821815   511 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:51.800590   998 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 36.26, Last appended by leader: 26, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.824142   998 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:51.824529 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:51.826416 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:51.828452 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:51.831504 20775 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:51.832504 32622 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader election won for term 42
I20260709 12:26:51.833030 32622 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 42 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:51.827365   999 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:51.833873 32622 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 41.24, Last appended by leader: 24, Current term: 42, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.821810   841 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.838030 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:51.827565 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:51.838799 21430 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:51.820945 20700 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:51.842025 20700 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:51.843518 21435 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:51.837378 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:51.849808 32316 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:51.803180   719 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.010s
I20260709 12:26:51.844384   974 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.854375   974 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.856747   974 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.852249   799 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:51.849364   511 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:51.866807 20775 leader_election.cc:400] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:51.867372 20775 leader_election.cc:403] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.859723   799 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.816342  1005 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:51.900672 20729 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
W20260709 12:26:51.900898 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.818035   728 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:51.884974 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:51.845970   920 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:51.808962   870 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:51.922629 20883 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 43.
I20260709 12:26:51.911729   725 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.919690 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:51.845808  1009 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:51.914964   841 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:51.907608 20732 raft_consensus.cc:2410] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 37 because replica has last-logged OpId of term: 36 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 35 index: 22.
I20260709 12:26:51.909078 20775 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:51.928745 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:51.932075 21430 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:51.930552 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 123617us
I20260709 12:26:51.935037 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.935632 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:51.903033 32561 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:51.905781   999 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election lost for term 37. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:51.922091   920 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.942173 20879 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 41: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:51.922340   800 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:51.946058 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 104134us
I20260709 12:26:51.948824 21429 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:51.930493   974 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:51.927739   705 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:51.947014 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.938511   728 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:51.934633   456 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:51.963373 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:51.964084 21429 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:26:51.955869 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.959673 21430 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 39.
I20260709 12:26:51.947817   920 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:51.994123  1003 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:51.935999   725 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:26:51.999836   719 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:52.000041  1008 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:26:52.001081 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45281.2us
I20260709 12:26:51.953859   974 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.007830 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:52.002228 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 74934.8us
I20260709 12:26:52.010452 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:52.011973   719 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:52.012478   719 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:52.012682 20701 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:52.013381   719 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.016425  1011 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:52.016970  1011 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 39 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:52.017844  1011 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 38.23, Last appended by leader: 23, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:51.961774   728 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.961617 32561 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:51.950198   841 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.053s
I20260709 12:26:52.027648 21426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2411) took 1811 ms. Trace:
I20260709 12:26:52.038061 21426 rpcz_store.cc:276] 0709 12:26:50.216052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.509846 (+293794us) service_pool.cc:224] Handling call
0709 12:26:50.510233 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.078707 (+568474us) raft_consensus.cc:1534] Early marking committed up to index 25
0709 12:26:51.080329 (+  1622us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.080531 (+   202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:51.080586 (+    55us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:51.080631 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.027352 (+946721us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54005f3a54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:52.027416 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.027577 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1512,"prepare.queue_time_us":1603,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":2851,"raft.queue_time_us":350,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4936,"spinlock_wait_cycles":1208586112}
I20260709 12:26:52.040480 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:52.041137 21426 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:52.002492 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:51.954123   810 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.068s
W20260709 12:26:52.041051 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 114028us
I20260709 12:26:51.952399   800 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.036859 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:52.031811 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:52.022639   974 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:51.961972   705 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.033105   728 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:51.965963   891 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.055536 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:52.060693 21426 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 28.
I20260709 12:26:52.063745   799 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.067677 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:52.068097 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:52.069166 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:52.068284 21427 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:26:52.034752   899 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:52.069758 21429 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
W20260709 12:26:52.071486 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 69178.7us
I20260709 12:26:52.072448 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:52.048326   922 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:52.118146 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:52.067407   511 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:52.128273 21427 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:52.120383   910 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:52.135226 20701 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:52.094146  1015 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:52.142557 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:52.153806 20876 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 34.
I20260709 12:26:52.099223   917 consensus_queue.cc:1059] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 38.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.047s
W20260709 12:26:52.155826 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 100644us
I20260709 12:26:52.156572 20699 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:52.135300   725 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.167483   841 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:52.167743   511 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.168107   841 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.170095   841 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.171715   705 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:52.172411  1009 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:52.155673   910 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.172977  1009 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:52.173847  1009 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.175439   725 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.136408  1013 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:52.183226 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:52.183782 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:52.141222   870 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.193898 20806 raft_consensus.cc:1275] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
I20260709 12:26:52.207845   719 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:52.208487   719 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:52.194492   800 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:52.199323  1010 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:26:52.209358   719 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.190269   872 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:52.211308   872 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.124033  1019 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.068s
I20260709 12:26:52.200002 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:52.190632   572 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.216744   572 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.219698   572 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:52.199913 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.228168 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.204320 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:52.229604 21430 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 39.
I20260709 12:26:52.224532 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.225574 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.189462   725 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:52.184835  1011 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:52.234174 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.260275 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.234860 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.241120 20726 raft_consensus.cc:1240] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:52.263671 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.175019   910 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.265447 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.214963   872 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:52.265480 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.274883 20806 raft_consensus.cc:2393] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 41: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:52.231945   877 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.277139 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.281051 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:52.281836 20804 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:52.281100 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.286931 20701 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:52.288942 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 201833us
I20260709 12:26:52.289364 20734 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:52.180002  1020 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.112s
I20260709 12:26:52.295059 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.296845 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.298067 20805 raft_consensus.cc:2393] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:52.285365   901 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.299877   901 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.236033   920 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.301929   901 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.265116   705 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 38.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.026s
I20260709 12:26:52.232786   799 consensus_queue.cc:1048] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 21, Time since last communication: 0.052s
I20260709 12:26:52.304637   891 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.307367 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.307911 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:52.308475 20876 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:52.305845 20730 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38. Ops: [37.19-37.19]
I20260709 12:26:52.310092 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.305850 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.313179 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.312425 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.317826 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.318580 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.318727 32316 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:52.221206  1013 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.319206 20728 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:52.320215 20876 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:52.320292 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.321483 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:52.303452   920 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.329667   511 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:52.331972  1026 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:52.332633  1026 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:52.333539  1026 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.18, Last appended by leader: 18, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.308610   810 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:52.326933 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.328224 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.326030 20735 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
W20260709 12:26:52.401075 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 72923.3us
I20260709 12:26:52.344262 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:52.403033 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 58848.1us
I20260709 12:26:52.404063 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.405941 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:52.406908 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 80475.8us
I20260709 12:26:52.408159 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.319465   870 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:52.291150   877 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.428473 32316 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:52.342365  1019 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:52.431408 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.433243 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.462602 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:52.468366 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 60271.5us
I20260709 12:26:52.470602 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.469121 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.403811   721 consensus_queue.cc:1059] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 41.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.068s
I20260709 12:26:52.477625 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:52.498772 20810 raft_consensus.cc:2393] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 34: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:52.499240 21433 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:26:52.502409 21432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44460 (request call id 2920) took 1893 ms. Trace:
I20260709 12:26:52.502831 21432 rpcz_store.cc:276] 0709 12:26:50.609478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.718196 (+108718us) service_pool.cc:224] Handling call
0709 12:26:50.718512 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.913454 (+194942us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:50.930566 (+ 17112us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.954107 (+ 23541us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b180034f540. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:26:51.117070 (+162963us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:51.117160 (+    90us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:51.117247 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.127749 (+1010502us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b180034f540. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419
0709 12:26:52.502180 (+374431us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54005a4954. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:52.502227 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.502349 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15632,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":140,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":10149,"spinlock_wait_cycles":465022464,"thread_start_us":162575,"threads_started":1,"wal-append.queue_time_us":217605,"wal-append.run_cpu_time_us":1092,"wal-append.run_wall_time_us":1010908}
I20260709 12:26:52.496886 20803 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: [36.23-36.23]
I20260709 12:26:52.510787 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54488 (request call id 4433) took 1126 ms. Trace:
I20260709 12:26:52.511226 20883 rpcz_store.cc:276] 0709 12:26:51.384598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.388039 (+  3441us) service_pool.cc:224] Handling call
0709 12:26:52.510600 (+1122561us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400267854. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:52.510702 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":855614208}
I20260709 12:26:52.511747   721 raft_consensus.cc:3055] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Stepping down as leader of term 41
I20260709 12:26:52.512138   721 raft_consensus.cc:740] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:52.512944   721 consensus_queue.cc:260] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 41.23, Last appended by leader: 23, Current term: 41, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.514312   721 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:52.412550   511 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:52.412526   910 consensus_queue.cc:1048] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.103s
I20260709 12:26:52.518805   723 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:52.523759  1025 log.cc:927] Time spent T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:26:52.472401 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.413894   920 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:52.414676   870 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.412940   496 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:52.471735 20806 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 41.
W20260709 12:26:52.529922 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59451.5us
I20260709 12:26:52.473999 20801 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:52.554311 20729 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:52.554972 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:52.559018 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:52.555634 20731 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:52.561769 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:52.480252 20774 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.19.36.195:45387 timed out after 1.902s (SENT) [suppressed 6 similar messages]
I20260709 12:26:52.563862 20730 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 41 index: 24. Preceding OpId from leader: term: 42 index: 25. (index mismatch)
I20260709 12:26:52.564208 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:52.471406   901 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:52.493968   572 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.565613 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.528204  1015 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:52.586500 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:52.558048   910 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:52.560767   870 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:52.571942  1021 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:52.558130   872 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 35.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:26:52.580787   705 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:52.548069   723 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.612694  1013 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.613754   799 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:52.623574   799 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:52.594086   778 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:52.584687   454 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.033s
I20260709 12:26:52.635771   778 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.591012  1019 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 22, Time since last communication: 0.037s
I20260709 12:26:52.581844   572 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.588617  1015 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.637630   456 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:52.686169   877 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.652588   705 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:52.688601   705 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.647923   778 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:52.650112   810 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:52.714005 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.691160   705 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.724416 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.725679 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:52.746246 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.677403   456 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:52.698845   454 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.751865  1020 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:52.755532  1027 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:52.759416 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:52.712291   872 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:52.762091 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:52.736413  1011 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.735975   511 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:52.781325 20804 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:52.756829   979 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.737182   810 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:52.851284 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 371041us
I20260709 12:26:52.863215  1005 raft_consensus.cc:3055] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:52.863873  1005 raft_consensus.cc:740] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:52.794659   979 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:52.833433   728 consensus_meta.cc:287] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.763s	user 0.007s	sys 0.004s
I20260709 12:26:52.766166   454 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.993283 20884 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:26:52.994619 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:52.794832  1041 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:52.783829  1027 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.065s
I20260709 12:26:52.814600  1020 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:52.947280 21427 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
W20260709 12:26:52.951889  1036 log.cc:927] Time spent T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:26:52.797042   456 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 37.19, Last appended by leader: 19, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.003890   910 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.032838   910 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:53.003942  1031 log.cc:927] Time spent T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:26:53.036851   810 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:52.960008  1015 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:52.963618  1034 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:26:52.965314   901 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:52.967761  1035 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 12:26:52.968894   979 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.970141   873 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:52.982259   850 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:52.783238  1011 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:52.983717 20774 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:52.985724   572 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:52.985905 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.986629 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:52.773123   872 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.987758   723 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:52.988418   997 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:52.990085 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:53.001196   974 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.932s	user 0.005s	sys 0.005s
I20260709 12:26:53.005887 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:53.005898 20883 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38. Ops: [37.19-37.19]
I20260709 12:26:53.042955   921 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:53.025946   721 consensus_meta.cc:287] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 12:26:53.026607   454 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.027966 20880 raft_consensus.cc:3055] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:52.784924   511 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.034756   728 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:52.955892   891 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:53.113780 20733 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.25-34.25]
W20260709 12:26:53.111271 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52989.7us
I20260709 12:26:53.046092 20876 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
W20260709 12:26:53.055792 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 72137.4us
I20260709 12:26:53.050721 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.054984 20881 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:53.078841 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:53.049598 21430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.007s	sys 0.026s
I20260709 12:26:53.003556  1041 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:53.084971 21435 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:53.057767 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.088938 21431 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:26:53.102497  1019 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:53.050275   901 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.059664   496 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.040016 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:53.117173 21430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.007s	sys 0.027s
I20260709 12:26:53.110553 20880 raft_consensus.cc:740] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:26:53.122148 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 64458us
I20260709 12:26:53.119333 20699 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:53.124640 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.058593   986 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:53.059538   800 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:53.133635 20849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.100s	sys 0.124s
I20260709 12:26:53.052330   873 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:53.058772   998 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:53.133987 20849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.100s	sys 0.124s
I20260709 12:26:53.096655   893 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.096390   572 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.132145 20880 consensus_queue.cc:260] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 38.23, Last appended by leader: 23, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:53.145618 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 717200us
I20260709 12:26:53.158416 20880 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.160393   873 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.161706   873 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:53.131943  1019 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.165542  1019 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.135965  1040 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.036s
I20260709 12:26:53.172240   979 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.108762   997 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.172914   979 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:53.173439   979 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:53.174505   997 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.174320   979 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.156952   998 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.176626   910 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.180537   572 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.120791  1041 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:53.179001   998 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.121881   721 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:53.174798  1030 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.708s	user 0.001s	sys 0.001s
I20260709 12:26:53.105011   850 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.127345 32561 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:53.189695 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:53.191154 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 57925.4us
I20260709 12:26:53.196530 20880 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 38 index: 23. Preceding OpId from leader: term: 39 index: 24. (index mismatch)
I20260709 12:26:53.196990 20884 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:53.137091   496 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.198829   998 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:53.200428 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.200668   496 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.201073 20884 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 42.
W20260709 12:26:53.201073   782 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 39 current term: 39 is higher.
I20260709 12:26:53.203262  1013 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:53.205519 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.110687   974 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.211205 20884 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 43: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:53.206046 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:53.116258   921 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.158210   917 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:53.127426   891 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.183054   572 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.227157 21429 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: [37.24-37.24]
I20260709 12:26:53.212409 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:53.231698 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.237731 20803 raft_consensus.cc:3055] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:53.249945 20803 raft_consensus.cc:740] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:53.215592   997 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:53.245836 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:53.193396   721 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.254053 20801 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:53.249243 20805 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:26:53.148406   800 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.223435   917 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:53.256896 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.257381 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.250787 20803 consensus_queue.cc:260] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.261175 20803 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:53.266212 20806 raft_consensus.cc:2393] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 43: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:53.252803   997 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.241214   920 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.032s
I20260709 12:26:53.255089   728 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.272981 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:53.261112   998 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:53.265888  1026 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election won for term 43
W20260709 12:26:53.268404  1047 log.cc:927] Time spent T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:26:53.274992 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:53.246726  1013 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.022s
I20260709 12:26:53.262791   917 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:53.262012   893 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.278849 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:53.291913 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.292193 21427 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
W20260709 12:26:53.283149 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 338397us
I20260709 12:26:53.270207   997 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.298229 21435 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:53.285099 20803 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 21. (term mismatch)
I20260709 12:26:53.299804 20803 pending_rounds.cc:85] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 20
I20260709 12:26:53.300398 20803 pending_rounds.cc:107] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 27.21
I20260709 12:26:53.300696 20803 raft_consensus.cc:2889] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:53.301838 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 177277us
I20260709 12:26:53.281906  1026 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.319697 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:53.319770  1050 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:26:53.324864   997 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.325858 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.328027 21428 raft_consensus.cc:1240] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:53.329191 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.329376   997 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:53.329856 20735 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 4266) took 1106 ms. Trace:
I20260709 12:26:53.329957 21427 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:53.330222 20735 rpcz_store.cc:276] 0709 12:26:52.223680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.224914 (+  1234us) service_pool.cc:224] Handling call
0709 12:26:52.225163 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.344971 (+119808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.329646 (+984675us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003ad9d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:53.329804 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231540480}
I20260709 12:26:53.332866 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:53.335652  1045 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:26:53.336805 21430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2430) took 1169 ms. Trace:
I20260709 12:26:53.321277 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.337157 21430 rpcz_store.cc:276] 0709 12:26:52.167146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.315835 (+148689us) service_pool.cc:224] Handling call
0709 12:26:52.335899 (+ 20064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.412129 (+ 76230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.412173 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.466156 (+ 53983us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.121890 (+655734us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:53.121928 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.121991 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.336618 (+214627us) raft_consensus.cc:1685] finished
0709 12:26:53.336638 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.336753 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70725,"raft.queue_time_us":72337,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":20546,"thread_start_us":597197,"threads_started":1,"wal-append.queue_time_us":633221}
I20260709 12:26:53.338899  1064 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:53.339342  1064 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.339717  1064 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:53.342343 20733 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33906 (request call id 2178) took 1030 ms. Trace:
I20260709 12:26:53.342728 20733 rpcz_store.cc:276] 0709 12:26:52.312392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.312511 (+   119us) service_pool.cc:224] Handling call
0709 12:26:52.324399 (+ 11888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.114167 (+789768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.342099 (+227932us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54003ad4d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:53.342288 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814837504}
I20260709 12:26:53.345204 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:53.345808 21434 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.347697 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.349326 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.352783   511 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.353284 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.356364   901 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:53.358623  1011 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.360191   850 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.360875 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.361843 21434 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:53.365140 20807 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2445) took 1070 ms. Trace:
I20260709 12:26:53.365571 20807 rpcz_store.cc:276] 0709 12:26:52.295033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.295155 (+   122us) service_pool.cc:224] Handling call
0709 12:26:52.295435 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.295623 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 24
0709 12:26:52.693004 (+397381us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.088634 (+395630us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.357343 (+268709us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b1800531e20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb802a6 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1
0709 12:26:53.357925 (+   582us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b1800531e20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:53.364590 (+  6665us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b1800531e20. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb2cc0a 00007f9e2526c219 00007f9e1bbb4a40 00007f9e1bbaf4e4 00007f9e1bbaf455 00007f9e1ec97060 00007f9e1eca9716 00007f9e1ecba287 00007f9e1ecba208
0709 12:26:53.364765 (+   175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.364816 (+    51us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:53.364862 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.364932 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.365051 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18625,"prepare.queue_time_us":29784,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":464,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":35772,"spinlock_wait_cycles":80843904,"thread_start_us":688069,"threads_started":2,"wal-append.queue_time_us":505796}
I20260709 12:26:53.367929   891 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:53.369267  1005 consensus_queue.cc:260] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 36.23, Last appended by leader: 23, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.375651   799 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:53.376664 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 261490us
W20260709 12:26:53.377403 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.378453   870 consensus_meta.cc:287] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.770s	user 0.007s	sys 0.002s
I20260709 12:26:53.378541   454 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:53.379819 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.427315 20776 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.380204   889 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:53.381872 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.383508 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2421) took 1537 ms. Trace:
I20260709 12:26:53.384279   974 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:53.386720 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:53.388309 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.389269   456 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:53.390636   872 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.391880   721 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:53.393469  1055 log.cc:927] Time spent T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:26:53.393677  1041 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:53.375104   778 consensus_meta.cc:287] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.643s	user 0.003s	sys 0.003s
I20260709 12:26:53.395105   800 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.396505   917 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.400028 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.457058 21431 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:26:53.457183 20850 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.304509   893 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.447135   889 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.461611   889 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.457228   721 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:53.462319 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.462989 21427 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:53.462726   721 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.464030 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.406023 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:53.406721 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 109170us
I20260709 12:26:53.407347  1009 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:53.409364 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.433943 21429 rpcz_store.cc:276] 0709 12:26:51.845729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.072161 (+226432us) service_pool.cc:224] Handling call
0709 12:26:52.092943 (+ 20782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.227746 (+1134803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.383267 (+155521us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54004d9c54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:53.383444 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607582592}
I20260709 12:26:53.417935  1026 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:53.449949 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:53.428555  1021 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.817s	user 0.008s	sys 0.003s
W20260709 12:26:53.450253 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 135255us
I20260709 12:26:53.483739 20882 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
I20260709 12:26:53.455211 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.400149   921 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.458439   800 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:53.465016 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33486 (request call id 4320) took 1019 ms. Trace:
W20260709 12:26:53.466675 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 254332us
I20260709 12:26:53.470357 20884 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:53.453737 20776 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:53.451771   799 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.521421 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.521642 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.522060 20804 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:53.522224 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:53.522504 20727 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:53.523293 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53222.2us
I20260709 12:26:53.488914 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:53.524462 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.525347 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:53.497644 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.516211 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.519167 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.526379 20728 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:53.526849 20731 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:53.527369 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:53.527971 20728 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.430974   870 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.519654 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.435480   454 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.530411 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.531116 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.532521 20804 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
I20260709 12:26:53.533814 20805 raft_consensus.cc:1240] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 36. Current term is 37. Ops: [36.23-36.23]
I20260709 12:26:53.481812   922 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.536232 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.509317 20802 rpcz_store.cc:276] 0709 12:26:52.445289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.445473 (+   184us) service_pool.cc:224] Handling call
0709 12:26:52.445886 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.446341 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:52.492443 (+ 46102us) log.cc:844] Serialized 14 byte log entry
0709 12:26:52.756858 (+264415us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b18004888c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f23a 00007f9e1eca9a2a
0709 12:26:53.087936 (+331078us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.100312 (+ 12376us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b18004888c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb23f3 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:53.113740 (+ 13428us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.113890 (+   150us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:53.113924 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.113989 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.181111 (+ 67122us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b4800002958. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:26:53.195735 (+ 14624us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b18004888c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:53.346995 (+151260us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b18004888c0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2557 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:53.464807 (+117812us) raft_consensus.cc:1685] finished
0709 12:26:53.464832 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.464963 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208661,"prepare.queue_time_us":45912,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":7986,"raft.run_cpu_time_us":1363,"raft.run_wall_time_us":98284,"spinlock_wait_cycles":211831936,"thread_start_us":323266,"threads_started":1,"wal-append.queue_time_us":712901}
I20260709 12:26:53.537227 20732 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 39: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:53.520166  1011 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.018s
I20260709 12:26:53.539037   454 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.540822 20730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:53.541369 20730 raft_consensus.cc:606] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:53.541635   917 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.542339 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.543562 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.544322   893 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.544539 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:53.545403  1059 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:53.546739 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.547392 20883 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:53.550952 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.551260 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.552675 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:53.553421 20803 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.548235 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.558841 20883 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
W20260709 12:26:53.559762 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 106154us
I20260709 12:26:53.541688 20730 raft_consensus.cc:614] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:53.560825 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:53.563690 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:53.564563 20802 raft_consensus.cc:3060] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.508301  1021 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:53.573470 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:53.576634 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 43792.3us
I20260709 12:26:53.578125 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:53.452551   778 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.573525 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:53.585093 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.532269  1017 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:53.593590 20803 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:53.584348 21399 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:53.531355   800 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:53.485863  1040 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.086s
I20260709 12:26:53.556094   920 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:53.600157 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.590715 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:53.532147   998 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:53.485114 32622 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.081s
I20260709 12:26:53.592773 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.584071 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:53.620644 21435 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:53.617056 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.618620 20884 raft_consensus.cc:2376] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 657bc62d20374e539b23641715f22745 in term 43. Re-sending same reply.
I20260709 12:26:53.626057 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.571892   850 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:53.572501   782 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.623157 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.554872   721 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.629801 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.624790 20883 raft_consensus.cc:2393] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 39: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:53.589857   913 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:53.629459 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:53.634573 21429 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:53.592180   974 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:53.635143 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.598803 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:53.641846 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.643760  1060 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:53.644910 32622 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:53.645335 32622 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:53.612619   877 raft_consensus.cc:2764] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election decision vote started in defunct term 38: won
I20260709 12:26:53.639252 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.613372   901 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:53.580482  1059 consensus_queue.cc:1059] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 34.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.051s
I20260709 12:26:53.593159   872 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:53.593578  1017 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:53.566488   922 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.657439 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.659111 20732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26ced453445642b09bda798a2963748b"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:53.687675 20732 raft_consensus.cc:606] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:53.688160 20732 raft_consensus.cc:614] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:53.651095 21399 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:53.660759 20802 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:53.638716   997 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:53.694600   997 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.694695   810 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:53.695032   997 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:53.695115   810 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:53.695562   728 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.696069   728 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.698210   728 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.647425   974 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:53.661983 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:53.695581   810 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:53.706283   922 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.634608   723 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:53.707477   723 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.708745 20728 raft_consensus.cc:2468] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:53.669888   778 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.672082 20878 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 41 index: 24. Preceding OpId from leader: term: 42 index: 25. (index mismatch)
I20260709 12:26:53.710506   870 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.639430   782 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.631367   998 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.711012 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.727782 21435 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:53.723907 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.728092 20731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:53.728644 20731 raft_consensus.cc:606] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:53.718761 21430 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:53.729377 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.724505 21431 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 38 index: 23. Preceding OpId from leader: term: 39 index: 24. (index mismatch)
I20260709 12:26:53.723012 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.721800 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.731922 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.724282 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.740608 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:53.723521 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.746829 20726 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:26:53.746970 21435 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:26:53.750168 21434 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:53.677237 21429 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:53.584816  1011 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.055s
I20260709 12:26:53.595901   800 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.24, Last appended by leader: 24, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.650628 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:53.783645 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.784287 21431 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 30.
I20260709 12:26:53.699365   974 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:53.596514   719 consensus_queue.cc:1059] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.079s
I20260709 12:26:53.652273   572 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.048s
W20260709 12:26:53.699976 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.691433 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 48640.1us
I20260709 12:26:53.702399  1064 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.686305  1017 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:53.775259 20878 raft_consensus.cc:2393] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 41: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:53.737388 20877 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:53.782284 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:53.782565 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:53.805496 20882 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:53.805682 20875 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 40
I20260709 12:26:53.785153 20806 pending_rounds.cc:85] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 18
I20260709 12:26:53.797253 21429 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: [32.23-34.24]
I20260709 12:26:53.806334 20806 pending_rounds.cc:107] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 37.19
I20260709 12:26:53.801121 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:53.806712 20806 raft_consensus.cc:2889] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:53.718472  1041 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.811303 20850 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.721781   725 raft_consensus.cc:3055] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:53.802150 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:53.807118 21431 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:53.826141 20882 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 41.
I20260709 12:26:53.818001 20876 raft_consensus.cc:2368] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 41. Current term is 42.
I20260709 12:26:53.827692 20801 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 41.
I20260709 12:26:53.821271   920 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
I20260709 12:26:53.813265  1062 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.816565   782 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.813203  1066 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:53.825260 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.832167 20774 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:26:53.834744 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 141193us
I20260709 12:26:53.813534  1060 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:26:53.843326 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 259309us
I20260709 12:26:53.844252 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:53.813421  1009 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:53.818238   725 raft_consensus.cc:740] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:53.842466 21399 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:53.849480 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:53.850847 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 260271us
W20260709 12:26:53.851619 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.829391   922 raft_consensus.cc:2804] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:53.852268  1060 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.867933 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:53.868647 21429 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:26:53.872561 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:53.824695   799 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 33.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.024s
I20260709 12:26:53.836499   889 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader election won for term 43
I20260709 12:26:53.872445 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:53.852799   921 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:53.877008 20849 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:53.851115   985 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:53.893545 20850 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
W20260709 12:26:53.894840 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 150568us
I20260709 12:26:53.869894   901 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:53.902436 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:53.871634  1060 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.905371 20801 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 37: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:53.875002   721 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:53.874696   454 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:53.872938   922 raft_consensus.cc:697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:53.910873   922 consensus_queue.cc:237] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 34.24, Last appended by leader: 24, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.911643  1041 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:53.912051  1064 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:53.912216  1041 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 41 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:53.913323  1041 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 40.24, Last appended by leader: 24, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.902866  1039 raft_consensus.cc:2804] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:53.916973 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.917742 20726 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 30.
I20260709 12:26:53.840765  1062 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.870337   456 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.033s
I20260709 12:26:53.956213 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:53.860298  1009 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:53.904161   496 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:53.892027   921 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:53.907050   901 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.907863   723 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:53.966086 20875 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:26:53.967267 20849 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:53.916420  1039 raft_consensus.cc:697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:53.918187   889 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:53.889838   997 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.972030  1013 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.972594 20849 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:53.973320   922 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:53.976845 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.977909 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.979892   920 consensus_queue.cc:1059] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:53.987524 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.983817 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:53.988173 20728 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:26:53.988406 20734 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:53.984757 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:53.989010 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:53.986500 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 134869us
W20260709 12:26:53.990625 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 148241us
I20260709 12:26:53.991533 20849 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:53.958686  1027 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:53.964346   496 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.994170 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:53.960740  1020 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:53.969635   921 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:53.965144   454 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:53.969051   901 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:53.998000 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:54.000507 20880 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 36: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:53.970659   870 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.969116   998 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.015445 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:53.999564   496 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:53.993429  1064 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:54.017067   454 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.995249  1013 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.906083   725 consensus_queue.cc:260] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 34.24, Last appended by leader: 24, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:53.994650   917 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:54.031623   913 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:53.994570   728 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.061s
I20260709 12:26:54.067330   454 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:54.069242  1075 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 12:26:54.029563   997 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.078073 21431 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:26:54.000882  1027 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.079706 20732 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:54.003829  1020 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.075696   725 raft_consensus.cc:3060] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:54.020423   870 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.095389 20726 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:54.096990   870 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.097687   870 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:54.098237   870 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.100658   870 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:54.101895  1083 log.cc:927] Time spent T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:54.073411  1087 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:54.078449  1058 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.106859 20734 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
W20260709 12:26:54.114058 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 135689us
I20260709 12:26:54.114598  1058 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.119659  1062 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.072408   889 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 42.24, Last appended by leader: 24, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.121495  1058 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.026705   810 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.107890  1087 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:54.130928  1058 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.003s
I20260709 12:26:54.134510   870 raft_consensus.cc:993] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:26:54.135005   870 raft_consensus.cc:1081] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:26:54.134979  1017 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.136117  1041 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:54.137212  1017 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.133513 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:54.138193  1017 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:54.138566  1017 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.138911  1017 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:54.145824 20733 raft_consensus.cc:1240] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 37. Current term is 38. Ops: []
W20260709 12:26:54.145951 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 147748us
I20260709 12:26:54.146258 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.147869 20849 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.072539  1039 consensus_queue.cc:237] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 37.22, Last appended by leader: 22, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.159032   974 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.156908 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.167652   974 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:54.167939  1084 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:54.173556   901 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.147208   496 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.163727   917 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:54.148340  1087 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:54.163913  1009 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.175133  1027 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.193629   921 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.193908 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.155406  1041 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:54.194759 21434 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 39.
I20260709 12:26:54.196827 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.197321 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.197793 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.197922 20803 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
W20260709 12:26:54.203553 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.211130 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.213809 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.221036 20734 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 42: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:54.214264 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57479.4us
I20260709 12:26:54.228631 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:54.231000 20734 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 36: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:54.231135 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.232045 21435 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:54.213158 20806 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:26:54.282212 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:54.283154 20806 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:54.178321   901 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.211400   810 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.219412 21434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44466
I20260709 12:26:54.313934 21434 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:54.314306 21434 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:54.222874   997 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:54.176986  1013 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.234179   782 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:54.319677 20699 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.295996 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 234254us
I20260709 12:26:54.325343 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:54.312350  1040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.026s	sys 0.033s
I20260709 12:26:54.329522 20726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6"
dest_uuid: "49323f66286f4798b268f216bb62b3be"
mode: GRACEFUL
new_leader_uuid: "657bc62d20374e539b23641715f22745"
 from {username='slave'} at 127.0.0.1:46772
I20260709 12:26:54.331291 20726 raft_consensus.cc:606] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Received request to transfer leadership to TS 657bc62d20374e539b23641715f22745
I20260709 12:26:54.331662 20726 raft_consensus.cc:614] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:54.335556 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:54.326131  1026 consensus_meta.cc:287] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.815s	user 0.008s	sys 0.004s
I20260709 12:26:54.342351 21432 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:54.344934 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.345624 21428 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:54.347848 21427 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:54.352963   921 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:54.330202   868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.003s	sys 0.007s
W20260709 12:26:54.333873  1040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.026s	sys 0.033s
I20260709 12:26:54.315372  1017 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.370786  1087 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.360428 21434 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.330735   782 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:54.380589   868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.004s	sys 0.008s
I20260709 12:26:54.391451  1005 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:54.415670 21427 raft_consensus.cc:1240] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:54.358623   999 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:54.423312 20879 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 35 index: 18. Preceding OpId from leader: term: 38 index: 19. (index mismatch)
I20260709 12:26:54.427371 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.428355 20726 raft_consensus.cc:2393] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:54.430111 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.430447 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.430889 20877 raft_consensus.cc:2368] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38.
I20260709 12:26:54.395658  1017 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.442689 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:54.444334 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:54.396296 20876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 2418) took 1049 ms. Trace:
I20260709 12:26:54.445120 20875 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:54.397396 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:54.398654   873 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.450990 21399 leader_election.cc:400] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38.
I20260709 12:26:54.451525 21399 leader_election.cc:403] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:54.406193   917 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:54.408227 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
W20260709 12:26:54.461433 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53310.9us
I20260709 12:26:54.410519 20880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2"
dest_uuid: "116023b4c0c94bfea737fc238bda3838"
mode: GRACEFUL
new_leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:41760
I20260709 12:26:54.462668 20880 raft_consensus.cc:606] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Received request to transfer leadership to TS 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:54.463001 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.409857   979 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.358520  1026 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.414835 20878 raft_consensus.cc:1240] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:54.417227   893 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:54.418541 21434 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.437104 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.437512 21432 raft_consensus.cc:1240] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 37. Current term is 38. Ops: [37.24-37.24]
I20260709 12:26:54.394905 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.445408 20876 rpcz_store.cc:276] 0709 12:26:53.347324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.347447 (+   123us) service_pool.cc:224] Handling call
0709 12:26:54.396041 (+1048594us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b540005b754. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:54.396217 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079416576}
I20260709 12:26:54.453230 20775 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:54.459285 21428 raft_consensus.cc:2393] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 31: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:54.460480 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:54.463120 20880 raft_consensus.cc:614] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:54.409772  1096 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.475700 20884 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:54.413920 20806 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 42.
I20260709 12:26:54.423130   999 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:54.478428 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 83621.5us
I20260709 12:26:54.481395 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.477346   496 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.456256   913 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.471869   810 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 38
I20260709 12:26:54.481585 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:54.492789 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 193491us
I20260709 12:26:54.475646   893 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:54.488783 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.489836 21400 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:54.499045 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.499212 20735 raft_consensus.cc:2468] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:26:54.483863 20775 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:54.490276 20729 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 42. Current term is 43. Ops: [42.25-42.25]
I20260709 12:26:54.472456  1017 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:54.493909 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.500212 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.487897 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:54.482108   974 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:54.480316   456 raft_consensus.cc:2764] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader pre-election decision vote started in defunct term 41: won
I20260709 12:26:54.501284 21400 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:54.490782 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:54.501717 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.502142  1060 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:54.502631  1087 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.492517  1089 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:54.506764   999 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.524261 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:54.524202 21428 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:26:54.463917   873 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.512471 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:54.516476 20878 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:54.516381 20880 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:54.489094  1099 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:54.521284 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:54.474370   979 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.522819 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.525709 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.528700 21431 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Advancing to term 42
I20260709 12:26:54.529186 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:54.529094 32622 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.533865 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:54.535748 21432 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 41: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:54.537686 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:54.538275 20805 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:54.533243 20884 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 31.
I20260709 12:26:54.540235 20875 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:54.542704 21426 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:54.481206  1096 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.513973   998 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:54.544605 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.554932 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 37.
I20260709 12:26:54.518392   901 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:54.492619   913 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.543859 20733 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 39: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:54.549027 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:54.550990 21431 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 42.
I20260709 12:26:54.546141 20849 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:54.529129  1105 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:54.559485 20877 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 42.
I20260709 12:26:54.560650  1096 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.561868   922 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:54.562407   922 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.564720   922 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.565411 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.565814   782 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.526459  1093 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:54.528951  1017 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:54.567435 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.567926   979 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.568040 20802 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 41.
I20260709 12:26:54.569033 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:54.526472  1091 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:54.546377   999 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:54.570160 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.527823  1062 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:54.590682 20880 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 41.
I20260709 12:26:54.590154 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:54.582726 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.594923 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.591610 20878 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:54.574574 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:54.596493 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 44 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.570812 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.597357 20807 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:54.598050 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.597318 20730 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 43.
I20260709 12:26:54.600579 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.601135 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.601776 20808 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:54.603220 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.604501 20809 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:54.598827 20733 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 42.
I20260709 12:26:54.605105 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:54.529935  1039 consensus_queue.cc:1059] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 37.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.060s
I20260709 12:26:54.613097 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 44 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.613883 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.616008 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:54.609159  1013 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election won for term 43
I20260709 12:26:54.559062  1105 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.619756  1039 raft_consensus.cc:3055] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:54.620151  1039 raft_consensus.cc:740] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:54.620313  1026 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.620990  1039 consensus_queue.cc:260] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 37.20, Last appended by leader: 20, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.622141  1039 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:54.588299   979 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:54.615517  1106 log.cc:927] Time spent T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:26:54.624517 20805 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:54.599330   782 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election won for term 42
I20260709 12:26:54.557111   921 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.055s
I20260709 12:26:54.646294 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:54.652266 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:54.626060   917 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader pre-election won for term 44
I20260709 12:26:54.657713 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:54.629988   997 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election won for term 43
I20260709 12:26:54.571038   998 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:54.689774  1108 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:54.587889   893 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 34.25, Last appended by leader: 25, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.666872   997 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.666638   917 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.693606 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.657325   999 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:54.708669   893 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:54.711432   917 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Advancing to term 44
I20260709 12:26:54.650502   782 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:54.706471   913 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.706764 20699 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.715167 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.639314   979 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.609688  1062 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:54.717967 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.704941   997 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:54.724401 20699 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:54.633773  1017 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.692436  1009 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 12:26:54.729650 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:54.719568  1058 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:54.722195   841 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:54.732618  1026 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:54.726203  1039 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:54.740314 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.729187  1057 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:54.742879  1105 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.736197  1026 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:54.734540   873 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.736204   725 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:54.748440   998 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.749759 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.758522 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.758999 20880 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:54.762259 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.760552 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.758477   725 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.764295   725 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:54.765755  1020 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:54.752404  1039 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:54.769064  1039 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.771694  1026 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.772444  1066 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.772953  1066 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:54.754070   913 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:54.775609   913 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.777261  1011 raft_consensus.cc:3055] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:54.777711  1011 raft_consensus.cc:740] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:54.778733   913 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:54.779421 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.780267 20733 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 24. Preceding OpId from leader: term: 35 index: 25. (index mismatch)
I20260709 12:26:54.785830   979 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:54.786495 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:54.791759 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:54.797475 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:54.798395 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: true dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:54.798974 20726 raft_consensus.cc:3055] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:54.799317 20726 raft_consensus.cc:740] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:54.804811 20730 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 34. Current term is 35. Ops: [34.25-34.25]
I20260709 12:26:54.808179 21435 raft_consensus.cc:1240] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:54.809276 21434 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:54.809481 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:54.809587 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.810086 21428 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:54.810326 21426 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:54.811086 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:54.812443 20883 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:54.814810 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.815460 21434 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:54.815112 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:54.800004 20726 consensus_queue.cc:260] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.825177 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:54.825531 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.826696 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.778569  1011 consensus_queue.cc:260] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.24, Last appended by leader: 24, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.826473 20776 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:54.830444 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 38 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:54.831470 20726 raft_consensus.cc:3060] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:54.815914  1062 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.840958  1039 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:54.843953 20850 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:54.869144 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:54.871834 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:54.752602   782 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 40.26, Last appended by leader: 26, Current term: 42, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:54.873580 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 47155.5us
I20260709 12:26:54.822870   778 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:54.875149 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:54.905526 21428 raft_consensus.cc:2468] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:54.916401 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:54.859925   998 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:54.916204 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:54.866132  1011 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:54.925836 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50814.2us
I20260709 12:26:54.872109   496 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:54.910622   572 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:54.927423 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:54.917946   850 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:54.931588  1115 log.cc:927] Time spent T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:54.929811 20726 raft_consensus.cc:2468] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:26:54.931788  1059 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:54.935998   998 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:54.914703   778 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:54.979238 20726 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 22. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
I20260709 12:26:54.957000  1058 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:54.959918   572 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:54.961354  1059 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.960608   850 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:55.014653 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 100962us
W20260709 12:26:55.017155 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.017851 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.015039  1005 consensus_meta.cc:287] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 12:26:54.989490  1059 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:54.983023   572 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:55.025832 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.029356 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.015465   810 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 37. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 37. Current term is 38.
I20260709 12:26:55.031591 20727 raft_consensus.cc:2410] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 38 because replica has last-logged OpId of term: 36 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 35 index: 22.
I20260709 12:26:55.040561   778 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:55.044440 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:55.049548 20882 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53814 (request call id 2428) took 1630 ms (client timeout 2036 ms). Trace:
W20260709 12:26:55.050128 20882 rpcz_store.cc:269] 0709 12:26:53.419126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.544333 (+125207us) service_pool.cc:224] Handling call
0709 12:26:55.049303 (+1504970us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540022e2d4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:55.049474 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598712576}
I20260709 12:26:55.051664 20884 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:55.047108   999 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:54.986019  1105 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.038s
I20260709 12:26:55.004784   725 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.060045 20805 raft_consensus.cc:1275] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 22. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
I20260709 12:26:55.037942   572 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:55.071640  1121 log.cc:927] Time spent T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:26:55.073020 20805 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 24. Preceding OpId from leader: term: 39 index: 25. (index mismatch)
I20260709 12:26:55.077301   850 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:55.057235   979 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:55.078166   979 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.079607  1013 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.080065  1013 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:55.080367   979 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:55.083703 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:55.084033 21400 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:55.084210  1059 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.085476   810 raft_consensus.cc:2804] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:55.085557 20729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37282 (request call id 4275) took 1595 ms. Trace:
I20260709 12:26:55.086007 20729 rpcz_store.cc:276] 0709 12:26:53.490587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.491290 (+   703us) service_pool.cc:224] Handling call
0709 12:26:53.491683 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.500213 (+1008530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.085294 (+585081us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54000e7ed4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:55.085498 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295872896}
I20260709 12:26:55.086037   810 raft_consensus.cc:697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:55.088004   810 consensus_queue.cc:237] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.089751 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:55.090312 21430 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:55.058631   872 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 40.24, Next index: 25, Last known committed idx: 23, Time since last communication: 0.033s
I20260709 12:26:55.095995 21428 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: [31.21-31.21]
I20260709 12:26:55.097708  1020 consensus_queue.cc:1059] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:55.098189 20727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37282 (request call id 4287) took 1376 ms. Trace:
I20260709 12:26:55.098565  1020 raft_consensus.cc:3055] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:55.098979  1020 raft_consensus.cc:740] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:55.099300   917 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.098631 20727 rpcz_store.cc:276] 0709 12:26:53.721415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.721544 (+   129us) service_pool.cc:224] Handling call
0709 12:26:55.097994 (+1376450us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400226ad4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:55.098133 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2988163968}
I20260709 12:26:55.100534 20881 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 43. Current term is 44. Ops: []
I20260709 12:26:55.101009 32622 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:55.099823  1020 consensus_queue.cc:260] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.102281  1020 raft_consensus.cc:3060] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:55.102600 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.103314 20730 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 35: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:55.106279 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.107065 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.108103 21430 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:26:55.114723 20701 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:55.057132   998 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.115623   728 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:55.116108   728 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:55.117022   728 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.120929   893 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:55.121670   893 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.109643   917 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:55.124723 21432 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:55.127964 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 44 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
W20260709 12:26:55.129169 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 56915.2us
I20260709 12:26:55.148110 20727 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Advancing to term 44
I20260709 12:26:55.148702 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:55.149386 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 62771.8us
W20260709 12:26:55.134410 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52379.6us
W20260709 12:26:55.151636 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.135836 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:55.141307   893 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:55.120976   997 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.146692 20804 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 31.
I20260709 12:26:55.134418  1059 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:55.152892 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 44 candidate_status { last_received { term: 42 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:55.158037 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.161163 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.168078 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.172636 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:55.165634 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:55.184808 20727 raft_consensus.cc:2468] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 44.
I20260709 12:26:55.198383 20849 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:55.199858 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.158596   913 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.194813   848 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:55.207532 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:55.219050 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46534.5us
I20260709 12:26:55.220221 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:55.171533   979 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:55.224493 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.194902  1094 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:55.231056 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.201707  1096 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:55.205088  1039 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:55.164633   999 consensus_queue.cc:1059] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 41.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.058s
I20260709 12:26:55.215255   921 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:55.215818   782 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:55.228590   979 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.206156   848 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:55.239840 20776 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:55.226438  1084 log.cc:927] Time spent T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:26:55.215569  1087 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:26:55.259210 21401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.228358   723 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:55.141791  1093 consensus_queue.cc:1059] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.134s
I20260709 12:26:55.149678   986 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 40.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.125s
I20260709 12:26:55.233564  1082 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:55.216879   913 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.241206   985 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:55.288040   979 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:55.282897   848 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:55.234833  1094 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.288506  1123 raft_consensus.cc:2804] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:26:55.305575 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 71245.4us
I20260709 12:26:55.307269   997 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:55.316222   725 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:55.318977  1094 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:55.325157 21429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44466 (request call id 2451) took 1104 ms. Trace:
I20260709 12:26:55.325768 21429 rpcz_store.cc:276] 0709 12:26:54.220792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.221006 (+   214us) service_pool.cc:224] Handling call
0709 12:26:54.296488 (+ 75482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.312145 (+ 15657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.312174 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:54.312651 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 12:26:54.873046 (+560395us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b1800508960. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bb94e07 00007f9e249ef9a9 00007f9e1bbb169b 00007f9e1bbaf93f 00007f9e1bbb1c02 00007f9e1e83920b 00007f9e1e83ed23 00007f9e1e83f0d1 00007f9e1ec5aa0f
0709 12:26:55.035743 (+162697us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:55.055730 (+ 19987us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.055932 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.056013 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.324736 (+268723us) raft_consensus.cc:1685] finished
0709 12:26:55.324868 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.325056 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15164,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":16028,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":6920,"spinlock_wait_cycles":7510784,"thread_start_us":162481,"threads_started":1,"wal-append.queue_time_us":908727}
I20260709 12:26:55.331898  1123 raft_consensus.cc:697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:55.249158   917 consensus_queue.cc:1048] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.053s
I20260709 12:26:55.332962   985 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:55.332882  1123 consensus_queue.cc:237] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.334491  1099 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:55.336531   868 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:55.333416   985 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:55.337074   868 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:55.337911  1066 consensus_meta.cc:287] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 12:26:55.337597   985 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 40.24, Last appended by leader: 24, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.338317  1066 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.338845   985 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:55.339131   868 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:55.334996  1099 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.340380  1066 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:55.341101 20879 raft_consensus.cc:2368] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:55.341629  1099 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:55.342772   456 consensus_queue.cc:1175] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "116023b4c0c94bfea737fc238bda3838" responder_term: 41 status { last_received { term: 40 index: 24 } last_committed_idx: 24 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 40. Current term is 41. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:55.348018   848 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.352003  1013 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.355299  1133 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:55.309008 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.351354   848 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:55.359966 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51049.2us
I20260709 12:26:55.365801 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.367275 20733 raft_consensus.cc:1240] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 43. Current term is 44. Ops: []
I20260709 12:26:55.369145 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:55.369903 20733 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 43: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:55.374517 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:55.399770 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.375598  1066 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:55.346483   572 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.406088   723 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:55.411331  1013 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:55.428193 20849 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:55.428345 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:55.430217 20732 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:55.416651   721 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:55.401758   985 consensus_queue.cc:1059] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.018s
I20260709 12:26:55.438665 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:55.439646 20775 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:55.440019 20775 leader_election.cc:403] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Cancelling election due to peer responding with higher term
I20260709 12:26:55.440886  1078 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:55.437562  1139 raft_consensus.cc:3055] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 LEADER]: Stepping down as leader of term 43
I20260709 12:26:55.445982  1139 raft_consensus.cc:740] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:55.447453 20803 raft_consensus.cc:3060] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:55.446990   979 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.449385   979 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:55.433732   572 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:55.446961  1139 consensus_queue.cc:260] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 43.25, Last appended by leader: 25, Current term: 43, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.453094  1139 raft_consensus.cc:3060] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Advancing to term 44
I20260709 12:26:55.456595   778 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
W20260709 12:26:55.456777 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 298826us
I20260709 12:26:55.457362   778 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.457803   778 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.458187 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:55.459820   778 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:55.460819   778 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:55.461151   778 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.461472   778 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:55.461942   913 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:55.465163   810 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.465715   810 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.473906 21429 raft_consensus.cc:1275] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 35 index: 25. (index mismatch)
I20260709 12:26:55.474151 21432 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 24. Preceding OpId from leader: term: 39 index: 25. (index mismatch)
I20260709 12:26:55.471676 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:55.477728 21427 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:55.478865 20803 raft_consensus.cc:2468] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 43.
I20260709 12:26:55.480716 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:55.481297 21430 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:55.484472 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56362us
I20260709 12:26:55.433030  1039 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:55.502483 21430 raft_consensus.cc:2468] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:26:55.492991   979 raft_consensus.cc:2804] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader election won for term 44
I20260709 12:26:55.506511   778 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.484007  1060 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:55.529014 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 71829us
I20260709 12:26:55.530983 21399 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.532742  1060 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.538441 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:55.538906  1060 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:55.539999  1134 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.004s
I20260709 12:26:55.542997 20803 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:55.532045   979 raft_consensus.cc:697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 44 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:55.546684 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:55.551072 21427 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:55.585724 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:55.586714 20803 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:55.542112   910 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:55.540006  1060 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:55.547619  1078 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election lost for term 37. Reason: Vote denied by peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:55.547355   572 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:55.560894   810 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:55.603505  1060 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:55.607411   572 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.612774  1060 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 40.24, Last appended by leader: 24, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.616006   979 consensus_queue.cc:237] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 42.24, Last appended by leader: 24, Current term: 44, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.621496 20806 raft_consensus.cc:1240] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:55.628306 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:55.629729   778 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:55.630321  1120 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.673s	user 0.003s	sys 0.000s
I20260709 12:26:55.638356 21430 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 38 index: 25. Preceding OpId from leader: term: 40 index: 26. (index mismatch)
I20260709 12:26:55.638490 20849 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.639487   572 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:55.640666  1124 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:55.641081  1124 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:55.642330  1137 raft_consensus.cc:2804] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader election won for term 43
W20260709 12:26:55.642346  1126 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:26:55.642891  1137 raft_consensus.cc:697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 43 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:55.643857  1133 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:55.643817  1137 consensus_queue.cc:237] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 41.23, Last appended by leader: 23, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:55.645744   922 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:55.646176   922 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:55.646258   986 raft_consensus.cc:3055] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:55.646696   986 raft_consensus.cc:740] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:55.647181   922 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.647624   986 consensus_queue.cc:260] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 34.25, Last appended by leader: 25, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.648414   922 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:55.652635 20884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54488 (request call id 4467) took 1195 ms. Trace:
I20260709 12:26:55.663008 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:55.663604 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.669924   800 consensus_queue.cc:1048] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:55.670747 20850 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.672945 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.673432 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:55.696869 20730 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:55.674504 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:55.677668 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.678412 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.698477 20735 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:26:55.680297 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.681433 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:55.684592 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:55.702322 20882 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 31.
I20260709 12:26:55.679036 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.690024 20884 rpcz_store.cc:276] 0709 12:26:54.457632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.540128 (+ 82496us) service_pool.cc:224] Handling call
0709 12:26:55.051373 (+511245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.052235 (+   862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.652414 (+600179us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b540022e2d0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9e8ad 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:55.652547 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174805376}
I20260709 12:26:55.706068  1105 raft_consensus.cc:2804] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:55.720152 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:55.693589 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.695075 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:55.740852 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.707432 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.698813 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.674775 20803 raft_consensus.cc:2468] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 42.
I20260709 12:26:55.715742 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:55.741067 20876 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:26:55.743268 20849 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.713444 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:55.744403 20882 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:55.742765 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:55.641981  1124 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 40.24, Last appended by leader: 24, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.539968  1039 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 36.26, Last appended by leader: 26, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.772204 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:55.774139 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.774871 20807 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:26:55.786733 20882 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:55.791109 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:55.791862 20802 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:55.793303 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.789677 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:55.795919 20806 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:55.671450  1064 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.079s
W20260709 12:26:55.799156 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 260765us
I20260709 12:26:55.746950   873 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.777370   997 raft_consensus.cc:2804] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election won for term 42
I20260709 12:26:55.810870 20699 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:55.818887 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.816169 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.778038  1105 raft_consensus.cc:697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:55.822779   873 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.829216 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 41 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:55.835848 20803 raft_consensus.cc:2393] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 43: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:55.844118 20803 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 40 index: 26. Preceding OpId from leader: term: 42 index: 27. (index mismatch)
I20260709 12:26:55.845619 20805 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:55.853073 20802 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:55.856437 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:55.869102 20807 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
W20260709 12:26:55.868062 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 52060.9us
I20260709 12:26:55.863837 20729 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:55.876828 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.856040   778 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:55.857390  1097 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader election lost for term 43. Reason: could not achieve majority
W20260709 12:26:55.882010 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 72492.8us
I20260709 12:26:55.865216   920 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election won for term 38
W20260709 12:26:55.882898 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.866991   910 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:55.887446   910 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.887921   910 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:55.884541 21399 leader_election.cc:400] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:55.889057 21399 leader_election.cc:403] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:26:55.864701   997 raft_consensus.cc:697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:55.870301   850 raft_consensus.cc:2764] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:26:55.829727   985 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 40.24, Next index: 25, Last known committed idx: 23, Time since last communication: 0.052s
W20260709 12:26:55.902594 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 130504us
I20260709 12:26:55.903426 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:55.906479 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:55.907204 20729 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
I20260709 12:26:55.883780  1066 consensus_queue.cc:1059] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.027s
W20260709 12:26:55.907436  1147 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:26:55.897197   778 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:55.928161 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:55.944696 21432 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:55.898756   920 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:55.931254 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:55.932698 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:55.934829 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:55.954449 21429 raft_consensus.cc:2368] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:55.936851 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:55.955441 21426 raft_consensus.cc:2393] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 32: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
W20260709 12:26:55.965186 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 154424us
I20260709 12:26:55.920152   810 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:55.917256   917 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:55.917738  1149 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) with higher term. Message: Invalid argument: T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 31. Current term is 32.
I20260709 12:26:55.961779   778 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:55.965586  1066 consensus_queue.cc:1243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer 49323f66286f4798b268f216bb62b3be log is divergent from this leader: its last log entry 40.26 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 25
W20260709 12:26:55.989491  1146 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
W20260709 12:26:56.001569  1153 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:26:56.062529 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:55.943367  1105 consensus_queue.cc:237] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:56.043354 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 216223us
I20260709 12:26:56.068825 32622 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:56.269300 32622 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:56.269719 32622 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:56.269675 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.068323 20776 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:55.972731  1062 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.088s
I20260709 12:26:55.978204   917 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.036125  1156 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:56.040836   572 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.059506  1138 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:56.040500   848 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:56.006844  1013 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.074s
I20260709 12:26:55.970139   997 consensus_queue.cc:237] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 40.24, Last appended by leader: 24, Current term: 42, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:56.279552   917 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.280084  1117 raft_consensus.cc:3055] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:26:56.064632   721 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:56.280583  1117 raft_consensus.cc:740] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:56.280866   721 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.281599  1117 consensus_queue.cc:260] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 39.26, Last appended by leader: 26, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.090656   778 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:56.093883 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 26 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:56.283885   848 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:56.080089   920 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 36.24, Last appended by leader: 24, Current term: 38, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.286018   848 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:56.282910   721 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.287979 32622 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.108422   910 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:56.110855  1150 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:26:56.055745  1066 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:56.166687 21429 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 40 index: 26. Preceding OpId from leader: term: 42 index: 27. (index mismatch)
I20260709 12:26:56.166554 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:26:56.139966  1153 log.cc:927] Time spent T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:26:56.170998 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.194998 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:56.171588 20732 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:56.274333   979 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.274610  1057 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:56.194422 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:56.204255 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:56.194023 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:56.228865 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:56.297781 20884 raft_consensus.cc:2376] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 41. Re-sending same reply.
W20260709 12:26:56.234015   922 consensus_meta.cc:287] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 12:26:56.302454   910 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:56.238929  1144 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.700s	user 0.003s	sys 0.009s
W20260709 12:26:56.239735 20847 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.242596  1124 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:56.305066   778 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:56.305565   778 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:56.306027   778 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:56.250358  1093 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:26:56.255877 20775 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.256623   998 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20260709 12:26:56.308490 20879 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:56.262210 20729 raft_consensus.cc:1240] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 42. Current term is 43. Ops: [42.25-42.25]
I20260709 12:26:56.263731  1087 raft_consensus.cc:3055] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 42 LEADER]: Stepping down as leader of term 42
I20260709 12:26:56.268399  1175 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.271790 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:56.273850 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 205600us
I20260709 12:26:56.275784  1081 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:56.295491 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 128955us
I20260709 12:26:56.294783  1117 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:56.296844 20876 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:56.331027 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.332131 21429 raft_consensus.cc:1240] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 39. Current term is 40. Ops: []
I20260709 12:26:56.332540 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:56.333160 20805 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:56.300521 20880 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 40 index: 24. Preceding OpId from leader: term: 41 index: 25. (index mismatch)
I20260709 12:26:56.300670  1066 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.338124  1066 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.310307 21399 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.339223   810 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:56.339478 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:56.340330  1062 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:56.340269 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:56.340715 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.340956  1062 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:56.341167 20729 raft_consensus.cc:2393] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 38: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:56.342087 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:56.276175   572 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:56.342815 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:56.343587 20726 raft_consensus.cc:3060] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:56.317019 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 45302.7us
I20260709 12:26:56.344807 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.326937 20804 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 41 index: 23. Preceding OpId from leader: term: 43 index: 24. (index mismatch)
I20260709 12:26:56.296181 32622 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.315987 20776 leader_election.cc:400] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 37. Current term is 38.
I20260709 12:26:56.328367  1013 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:56.347502 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:56.348281 20730 raft_consensus.cc:2393] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 35: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:56.323107 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:56.342861  1158 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:56.341840  1062 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 40.24, Last appended by leader: 24, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.274729  1156 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.354524 21434 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:56.354673 20809 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:56.352542 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:56.355530 20876 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:26:56.357331 20726 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
W20260709 12:26:56.362361 21433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44462 (request call id 2339) took 1839 ms (client timeout 2353 ms). Trace:
I20260709 12:26:56.313633  1087 raft_consensus.cc:740] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 42 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:56.366366 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:56.362802 21433 rpcz_store.cc:269] 0709 12:26:54.522743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.522886 (+   143us) service_pool.cc:224] Handling call
0709 12:26:56.362151 (+1839265us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400592254. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1eca32fa 00007f9e23c64b17 00007f9e1d1df6d4 00007f9e1d1df662 00007f9e1d1df591 00007f9e1d1df50c 00007f9e1d1de7c2 00007f9e1cfda48c 00007f9e1cfd98c6 00007f9e1cfd926f
0709 12:26:56.362284 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3126372608}
I20260709 12:26:56.345580 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.375785 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:56.391921 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46411.4us
I20260709 12:26:56.399036 21429 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:56.400465 20809 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 40 index: 24. Preceding OpId from leader: term: 41 index: 25. (index mismatch)
I20260709 12:26:56.311558   979 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.400501 20776 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:56.387132  1133 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:56.416431 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:56.393976   910 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:56.419735 20877 raft_consensus.cc:1240] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:56.398543  1027 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader election lost for term 43. Reason: could not achieve majority
I20260709 12:26:56.328974  1175 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:56.432207 20876 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:56.436087 20726 raft_consensus.cc:1275] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:56.390708  1125 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.033s
I20260709 12:26:56.441124 20805 raft_consensus.cc:2468] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:56.425835  1138 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:56.433244  1175 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:56.406916  1160 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.030s
I20260709 12:26:56.413877  1167 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.026s
I20260709 12:26:56.448235 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:56.447279 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 118693us
I20260709 12:26:56.468065 20804 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:26:56.434826   910 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:56.422259  1159 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.036s
W20260709 12:26:56.460629 20774 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.421842  1181 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.023s
I20260709 12:26:56.483376  1156 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:26:56.484583  1174 log.cc:927] Time spent T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:26:56.451278 21400 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:56.459848   778 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.425279  1133 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.489292 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:56.489699 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:56.470177 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:56.478811   873 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:56.490273 20774 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:56.433631  1087 consensus_queue.cc:260] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 42.25, Last appended by leader: 25, Current term: 42, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:56.499792 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.500020 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:56.503144  1182 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.001s
I20260709 12:26:56.505072   778 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:56.469807  1138 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:56.507846  1138 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 34.25, Last appended by leader: 25, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.508489  1168 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:56.508901  1168 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.509238  1168 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:56.514693   572 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:56.519747 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:56.524701 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:56.510915   979 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.528214 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:56.496816  1156 raft_consensus.cc:2804] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:56.517112  1078 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:56.553064  1156 raft_consensus.cc:697] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:56.560029  1078 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:56.560492  1156 consensus_queue.cc:237] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 36.23, Last appended by leader: 23, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.560976   986 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:56.560911  1078 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.553555  1182 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:56.556474  1133 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.556859 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.563731 21430 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:26:56.537761  1066 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:56.564247   979 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.537132  1087 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:56.564359 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.564851   979 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.568357 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:56.568939 20805 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:56.522671  1020 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:56.563339   910 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.558109   873 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:56.572175 20884 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 39 index: 26. Preceding OpId from leader: term: 40 index: 26. (term mismatch)
I20260709 12:26:56.658833 20884 pending_rounds.cc:85] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Aborting all ops after (but not including) 25
I20260709 12:26:56.659307 20884 pending_rounds.cc:107] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: Aborting uncommitted NO_OP operation due to leader change: 39.26
I20260709 12:26:56.659626 20884 raft_consensus.cc:2889] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:56.572444 20880 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:56.573352 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:56.577040 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.576993 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:56.566996   979 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.667407 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:56.682709 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 105832us
I20260709 12:26:56.683015 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:56.683585 20850 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.658927  1060 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.695050 20879 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:56.658458   810 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:56.695050 21428 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:56.729315 20881 raft_consensus.cc:1275] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:26:56.658505  1020 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:26:56.704936  1187 log.cc:927] Time spent T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:26:56.700677  1182 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:56.700977   979 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:56.741715  1159 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.741626  1158 consensus_queue.cc:1048] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:26:56.742326  1159 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.745034  1159 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.751718 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:56.763816 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:56.765254  1020 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:56.777661  1184 log.cc:927] Time spent T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:26:56.779786  1185 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:26:56.724623   810 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.782866   810 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:56.701788  1060 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.785076  1060 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:56.739630  1172 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.785830  1172 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.787945  1172 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.740660  1182 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.732020  1013 consensus_queue.cc:1059] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 22, Time since last communication: 0.028s
I20260709 12:26:56.812299 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:56.810158  1182 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:56.818276  1013 consensus_queue.cc:1243] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 42.25 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 22
I20260709 12:26:56.824334 20699 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.841075  1026 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:56.865062  1160 consensus_queue.cc:1059] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:56.865813  1160 consensus_queue.cc:1243] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer 116023b4c0c94bfea737fc238bda3838 log is divergent from this leader: its last log entry 32.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:26:56.866312  1160 raft_consensus.cc:3055] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:56.866655  1160 raft_consensus.cc:740] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:56.871565  1168 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:56.873838  1168 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:56.890443   870 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.891013   870 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.897603 21427 raft_consensus.cc:1240] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:56.899475   870 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:56.892849  1027 raft_consensus.cc:3055] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 41 LEADER]: Stepping down as leader of term 41
I20260709 12:26:56.900657  1027 raft_consensus.cc:740] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 41 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:56.901571  1027 consensus_queue.cc:260] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 41.25, Last appended by leader: 25, Current term: 41, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:56.902910  1027 raft_consensus.cc:3060] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:56.909196 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:56.932806 20879 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:26:56.911114 21427 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 38 index: 25. Preceding OpId from leader: term: 40 index: 26. (index mismatch)
I20260709 12:26:56.923115 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.936188 21432 raft_consensus.cc:1240] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:56.942762 20625 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 206) took 47671 ms (client timeout 15000 ms). Trace:
I20260709 12:26:56.945276 20735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 26 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:56.949401  1158 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:56.950260   872 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.950749   872 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.958603 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.956631 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:56.963042 21432 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:26:56.964775 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:56.965349 20802 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 39
I20260709 12:26:56.968940 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:56.971179 20805 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 40 index: 24. Preceding OpId from leader: term: 42 index: 25. (index mismatch)
I20260709 12:26:56.959017 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:56.888622  1182 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.962894 20699 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:26:56.984539 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:56.985219 20802 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:26:56.987999 20882 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
I20260709 12:26:56.988713 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:56.989491 20734 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:56.986653  1167 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:56.993600  1167 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.994544  1057 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:56.952112 20627 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 30, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 30 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:56.994957  1057 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.995362  1057 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:56.996284 20627 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 31 to 34. New cstate: current_term: 34 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:56.997308  1020 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:26:56.997689 20627 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 31 to 33. New cstate: current_term: 33 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:56.998672  1020 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:57.000094 20701 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:56.952847   872 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.000458  1134 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.009274  1060 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.009824  1060 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.011929  1060 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:56.986878  1059 consensus_queue.cc:1059] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.026s
W20260709 12:26:56.976729  1197 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:26:57.008309  1134 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.013794 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:57.014104 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:57.014853 20807 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 39 because replica has last-logged OpId of term: 37 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 35 index: 23.
I20260709 12:26:57.015666  1167 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:56.999590   728 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.013s
I20260709 12:26:57.016144  1162 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.016436  1134 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.016691  1162 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.017903  1059 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:57.018572  1159 raft_consensus.cc:3055] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:57.018967  1159 raft_consensus.cc:740] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:57.019876 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:57.022189   810 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:57.022580   810 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.022904   810 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:57.023586  1159 consensus_queue.cc:260] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:56.953040   572 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.024801  1159 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:57.025141   572 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:57.026351 20801 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 43 index: 25. Preceding OpId from leader: term: 44 index: 25. (term mismatch)
I20260709 12:26:57.026674 20801 pending_rounds.cc:85] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 24
I20260709 12:26:57.026878 20801 pending_rounds.cc:107] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 43.25
I20260709 12:26:57.027098 20801 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:57.029448 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70497.8us
I20260709 12:26:57.031662 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:57.031864 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:57.032462 20726 raft_consensus.cc:2410] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 39 because replica has last-logged OpId of term: 37 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 35 index: 23.
I20260709 12:26:57.032526   848 raft_consensus.cc:2764] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election decision vote started in defunct term 37: won
I20260709 12:26:57.035409 20726 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 35 index: 26. (index mismatch)
I20260709 12:26:57.036654 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 45 candidate_status { last_received { term: 43 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:57.038758 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:57.039412 20879 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41.
W20260709 12:26:56.972820  1192 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:26:57.046062 20884 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 40 index: 24. Preceding OpId from leader: term: 41 index: 25. (index mismatch)
I20260709 12:26:57.047169 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:57.047577 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:57.007259  1160 consensus_queue.cc:260] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.052639 20877 raft_consensus.cc:1240] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 41. Current term is 42. Ops: [41.25-41.25]
I20260709 12:26:57.059026 20726 raft_consensus.cc:1275] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 41 index: 23. Preceding OpId from leader: term: 43 index: 24. (index mismatch)
I20260709 12:26:57.058068 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:57.059757 20803 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 26. Preceding OpId from leader: term: 38 index: 27. (index mismatch)
I20260709 12:26:57.060904 20801 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:26:57.053509 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:57.067546 20881 raft_consensus.cc:2393] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 33: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:57.070334 20726 raft_consensus.cc:1275] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 42 index: 24. Preceding OpId from leader: term: 44 index: 25. (index mismatch)
I20260709 12:26:57.082852 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:57.082986 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:57.043265   572 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:57.107179 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:56.943419 20625 rpcz_store.cc:269] 0709 12:26:09.245082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.402140 (+13157058us) service_pool.cc:224] Handling call
0709 12:26:56.916010 (+34513870us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:50.504957 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:50.507296 (+  2339us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:50.507341 (+    45us) write_op.cc:435] Acquired schema lock
0709 12:26:50.507362 (+    21us) tablet.cc:662] Decoding operations
0709 12:26:50.542182 (+ 34820us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:50.542294 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:50.542319 (+    25us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:26:50.615177 (+ 72858us) tablet.cc:701] Row locks acquired
0709 12:26:50.615250 (+    73us) write_op.cc:260] PREPARE: finished
0709 12:26:50.615406 (+   156us) write_op.cc:270] Start()
0709 12:26:50.697665 (+ 82259us) write_op.cc:276] Timestamp: P: 1783600010615342 usec, L: 0
0709 12:26:50.701767 (+  4102us) op_driver.cc:348] REPLICATION: starting
0709 12:26:50.910181 (+208414us) log.cc:844] Serialized 8761 byte log entry
0709 12:26:51.699563 (+789382us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
0709 12:26:51.817627 (+118064us) op_driver.cc:464] REPLICATION: finished
0709 12:26:51.994014 (+176387us) write_op.cc:301] APPLY: starting
0709 12:26:51.994128 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:52.757408 (+763280us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:52.767971 (+ 10563us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:53.464355 (+696384us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:53.481382 (+ 17027us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 12:26:53.506579 (+ 25197us) write_op.cc:312] APPLY: finished
0709 12:26:53.548001 (+ 41422us) log.cc:844] Serialized 196 byte log entry
0709 12:26:53.641481 (+ 93480us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:53.759282 (+117801us) write_op.cc:454] Released schema lock
0709 12:26:53.792754 (+ 33472us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:53.827076 (+ 34322us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:53.830813 (+  3737us) write_op.cc:454] Released schema lock
0709 12:26:54.068680 (+237867us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bad8c85 00007f9e1bad96a2 00007f9e1e84b1db 00007f9e1e846aaf 00007f9e1e83a86e 00007f9e1e839bb6 00007f9e1e8397d8 00007f9e1e843221 00007f9e1e8431d9 00007f9e1e843169
Metrics: {"mutex_wait_us":129117,"reported_tablets":38,"spinlock_wait_cycles":992512,"child_traces":[["op",{"apply.queue_time_us":176114,"apply.run_cpu_time_us":46538,"apply.run_wall_time_us":1824615,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96087,"num_ops":22,"prepare.queue_time_us":110221,"prepare.run_cpu_time_us":35189,"prepare.run_wall_time_us":639535,"raft.queue_time_us":82832,"raft.run_cpu_time_us":79232,"raft.run_wall_time_us":113052,"replication_time_us":1115699,"spinlock_wait_cycles":45110272,"thread_start_us":472079,"threads_started":5,"wal-append.queue_time_us":604951,"wal-append.run_cpu_time_us":18660,"wal-append.run_wall_time_us":2822539}]]}
W20260709 12:26:57.082418   920 consensus_queue.cc:1175] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "50c1ab13351d4427b836fbb449c6d7bb" responder_term: 32 status { last_received { term: 29 index: 19 } last_committed_idx: 19 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 31. Current term is 32. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:57.089558 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 102008us
I20260709 12:26:57.091341 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:57.092159 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:57.093058   810 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.093567 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:57.094601 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:57.112690 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:26:57.114188 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 122839us
I20260709 12:26:57.095381  1062 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:57.095820   893 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:57.098762 20849 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:57.099144 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:57.103140  1162 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.121222 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:57.130059 21429 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:57.111047 20776 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:57.118160 20730 raft_consensus.cc:3060] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:57.120296 20875 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:57.132053 21435 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 26. Preceding OpId from leader: term: 38 index: 27. (index mismatch)
I20260709 12:26:57.125613 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:57.130904 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:57.133379 20810 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:57.133553 20849 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:57.139678 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:57.141700 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:57.141912 20803 raft_consensus.cc:1275] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
W20260709 12:26:57.143411 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51347.9us
I20260709 12:26:57.148888 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:57.171442 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:57.106541  1172 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election won for term 42
W20260709 12:26:57.200695 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 58375.3us
W20260709 12:26:57.206868 20883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.019s	sys 0.021s
W20260709 12:26:57.207526 20883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.020s	sys 0.021s
W20260709 12:26:57.210424 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 44881.5us
I20260709 12:26:57.082293  1064 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.082s
I20260709 12:26:57.006126  1057 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.073166   979 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.069s
I20260709 12:26:57.119659   910 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:57.163697  1168 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:57.172340  1156 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:57.171664   848 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:57.084599  1149 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.128s
I20260709 12:26:57.163684   986 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:57.212301  1172 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.109288  1167 consensus_queue.cc:1059] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 40.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.109s
I20260709 12:26:57.238718  1062 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:26:57.213016  1192 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:26:57.163288   922 consensus_queue.cc:1048] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.074s
I20260709 12:26:56.999081 20627 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 29 to 32. New cstate: current_term: 32 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:57.252713   848 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.217000   999 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:57.237294  1173 consensus_queue.cc:1048] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.045s
W20260709 12:26:57.246881 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 75522.3us
I20260709 12:26:57.224773   723 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.041s
W20260709 12:26:57.306167  1199 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:26:57.328127  1182 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.329802 20627 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 23 to 29. New cstate: current_term: 29 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:26:57.356403 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 52618.2us
I20260709 12:26:57.357470 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:26:57.277774 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50410.7us
I20260709 12:26:57.173036  1124 consensus_queue.cc:1048] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.073s
I20260709 12:26:57.279250  1172 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:57.216799   800 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:57.142931   572 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:57.371627 21427 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
I20260709 12:26:57.119315  1158 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:57.219152  1134 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:57.301020   848 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:57.350849  1156 raft_consensus.cc:3055] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Stepping down as leader of term 41
I20260709 12:26:57.405480 20808 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
W20260709 12:26:57.386571 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 253095us
I20260709 12:26:57.410295 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:57.378754   456 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.289273  1101 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.110s
I20260709 12:26:57.416797 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:57.396229   572 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.435497 20803 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:57.454228 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:57.378512   999 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.413821  1156 raft_consensus.cc:740] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:26:57.474401 20698 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.432096 20849 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:57.417555   893 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election won for term 39
W20260709 12:26:57.476773 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 44759.7us
I20260709 12:26:57.434540   910 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:57.438725   572 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Advancing to term 42
W20260709 12:26:57.494499  1204 log.cc:927] Time spent T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:26:57.426416  1149 consensus_queue.cc:1048] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.058s
I20260709 12:26:57.424297   456 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.482241   893 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.466274  1181 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 12:26:57.469689   922 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.040s
W20260709 12:26:57.504983  1190 log.cc:927] Time spent T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:26:57.504961  1208 raft_consensus.cc:2804] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:57.519640   893 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:57.526057  1160 raft_consensus.cc:3060] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:57.530282  1206 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.036s
W20260709 12:26:57.543134  1198 log.cc:927] Time spent T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 12:26:57.530159  1203 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:26:57.568053  1202 log.cc:927] Time spent T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:57.549142  1057 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.528455  1156 consensus_queue.cc:260] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 41.25, Last appended by leader: 25, Current term: 41, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.530728  1137 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 25, Time since last communication: 0.049s
W20260709 12:26:57.581820  1219 log.cc:927] Time spent T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:26:57.599480 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 86787.2us
I20260709 12:26:57.608212 20730 raft_consensus.cc:2468] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
W20260709 12:26:57.622056 21398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.086s	sys 0.164s
W20260709 12:26:57.622730 21398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.086s	sys 0.164s
W20260709 12:26:57.623541 21398 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.624136 20849 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:57.633797 20883 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54476 (request call id 4540) took 1059 ms. Trace:
I20260709 12:26:57.613999  1208 raft_consensus.cc:697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:57.634471 20883 rpcz_store.cc:276] 0709 12:26:56.574814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.574956 (+   142us) service_pool.cc:224] Handling call
0709 12:26:56.575287 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.575500 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.575523 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.631554 (+ 56031us) log.cc:844] Serialized 26 byte log entry
0709 12:26:57.208543 (+576989us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:57.208627 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.208800 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.633522 (+424722us) raft_consensus.cc:1685] finished
0709 12:26:57.633550 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.633726 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":513526,"threads_started":1,"wal-append.queue_time_us":714134}
I20260709 12:26:57.649843 20849 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:57.489069 20627 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:57.643049  1027 consensus_meta.cc:287] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.001s
I20260709 12:26:57.631453  1167 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:57.703682 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 228074us
I20260709 12:26:57.713968 20729 raft_consensus.cc:1275] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 41 index: 25. Preceding OpId from leader: term: 42 index: 25. (term mismatch)
I20260709 12:26:57.714563 20729 pending_rounds.cc:85] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 24
I20260709 12:26:57.714861 20729 pending_rounds.cc:107] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 41.25
I20260709 12:26:57.715200 20729 raft_consensus.cc:2889] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.676630  1181 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:57.668236  1162 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.669505   998 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election lost for term 42. Reason: could not achieve majority
I20260709 12:26:57.711624  1136 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.757417   456 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.745484   999 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.719887  1167 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:57.769140  1196 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.243s	user 0.001s	sys 0.001s
I20260709 12:26:57.739811  1162 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.762130  1136 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:57.764165  1208 consensus_queue.cc:237] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 37.20, Last appended by leader: 20, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:57.828903  1217 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:57.841493  1136 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:57.880502 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:57.916239   810 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:57.694247 20627 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 28 to 31, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 31 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:57.882745 20699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.002s	sys 0.010s
W20260709 12:26:57.920228 20699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.002s	sys 0.010s
I20260709 12:26:57.920137   917 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:57.920945 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 787343us
I20260709 12:26:57.921350   917 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:57.925524   917 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:57.848160  1167 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:57.839861  1217 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.928622 20627 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 28, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 28 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:57.930328  1217 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.842283  1172 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:57.933180 20875 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 41. Current term is 42. Ops: [41.25-41.25]
I20260709 12:26:57.890391 20732 raft_consensus.cc:1275] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 20. (term mismatch)
I20260709 12:26:57.934278  1156 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:57.904573 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:57.935593 20877 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 42.
I20260709 12:26:57.908319 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:57.937252 20883 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:26:57.846303  1162 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.933476 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:57.938630 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:57.939059 21434 raft_consensus.cc:2393] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 35: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:57.934273 20732 pending_rounds.cc:85] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Aborting all ops after (but not including) 19
I20260709 12:26:57.940081 20732 pending_rounds.cc:107] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: Aborting uncommitted NO_OP operation due to leader change: 31.20
I20260709 12:26:57.940294  1172 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.940461 20732 raft_consensus.cc:2889] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.942288   800 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:57.942924   800 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:57.947834 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:57.950377 20627 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 21 to 29. New cstate: current_term: 29 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:57.950730   910 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:57.979586 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
W20260709 12:26:57.998296 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 592797us
I20260709 12:26:58.015880  1160 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.005733   721 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:58.016791  1160 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.018049   721 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.020504  1211 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.021427  1211 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.024430   721 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:57.991037   800 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.026252 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.027897   800 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.027993  1214 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.028934  1214 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.029228 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:58.032447  1230 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.033257  1230 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.038170  1020 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:58.038997  1020 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.056499  1020 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.060621  1211 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.061328  1039 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.062078  1039 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.062407  1160 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:58.052434  1221 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.495s	user 0.006s	sys 0.000s
I20260709 12:26:58.053081   870 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:58.064463   870 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.065822  1214 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.050305  1059 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.066648  1059 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.053797  1013 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.070669  1013 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.074128  1013 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.062412  1230 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.068600  1093 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:58.084448  1093 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.085403  1059 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.067306   870 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.052287   920 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.086663   920 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.088685  1093 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.029148 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:58.089021   920 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:58.092175 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 45015.7us
W20260709 12:26:58.098330  1234 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
I20260709 12:26:58.052784 20627 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 31. New cstate: current_term: 31 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:58.107789 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 78900.1us
I20260709 12:26:58.103066  1093 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.115854  1158 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.116679  1158 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.111243   920 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.127506  1237 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.128166  1039 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.128300  1237 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.129498  1236 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.130223  1236 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.127490  1158 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:58.131125  1237 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.132824  1236 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:58.134701   893 consensus_meta.cc:287] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.012s
W20260709 12:26:58.134857   848 consensus_meta.cc:287] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.715s	user 0.004s	sys 0.017s
I20260709 12:26:58.135396   848 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.137502 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 43 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.138722 20729 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 43: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
I20260709 12:26:58.122663 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:58.147797 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 740500us
I20260709 12:26:58.151374  1093 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.154172  1093 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.160202   920 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.162861   920 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.163651 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:58.175365 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 66071.5us
I20260709 12:26:58.166582 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.167996 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.139642   893 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.181186 21434 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 38. Current term is 39. Ops: [38.27-38.27]
I20260709 12:26:58.182403   893 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.183691 21428 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:58.184875 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.186355 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.188110 21429 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:58.188942 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.190456 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.191519 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.193035 20627 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 25 to 28, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 28 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:58.194131 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.195847 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.195369 20627 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 33 to 37. New cstate: current_term: 37 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.196193 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.197043 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.197494 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.197319 20627 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 30 to 37, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 37 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:58.198105 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.198694 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.199541 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.171638 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.178315 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.178589 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.142230   848 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.140748  1236 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.206599 20807 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:26:58.203766 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:26:58.214742  1231 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.096s	user 0.002s	sys 0.000s
I20260709 12:26:58.151407  1162 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader pre-election won for term 44
I20260709 12:26:58.225936  1162 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.226352  1162 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Advancing to term 44
I20260709 12:26:58.190479 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.204684  1237 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.228693  1237 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.231017  1237 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.218840 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:58.187052 21431 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:58.220916 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:58.222044  1236 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.198868 20627 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 35 to 37, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 37 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:58.253592  1181 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.254155 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 43 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.254546  1181 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.257829  1181 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.261912  1027 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:58.264725  1236 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:58.265938  1239 log.cc:927] Time spent T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:58.266186 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.267503 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.269608 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:58.270298 21434 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:58.275535 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.281418 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.283943 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.292021 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.285197 20627 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 30 to 32, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:58.301818 21434 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:58.308470 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:58.311326  1039 consensus_queue.cc:1048] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:58.313154  1039 consensus_queue.cc:1048] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:58.319585 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.320421 20805 raft_consensus.cc:2368] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36.
W20260709 12:26:58.322880   572 consensus_meta.cc:287] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: Time spent flushing consensus metadata: real 0.813s	user 0.014s	sys 0.007s
I20260709 12:26:58.325353 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.347349 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.349659 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.327343  1027 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.300931  1237 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.351583 20627 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 33, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 33 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.367381 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.369866 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:58.358870   920 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.368711 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.344911  1236 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.373222   922 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 36.26, Next index: 27, Last known committed idx: 26, Time since last communication: 0.032s
I20260709 12:26:58.379945 20729 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: [36.23-36.23]
I20260709 12:26:58.381191 20729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 45 candidate_status { last_received { term: 43 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.383160  1027 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.343055   572 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.383729 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.384886 20881 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 36
I20260709 12:26:58.385681   572 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.386318   872 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:58.388123   872 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.388602   872 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:58.388698 20802 raft_consensus.cc:1240] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 41. Current term is 42. Ops: []
I20260709 12:26:58.390180 20802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43456 (request call id 2497) took 1257 ms. Trace:
I20260709 12:26:58.367714 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.393914 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.371325  1238 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.390540 20802 rpcz_store.cc:276] 0709 12:26:57.132600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.132743 (+   143us) service_pool.cc:224] Handling call
0709 12:26:57.132994 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:58.389184 (+1256190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.389889 (+   705us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002b5cd4. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec08571 00007f9e1ec9fe01 00007f9e1ec9e83e 00007f9e23c63419 00007f9e1d1dabd4 00007f9e1d1dab62 00007f9e1d1daa91 00007f9e1d1daa0c 00007f9e1d1d9cc2 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:26:58.390104 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2886928768}
I20260709 12:26:58.399369 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.399442 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:58.394114  1230 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:58.398123 20627 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 31, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 31 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:58.413582 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.414659 20884 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:58.417865 20732 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:58.418043 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.406251  1059 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:58.393878  1237 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.417595 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.406391   870 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:58.423616 20730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.431496 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.432471   870 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.435290  1181 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:58.435751  1181 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:26:58.436882  1181 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 38.27, Last appended by leader: 27, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.438278  1181 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:58.441416 20734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.443368  1160 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:58.444397 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.445935 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.447912  1211 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:58.448724  1211 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:58.449923  1027 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.449368 20627 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.450127   723 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.450783   723 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.451359 20627 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 29 to 33. New cstate: current_term: 33 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.413297   920 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.452906 20627 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 30 to 31, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 31 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:58.454043   723 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.455238 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:58.458850 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66440.3us
I20260709 12:26:58.463028 20849 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:58.432946 20728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:58.464566 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:58.464985 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.447085 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.468292 20700 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.470292 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.471479  1168 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:58.472213  1168 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
I20260709 12:26:58.474859  1230 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.409657  1236 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.478291 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:58.419781  1238 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.483207  1162 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.484513 20809 raft_consensus.cc:2368] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 43. Current term is 44.
I20260709 12:26:58.484647 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.485962 20728 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 43.
I20260709 12:26:58.493539 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.494508 20877 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 42: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:58.495364  1039 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:58.495811 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:58.496454 20876 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Advancing to term 44
I20260709 12:26:58.495374 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:58.498998 20734 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:26:58.467072 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:58.517545 20884 raft_consensus.cc:2393] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 42: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:58.496016  1039 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.523553 20732 raft_consensus.cc:2468] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
W20260709 12:26:58.523444 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 57291.1us
I20260709 12:26:58.496655  1214 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:58.530186 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.527058 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:58.499219  1162 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.540791 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:26:58.559448 20700 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 91248.5us
I20260709 12:26:58.537201  1236 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.603272 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:58.599089 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66964.4us
I20260709 12:26:58.558465  1059 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:58.619246 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.537612 20627 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 27 to 29, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193), now has a pending config: VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193), VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194), VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195), VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 29 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:26:58.591166  1243 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:26:58.626581 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:58.627255 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.625936 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.591157   848 consensus_queue.cc:1059] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.038s
I20260709 12:26:58.573983 20881 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:26:58.578722  1230 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.580377 20875 raft_consensus.cc:1240] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:58.580823 20728 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 20. Preceding OpId from leader: term: 39 index: 21. (index mismatch)
I20260709 12:26:58.581187  1039 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.588068  1238 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.588189  1237 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.595347 20731 raft_consensus.cc:1240] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 35. Current term is 36. Ops: [34.25-35.26]
I20260709 12:26:58.600376   723 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:58.632933 20877 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 39 because replica has last-logged OpId of term: 38 index: 27, which is greater than that of the candidate, which has last-logged OpId of term: 36 index: 26.
I20260709 12:26:58.633628   728 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.633875 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.639309 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.654904 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 44 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.634418  1211 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:58.652398 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.655181   920 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.629742   848 raft_consensus.cc:3055] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:58.675585   848 raft_consensus.cc:740] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:58.676991   848 consensus_queue.cc:260] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.26, Last appended by leader: 26, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.636557  1236 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:58.679947 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 199755us
I20260709 12:26:58.664541  1156 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:58.665091   800 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:58.667201  1149 consensus_queue.cc:1059] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.006s
I20260709 12:26:58.671965   728 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.663201 20804 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:58.656240  1230 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:58.651624 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 44 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.680796 20849 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.678821   848 raft_consensus.cc:3060] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:58.681306 20774 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:58.659291 20627 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 26 to 32. New cstate: current_term: 32 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.701995 20805 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 44: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:58.703213 20876 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 44.
I20260709 12:26:58.716131 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:58.715780 20627 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:26:58.719240 20774 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:58.727293 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 46597.6us
I20260709 12:26:58.690814   800 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.725358  1230 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:58.755589 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:58.696795   872 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.759343 20849 leader_election.cc:400] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36.
I20260709 12:26:58.760305 20849 leader_election.cc:403] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:58.731335   922 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election lost for term 42. Reason: could not achieve majority
I20260709 12:26:58.734678  1093 raft_consensus.cc:2804] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Leader election won for term 44
I20260709 12:26:58.764720 20627 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 35 to 36, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 36 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:26:58.735975  1149 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:58.751041 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:58.775974 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 43995.7us
I20260709 12:26:58.722564  1156 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.759832  1230 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.788380  1093 raft_consensus.cc:697] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:58.812302  1093 consensus_queue.cc:237] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 42.25, Last appended by leader: 25, Current term: 44, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:58.814605   986 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:58.825137  1156 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:58.829258   728 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.835291  1149 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.838028  1149 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:58.793326   922 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 35. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36.
W20260709 12:26:58.845945 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 243240us
I20260709 12:26:58.846515  1136 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:58.846750 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:58.847141  1136 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.847680  1136 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:58.849028   800 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:58.849345  1162 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
W20260709 12:26:58.851636 20850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.854619 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:58.860502   986 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:58.861790   986 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 34.25, Last appended by leader: 25, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.843178  1158 raft_consensus.cc:3055] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Stepping down as leader of term 41
I20260709 12:26:58.865844  1158 raft_consensus.cc:740] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:58.866122   872 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:26:58.867864 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 91066.5us
I20260709 12:26:58.874265 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:58.875057 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.875028  1017 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:58.793298  1230 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:58.891767 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 272617us
I20260709 12:26:58.893419 20849 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.915040 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:58.954514 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.955495 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 44 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.957448 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.958253 20884 raft_consensus.cc:2393] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 44: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:58.957029 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 44 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:58.913300  1039 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:58.945482 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:58.873315  1158 consensus_queue.cc:260] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 41.25, Last appended by leader: 25, Current term: 41, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:58.920552   572 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:58.917532   870 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:58.921941   872 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:58.998835 20849 leader_election.cc:400] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 43. Current term is 44.
W20260709 12:26:58.998890 20699 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53508.2us
I20260709 12:26:59.000159   728 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:59.009982 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:59.016721 20884 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:59.015390 20849 leader_election.cc:403] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:59.015822   810 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:26:59.031638 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 265396us
I20260709 12:26:59.040822 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.013940   572 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:59.031129  1181 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader pre-election lost for term 43. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 43. Current term is 44.
I20260709 12:26:59.032559  1173 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:59.033999  1057 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:59.064601   572 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:59.064790  1266 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:59.065331 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:26:59.065699  1252 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:26:59.066565   920 consensus_queue.cc:1059] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 40.26, Next index: 27, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:26:59.069094 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.050978  1158 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:59.057469  1236 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:59.075652 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:59.086355 20733 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 37: Already voted for candidate 49323f66286f4798b268f216bb62b3be in this term.
W20260709 12:26:59.087334  1253 log.cc:927] Time spent T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:26:59.089408  1258 log.cc:927] Time spent T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:59.031966   872 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:59.084175 20884 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:26:59.084707 20809 raft_consensus.cc:1275] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 37 index: 20. Preceding OpId from leader: term: 39 index: 21. (index mismatch)
I20260709 12:26:59.085212 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
W20260709 12:26:59.097285 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 257537us
I20260709 12:26:59.102248  1236 raft_consensus.cc:3060] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Advancing to term 44
W20260709 12:26:59.109846 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 215684us
I20260709 12:26:59.110853 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:26:59.139671 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 56811.6us
I20260709 12:26:59.149125 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.108373  1264 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.148504 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.156800 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.159576 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.160668 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.073348   810 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.133751   920 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:59.163669 20699 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:26:59.169287 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 70859.4us
I20260709 12:26:59.191049 20733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:59.189559 20727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:59.192497 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.194446 20732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:59.195808 20701 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.196256   728 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
W20260709 12:26:59.199694  1244 log.cc:927] Time spent T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:59.200275 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.200973 21433 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:26:59.194563 20731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be"
I20260709 12:26:59.211301 20699 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.212867 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.191311  1168 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:59.246621 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 50890.3us
W20260709 12:26:59.245440 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 51232us
I20260709 12:26:59.216131  1059 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
I20260709 12:26:59.256567  1262 raft_consensus.cc:3055] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Stepping down as leader of term 40
I20260709 12:26:59.253273 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:59.172629  1266 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.217707  1201 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:59.260977 20805 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:26:59.256583 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.245958  1160 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:59.173866  1264 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.249711  1020 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:59.272292 21429 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 35.
I20260709 12:26:59.257027  1262 raft_consensus.cc:740] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:26:59.284299 20726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37276 (request call id 4553) took 1701 ms. Trace:
I20260709 12:26:59.230469  1136 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:59.277782  1257 log.cc:927] Time spent T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 12:26:59.339740 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 75682.6us
I20260709 12:26:59.276199  1168 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.352301  1262 consensus_queue.cc:260] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 40.26, Last appended by leader: 26, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:59.387445 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 142174us
W20260709 12:26:59.399900 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 197706us
I20260709 12:26:59.400780   810 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:59.395804  1262 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:59.316740 20726 rpcz_store.cc:276] 0709 12:26:57.411774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.412459 (+   685us) service_pool.cc:224] Handling call
0709 12:26:57.412729 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.234827 (+822098us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:26:58.404456 (+169629us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.733020 (+328564us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.787341 (+ 54321us) log.cc:844] Serialized 26 byte log entry
0709 12:26:58.787554 (+   213us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:58.787877 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.788198 (+   321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.794514 (+  6316us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:58.864257 (+ 69743us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b48000d0458. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e23bc8aa1 00007f9e23bbfb37 00007f9e23bbf9ca 00007f9e23cb726f 00007f9e23cb6f37 00007f9e23ccb55c 00007f9e23ccb508 00007f9e23ccb488 00007f9e23ccb443 00007f9e23cca6ec 00007f9e22bf4a96
0709 12:26:59.089641 (+225384us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b18004cafe0. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e24a7aab1 00007f9e24a87281 00007f9e1bbb2522 00007f9e1bbb47c1 00007f9e1bbb4779 00007f9e1bbb4709 00007f9e1bbb46d1 00007f9e1bbb39cd 0000000000388634 0000000000388479 00007f9e1bb9583a
0709 12:26:59.093373 (+  3732us) raft_consensus.cc:1685] finished
0709 12:26:59.093392 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.113073 (+ 19681us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":636956,"prepare.queue_time_us":229221,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":641050,"raft.run_cpu_time_us":1612,"raft.run_wall_time_us":211927,"spinlock_wait_cycles":29101696,"thread_start_us":472087,"threads_started":3,"wal-append.queue_time_us":490023}
I20260709 12:26:59.407589 20849 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:59.410328  1181 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:59.404099   872 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:59.424547  1273 log.cc:927] Time spent T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:26:59.426851  1246 log.cc:927] Time spent T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:26:59.430573 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.426014  1275 consensus_queue.cc:1059] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: INVALID_TERM, Last received: 34.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.028s
I20260709 12:26:59.423092  1168 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:59.426312  1284 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:26:59.438278 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:59.440055  1136 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:59.440907  1288 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.016s
I20260709 12:26:59.441614  1275 raft_consensus.cc:3055] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:59.440121 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.442067  1275 raft_consensus.cc:740] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:59.442550  1288 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:59.427289  1264 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:59.442948  1275 consensus_queue.cc:260] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 36.23, Last appended by leader: 23, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:59.447516  1250 log.cc:927] Time spent T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.452s	user 0.000s	sys 0.002s
I20260709 12:26:59.449486  1275 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:59.449630  1236 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader pre-election lost for term 44. Reason: could not achieve majority
I20260709 12:26:59.452626 21400 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:26:59.454749  1158 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:59.455423  1158 raft_consensus.cc:2804] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:59.455977  1158 raft_consensus.cc:697] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:461: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 2
I20260709 12:26:59.466527 20731 raft_consensus.cc:2468] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 44.
I20260709 12:26:59.468199 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.472440 20726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "49323f66286f4798b268f216bb62b3be" is_pre_election: true
I20260709 12:26:59.484524  1168 consensus_queue.cc:1048] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:59.481854 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
I20260709 12:26:59.507908  1158 consensus_queue.cc:237] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 34.25, Last appended by leader: 25, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.515374  1286 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:59.516716  1266 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:59.514997  1181 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:59.527216 20877 raft_consensus.cc:1240] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:59.528442 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.531970 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 40 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:59.535979 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.536794 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 42.
I20260709 12:26:59.538532 21431 raft_consensus.cc:2410] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for term 42 because replica has last-logged OpId of term: 41 index: 25, which is greater than that of the candidate, which has last-logged OpId of term: 40 index: 24.
I20260709 12:26:59.542214 21399 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:26:59.547416 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.548076 21433 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 42.
W20260709 12:26:59.547735 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 392022us
I20260709 12:26:59.550236 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.545279 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:26:59.550470 20774 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:26:59.553143 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.547108  1275 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:59.562196   872 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:59.562790   872 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:59.539649  1289 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:59.564997 20880 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: [32.23-32.23]
I20260709 12:26:59.563731   872 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.23, Last appended by leader: 23, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.568569   723 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.569106   723 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:59.569548 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:59.570850  1266 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:59.557422 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:59.588891 20807 raft_consensus.cc:1240] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:59.578925 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:59.575233 20776 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:59.573072 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:59.592571 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:26:59.590204 20808 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 41.
I20260709 12:26:59.602713 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:26:59.585647 21400 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20260709 12:26:59.605880 20700 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.575767   893 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader pre-election won for term 43
I20260709 12:26:59.613492 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:59.571113   723 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:26:59.614007 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 54403.1us
I20260709 12:26:59.622157   850 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader election lost for term 42. Reason: could not achieve majority
I20260709 12:26:59.603582   800 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:59.628160 20776 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:26:59.630319 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:59.636669 21433 raft_consensus.cc:3060] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:59.637215 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:59.650516 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.653119 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
W20260709 12:26:59.650090 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97497.4us
I20260709 12:26:59.657055 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.615860   893 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.672590 21434 raft_consensus.cc:2393] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 35: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:59.674765 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.640053   800 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:26:59.646585  1236 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:59.695161 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73904us
I20260709 12:26:59.680718  1059 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:59.655450   723 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:59.686031   893 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:59.709946  1261 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:59.698170   800 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:59.699092  1059 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.650090  1286 consensus_queue.cc:1059] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.056s
I20260709 12:26:59.702211   723 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:59.740020  1230 consensus_meta.cc:287] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: Time spent flushing consensus metadata: real 0.822s	user 0.006s	sys 0.005s
I20260709 12:26:59.740689  1230 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.731001   723 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:59.725139  1059 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:59.749579 21435 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:26:59.751413  1298 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:26:59.729879   986 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:59.757676  1277 raft_consensus.cc:3055] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:59.758111  1277 raft_consensus.cc:740] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:26:59.758903  1277 consensus_queue.cc:260] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.763447  1277 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:59.765684 21433 raft_consensus.cc:2468] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:59.767128 20776 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:26:59.768517  1162 raft_consensus.cc:2804] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:59.769026  1162 raft_consensus.cc:697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:26:59.769933  1162 consensus_queue.cc:237] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.24, Last appended by leader: 24, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.755070   986 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.775416   986 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:59.779748 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:26:59.780824 21432 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 39: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:26:59.791071 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.794118 20876 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:26:59.801445  1059 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.802548  1230 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:59.807459 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:26:59.815872  1277 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:26:59.829463 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 91574.3us
W20260709 12:26:59.868258 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 282627us
W20260709 12:26:59.869817 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 62488.3us
W20260709 12:26:59.874949 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 193450us
I20260709 12:26:59.877969   723 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.881143 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:59.883113 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:26:59.883911 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:59.884673 21434 raft_consensus.cc:2393] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 39: Already voted for candidate 657bc62d20374e539b23641715f22745 in this term.
I20260709 12:26:59.885150 20876 raft_consensus.cc:2393] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 42: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:26:59.867147   728 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:59.887811 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.876003   986 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:59.892956 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:26:59.875353  1160 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:59.884395   723 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:59.868253   800 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:59.908701  1254 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:59.911376  1263 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:59.911561  1284 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:26:59.911743 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 156650us
I20260709 12:26:59.894419   986 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:59.872141  1277 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.899919  1230 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:26:59.920090  1059 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:26:59.894193   728 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:59.916128   986 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:59.914950  1307 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:26:59.943538 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:26:59.912345  1160 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:26:59.949373 21432 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:59.949083 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 50514.7us
I20260709 12:26:59.957819 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:59.963562 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:26:59.964172 20805 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 36
I20260709 12:26:59.968945 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:26:59.937114   893 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.971589 20884 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:26:59.960531   728 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:26:59.936919  1230 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:26:59.972226 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:59.973332 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:26:59.934893  1284 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:26:59.983034 20882 raft_consensus.cc:1275] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 42 index: 25. Preceding OpId from leader: term: 44 index: 26. (index mismatch)
I20260709 12:26:59.984982 21430 raft_consensus.cc:2368] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 42. Current term is 43.
I20260709 12:26:59.981820  1093 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.994091   800 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:26:59.995620  1299 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.018s
I20260709 12:26:59.997377 20884 raft_consensus.cc:1240] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting Update request from peer 49323f66286f4798b268f216bb62b3be for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:59.980464  1017 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:26:59.967859  1307 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:00.006991  1311 log.cc:927] Time spent T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:27:00.008143  1160 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.000901   872 consensus_queue.cc:1048] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.011s
I20260709 12:27:00.018666 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:00.019310 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.020901 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47777.8us
W20260709 12:27:00.026648 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.003206  1093 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.041306 20776 leader_election.cc:400] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 42. Current term is 43.
I20260709 12:27:00.041683 20776 leader_election.cc:403] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 election: Cancelling election due to peer responding with higher term
W20260709 12:27:00.036660 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.020532   728 consensus_queue.cc:1059] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 38.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.021s
W20260709 12:27:00.045976 20773 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.024469  1017 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:27:00.046053 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.047343  1136 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Advancing to term 43
W20260709 12:27:00.056958 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.061129 20775 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:00.065855 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.069311 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88257.9us
I20260709 12:27:00.058277   893 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.072227 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.072674  1284 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.075296  1162 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:00.073580 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:00.079553  1277 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.082260  1093 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.090365 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:00.122223 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 53401.5us
W20260709 12:27:00.131700 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 72236us
I20260709 12:27:00.132660 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.140537 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:00.123647 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.143416 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 73027.5us
I20260709 12:27:00.148723 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:00.149330 20877 raft_consensus.cc:3060] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:00.156344 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:00.167764 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.167223  1020 raft_consensus.cc:3055] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:27:00.175207  1020 raft_consensus.cc:740] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:27:00.158895  1261 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:00.176085  1020 consensus_queue.cc:260] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 38.19, Last appended by leader: 19, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.177359  1020 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:00.178707  1230 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.176201  1261 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:00.187834 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.173485 21399 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:27:00.199715 21400 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.204757   893 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:00.209757  1136 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader election lost for term 42. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 for earlier term 42. Current term is 43.
W20260709 12:27:00.210556 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.211265 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.212749 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.217983 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.220008 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:00.230564 20805 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:27:00.236793 20809 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 43.
I20260709 12:27:00.240779 20877 raft_consensus.cc:2468] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 43.
W20260709 12:27:00.238392 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 63742.9us
I20260709 12:27:00.211756   986 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.242288  1013 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.247747  1013 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.250669  1230 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.251247  1230 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.267059 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.268628 21399 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:00.273276 20701 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:00.273561 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.274150  1059 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:00.284912 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.285691 21432 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 36.
I20260709 12:27:00.295377 20878 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:27:00.300311 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.297952 20701 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:00.307469 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.296552   721 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.298614  1149 raft_consensus.cc:2804] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader election won for term 43
I20260709 12:27:00.298172  1279 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.313016 20775 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:00.308781  1059 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:27:00.310811   922 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:00.315104   721 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.325213  1013 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.307296  1136 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.013s
I20260709 12:27:00.331945  1261 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.333266  1230 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.316679  1149 raft_consensus.cc:697] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:00.319468  1307 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.335752  1059 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:27:00.336256   870 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.334426  1017 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.342720 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.365315   870 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.346781 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.347635 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.367625   870 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.359777  1149 consensus_queue.cc:237] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 41.25, Last appended by leader: 25, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.360432 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.347623 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 44 candidate_status { last_received { term: 43 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.372083 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.317687  1279 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.379743  1279 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.350324  1266 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:27:00.381608 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.381347  1059 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.382699 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.384344 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.384982 20805 raft_consensus.cc:1275] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 26. Preceding OpId from leader: term: 36 index: 26. (term mismatch)
I20260709 12:27:00.385552 20805 pending_rounds.cc:85] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Aborting all ops after (but not including) 25
I20260709 12:27:00.382243 20882 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 42.
I20260709 12:27:00.385838 20805 pending_rounds.cc:107] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: Aborting uncommitted NO_OP operation due to leader change: 35.26
I20260709 12:27:00.386662 20805 raft_consensus.cc:2889] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:00.384982 20803 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 42.
I20260709 12:27:00.385664 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:00.370971   986 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.387640 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:00.393169  1158 consensus_queue.cc:1048] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:27:00.396543  1158 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader pre-election won for term 43
I20260709 12:27:00.396930  1158 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.397334  1158 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:00.342216   721 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.403362  1017 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:00.419960 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:27:00.421952 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 109017us
I20260709 12:27:00.426795  1017 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:00.427429  1017 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:27:00.429183 20701 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.420531   870 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:00.452677   870 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.439654  1017 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.455070   870 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.504395 20877 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:27:00.507395 20774 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:27:00.521958 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 210333us
W20260709 12:27:00.522836 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 209996us
I20260709 12:27:00.524940 21435 raft_consensus.cc:1275] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 39 index: 25. (index mismatch)
I20260709 12:27:00.530953 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:00.531708 21435 raft_consensus.cc:2393] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 39: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:27:00.534667  1320 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:27:00.535284  1320 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:00.536244  1320 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 38.27, Last appended by leader: 27, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.559702 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.533977  1017 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:00.560635  1017 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.560925 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:00.578727 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 154120us
W20260709 12:27:00.582329 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.584458 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.586150 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.540805  1162 consensus_queue.cc:1048] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 23, Time since last communication: 0.041s
I20260709 12:27:00.602547  1017 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.603533 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.604791  1158 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.606914  1158 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.609754 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:00.620513 20806 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Advancing to term 43
W20260709 12:27:00.610778 20849 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.613585 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.614771 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.624401  1020 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:00.624588 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.624989  1020 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.625887 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.627454 20809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 44 candidate_status { last_received { term: 43 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.620523  1307 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.628655 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.625002 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.629886 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.631225 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.628548  1307 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.592741  1059 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
W20260709 12:27:00.638135 20775 leader_election.cc:336] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.635278 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.621940 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.640928 20775 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:00.646265 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 41 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:00.646833 20877 raft_consensus.cc:3060] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:00.644578 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.652462 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:27:00.660768 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 50013.6us
W20260709 12:27:00.662102 20849 leader_election.cc:336] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:00.666144  1325 log.cc:927] Time spent T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: Append to log took a long time: real 0.146s	user 0.003s	sys 0.000s
I20260709 12:27:00.643005  1059 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.673982 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.650789  1020 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.654866  1266 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:00.655012  1201 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:00.658895   728 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.659123  1237 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
W20260709 12:27:00.685896 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47788.3us
I20260709 12:27:00.662087  1313 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:27:00.691354   728 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.708789 20806 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 43.
I20260709 12:27:00.672767  1307 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.681447  1160 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.727116   728 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:00.728231 20701 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 106657us
I20260709 12:27:00.732366 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.732416 20808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.732762 20803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.737977 20877 raft_consensus.cc:2468] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 43.
W20260709 12:27:00.746354 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.754269 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:27:00.754309 20849 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:00.756299 20775 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.756816 20775 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:00.756407 20849 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:00.759884 20775 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:00.760859 20775 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.758123  1307 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:00.741441  1262 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.762018  1307 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.762424  1262 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:00.764699  1326 log.cc:927] Time spent T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.245s	user 0.002s	sys 0.001s
I20260709 12:27:00.766027  1262 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:00.767707  1330 log.cc:927] Time spent T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:27:00.769613 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:00.772166 20701 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:00.773038 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.775120 21399 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:00.776818 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.761233 20775 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:00.785964 20775 leader_election.cc:336] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.787385 20700 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.787662 20775 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:00.737473  1160 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.796895  1160 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.774380  1266 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:00.790400 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.780264  1333 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:00.793270  1261 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:00.778172   800 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:00.790899  1017 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:00.780053  1332 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:00.815258 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.816967  1307 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.820258  1307 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:00.811614 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:00.786620   810 raft_consensus.cc:2804] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader election won for term 43
I20260709 12:27:00.814007  1059 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.788213  1162 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:00.814646  1333 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.824978 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.840373 20877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.840438 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:00.789846 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:00.815049 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.839473  1017 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:00.842803  1017 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.837266   810 raft_consensus.cc:697] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:00.844944  1017 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.845194  1333 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.839331 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:00.839715  1307 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:00.846350 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.847002  1307 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:00.847751  1059 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader pre-election lost for term 44. Reason: could not achieve majority
I20260709 12:27:00.849608 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.850190 20879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.850401  1059 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:00.845077   810 consensus_queue.cc:237] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 41.25, Last appended by leader: 25, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.851038 20879 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 41: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:27:00.851563 20700 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.851889 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.852236  1059 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:00.853356 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.854075  1059 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:00.856267 20700 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.856806  1059 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:00.859237  1059 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.861352  1059 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.862555 20700 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.862560 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.865028  1332 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:00.863112  1059 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:00.867439 20701 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:00.868767  1017 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:00.870008  1017 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.870563  1017 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.873770 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.879354  1059 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:00.879927  1059 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:00.883359  1059 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.893875  1017 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:00.903277 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 44 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:00.907125 20807 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 41 index: 25. Preceding OpId from leader: term: 43 index: 26. (index mismatch)
I20260709 12:27:00.825256  1320 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
W20260709 12:27:00.931491 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 141677us
W20260709 12:27:00.940061 20619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 127464us
I20260709 12:27:00.934367  1263 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.006s
I20260709 12:27:00.948235 20878 raft_consensus.cc:1275] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 40 index: 24. Preceding OpId from leader: term: 43 index: 26. (index mismatch)
W20260709 12:27:00.949309 21400 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.952173 21400 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:27:00.932209 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.953177 20775 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:00.968271 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:00.962064  1263 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:27:00.973599  1320 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:00.976752   728 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.961639  1307 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:00.981014 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:00.989687  1201 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:00.990278  1201 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.992688  1201 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:00.991820 20628 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 20 to 27, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 27 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:00.985739  1059 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:00.966640  1156 consensus_queue.cc:1048] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.014s
I20260709 12:27:01.001236 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.026646 20628 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 27 to 29, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 29 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:00.977308   728 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:00.986585  1307 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.029925  1307 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.030277   728 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.013394 21435 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:27:01.040756 21432 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:27:01.031112 20628 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 27 to 30, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 30 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:01.044370 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.057576 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.058806 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.058288 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 41.
I20260709 12:27:01.048249  1307 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:01.065945 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:27:01.066215  1307 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:01.068490  1307 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:01.073302 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.076280  1263 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:27:01.077482 21433 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 41.
I20260709 12:27:01.066840   728 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:27:01.079577   728 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.079942   728 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:27:01.085125 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.086848 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.089991  1320 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:27:01.096007  1059 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:27:01.100222 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.100984 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.099962 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.102557 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:01.104341 20627 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 279) took 34961 ms (client timeout 15000 ms). Trace:
I20260709 12:27:01.109012 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.112074 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 44 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.112944 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:01.098080   728 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.115108 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.111917   810 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.105406 20627 rpcz_store.cc:269] 0709 12:26:26.143661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.900057 (+8756396us) service_pool.cc:224] Handling call
0709 12:27:01.104256 (+26204199us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:59.340482 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:59.420065 (+ 79583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:59.420140 (+    75us) write_op.cc:435] Acquired schema lock
0709 12:26:59.420190 (+    50us) tablet.cc:662] Decoding operations
0709 12:26:59.425925 (+  5735us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:59.426030 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:59.426062 (+    32us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:26:59.429241 (+  3179us) tablet.cc:701] Row locks acquired
0709 12:26:59.429291 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:26:59.429471 (+   180us) write_op.cc:270] Start()
0709 12:26:59.429636 (+   165us) write_op.cc:276] Timestamp: P: 1783600019429403 usec, L: 0
0709 12:26:59.429674 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:26:59.430463 (+   789us) log.cc:844] Serialized 9701 byte log entry
0709 12:26:59.762514 (+332051us) op_driver.cc:464] REPLICATION: finished
0709 12:26:59.947588 (+185074us) write_op.cc:301] APPLY: starting
0709 12:26:59.954762 (+  7174us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:00.334364 (+379602us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:00.334493 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:00.555047 (+220554us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:00.555677 (+   630us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:27:00.555766 (+    89us) write_op.cc:312] APPLY: finished
0709 12:27:00.556591 (+   825us) log.cc:844] Serialized 212 byte log entry
0709 12:27:00.581166 (+ 24575us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:00.599105 (+ 17939us) write_op.cc:454] Released schema lock
0709 12:27:00.624034 (+ 24929us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:00.664819 (+ 40785us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:00.664853 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":152107,"apply.run_cpu_time_us":38759,"apply.run_wall_time_us":739057,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104447,"num_ops":24,"prepare.queue_time_us":194300,"prepare.run_cpu_time_us":23995,"prepare.run_wall_time_us":226848,"raft.queue_time_us":199090,"raft.run_cpu_time_us":39744,"raft.run_wall_time_us":135018,"replication_time_us":332625,"thread_start_us":423044,"threads_started":4,"wal-append.queue_time_us":107301}]]}
I20260709 12:27:01.124756 20806 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:27:01.126956 20882 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 27. Preceding OpId from leader: term: 39 index: 28. (index mismatch)
I20260709 12:27:01.116699   920 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:01.127439   810 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.138748   810 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:01.139747   728 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.141103 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:01.139739 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.141760 20878 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:27:01.142244 20882 raft_consensus.cc:2410] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for term 40 because replica has last-logged OpId of term: 38 index: 27, which is greater than that of the candidate, which has last-logged OpId of term: 36 index: 26.
I20260709 12:27:01.154646 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.152040 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 42.
I20260709 12:27:01.156148 20699 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:27:01.160218   728 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader election won for term 42
I20260709 12:27:01.146903  1320 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 26, Time since last communication: 0.012s
I20260709 12:27:01.160908   728 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 42 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:27:01.163283   728 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 40.26, Last appended by leader: 26, Current term: 42, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.175949 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.177248  1149 raft_consensus.cc:2749] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:01.182343 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:01.182922 21433 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:27:01.184940 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.205477 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:27:01.207559 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.213539 20700 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.225879 20700 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.233100 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:27:01.214634 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.238518  1237 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:01.231158   920 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:01.240712 20700 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:27:01.236181 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.245946 21426 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 36 index: 26. Preceding OpId from leader: term: 39 index: 28. (index mismatch)
I20260709 12:27:01.247160 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.247364   728 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:01.245337 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.249284 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.249817 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.250497 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.251849 20700 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.230845  1059 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:01.255026 20700 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.244246  1017 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.239741  1237 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
W20260709 12:27:01.252525 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.266928 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.259416  1160 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:01.261795  1332 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.007s
I20260709 12:27:01.266783  1017 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:01.269798 21433 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 42.
I20260709 12:27:01.271211  1017 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:01.254755  1162 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 26, Time since last communication: 0.015s
I20260709 12:27:01.275521 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.263362  1201 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:01.252876  1331 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:01.254354  1290 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.267555 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.301651  1277 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.311825 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:01.307353 20775 consensus_peers.cc:597] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.300282  1290 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.306035  1277 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.326045  1277 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:01.317528 20700 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.328465 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.330945  1307 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:01.334221  1290 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.335246 20881 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 37.
I20260709 12:27:01.338861 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:01.340368 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:01.347065 20849 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:01.341506 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.355831 21400 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:01.355588  1277 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:27:01.370736  1277 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.371115  1277 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:27:01.371443 20806 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 40 index: 24. Preceding OpId from leader: term: 43 index: 26. (index mismatch)
W20260709 12:27:01.374431 20681 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:01.374840 20681 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:01.375130 20681 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:01.374928  1149 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:27:01.397351  1237 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:01.397900  1237 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.403236  1237 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.404091 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.404502 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.405957 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.407158  1237 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:27:01.415807  1347 log.cc:927] Time spent T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:27:01.460366  1331 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:01.460956  1331 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.472529 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.478602 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.478686 20878 raft_consensus.cc:1275] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 40 index: 24. Preceding OpId from leader: term: 43 index: 26. (index mismatch)
I20260709 12:27:01.481572 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.483961 20700 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.485687   810 consensus_queue.cc:1048] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 23, Time since last communication: 0.006s
I20260709 12:27:01.458186  1277 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.502342  1277 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:01.507824 21426 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 38: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:27:01.489349  1201 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:01.471060  1331 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.516047 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:01.517251 20875 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:27:01.523919 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:01.541483 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 18)) from {username='slave'} at 127.0.0.1:34158
I20260709 12:27:01.547313 20875 raft_consensus.cc:2468] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:27:01.547884   922 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:27:01.554628 20627 catalog_manager.cc:5697] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 30 to 35. New cstate: current_term: 35 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:27:01.546962 21400 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:01.556026 20627 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 29 to 33, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 33 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:01.546046 20628 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 299) took 65627 ms (client timeout 15000 ms). Trace:
I20260709 12:27:01.559736 21399 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:27:01.558826 20628 rpcz_store.cc:269] 0709 12:25:55.914524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.510121 (+14595597us) service_pool.cc:224] Handling call
0709 12:27:01.541605 (+51031484us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:01.215510 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:01.216042 (+   532us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:01.216072 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:27:01.216091 (+    19us) tablet.cc:662] Decoding operations
0709 12:27:01.216594 (+   503us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:01.216671 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:01.216697 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:27:01.217020 (+   323us) tablet.cc:701] Row locks acquired
0709 12:27:01.217042 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:27:01.217171 (+   129us) write_op.cc:270] Start()
0709 12:27:01.217298 (+   127us) write_op.cc:276] Timestamp: P: 1783600021217124 usec, L: 0
0709 12:27:01.217328 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:27:01.227865 (+ 10537us) log.cc:844] Serialized 1312 byte log entry
0709 12:27:01.342949 (+115084us) op_driver.cc:464] REPLICATION: finished
0709 12:27:01.422968 (+ 80019us) write_op.cc:301] APPLY: starting
0709 12:27:01.423054 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:01.427427 (+  4373us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:01.427557 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:01.432233 (+  4676us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:01.432425 (+   192us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:27:01.432487 (+    62us) write_op.cc:312] APPLY: finished
0709 12:27:01.432682 (+   195us) log.cc:844] Serialized 41 byte log entry
0709 12:27:01.432849 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:01.432940 (+    91us) write_op.cc:454] Released schema lock
0709 12:27:01.432984 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:01.435530 (+  2546us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:01.435566 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":741,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":78800,"apply.run_cpu_time_us":5681,"apply.run_wall_time_us":12526,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16667,"num_ops":3,"prepare.queue_time_us":109160,"prepare.run_cpu_time_us":3106,"prepare.run_wall_time_us":12706,"replication_time_us":119005,"thread_start_us":123791,"threads_started":2}]]}
I20260709 12:27:01.572896  1149 raft_consensus.cc:2804] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:27:01.557547 20627 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 32, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.593770  1149 raft_consensus.cc:697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:01.594619 20627 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 36 to 38. New cstate: current_term: 38 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.598151 20627 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 30 to 36, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 36 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:01.602561 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet f15b5f1ac1444b63bad60a89fa9f8a29 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:27:01.608080 20627 catalog_manager.cc:5697] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 30 to 35, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.609977 20627 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 35 to 42. New cstate: current_term: 42 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.612165  1331 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:01.612764  1331 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.594745  1149 consensus_queue.cc:237] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 36.23, Last appended by leader: 23, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.613797 20627 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 27 to 41. New cstate: current_term: 41 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.625089 20627 catalog_manager.cc:5697] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.626462 20627 catalog_manager.cc:5697] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 35, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.628656 20627 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 32, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 32 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.639336  1331 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.640394 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.630491 20627 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 34 to 40, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 40 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.645900 21426 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 40 index: 26. Preceding OpId from leader: term: 42 index: 27. (index mismatch)
I20260709 12:27:01.642493 20627 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 12 to 20, term changed from 29 to 30, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193), VOTER 657bc62d20374e539b23641715f22745 (127.19.36.194), VOTER 116023b4c0c94bfea737fc238bda3838 (127.19.36.195), VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 30 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.658255 20627 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 32 to 38. New cstate: current_term: 38 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.659653 20627 catalog_manager.cc:5697] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 33. New cstate: current_term: 33 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.660724  1237 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.001s
I20260709 12:27:01.665529 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.673655 20627 catalog_manager.cc:5697] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 37 to 41, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 41 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:01.675911 20699 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:01.683911  1237 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:01.686291 20627 catalog_manager.cc:5697] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 29 to 36. New cstate: current_term: 36 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:01.694941 20627 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 36 to 38, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 38 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.696503 20627 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 30 to 38, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 38 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:01.700568 20627 catalog_manager.cc:5697] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 29 to 34. New cstate: current_term: 34 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.705513 20881 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 40 index: 26. Preceding OpId from leader: term: 42 index: 27. (index mismatch)
I20260709 12:27:01.707333  1307 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:27:01.707995 20627 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 33 to 37, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 37 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.727016 20627 catalog_manager.cc:5697] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 34 to 36. New cstate: current_term: 36 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:01.730237 20627 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 27 to 30, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 30 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.749989 20627 catalog_manager.cc:5697] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 26 to 30, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 30 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.764331 20627 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 33 to 35. New cstate: current_term: 35 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.767166 20627 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 33 to 35, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.779677   728 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:01.780269   728 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.782295   728 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.784101 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.779904 20627 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 34, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 34 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:01.793998 20627 catalog_manager.cc:5697] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 32, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 32 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:01.795909 20627 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 37 to 41, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 41 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:01.797802 20627 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 27 to 35, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:01.799557 20627 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 34, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 34 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:01.822342 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.824126 20700 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:01.825461  1307 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:01.904752  1307 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:01.905365  1307 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:01.910887  1307 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:01.917455 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.918999 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:01.926872 20699 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:01.931111   728 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:01.959172  1307 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.959800  1307 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:01.972729 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:01.973374 20881 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 39.
I20260709 12:27:01.974941 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:27:01.976568  1237 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:27:01.976969  1237 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.977320  1237 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:27:01.978053 19603 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 20760)
I20260709 12:27:01.980892 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:01.981597 20807 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 38.
I20260709 12:27:01.989778  1307 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.022190  1237 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.041719 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:02.042420 20807 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 40
I20260709 12:27:02.053213 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 38 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:02.053822 20881 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:02.062335 20881 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:27:02.063592 20699 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be; no voters: 
I20260709 12:27:02.064538  1307 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:27:02.065068  1307 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Becoming Leader. State: Replica: 49323f66286f4798b268f216bb62b3be, State: Running, Role: LEADER
I20260709 12:27:02.074961 20807 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49323f66286f4798b268f216bb62b3be in term 40.
I20260709 12:27:02.065924  1307 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 38.19, Last appended by leader: 19, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.073036  1237 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.110323  1237 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.123710  1237 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:02.113222 20881 raft_consensus.cc:1275] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 36 index: 23. Preceding OpId from leader: term: 38 index: 24. (index mismatch)
I20260709 12:27:02.126909  1237 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:02.127347 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:02.129117 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 38 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.130111 20881 raft_consensus.cc:2393] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49323f66286f4798b268f216bb62b3be in current term 38: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:27:02.131956   810 consensus_queue.cc:1048] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:27:02.136575 20701 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:02.140095  1307 raft_consensus.cc:3060] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:27:02.154068  1307 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:27:02.191056 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.209311  1237 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:02.217669  1237 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:02.220489  1237 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:02.220688 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.222478 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:02.222922 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.226810 20700 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
W20260709 12:27:02.229637 20627 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36978 (request call id 309) took 11750 ms (client timeout 15000 ms). Trace:
I20260709 12:27:02.229552  1307 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:02.230218 20627 rpcz_store.cc:269] 0709 12:26:50.479557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.140448 (+10660891us) service_pool.cc:224] Handling call
0709 12:27:02.229556 (+1089108us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:01.864710 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:01.865286 (+   576us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:01.865326 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:27:01.865346 (+    20us) tablet.cc:662] Decoding operations
0709 12:27:01.867867 (+  2521us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:01.867956 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:01.867985 (+    29us) tablet.cc:685] Acquiring locks for 31 operations
0709 12:27:01.870241 (+  2256us) tablet.cc:701] Row locks acquired
0709 12:27:01.870262 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:27:01.870405 (+   143us) write_op.cc:270] Start()
0709 12:27:01.870524 (+   119us) write_op.cc:276] Timestamp: P: 1783600021870357 usec, L: 0
0709 12:27:01.870550 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:27:01.919059 (+ 48509us) log.cc:844] Serialized 12291 byte log entry
0709 12:27:01.927547 (+  8488us) op_driver.cc:464] REPLICATION: finished
0709 12:27:01.927942 (+   395us) write_op.cc:301] APPLY: starting
0709 12:27:01.928020 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:02.018018 (+ 89998us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:02.018120 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:02.153904 (+135784us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:02.154251 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=0,mrs_lookups=0
0709 12:27:02.154296 (+    45us) write_op.cc:312] APPLY: finished
0709 12:27:02.154805 (+   509us) log.cc:844] Serialized 268 byte log entry
0709 12:27:02.157155 (+  2350us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:02.157395 (+   240us) write_op.cc:454] Released schema lock
0709 12:27:02.157471 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:02.157844 (+   373us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:02.157880 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":562,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":169,"apply.run_cpu_time_us":36955,"apply.run_wall_time_us":229857,"cfile_cache_hit":124,"cfile_cache_hit_bytes":129580,"num_ops":31,"prepare.queue_time_us":53841,"prepare.run_cpu_time_us":7067,"prepare.run_wall_time_us":54721,"replication_time_us":56847,"thread_start_us":53476,"threads_started":1}]]}
W20260709 12:27:02.244633 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:02.255426 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:02.261590 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:02.336366 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:02.395880 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:02.466933 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.486191  1201 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.486737  1201 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:02.488947  1201 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:02.492928 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:02.493535 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:02.494130 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.494952 20700 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.496640  1307 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
W20260709 12:27:02.497413 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.519160  1201 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.519722  1201 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.520200 20626 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.521870  1201 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.523473 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:02.524353 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.524827 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.526399 20699 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.527238  1201 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:02.543574 20626 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 35 to 36, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 36 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:02.553211 20626 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 32 to 33, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 33 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.559453 20806 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 38 index: 19. Preceding OpId from leader: term: 40 index: 20. (index mismatch)
I20260709 12:27:02.559649 20878 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 49323f66286f4798b268f216bb62b3be: Log matching property violated. Preceding OpId in replica: term: 38 index: 19. Preceding OpId from leader: term: 40 index: 20. (index mismatch)
I20260709 12:27:02.562561  1201 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:02.564240  1201 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:27:02.567138 20629 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 205) took 72340 ms (client timeout 15000 ms). Trace:
W20260709 12:27:02.567642 20629 rpcz_store.cc:269] 0709 12:25:50.227034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.391703 (+12164669us) service_pool.cc:224] Handling call
0709 12:27:02.566961 (+60175258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85101,"reported_tablets":38}
W20260709 12:27:02.575343 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.588851  1307 raft_consensus.cc:493] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:02.589430  1307 raft_consensus.cc:515] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.592265  1307 leader_election.cc:290] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.593997 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.597417 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc725b7c88e47f693e3a16879798246" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:02.598992 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.600467 20700 leader_election.cc:304] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.601500  1201 raft_consensus.cc:2749] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:02.613637  1307 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:02.616607  1307 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.618817  1307 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.622396 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.622923 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.625790 20699 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.629535  1201 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:02.665423 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:02.671912 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:02.684989 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.709465  1237 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:02.710040  1237 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:27:02.732225 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.740281 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.740995 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.745539  1237 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.749253 20700 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.750417  1237 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:02.756803  1307 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:02.757323  1307 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.762003  1307 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.763442 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 44 candidate_status { last_received { term: 43 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.768445 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 44 candidate_status { last_received { term: 43 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:02.771492 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.773531 20700 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.775377  1237 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Leader pre-election lost for term 44. Reason: could not achieve majority
I20260709 12:27:02.775926  1307 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:02.776556  1307 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.780035 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.781412  1307 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.791265  1237 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:02.791814  1237 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:02.793962  1237 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:02.796689 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:02.798382 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.799988 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.801734 20700 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.803290  1307 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:02.809695 20700 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.810825  1307 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:02.864814  1307 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:27:02.865334 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.865391  1307 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.868121  1307 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.870179 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.870610 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 43 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.872303 20629 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 27 to 33. New cstate: current_term: 33 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.873934 20700 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.874096 20629 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 33 to 34, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 34 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:02.875468 20629 catalog_manager.cc:5697] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 25 to 32, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 32 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.877053 20629 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 38 to 42, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 42 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.877931  1237 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
I20260709 12:27:02.878345 20629 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 36 to 39, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 39 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.881870 20629 catalog_manager.cc:5697] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 42 to 44. New cstate: current_term: 44 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.886677 20629 catalog_manager.cc:5697] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 41 to 44, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 44 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:02.887244 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 18)) from {username='slave'} at 127.0.0.1:34158
W20260709 12:27:02.888345 20626 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 300) took 41911 ms (client timeout 15000 ms). Trace:
I20260709 12:27:02.907590 20629 catalog_manager.cc:5697] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 32 to 43. New cstate: current_term: 43 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.913278 20629 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 32 to 38. New cstate: current_term: 38 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:27:02.905460 20626 rpcz_store.cc:269] 0709 12:26:20.985578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.638032 (+8652454us) service_pool.cc:224] Handling call
0709 12:27:02.888274 (+33250242us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:02.691867 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:02.692466 (+   599us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:02.692504 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:27:02.692525 (+    21us) tablet.cc:662] Decoding operations
0709 12:27:02.693093 (+   568us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:02.693175 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:02.693208 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:27:02.693557 (+   349us) tablet.cc:701] Row locks acquired
0709 12:27:02.693587 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:27:02.693730 (+   143us) write_op.cc:270] Start()
0709 12:27:02.693841 (+   111us) write_op.cc:276] Timestamp: P: 1783600022693683 usec, L: 0
0709 12:27:02.693864 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:27:02.694543 (+   679us) log.cc:844] Serialized 1307 byte log entry
0709 12:27:02.728270 (+ 33727us) op_driver.cc:464] REPLICATION: finished
0709 12:27:02.811564 (+ 83294us) write_op.cc:301] APPLY: starting
0709 12:27:02.811659 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:02.813725 (+  2066us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:02.813826 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:02.815665 (+  1839us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:02.815889 (+   224us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:27:02.815940 (+    51us) write_op.cc:312] APPLY: finished
0709 12:27:02.816163 (+   223us) log.cc:844] Serialized 41 byte log entry
0709 12:27:02.824167 (+  8004us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:02.824273 (+   106us) write_op.cc:454] Released schema lock
0709 12:27:02.824318 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:02.824738 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:02.824770 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":461423,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":83059,"apply.run_cpu_time_us":5360,"apply.run_wall_time_us":13166,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16667,"num_ops":3,"prepare.queue_time_us":78451,"prepare.run_cpu_time_us":2999,"prepare.run_wall_time_us":2999,"replication_time_us":34244,"thread_start_us":105053,"threads_started":2}]]}
I20260709 12:27:02.918228 20629 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 32 to 33, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 33 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.920203 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet f15b5f1ac1444b63bad60a89fa9f8a29 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:27:02.920198 20629 catalog_manager.cc:5697] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 38, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 38 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.922044 20629 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 32 to 38, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 38 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.925029 20629 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 33 to 39, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 39 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.928184 20629 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 38 to 39, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 39 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.930792 20629 catalog_manager.cc:5697] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 42, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 42 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.932397 20629 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 38 to 40, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 40 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.941250 20629 catalog_manager.cc:5697] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 36 to 41, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 41 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.942791 20629 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 30 to 33. New cstate: current_term: 33 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.948956 20629 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 32 to 34, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 34 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.950057  1307 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:02.950630  1307 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.950344 20629 catalog_manager.cc:5697] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 37 to 39. New cstate: current_term: 39 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.951931 20629 catalog_manager.cc:5697] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 35 to 36, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 36 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.952698  1307 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.953502 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.953353 20629 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 31 to 37, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 37 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
I20260709 12:27:02.954617 20629 catalog_manager.cc:5697] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 28 to 33, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 33 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.963735  1201 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:02.967492  1201 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:02.969157   728 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:02.969620  1201 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:02.965199 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.970724 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:02.969667   728 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:02.972544 20629 catalog_manager.cc:5697] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 30 to 39. New cstate: current_term: 39 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.973955   728 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:02.974031 20629 catalog_manager.cc:5697] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 33 to 38, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 38 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.976404 20629 catalog_manager.cc:5697] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 41 to 43. New cstate: current_term: 43 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.977629 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f61efc9c6d5f43ac84cfe1def41de8a0" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:02.978358 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:02.978526 19603 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 20760)
I20260709 12:27:02.977778 20629 catalog_manager.cc:5697] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 35 to 43, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 43 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:02.980372 20699 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.987140 20700 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.983578 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:02.988220  1237 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:02.988858  1237 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:02.988217 20629 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 34 to 36. New cstate: current_term: 36 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:02.991155 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:02.991847 20700 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:02.992583  1237 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:02.990644 20629 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 40 to 42. New cstate: current_term: 42 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.004653 20629 catalog_manager.cc:5697] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 26 to 35. New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.007443  1237 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:03.008010  1237 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.010149  1237 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.012369 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.017935  1237 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:03.018481  1237 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:03.019527 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.021978  1237 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:03.023660 20700 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.024322 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.027874 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.026976  1237 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:03.029989 20700 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.031234  1237 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:03.033608  1237 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:03.034108  1237 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:03.037586 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 44 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.039676  1237 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:03.040107 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 44 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.042033 20700 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.043094  1307 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:03.043372  1237 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:03.043675  1307 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:03.046056  1307 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:03.051685 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:03.067610 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.095844  1237 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:03.096519  1237 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.097286 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.098732 20700 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.100811 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.102193   728 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:03.098891  1237 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.107244 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.108789 20699 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:03.109545   728 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:03.116880  1237 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:03.117503  1237 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.120983 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.123014  1237 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.125063 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.128046 20699 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:03.129567   728 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:27:03.130975 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.143355 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 57292us
I20260709 12:27:03.171763  1237 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:03.172359  1237 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.178195 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:03.175171 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.210800 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.210942  1237 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.212644 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.212750 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.213403   728 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:03.272688  1237 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:03.273319  1237 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.277040 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.312599  1237 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.316582 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.335570 20701 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.340624  1237 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:27:03.400395 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.407740 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.426085 20626 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 30 to 40. New cstate: current_term: 40 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } }
W20260709 12:27:03.503307 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.537039   728 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.537540   728 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.539368   728 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.540767 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:03.544368 20775 consensus_peers.cc:597] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.546167 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.565076 20701 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.565878   728 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:27:03.585281 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.642204 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.666898  1237 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:03.667574  1237 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.670760 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.676425  1237 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.678191 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.681157 20699 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.682785   728 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:03.687574   728 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:03.688037   728 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.689970   728 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.693102 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.694496 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:03.696902 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.718832 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.730227   728 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:03.730757   728 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:03.732720   728 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:03.734993 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.740504 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.742022 20699 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.743539   728 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:03.758510 20701 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:03.759907 20631 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 203) took 122475 ms (client timeout 15000 ms). Trace:
W20260709 12:27:03.760417 20631 rpcz_store.cc:269] 0709 12:25:01.284928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.285259 (+   331us) service_pool.cc:224] Handling call
0709 12:27:03.759701 (+122474442us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":465108,"reported_tablets":38}
I20260709 12:27:03.764703   728 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:03.798538 20632 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 199) took 219360 ms (client timeout 15000 ms). Trace:
W20260709 12:27:03.799108 20632 rpcz_store.cc:269] 0709 12:23:24.438275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.438564 (+   289us) service_pool.cc:224] Handling call
0709 12:27:03.798004 (+219359440us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1696694,"reported_tablets":36}
W20260709 12:27:03.805846 20630 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 275) took 141895 ms (client timeout 15000 ms). Trace:
W20260709 12:27:03.806262 20630 rpcz_store.cc:269] 0709 12:24:41.953734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.954109 (+   375us) service_pool.cc:224] Handling call
0709 12:27:03.805666 (+141851557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171116,"reported_tablets":48}
W20260709 12:27:03.807708 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.812763 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.818958 20633 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 26 to 30. New cstate: current_term: 30 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:03.847150 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.875834 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.895562   728 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:03.896103   728 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:03.899775 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.899788 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:03.901783   728 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:03.902767 20699 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.904997   728 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:27:03.932268 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.964385   728 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:03.964927   728 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:03.969741 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:03.966943   728 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:03.973204 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:03.974766 20701 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:03.976147   728 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:03.981617 20624 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.009524 20624 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 37 to 40. New cstate: current_term: 40 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:03.987217 19603 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 20760)
W20260709 12:27:03.983613 20633 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 207) took 30953 ms (client timeout 15000 ms). Trace:
W20260709 12:27:04.010819 20633 rpcz_store.cc:269] 0709 12:26:33.048846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.110352 (+10061506us) service_pool.cc:224] Handling call
0709 12:27:03.983536 (+20873184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:03.850214 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:03.850849 (+   635us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:03.850909 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:27:03.850931 (+    22us) tablet.cc:662] Decoding operations
0709 12:27:03.851346 (+   415us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:03.851488 (+   142us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:03.851533 (+    45us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:27:03.851751 (+   218us) tablet.cc:701] Row locks acquired
0709 12:27:03.851771 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:27:03.851948 (+   177us) write_op.cc:270] Start()
0709 12:27:03.852072 (+   124us) write_op.cc:276] Timestamp: P: 1783600023851897 usec, L: 0
0709 12:27:03.852096 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:27:03.852782 (+   686us) log.cc:844] Serialized 536 byte log entry
0709 12:27:03.854834 (+  2052us) op_driver.cc:464] REPLICATION: finished
0709 12:27:03.855289 (+   455us) write_op.cc:301] APPLY: starting
0709 12:27:03.855374 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:03.856270 (+   896us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:03.856377 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:03.857128 (+   751us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:03.857309 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:27:03.857369 (+    60us) write_op.cc:312] APPLY: finished
0709 12:27:03.857566 (+   197us) log.cc:844] Serialized 25 byte log entry
0709 12:27:03.857730 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:03.857842 (+   112us) write_op.cc:454] Released schema lock
0709 12:27:03.857885 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:03.858247 (+   362us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:03.858280 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":238,"apply.run_cpu_time_us":2933,"apply.run_wall_time_us":2932,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4180,"num_ops":1,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":2863,"prepare.run_wall_time_us":2863,"replication_time_us":2617}]]}
I20260709 12:27:04.017892 20624 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 31 to 35. New cstate: current_term: 35 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.036125 20624 catalog_manager.cc:5697] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 31 to 34, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 34 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.078656  1307 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:04.079324  1307 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:04.082597 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:04.084697  1307 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:04.088925 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:04.090628 20701 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:04.096244  1237 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:04.114454  1307 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:04.115106  1307 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:04.118723 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:04.128942 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.130867 20699 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:04.131748  1374 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:04.126993  1307 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:04.135746 20624 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 27 to 33, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 33 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.223376  1307 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:04.224023  1307 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:04.228487 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:04.232201 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.233794 20699 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:04.234732  1374 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:04.236482  1307 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:04.317860 20625 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 31 to 36, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 36 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.346733 20625 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 29 to 32, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 32 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.363860 20625 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be reported cstate change: term changed from 35 to 37. New cstate: current_term: 37 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.416347 20800 tablet_service.cc:1558] Processing DeleteTablet for tablet f15b5f1ac1444b63bad60a89fa9f8a29 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 18)) from {username='slave'} at 127.0.0.1:34158
W20260709 12:27:04.417472 20624 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 301) took 21962 ms (client timeout 15000 ms). Trace:
W20260709 12:27:04.418104 20624 rpcz_store.cc:269] 0709 12:26:42.462775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.241307 (+6778532us) service_pool.cc:224] Handling call
0709 12:26:49.716419 (+475112us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b5400002490. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7b0b2 00007f9e24af4759 00007f9e24a8ca76 00007f9e24a5dd58 00007f9e24b5c04e 00007f9e2301d2a4 00007f9e2301d232 00007f9e2301d161 00007f9e2301d0dc
0709 12:27:04.186480 (+14470061us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.186508 (+    28us) write_op.cc:454] Released schema lock
0709 12:27:04.417347 (+230839us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:04.170558 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:04.171178 (+   620us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:04.171251 (+    73us) write_op.cc:435] Acquired schema lock
0709 12:27:04.171271 (+    20us) tablet.cc:662] Decoding operations
0709 12:27:04.171939 (+   668us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:04.172010 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:04.172046 (+    36us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:27:04.172611 (+   565us) tablet.cc:701] Row locks acquired
0709 12:27:04.172625 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:27:04.172779 (+   154us) write_op.cc:270] Start()
0709 12:27:04.172898 (+   119us) write_op.cc:276] Timestamp: P: 1783600024172729 usec, L: 0
0709 12:27:04.172917 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:27:04.173599 (+   682us) log.cc:844] Serialized 2069 byte log entry
0709 12:27:04.176032 (+  2433us) op_driver.cc:464] REPLICATION: finished
0709 12:27:04.176462 (+   430us) write_op.cc:301] APPLY: starting
0709 12:27:04.176548 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:04.179905 (+  3357us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:04.180035 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:04.184707 (+  4672us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:04.184885 (+   178us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:27:04.184926 (+    41us) write_op.cc:312] APPLY: finished
0709 12:27:04.185160 (+   234us) log.cc:844] Serialized 57 byte log entry
0709 12:27:04.185932 (+   772us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.186029 (+    97us) write_op.cc:454] Released schema lock
0709 12:27:04.186073 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":49,"spinlock_wait_cycles":5633920,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":7565,"apply.run_wall_time_us":9872,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20900,"num_ops":5,"prepare.queue_time_us":9550,"prepare.run_cpu_time_us":3358,"prepare.run_wall_time_us":12494,"replication_time_us":2980}]]}
I20260709 12:27:04.449421  1374 raft_consensus.cc:493] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:04.449949  1374 raft_consensus.cc:515] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:04.452071  1374 leader_election.cc:290] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:04.452502  1374 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:04.452801 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.454366 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f3454624b7c40c6934257c77f2174de" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:04.452963  1374 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:04.460189 20701 leader_election.cc:304] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:04.461650  1237 raft_consensus.cc:2749] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:04.468256 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:04.472887 20619 catalog_manager.cc:5028] TS 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): tablet f15b5f1ac1444b63bad60a89fa9f8a29 (table test-workload [id=4e3095a8e68a4b08934777e8c3ab461d]) successfully deleted
I20260709 12:27:04.481113 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.491532 20699 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:04.493747  1307 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:04.523864  1374 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:04.535104  1374 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.535853  1374 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:04.556712 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:04.559329 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:04.562345 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 45 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.564107 20699 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:04.565099  1307 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:04.571305  1374 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:04.594800 20849 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 306 similar messages]
I20260709 12:27:04.644317 20627 catalog_manager.cc:5697] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 36 to 38, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 38 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.651810  1307 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:04.662655  1307 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:04.664570  1307 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:04.665292 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.665638 20627 catalog_manager.cc:5697] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 40 to 41, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 41 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.667608 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:04.669324 20701 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:04.670056  1307 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:04.670570 20627 catalog_manager.cc:5697] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 30 to 32, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 32 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.673651 20627 catalog_manager.cc:5697] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 37 to 39, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 39 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.701781 20627 catalog_manager.cc:5697] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 32 to 33, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 33 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:04.717538 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:04.722239 20625 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36964 (request call id 280) took 14865 ms (client timeout 15000 ms). Trace:
I20260709 12:27:04.704339 20627 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 26 to 28, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 49323f66286f4798b268f216bb62b3be (127.19.36.193). New cstate: current_term: 28 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.724020 20627 catalog_manager.cc:5697] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 33 to 36, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 36 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:04.722688 20625 rpcz_store.cc:269] 0709 12:26:49.857186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.233109 (+7375923us) service_pool.cc:224] Handling call
0709 12:27:04.722153 (+7489044us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:04.424683 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:04.425270 (+   587us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:04.425312 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:27:04.425332 (+    20us) tablet.cc:662] Decoding operations
0709 12:27:04.425880 (+   548us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:04.425942 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:04.425976 (+    34us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:27:04.426319 (+   343us) tablet.cc:701] Row locks acquired
0709 12:27:04.426338 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:27:04.426486 (+   148us) write_op.cc:270] Start()
0709 12:27:04.426595 (+   109us) write_op.cc:276] Timestamp: P: 1783600024426438 usec, L: 0
0709 12:27:04.426617 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:27:04.427366 (+   749us) log.cc:844] Serialized 1302 byte log entry
0709 12:27:04.482011 (+ 54645us) op_driver.cc:464] REPLICATION: finished
0709 12:27:04.483177 (+  1166us) write_op.cc:301] APPLY: starting
0709 12:27:04.483280 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:04.485088 (+  1808us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:04.485211 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:04.487050 (+  1839us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:04.489510 (+  2460us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:27:04.489567 (+    57us) write_op.cc:312] APPLY: finished
0709 12:27:04.489807 (+   240us) log.cc:844] Serialized 41 byte log entry
0709 12:27:04.505357 (+ 15550us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.505452 (+    95us) write_op.cc:454] Released schema lock
0709 12:27:04.505496 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:04.515822 (+ 10326us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.515857 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10067,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":936,"apply.run_cpu_time_us":5069,"apply.run_wall_time_us":32593,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12540,"num_ops":3,"prepare.queue_time_us":1006,"prepare.run_cpu_time_us":3042,"prepare.run_wall_time_us":57115,"replication_time_us":55261}]]}
I20260709 12:27:04.755885 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:04.757457 19603 tablet_replica.cc:333] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.758467 19603 raft_consensus.cc:2243] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:27:04.760037 19603 raft_consensus.cc:2272] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:04.762957 19603 tablet_replica.cc:333] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.763870 19603 raft_consensus.cc:2243] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:04.764519 19603 raft_consensus.cc:2272] T 8f3454624b7c40c6934257c77f2174de P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:04.771883 19603 tablet_replica.cc:333] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.772873 19603 raft_consensus.cc:2243] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:27:04.774137 19603 raft_consensus.cc:2272] T c00fffbb02ff4bc2a974d8a5b179a97d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:04.805790 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:04.806051 19603 tablet_replica.cc:333] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.826334  1374 raft_consensus.cc:493] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:04.829322  1374 raft_consensus.cc:515] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:04.832233  1374 leader_election.cc:290] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:04.833391 19603 raft_consensus.cc:2243] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:04.834111 19603 raft_consensus.cc:2272] T 3894130cfc2549b5a8c99d056675ef94 P 49323f66286f4798b268f216bb62b3be [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:04.836972 19603 tablet_replica.cc:333] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.837567 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:04.838685 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210839e60f364f8cab9f53ff440c461d" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.848006 20802 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:04.857285 20627 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36968 (request call id 302) took 4232 ms. Trace:
I20260709 12:27:04.857751 20627 rpcz_store.cc:276] 0709 12:27:00.624674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.242761 (+1618087us) service_pool.cc:224] Handling call
0709 12:27:02.274926 (+ 32165us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b540007dd54. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ec09090 00007f9e1eca7f3a 00007f9e24a5de9c 00007f9e24b5c04e 00007f9e2301d2a4 00007f9e2301d232 00007f9e2301d161 00007f9e2301d0dc 00007f9e2301c392 00007f9e1cfda48c 00007f9e1cfd98c6
0709 12:27:04.857190 (+2582264us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:04.771802 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:04.772493 (+   691us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:04.772535 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:27:04.772557 (+    22us) tablet.cc:662] Decoding operations
0709 12:27:04.773548 (+   991us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:04.773617 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:04.773659 (+    42us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:27:04.774515 (+   856us) tablet.cc:701] Row locks acquired
0709 12:27:04.774539 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:27:04.774705 (+   166us) write_op.cc:270] Start()
0709 12:27:04.774826 (+   121us) write_op.cc:276] Timestamp: P: 1783600024774650 usec, L: 0
0709 12:27:04.774848 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:27:04.775563 (+   715us) log.cc:844] Serialized 2835 byte log entry
0709 12:27:04.777966 (+  2403us) op_driver.cc:464] REPLICATION: finished
0709 12:27:04.778396 (+   430us) write_op.cc:301] APPLY: starting
0709 12:27:04.778479 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:04.782318 (+  3839us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:04.782418 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:04.786822 (+  4404us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:04.787037 (+   215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:27:04.787079 (+    42us) write_op.cc:312] APPLY: finished
0709 12:27:04.787334 (+   255us) log.cc:844] Serialized 73 byte log entry
0709 12:27:04.788065 (+   731us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.788170 (+   105us) write_op.cc:454] Released schema lock
0709 12:27:04.788218 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:04.788529 (+   311us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.788559 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":54032,"reported_tablets":49,"spinlock_wait_cycles":2374528,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":9335,"apply.run_wall_time_us":10094,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29260,"num_ops":7,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":4081,"prepare.run_wall_time_us":11596,"replication_time_us":2990}]]}
I20260709 12:27:04.868394 20628 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 32 to 34. New cstate: current_term: 34 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.872895 20701 leader_election.cc:304] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:04.876421 21432 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:27:04.878718  1373 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:27:04.837761 19603 raft_consensus.cc:2243] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:04.893685  1307 raft_consensus.cc:2749] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:04.894604 20881 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:27:04.896096 19603 raft_consensus.cc:2272] T 87775c62d4bc48d3b1831b4bdeff2870 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:04.900178 19603 tablet_replica.cc:333] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.901015 19603 raft_consensus.cc:2243] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:04.901669 19603 raft_consensus.cc:2272] T 7cd857e30c3240c5a4fdd11bc0602c00 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:04.900715   986 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 12:27:04.907346 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.927728 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:04.932512 19603 tablet_replica.cc:333] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.933481 19603 raft_consensus.cc:2243] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:04.934269 19603 raft_consensus.cc:2272] T a9c17d2fc98f481597ce7b97e8f98c43 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:04.933367   872 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Committing config change with OpId 30.23: config changed from index 20 to 23, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.935606 21426 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Committing config change with OpId 30.23: config changed from index 20 to 23, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:04.943434  1307 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:04.943992  1307 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } attrs { replace: true } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:04.947229 20619 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1047135f71834dbab6b869eb3c081b81 with cas_config_opid_index 20: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:27:04.948614 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:04.904408 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:04.951694 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 47336.2us
I20260709 12:27:04.955422  1307 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:04.956424 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:04.958434 20699 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49323f66286f4798b268f216bb62b3be; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:04.960235 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "49323f66286f4798b268f216bb62b3be" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:04.962716  1237 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:04.967828 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:04.968549 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:04.985721 19603 tablet_replica.cc:333] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:04.986783 19603 raft_consensus.cc:2243] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.000202 20627 catalog_manager.cc:5697] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 32 to 36, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 36 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:05.007477 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.008455 19603 raft_consensus.cc:2272] T d5c70f45aef146f4bcb47f1498221b99 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.012775 20882 raft_consensus.cc:2955] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Committing config change with OpId 30.23: config changed from index 20 to 23, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:05.021013 20627 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 34 to 36. New cstate: current_term: 36 leader_uuid: "49323f66286f4798b268f216bb62b3be" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:05.025511 19603 tablet_replica.cc:333] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.027772 19603 raft_consensus.cc:2243] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.028990 19603 raft_consensus.cc:2272] T 72e60e5ddf96445ab5e284dbeed45c81 P 49323f66286f4798b268f216bb62b3be [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.031936 19603 tablet_replica.cc:333] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.032773 19603 raft_consensus.cc:2243] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.033386 19603 raft_consensus.cc:2272] T 6dc725b7c88e47f693e3a16879798246 P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.035979 20628 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 208) took 8023 ms. Trace:
W20260709 12:27:05.042214 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.050320 19603 tablet_replica.cc:333] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
W20260709 12:27:05.050833 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.050169 20628 rpcz_store.cc:276] 0709 12:26:57.012727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.591257 (+4578530us) service_pool.cc:224] Handling call
0709 12:27:01.612513 (+ 21256us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5400002490. stack: 00007f9e1bb80343 00007f9e1bb80240 00007f9e1b50f1d4 00007f9e2524d304 00007f9e2524d299 00007f9e1ba3adb1 00007f9e1bb7ab71 00007f9e1bb7b0b2 00007f9e24af4759 00007f9e24a8ca76 00007f9e24a5dd58 00007f9e24b5c04e 00007f9e2301d2a4 00007f9e2301d232 00007f9e2301d161 00007f9e2301d0dc
0709 12:27:05.035901 (+3423388us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:04.907290 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:04.907884 (+   594us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:04.907922 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:27:04.907955 (+    33us) tablet.cc:662] Decoding operations
0709 12:27:04.908283 (+   328us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:04.908340 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:04.908375 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:27:04.908559 (+   184us) tablet.cc:701] Row locks acquired
0709 12:27:04.908579 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:27:04.908716 (+   137us) write_op.cc:270] Start()
0709 12:27:04.908835 (+   119us) write_op.cc:276] Timestamp: P: 1783600024908670 usec, L: 0
0709 12:27:04.908857 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:27:04.909486 (+   629us) log.cc:844] Serialized 536 byte log entry
0709 12:27:04.912027 (+  2541us) op_driver.cc:464] REPLICATION: finished
0709 12:27:04.913631 (+  1604us) write_op.cc:301] APPLY: starting
0709 12:27:04.913715 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:04.914526 (+   811us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:04.914617 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:04.915456 (+   839us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:04.915634 (+   178us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:27:04.915688 (+    54us) write_op.cc:312] APPLY: finished
0709 12:27:04.915882 (+   194us) log.cc:844] Serialized 25 byte log entry
0709 12:27:04.916042 (+   160us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:04.916122 (+    80us) write_op.cc:454] Released schema lock
0709 12:27:04.916159 (+    37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":51,"reported_tablets":38,"spinlock_wait_cycles":31620480,"child_traces":[["op",{"apply.queue_time_us":961,"apply.run_cpu_time_us":2816,"apply.run_wall_time_us":2817,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4180,"num_ops":1,"prepare.queue_time_us":13604,"prepare.run_cpu_time_us":2470,"prepare.run_wall_time_us":2469,"replication_time_us":3024}]]}
I20260709 12:27:05.056491 19603 raft_consensus.cc:2243] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.057191 19603 raft_consensus.cc:2272] T 210839e60f364f8cab9f53ff440c461d P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.063292 19603 tablet_replica.cc:333] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.064265 19603 raft_consensus.cc:2243] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.067515 19603 raft_consensus.cc:2272] T 3efcc9a1f14042de975fa0b662cad24d P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.075011 20625 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 20 to 23, VOTER 49323f66286f4798b268f216bb62b3be (127.19.36.193) evicted. New cstate: current_term: 30 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:05.075011 19603 tablet_replica.cc:333] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
W20260709 12:27:05.091449 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.105834 19603 raft_consensus.cc:2243] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.107785 19603 raft_consensus.cc:2272] T c6fe576036c04c149240cd6107285eca P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.110754 19603 tablet_replica.cc:333] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
W20260709 12:27:05.121160 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.122077 19603 raft_consensus.cc:2243] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.124055 19603 raft_consensus.cc:2272] T 0a4e4474d2fd45438665993211ae862a P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:05.125542 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.130064 19603 tablet_replica.cc:333] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.143978 19603 raft_consensus.cc:2243] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.148768 19603 raft_consensus.cc:2272] T b7040eacc899473abc05a913812e82a4 P 49323f66286f4798b268f216bb62b3be [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.153110 19603 tablet_replica.cc:333] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.160564 19603 raft_consensus.cc:2243] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.167625 19603 raft_consensus.cc:2272] T 626209a224524080801f3a9ad7307fbe P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.171680 19603 tablet_replica.cc:333] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.173179 19603 raft_consensus.cc:2243] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.173834 19603 raft_consensus.cc:2272] T 942b8c65bd1e42a99f971b099d0e19e6 P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.176632 19603 tablet_replica.cc:333] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.177431 19603 raft_consensus.cc:2243] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.178052 19603 raft_consensus.cc:2272] T c5ef54d60e4646b68d43519ad3fffa6d P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.181262 19603 tablet_replica.cc:333] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.182255 19603 raft_consensus.cc:2243] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.182967 19603 raft_consensus.cc:2272] T 761ea542665e419c8a63392e3014efa7 P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.185892 19603 tablet_replica.cc:333] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.186736 19603 raft_consensus.cc:2243] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.187520 19603 raft_consensus.cc:2272] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.190127 19603 tablet_replica.cc:333] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.190917 19603 raft_consensus.cc:2243] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.191529 19603 raft_consensus.cc:2272] T 1047135f71834dbab6b869eb3c081b81 P 49323f66286f4798b268f216bb62b3be [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:05.193076 20619 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 70928.3us
I20260709 12:27:05.194470 19603 tablet_replica.cc:333] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.195441 19603 raft_consensus.cc:2243] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:05.195801 20619 catalog_manager.cc:4755] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1047135f71834dbab6b869eb3c081b81: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 12:27:05.196013 19603 pending_rounds.cc:70] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Trying to abort 1 pending ops.
I20260709 12:27:05.196453 19603 pending_rounds.cc:77] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be: Aborting op as it isn't in flight: id { term: 42 index: 25 } timestamp: 7305625666682064896 op_type: NO_OP noop_request { }
I20260709 12:27:05.196900 19603 raft_consensus.cc:2889] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:27:05.197288 19603 raft_consensus.cc:2272] T a32323ce10e140bb895085f97c97fa46 P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.199923 19603 tablet_replica.cc:333] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.200744 19603 raft_consensus.cc:2243] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:05.200989 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.201329 19603 raft_consensus.cc:2272] T 4415902e5b30489691f67f4a635305b4 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.204435 19603 tablet_replica.cc:333] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.205210 19603 raft_consensus.cc:2243] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.205868 19603 raft_consensus.cc:2272] T a48bec319402467881543ac686fb8f0c P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.208563 19603 tablet_replica.cc:333] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.209358 19603 raft_consensus.cc:2243] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.210005 19603 pending_rounds.cc:70] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Trying to abort 1 pending ops.
I20260709 12:27:05.210309 19603 pending_rounds.cc:77] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be: Aborting op as it isn't in flight: id { term: 44 index: 25 } timestamp: 7305625665565089792 op_type: NO_OP noop_request { }
I20260709 12:27:05.210707 19603 raft_consensus.cc:2889] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:27:05.211104 19603 raft_consensus.cc:2272] T 6a458432a57843e4af30e5d9af5dfcfe P 49323f66286f4798b268f216bb62b3be [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.213665 19603 tablet_replica.cc:333] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.214428 19603 raft_consensus.cc:2243] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.215050 19603 raft_consensus.cc:2272] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.217904 19603 tablet_replica.cc:333] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.220415 19603 raft_consensus.cc:2243] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.221128 19603 raft_consensus.cc:2272] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.223852 19603 tablet_replica.cc:333] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.224629 19603 raft_consensus.cc:2243] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.225440 19603 raft_consensus.cc:2272] T e36a8df2d02f44ca965324882aba3ef2 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.228790 19603 tablet_replica.cc:333] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.229575 19603 raft_consensus.cc:2243] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.230361 19603 raft_consensus.cc:2272] T 986267de598f431fb25840d9d741436d P 49323f66286f4798b268f216bb62b3be [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.233026 19603 tablet_replica.cc:333] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.233803 19603 raft_consensus.cc:2243] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.234637 19603 raft_consensus.cc:2272] T 13cfeabebc54415b87ab9ce1d8a4caab P 49323f66286f4798b268f216bb62b3be [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.237358 19603 tablet_replica.cc:333] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.238112 19603 raft_consensus.cc:2243] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.239179 19603 raft_consensus.cc:2272] T be08f375c22d436fa6e99ac7bd7f6c93 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.241918 19603 tablet_replica.cc:333] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.242705 19603 raft_consensus.cc:2243] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.243796 19603 raft_consensus.cc:2272] T c1303aa66bdb4474b6bcb604461e74a5 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.246369 19603 tablet_replica.cc:333] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.247129 19603 raft_consensus.cc:2243] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.247711 19603 raft_consensus.cc:2272] T f15b5f1ac1444b63bad60a89fa9f8a29 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.250133 19603 tablet_replica.cc:333] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.250856 19603 raft_consensus.cc:2243] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.251524 19603 raft_consensus.cc:2272] T d5374e2302374917828294b0a4e3313a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.254312 19603 tablet_replica.cc:333] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.255059 19603 raft_consensus.cc:2243] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.256153 19603 raft_consensus.cc:2272] T 70cdcdfe95804895979397c927fae46f P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.258756 19603 tablet_replica.cc:333] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.259577 19603 raft_consensus.cc:2243] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.260224 19603 raft_consensus.cc:2272] T ec190d02960a4e88af0807817b789e6c P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.262741 19603 tablet_replica.cc:333] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.263506 19603 raft_consensus.cc:2243] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 36 LEADER]: Raft consensus shutting down.
W20260709 12:27:05.265491 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.266430 19603 raft_consensus.cc:2272] T 75e78bb14a7646c58f2f948b2d33b2cf P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:05.276054 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:05.281499 19603 tablet_replica.cc:333] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.282399 19603 raft_consensus.cc:2243] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.283077 19603 raft_consensus.cc:2272] T 37c593b286274096a8c431979866be8a P 49323f66286f4798b268f216bb62b3be [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.285902 19603 tablet_replica.cc:333] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.286680 19603 raft_consensus.cc:2243] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.287823 19603 raft_consensus.cc:2272] T f61efc9c6d5f43ac84cfe1def41de8a0 P 49323f66286f4798b268f216bb62b3be [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.290915 19603 tablet_replica.cc:333] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.291704 19603 raft_consensus.cc:2243] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 42 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.292799 19603 raft_consensus.cc:2272] T 26ced453445642b09bda798a2963748b P 49323f66286f4798b268f216bb62b3be [term 42 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.295249 19603 tablet_replica.cc:333] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.295966 19603 raft_consensus.cc:2243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.296993 19603 raft_consensus.cc:2272] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 49323f66286f4798b268f216bb62b3be [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.299731 19603 tablet_replica.cc:333] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.300459 19603 raft_consensus.cc:2243] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.301254 19603 raft_consensus.cc:2272] T 3199c6fc2e3441849ac7dd78244a3a18 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.305074 19603 tablet_replica.cc:333] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.306152 19603 raft_consensus.cc:2243] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.307300 19603 raft_consensus.cc:2272] T 0f517f5d84b3437eb2fd2ae144fc6380 P 49323f66286f4798b268f216bb62b3be [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.315918 19603 tablet_replica.cc:333] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.316639 19603 raft_consensus.cc:2243] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:05.317910 19603 raft_consensus.cc:2272] T 62396122903848d4a49e6d637c5c8b49 P 49323f66286f4798b268f216bb62b3be [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.320811 19603 tablet_replica.cc:333] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.321532 19603 raft_consensus.cc:2243] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.322314 19603 raft_consensus.cc:2272] T ff0a54292b924069aae5a0cd6c1366d1 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.325536 19603 tablet_replica.cc:333] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.326217 19603 raft_consensus.cc:2243] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.326839 19603 raft_consensus.cc:2272] T d82740a2c094456a91cf0e411dd72159 P 49323f66286f4798b268f216bb62b3be [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:05.330034 19603 tablet_replica.cc:333] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be: stopping tablet replica
I20260709 12:27:05.334704 19603 raft_consensus.cc:2243] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:05.347711 19603 raft_consensus.cc:2272] T 38d64dd1339c4abfb4ef05669402e801 P 49323f66286f4798b268f216bb62b3be [term 34 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:05.422611 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:05.435868 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:05.450043 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:05.579420 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:05.628762 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.011909 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.012439 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.012820 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.107342 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.132195  1383 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.132968  1383 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.133823  1396 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.136979  1396 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:27:06.139387 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.139652 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.142016  1396 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:06.143476 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.146065 21400 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.150453 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.149811 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.151243  1391 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.151921  1391 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.146580  1383 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.154150  1391 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.155712 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.158949  1383 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.159729  1395 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:06.159571  1383 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.162787 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.163292 20775 leader_election.cc:336] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.163698 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:06.164342  1391 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:06.164400 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:06.169544 21400 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.171344 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:06.161846  1383 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:06.180115 20849 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.181263  1395 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:06.183251 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:06.189342  1396 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:27:06.211954  1396 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.212625  1396 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.214908  1396 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:06.221089 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 20 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.226662  1391 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.227205  1391 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.229266  1391 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.231943 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.232604 20878 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:27:06.233803 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:06.234668  1391 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:27:06.235029  1391 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:27:06.235154 20775 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.235401  1391 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:27:06.240881  1383 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:27:06.241652 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.241443  1383 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.244808 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.245009  1391 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.247519  1383 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:06.248615 21400 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.249135 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.250749  1383 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:06.250113 20849 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.251680 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:06.254853  1394 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:27:06.263305  1396 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.263871  1396 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.267719 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.268362 21435 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
W20260709 12:27:06.269882 20775 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.269990 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:06.270337 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.270673 20881 raft_consensus.cc:3060] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:27:06.271334  1396 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.291009 20881 raft_consensus.cc:2468] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 41.
W20260709 12:27:06.292045 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.300994  1396 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:27:06.301542  1396 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.301923  1396 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:06.308626 20774 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.309576  1392 raft_consensus.cc:2804] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:27:06.310130  1392 raft_consensus.cc:697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:06.310973  1392 consensus_queue.cc:237] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 40.20, Last appended by leader: 20, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.320878  1396 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.321791  1383 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.332729  1383 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.334939  1383 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:06.324322  1391 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.338958 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.343870 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:06.344630 21435 raft_consensus.cc:3060] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:27:06.347055 20849 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.348420  1396 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:06.353060 21400 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.353528 20628 catalog_manager.cc:5697] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 40 to 41, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 41 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:06.354827 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:06.353747  1392 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.355795  1392 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.358904  1383 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:06.359026 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.359812 20881 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
I20260709 12:27:06.360296  1392 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.360811 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:27:06.361027 20849 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.361650   872 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:27:06.362027   872 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.362380   872 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:27:06.362764 20775 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.370539   872 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.372247 21435 raft_consensus.cc:2468] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:27:06.373500 20850 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.374585  1396 raft_consensus.cc:2804] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:27:06.375054  1396 raft_consensus.cc:697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 40 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:06.375917  1396 consensus_queue.cc:237] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 39.24, Last appended by leader: 24, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.360813  1392 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.382474  1392 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.384792  1392 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.387471 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 38 candidate_status { last_received { term: 37 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:06.388212 20878 raft_consensus.cc:3060] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:27:06.395310 20775 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:06.395915 20775 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.396574 20878 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:27:06.397949 20774 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.398826  1392 raft_consensus.cc:2804] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:27:06.399639  1392 raft_consensus.cc:697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:06.400931  1392 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 37.21, Last appended by leader: 21, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
W20260709 12:27:06.402529 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.399628 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.404568 21435 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:27:06.405853 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.406734  1391 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:06.407639   872 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 38 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.407121  1391 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.414651  1391 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:06.420053   872 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.420595   872 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.422569   872 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.423882 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.435540 20625 catalog_manager.cc:5697] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 39 to 40, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 40 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:06.442560 20775 leader_election.cc:336] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.443087 20775 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.444192  1396 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.444763  1396 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.446272  1373 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:27:06.448873 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.449353 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.455673  1378 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.456234  1378 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.460619 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.461236 21426 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:27:06.463203  1396 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.463526 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.464581  1396 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:27:06.464947  1396 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.465435  1396 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:27:06.458971  1378 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.464581  1391 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.466821 20849 leader_election.cc:336] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.469132 20849 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:06.471645  1384 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:06.480017 20849 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.485134  1391 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:06.491153 20775 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.492375 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:06.493079 21426 raft_consensus.cc:3060] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:06.506242 21426 raft_consensus.cc:2468] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 33.
I20260709 12:27:06.507596 20776 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.509438  1391 raft_consensus.cc:2804] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:27:06.518751  1391 raft_consensus.cc:697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:06.521919  1391 consensus_queue.cc:237] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 32.25, Last appended by leader: 25, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.529534  1396 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.532716  1396 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.533198 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:06.533820 21426 raft_consensus.cc:3060] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:27:06.537578 20628 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 37 to 38, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 38 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:06.540784 21426 raft_consensus.cc:2468] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:27:06.542063 20850 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.543239  1396 raft_consensus.cc:2804] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:27:06.544628  1396 raft_consensus.cc:697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:06.545903  1396 consensus_queue.cc:237] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 34.26, Last appended by leader: 26, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.562374  1383 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.562975  1383 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.567065   810 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.567595   810 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.569495   810 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.570371  1378 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.570914  1378 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.571555 20849 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.572969  1378 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:06.574641 20619 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 49031us
I20260709 12:27:06.574994 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.575642 20878 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 42.
I20260709 12:27:06.576864 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.577900  1395 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader pre-election won for term 43
I20260709 12:27:06.578552  1395 raft_consensus.cc:493] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:27:06.577832 20849 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.578974  1395 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:06.579509  1384 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.580098  1384 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.582424  1384 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:06.574641 21400 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.585923 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.591773 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.592451 20801 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:27:06.592801  1392 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.593470  1392 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.595079 21400 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.596235 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:06.598133  1384 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:27:06.598551  1384 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.598942  1384 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:06.599275  1383 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.600072  1402 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.600773  1402 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.603989 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.602938  1402 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.608405 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.613649 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:06.615005  1383 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:06.617007  1392 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:06.617905 20775 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:06.618682 20775 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.619241 20775 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:27:06.611320 20849 leader_election.cc:336] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.620100  1392 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:06.577803 20619 catalog_manager.cc:4755] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1047135f71834dbab6b869eb3c081b81: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:06.622890 20619 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45117.2us
I20260709 12:27:06.623512  1395 raft_consensus.cc:515] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.625312 21426 raft_consensus.cc:2393] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 43: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:27:06.607923  1384 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.627828 20850 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:06.628855  1378 raft_consensus.cc:3060] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:06.637753  1378 raft_consensus.cc:2749] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
W20260709 12:27:06.650830 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59481.7us
I20260709 12:27:06.641434  1384 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:06.654403   810 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.654096 20628 catalog_manager.cc:5697] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 32 to 33, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 33 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:06.655018   810 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.657251   810 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:06.658690 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.643908  1395 leader_election.cc:290] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.663625 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ced453445642b09bda798a2963748b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:06.667783 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:06.668445 20801 raft_consensus.cc:3060] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:06.671593 20881 raft_consensus.cc:2468] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 43.
I20260709 12:27:06.675339 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.675990 20802 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 28.
I20260709 12:27:06.679425 20801 raft_consensus.cc:2468] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 39.
I20260709 12:27:06.681222 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.681850 21426 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 36.
I20260709 12:27:06.688263 20849 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:06.689046  1384 raft_consensus.cc:2804] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:27:06.689572  1384 raft_consensus.cc:697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:06.690407  1384 consensus_queue.cc:237] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 38.22, Last appended by leader: 22, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.692317 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:27:06.692647 20849 leader_election.cc:336] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.693413  1391 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:27:06.693804  1391 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.694172  1391 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:27:06.696578 20628 catalog_manager.cc:5697] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 34 to 35, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:06.705338 20775 consensus_peers.cc:597] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:06.707356  1391 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.713277  1378 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.713868  1378 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.716427 20775 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.717002  1378 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.722230  1409 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.722879  1409 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.725008 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.725670 20802 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:27:06.726092  1402 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
W20260709 12:27:06.725682 19603 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 20756)
I20260709 12:27:06.728946  1402 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.730007 20849 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.730031  1378 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.730907  1378 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.733227  1378 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.728685  1409 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.734262  1378 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.734825  1378 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.736578  1402 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.737167  1378 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:06.742172  1395 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.743336  1391 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.737838 21399 leader_election.cc:304] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.740773 20878 raft_consensus.cc:1275] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 40 index: 20. Preceding OpId from leader: term: 41 index: 21. (index mismatch)
I20260709 12:27:06.741111 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:06.749210 20802 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
I20260709 12:27:06.737699 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.751176  1410 raft_consensus.cc:2804] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader election won for term 43
I20260709 12:27:06.751793  1410 raft_consensus.cc:697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 43 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
W20260709 12:27:06.753989 20849 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.752667  1410 consensus_queue.cc:237] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 42.27, Last appended by leader: 27, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.759088 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.758838 20628 catalog_manager.cc:5697] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 38 to 39, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 39 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.759815 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
I20260709 12:27:06.750720 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:27:06.761410  1402 consensus_queue.cc:1048] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:27:06.744179 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:06.742779  1395 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.763345 21400 leader_election.cc:336] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 43 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.763337  1409 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:06.765219  1395 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:06.746939 20775 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.766557  1409 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.767000  1409 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:06.767030 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:06.767791 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:06.768343 21426 raft_consensus.cc:3060] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:27:06.765565 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.769683  1384 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:27:06.769848 21435 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 38.
I20260709 12:27:06.770316  1384 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.770763  1384 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:27:06.775315 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:06.776057 20878 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:27:06.777657 20776 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
W20260709 12:27:06.778481 20849 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.783715  1402 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:27:06.784188  1402 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.784592  1402 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:06.797070 21426 raft_consensus.cc:2468] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 37.
W20260709 12:27:06.789074 21400 leader_election.cc:336] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.792970 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.806607  1395 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:27:06.807044  1395 raft_consensus.cc:493] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.807428  1395 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:27:06.824815 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.830633   810 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:27:06.831164   810 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 49323f66286f4798b268f216bb62b3be)
I20260709 12:27:06.831633   810 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:06.832930  1402 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:06.840212 20775 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.843868  1384 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.846254  1384 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:06.848639  1395 raft_consensus.cc:515] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.849246 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:06.849799 20802 raft_consensus.cc:3060] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:27:06.851781 20849 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.852885   810 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.853240 20776 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.854756   810 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.856571  1373 raft_consensus.cc:2804] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:27:06.857147  1373 raft_consensus.cc:697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:06.857803 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.858590 21434 raft_consensus.cc:2393] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 41: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:27:06.859957 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:06.857996  1373 consensus_queue.cc:237] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 36.23, Last appended by leader: 23, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.860527 21432 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:06.862003 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:06.862512 20628 catalog_manager.cc:5697] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 42 to 43, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 43 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.867388 21432 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:27:06.848641  1409 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.868809  1395 leader_election.cc:290] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:06.871775  1409 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:06.873241 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8b742e1fe24e6d82f1a1498f17a5fb" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:06.873939 20878 raft_consensus.cc:3060] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:27:06.848320  1402 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:06.877466 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:06.880064 20881 raft_consensus.cc:3060] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:06.885622 20878 raft_consensus.cc:2468] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 41.
I20260709 12:27:06.887338 20881 raft_consensus.cc:2468] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:27:06.887925 20802 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 35.
I20260709 12:27:06.889843 21399 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.893807 20849 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.895099  1384 raft_consensus.cc:2804] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:27:06.896762 20850 leader_election.cc:304] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:06.898664 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:06.899143 21400 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.899339  1396 raft_consensus.cc:2749] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
W20260709 12:27:06.900283 21400 leader_election.cc:336] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:06.901177 20775 consensus_peers.cc:597] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:06.903072  1384 raft_consensus.cc:697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:06.904265  1384 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:06.907512 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:06.908088 20802 raft_consensus.cc:3060] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:06.908034 21399 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:06.909394  1377 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:06.911170  1409 raft_consensus.cc:2804] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader election won for term 41
W20260709 12:27:06.912706 21400 leader_election.cc:336] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:06.912818  1395 raft_consensus.cc:2804] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election won for term 34
W20260709 12:27:06.913587 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 88534.4us
I20260709 12:27:06.927219  1395 raft_consensus.cc:697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:06.927405  1409 raft_consensus.cc:697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:06.929214  1395 consensus_queue.cc:237] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.929256  1409 consensus_queue.cc:237] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 40.26, Last appended by leader: 26, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:06.930081 20802 raft_consensus.cc:2468] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 29.
W20260709 12:27:06.941635 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:06.941845 20628 catalog_manager.cc:5697] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 36 to 37, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 37 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:06.943451 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9d40 after lost signal to thread 19606
W20260709 12:27:06.944633 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800350740 after lost signal to thread 20667
W20260709 12:27:06.946075 20664 debug-util.cc:398] Leaking SignalData structure 0x7b0800294780 after lost signal to thread 20834
W20260709 12:27:06.923934 21400 leader_election.cc:336] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:06.951653 21400 leader_election.cc:336] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:06.955382 20664 debug-util.cc:398] Leaking SignalData structure 0x7b08003a43c0 after lost signal to thread 20908
W20260709 12:27:06.956720 20664 debug-util.cc:398] Leaking SignalData structure 0x7b080035adc0 after lost signal to thread 21459
I20260709 12:27:06.952952 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:06.968357 20881 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:06.974826 21426 raft_consensus.cc:1275] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 24. Preceding OpId from leader: term: 40 index: 25. (index mismatch)
I20260709 12:27:06.979132 20628 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 34 to 35, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:06.981860 20775 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:06.982725 21400 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:06.986474  1383 raft_consensus.cc:2804] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:06.987061  1383 raft_consensus.cc:697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:06.987968  1383 consensus_queue.cc:237] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:07.000851 21426 raft_consensus.cc:1275] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 32 index: 25. Preceding OpId from leader: term: 33 index: 26. (index mismatch)
I20260709 12:27:07.003166 20882 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 37 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
I20260709 12:27:07.003115  1391 consensus_queue.cc:1048] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:27:07.005528  1414 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:27:07.031008 20881 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:27:07.026187  1377 consensus_queue.cc:1048] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.010s
I20260709 12:27:07.034590 20774 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:07.047251  1414 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
W20260709 12:27:07.053694 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:07.060027  1414 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:07.061086  1414 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 38.25, Last appended by leader: 25, Current term: 39, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:07.087497 21435 raft_consensus.cc:1275] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 26. Preceding OpId from leader: term: 35 index: 27. (index mismatch)
I20260709 12:27:07.114444  1378 consensus_queue.cc:1048] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.001s
I20260709 12:27:07.119680 20628 catalog_manager.cc:5697] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 33 to 34, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 34 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:07.126142 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.132553 20628 catalog_manager.cc:5697] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 40 to 41, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 41 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:07.162047 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 62503.5us
W20260709 12:27:07.190623 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 99744.8us
I20260709 12:27:07.196477 20802 raft_consensus.cc:1275] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 38 index: 22. Preceding OpId from leader: term: 39 index: 23. (index mismatch)
I20260709 12:27:07.220894  1427 consensus_queue.cc:1048] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
W20260709 12:27:07.247327 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:07.258751 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:07.259809 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:07.262406 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.248726 20625 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 38 to 39, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 39 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:07.285701 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:07.295912 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:07.303330 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:07.302839 20628 catalog_manager.cc:5697] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 28 to 29, leader changed from 49323f66286f4798b268f216bb62b3be (127.19.36.193) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 29 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:07.312394 20882 raft_consensus.cc:1275] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 42 index: 27. Preceding OpId from leader: term: 43 index: 28. (index mismatch)
I20260709 12:27:07.313609 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34158
I20260709 12:27:07.314079  1395 consensus_queue.cc:1048] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20260709 12:27:07.319093 21426 raft_consensus.cc:1275] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 36 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:27:07.320705  1373 consensus_queue.cc:1048] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:27:07.322024 20802 raft_consensus.cc:606] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Received request to transfer leadership
I20260709 12:27:07.327713 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34158
I20260709 12:27:07.328317 20802 raft_consensus.cc:606] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Received request to transfer leadership
W20260709 12:27:07.342844 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.374316 20775 consensus_peers.cc:597] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:07.375325 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.385522 20881 raft_consensus.cc:1275] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:27:07.387144  1410 consensus_queue.cc:1048] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:27:07.414379 20802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "657bc62d20374e539b23641715f22745"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34158
I20260709 12:27:07.414937 20802 raft_consensus.cc:606] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Received request to transfer leadership
W20260709 12:27:07.449438 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:07.506531 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:07.513044 20802 raft_consensus.cc:1275] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:27:07.517899 20881 raft_consensus.cc:1275] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 40 index: 26. Preceding OpId from leader: term: 41 index: 27. (index mismatch)
I20260709 12:27:07.521080  1415 consensus_queue.cc:1048] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
W20260709 12:27:07.531240 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:07.539716 20775 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:07.562443 20807 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:27:07.564751  1409 consensus_queue.cc:1048] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:27:07.567013 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:07.582608 21426 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 25. Preceding OpId from leader: term: 39 index: 26. (index mismatch)
W20260709 12:27:07.587615 20681 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:27:07.588096 20681 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 62396122903848d4a49e6d637c5c8b49 already in progress
W20260709 12:27:07.588394 20681 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 62396122903848d4a49e6d637c5c8b49 already in progress
I20260709 12:27:07.588554  1414 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:27:07.610755  1426 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:27:07.625684  1391 raft_consensus.cc:993] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: : Instructing follower 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:27:07.626339  1402 raft_consensus.cc:1081] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Signalling peer 50c1ab13351d4427b836fbb449c6d7bb to start an election
I20260709 12:27:07.628165 21432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49"
dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
 from {username='slave'} at 127.0.0.1:44460
I20260709 12:27:07.629120 21432 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:07.629539 21432 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:07.663479 21432 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting forced leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:07.669551 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.671514 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: true dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:07.672134 20878 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:07.680365 20878 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
W20260709 12:27:07.682587 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.683076 21432 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Requested vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:07.690749 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.714108 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: true dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:07.714732 20801 raft_consensus.cc:3055] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:27:07.715056 20801 raft_consensus.cc:740] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:07.715844 20801 consensus_queue.cc:260] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 39.26, Last appended by leader: 26, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:07.718361 20882 raft_consensus.cc:1240] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:27:07.721516  1402 consensus_queue.cc:1059] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
W20260709 12:27:07.727138 19603 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:07.727769 21400 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:07.738461 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.739009 20801 raft_consensus.cc:3060] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:27:07.760803 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:07.765693 20801 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
W20260709 12:27:07.768325 20619 catalog_manager.cc:4755] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1047135f71834dbab6b869eb3c081b81: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:07.775111 21400 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be
I20260709 12:27:07.780054  1395 raft_consensus.cc:2804] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:27:07.780618  1395 raft_consensus.cc:697] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:07.781577  1395 consensus_queue.cc:237] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 39.26, Last appended by leader: 26, Current term: 40, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:07.782647 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:07.792162 20619 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62396122903848d4a49e6d637c5c8b49 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER failed with leader 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 657bc62d20374e539b23641715f22745 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 40 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:07.803838 20625 catalog_manager.cc:5697] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 39 to 40, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 40 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:07.814440 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:07.836025 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:07.972357 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.004868 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.044543 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.061870 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:08.178360 20882 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 38 index: 25. Preceding OpId from leader: term: 40 index: 27. (index mismatch)
I20260709 12:27:08.183141  1409 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:27:08.270082 20801 raft_consensus.cc:1275] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 39 index: 26. Preceding OpId from leader: term: 40 index: 27. (index mismatch)
I20260709 12:27:08.275791  1448 consensus_queue.cc:1048] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 25, Time since last communication: 0.001s
W20260709 12:27:08.284540 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.305333 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 61218.7us
W20260709 12:27:08.339205 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.419728 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:08.507387 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.614033 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.676582 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.677238 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.684906 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.728571 19603 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:08.789541 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.810357 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.841624 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.884289 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:08.908191 20882 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 20, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:27:08.926833 20850 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae342ffbd6894fb1aa23aa48f4d2dd1b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:08.927335 20801 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:27:08.931387  1426 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:27:08.934723 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:09.003474 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 59815.8us
I20260709 12:27:09.017110  1461 raft_consensus.cc:2955] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Committing config change with OpId 35.22: config changed from index -1 to 22, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.029281 20807 raft_consensus.cc:2955] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Committing config change with OpId 35.22: config changed from index -1 to 22, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.052474 20801 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 34.21, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:09.068056 20625 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 22, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:09.078140 20882 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:27:09.079890  1463 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:27:09.085227 20776 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0b73d72c3ae4b17abbf64b777ddb8c2. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:09.102662 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:09.120201  1373 raft_consensus.cc:2955] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Committing config change with OpId 34.22: config changed from index -1 to 22, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.129851 20878 raft_consensus.cc:2955] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Committing config change with OpId 34.22: config changed from index -1 to 22, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:27:09.162230 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:09.212917 20801 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 38.22, Last appended by leader: 21, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:09.234171 20878 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 22. Preceding OpId from leader: term: 38 index: 23. (index mismatch)
W20260709 12:27:09.234964 20776 consensus_peers.cc:597] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72e60e5ddf96445ab5e284dbeed45c81. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:09.236773  1462 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 12:27:09.248047 20775 consensus_peers.cc:597] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:09.268260  1462 raft_consensus.cc:2955] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Committing config change with OpId 38.23: config changed from index -1 to 23, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:27:09.273566 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:09.280110 20878 raft_consensus.cc:2955] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Committing config change with OpId 38.23: config changed from index -1 to 23, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.289029 20627 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 22, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 34 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.363857 20627 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index -1 to 23, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 38 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.372929 20878 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:27:09.383512 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:09.401993  1472 ts_tablet_manager.cc:933] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:09.411073 20775 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:09.412302 20850 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): Couldn't send request to peer 50c1ab13351d4427b836fbb449c6d7bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cdcdfe95804895979397c927fae46f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:09.424130 20802 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 35 index: 25. (index mismatch)
I20260709 12:27:09.425771  1426 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
W20260709 12:27:09.432862 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:09.435660 20775 consensus_peers.cc:597] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:09.448813  1472 tablet_copy_client.cc:323] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
I20260709 12:27:09.450947 20893 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:27:09.451663 20893 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet ae342ffbd6894fb1aa23aa48f4d2dd1b from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-ae342ffbd6894fb1aa23aa48f4d2dd1b
I20260709 12:27:09.462297 20893 tablet_copy_source_session.cc:215] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:09.465272  1426 raft_consensus.cc:2955] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Committing config change with OpId 35.25: config changed from index -1 to 25, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.468302 20801 raft_consensus.cc:2955] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Committing config change with OpId 35.25: config changed from index -1 to 25, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New config: { opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
W20260709 12:27:09.504944 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:09.523226  1472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae342ffbd6894fb1aa23aa48f4d2dd1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:09.538590 20627 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index -1 to 25, NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) added. New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } } }
I20260709 12:27:09.549440  1472 tablet_copy_client.cc:806] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:09.552927  1472 tablet_copy_client.cc:670] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:09.560633  1472 tablet_copy_client.cc:538] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:09.579653 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:09.592514  1472 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:27:09.611912  1476 ts_tablet_manager.cc:933] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:09.626355  1476 tablet_copy_client.cc:323] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
W20260709 12:27:09.630851 20775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111) [suppressed 442 similar messages]
I20260709 12:27:09.636852 20819 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet c0b73d72c3ae4b17abbf64b777ddb8c2 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:27:09.637439 20819 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet c0b73d72c3ae4b17abbf64b777ddb8c2 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-c0b73d72c3ae4b17abbf64b777ddb8c2
W20260709 12:27:09.653589 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:09.660483 20819 tablet_copy_source_session.cc:215] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:27:09.667516 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:09.686992  1476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b73d72c3ae4b17abbf64b777ddb8c2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:09.695555 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:09.729238 19603 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:09.791070 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:09.810843  1472 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=4 applied=22 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:09.815594  1479 ts_tablet_manager.cc:933] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:09.816990  1476 tablet_copy_client.cc:806] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:09.820698  1476 tablet_copy_client.cc:670] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:09.822255  1472 tablet_bootstrap.cc:492] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:27:09.823092  1472 ts_tablet_manager.cc:1403] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.231s	user 0.062s	sys 0.011s
I20260709 12:27:09.837621  1472 raft_consensus.cc:359] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:27:09.839706 20775 consensus_peers.cc:597] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:09.839022  1472 raft_consensus.cc:740] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:27:09.842756  1472 consensus_queue.cc:260] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 22, Last appended: 35.22, Last appended by leader: 22, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
W20260709 12:27:09.852248 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:09.839864  1476 tablet_copy_client.cc:538] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:09.860280 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:09.866281  1479 tablet_copy_client.cc:323] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.194:43085
W20260709 12:27:09.868834 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:09.868956 20819 tablet_copy_service.cc:140] P 657bc62d20374e539b23641715f22745: Received BeginTabletCopySession request for tablet 72e60e5ddf96445ab5e284dbeed45c81 from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:43456)
I20260709 12:27:09.869570 20819 tablet_copy_service.cc:161] P 657bc62d20374e539b23641715f22745: Beginning new tablet copy session on tablet 72e60e5ddf96445ab5e284dbeed45c81 from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:43456: session id = 50c1ab13351d4427b836fbb449c6d7bb-72e60e5ddf96445ab5e284dbeed45c81
I20260709 12:27:09.923333 21426 raft_consensus.cc:1217] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Deduplicated request from leader. Original: 35.21->[35.22-35.22]   Dedup: 35.22->[]
I20260709 12:27:09.931452  1472 ts_tablet_manager.cc:1434] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.003s
I20260709 12:27:09.932969  1476 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
I20260709 12:27:09.941016 20893 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ae342ffbd6894fb1aa23aa48f4d2dd1b received from {username='slave'} at 127.0.0.1:53814
I20260709 12:27:09.941588 20893 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-ae342ffbd6894fb1aa23aa48f4d2dd1b on tablet ae342ffbd6894fb1aa23aa48f4d2dd1b with peer 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:09.943363 20819 tablet_copy_source_session.cc:215] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:09.955448  1472 ts_tablet_manager.cc:933] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: Initiating tablet copy from peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:09.972671  1479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e60e5ddf96445ab5e284dbeed45c81. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:09.994081 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.000125  1472 tablet_copy_client.cc:323] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Beginning tablet copy session from remote peer at address 127.19.36.195:45387
W20260709 12:27:10.000638 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.001893 20893 tablet_copy_service.cc:140] P 116023b4c0c94bfea737fc238bda3838: Received BeginTabletCopySession request for tablet 70cdcdfe95804895979397c927fae46f from peer 50c1ab13351d4427b836fbb449c6d7bb ({username='slave'} at 127.0.0.1:53814)
I20260709 12:27:10.002504 20893 tablet_copy_service.cc:161] P 116023b4c0c94bfea737fc238bda3838: Beginning new tablet copy session on tablet 70cdcdfe95804895979397c927fae46f from peer 50c1ab13351d4427b836fbb449c6d7bb at {username='slave'} at 127.0.0.1:53814: session id = 50c1ab13351d4427b836fbb449c6d7bb-70cdcdfe95804895979397c927fae46f
I20260709 12:27:10.009837  1479 tablet_copy_client.cc:806] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:10.010636  1479 tablet_copy_client.cc:670] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:10.011063 20893 tablet_copy_source_session.cc:215] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:10.014467  1472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70cdcdfe95804895979397c927fae46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:10.014714  1479 tablet_copy_client.cc:538] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:10.019047 20619 catalog_manager.cc:4755] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1047135f71834dbab6b869eb3c081b81: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:10.019903 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:10.032853 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.038690  1479 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
W20260709 12:27:10.044140 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:10.064215 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:10.099725  1472 tablet_copy_client.cc:806] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:10.100956  1472 tablet_copy_client.cc:670] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:10.134892  1472 tablet_copy_client.cc:538] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:10.223875 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 51564.5us
W20260709 12:27:10.231629 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:10.295882  1472 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap starting.
W20260709 12:27:10.399538 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 87374.6us
W20260709 12:27:10.454758 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:10.493639 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 321203us
W20260709 12:27:10.513058 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45525.3us
W20260709 12:27:10.540119 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:10.544495 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.613503  1476 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=2 applied=22 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:10.615658  1476 tablet_bootstrap.cc:492] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:27:10.616387  1476 ts_tablet_manager.cc:1403] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.684s	user 0.062s	sys 0.017s
W20260709 12:27:10.626893 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.648109  1476 raft_consensus.cc:359] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:10.649127  1476 raft_consensus.cc:740] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
W20260709 12:27:10.643930 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.649819  1476 consensus_queue.cc:260] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:10.664783  1476 ts_tablet_manager.cc:1434] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.003s
I20260709 12:27:10.669027 20819 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-c0b73d72c3ae4b17abbf64b777ddb8c2 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:27:10.669613 20819 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-c0b73d72c3ae4b17abbf64b777ddb8c2 on tablet c0b73d72c3ae4b17abbf64b777ddb8c2 with peer 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:10.695219 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.701152  1426 raft_consensus.cc:1064] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
W20260709 12:27:10.729910 19603 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 20756)
I20260709 12:27:10.756618 21432 raft_consensus.cc:1217] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEARNER]: Deduplicated request from leader. Original: 34.21->[34.22-34.22]   Dedup: 34.22->[]
I20260709 12:27:10.768316  1426 consensus_queue.cc:237] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.22, Last appended by leader: 20, Current term: 35, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:10.808393 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.808706  1479 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=3 applied=23 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:10.810016  1479 tablet_bootstrap.cc:492] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:27:10.810774  1479 ts_tablet_manager.cc:1403] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.772s	user 0.067s	sys 0.018s
W20260709 12:27:10.818152 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:10.826632 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:10.851042 21435 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:27:10.854297  1426 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:27:10.854303  1479 raft_consensus.cc:359] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:10.857995  1479 raft_consensus.cc:740] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:27:10.858772  1479 consensus_queue.cc:260] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 38.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:10.862185  1479 ts_tablet_manager.cc:1434] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 12:27:10.915704 20819 tablet_copy_service.cc:342] P 657bc62d20374e539b23641715f22745: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-72e60e5ddf96445ab5e284dbeed45c81 received from {username='slave'} at 127.0.0.1:43456
I20260709 12:27:10.916347 20819 tablet_copy_service.cc:434] P 657bc62d20374e539b23641715f22745: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-72e60e5ddf96445ab5e284dbeed45c81 on tablet 72e60e5ddf96445ab5e284dbeed45c81 with peer 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:10.930944 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 65018.6us
I20260709 12:27:10.937350 20807 raft_consensus.cc:1275] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
W20260709 12:27:10.941754  1472 log.cc:927] Time spent T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: Append to log took a long time: real 0.088s	user 0.003s	sys 0.000s
I20260709 12:27:10.977874  1428 consensus_queue.cc:1048] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
W20260709 12:27:10.994105 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.012387 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.025779 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:11.074918 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.105090 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:11.111361  1472 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=2 applied=25 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:11.112335  1472 tablet_bootstrap.cc:492] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: Bootstrap complete.
I20260709 12:27:11.113035  1472 ts_tablet_manager.cc:1403] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: Time spent bootstrapping tablet: real 0.817s	user 0.070s	sys 0.015s
W20260709 12:27:11.118510 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 70580.1us
I20260709 12:27:11.120416  1427 raft_consensus.cc:2955] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Committing config change with OpId 35.23: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:11.124455 20802 raft_consensus.cc:2955] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Committing config change with OpId 35.23: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:11.191919  1391 raft_consensus.cc:1064] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:27:11.192034  1472 raft_consensus.cc:359] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:11.193121  1472 raft_consensus.cc:740] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Initialized, Role: LEARNER
I20260709 12:27:11.193845  1472 consensus_queue.cc:260] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 35.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 25 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: NON_VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: true } }
I20260709 12:27:11.201787  1472 ts_tablet_manager.cc:1434] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.000s
I20260709 12:27:11.211565 20893 tablet_copy_service.cc:342] P 116023b4c0c94bfea737fc238bda3838: Request end of tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-70cdcdfe95804895979397c927fae46f received from {username='slave'} at 127.0.0.1:53814
I20260709 12:27:11.212062 20893 tablet_copy_service.cc:434] P 116023b4c0c94bfea737fc238bda3838: ending tablet copy session 50c1ab13351d4427b836fbb449c6d7bb-70cdcdfe95804895979397c927fae46f on tablet 70cdcdfe95804895979397c927fae46f with peer 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:11.216534 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:11.195047  1391 consensus_queue.cc:237] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 20, Current term: 34, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:11.226819 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 66886.5us
I20260709 12:27:11.237035 20625 catalog_manager.cc:5697] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:11.243948 21433 raft_consensus.cc:2955] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Committing config change with OpId 35.23: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:11.305912 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:11.315245 20881 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:27:11.317327  1501 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:27:11.321933 21427 raft_consensus.cc:1275] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:27:11.324294  1373 consensus_queue.cc:1048] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:27:11.352506  1373 raft_consensus.cc:2955] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Committing config change with OpId 34.23: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:11.372833 21427 raft_consensus.cc:2955] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Committing config change with OpId 34.23: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:11.380518 20881 raft_consensus.cc:2955] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Committing config change with OpId 34.23: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:11.389389 21433 raft_consensus.cc:1217] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEARNER]: Deduplicated request from leader. Original: 38.22->[38.23-38.23]   Dedup: 38.23->[]
W20260709 12:27:11.419158 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:11.423663 21427 raft_consensus.cc:1217] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Deduplicated request from leader. Original: 35.24->[35.25-35.25]   Dedup: 35.25->[]
W20260709 12:27:11.433176 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.433833 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.434336 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.434782 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:11.538362 20628 catalog_manager.cc:5697] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: config changed from index 22 to 23, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 34 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:11.599857 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.610281 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.730481 19603 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 20756)
I20260709 12:27:11.898840  1373 raft_consensus.cc:1064] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:27:11.901767  1373 consensus_queue.cc:237] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 38.23, Last appended by leader: 21, Current term: 38, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:11.913919 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:11.922657 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:11.964814  1427 raft_consensus.cc:1064] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: attempting to promote NON_VOTER 50c1ab13351d4427b836fbb449c6d7bb to VOTER
I20260709 12:27:11.969887  1427 consensus_queue.cc:237] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.25, Last appended by leader: 23, Current term: 35, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:12.073676 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:12.090713 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 85721.7us
W20260709 12:27:12.167716 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 72858.2us
I20260709 12:27:12.183276 20802 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 25. Preceding OpId from leader: term: 35 index: 26. (index mismatch)
W20260709 12:27:12.186124 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:12.188261  1427 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
W20260709 12:27:12.193620 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:12.193132 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:12.197016 21433 raft_consensus.cc:1275] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 LEARNER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 25. Preceding OpId from leader: term: 35 index: 26. (index mismatch)
W20260709 12:27:12.202625 20775 consensus_peers.cc:597] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:12.203372 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.204053 20775 consensus_peers.cc:597] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:12.226099  1535 consensus_queue.cc:1048] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:27:12.226397 21433 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEARNER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 23. Preceding OpId from leader: term: 38 index: 24. (index mismatch)
W20260709 12:27:12.230746 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:12.246512 20878 raft_consensus.cc:1275] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 23. Preceding OpId from leader: term: 38 index: 24. (index mismatch)
I20260709 12:27:12.253747  1531 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:27:12.256938  1391 consensus_queue.cc:1048] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
W20260709 12:27:12.274010 20618 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37c593b286274096a8c431979866be8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 116023b4c0c94bfea737fc238bda3838 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:12.286469 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:12.288702  1536 raft_consensus.cc:2955] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Committing config change with OpId 35.26: config changed from index 25 to 26, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:12.292752 20802 raft_consensus.cc:2955] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Committing config change with OpId 35.26: config changed from index 25 to 26, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:12.320922 20628 catalog_manager.cc:5697] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 reported cstate change: config changed from index 25 to 26, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 35 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:12.329661 20775 consensus_peers.cc:597] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:12.337575 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:12.373675  1532 raft_consensus.cc:2955] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Committing config change with OpId 38.24: config changed from index 23 to 24, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:12.382143 21433 raft_consensus.cc:2955] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Committing config change with OpId 38.24: config changed from index 23 to 24, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:12.384940 21435 raft_consensus.cc:2955] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Committing config change with OpId 35.26: config changed from index 25 to 26, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:12.403743 20775 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:12.421521 20881 raft_consensus.cc:2955] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Committing config change with OpId 38.24: config changed from index 23 to 24, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New config: { opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
I20260709 12:27:12.426970 20625 catalog_manager.cc:5697] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: config changed from index 23 to 24, 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) changed from NON_VOTER to VOTER. New cstate: current_term: 38 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } } }
W20260709 12:27:12.456235 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.557747 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:12.561817 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:12.592252 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.622699 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.725252 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:12.730988 19603 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:12.735535 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.738312 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.746469 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.749528 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.820811 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.954304 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.994930 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:12.999573 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.066551 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.120424 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.161437 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.193017 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.208145 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.312122 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.424965 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.720916 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.731712 19603 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:13.753523 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:13.783371 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.800215 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.847426 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.926398 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.929422 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.935753 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.032833 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:14.100056 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.175951 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.202545 20619 catalog_manager.cc:4755] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1047135f71834dbab6b869eb3c081b81: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:14.375370 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:14.426348 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:14.467159 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:14.549600 20775 consensus_peers.cc:597] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:14.639170 21400 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111) [suppressed 454 similar messages]
W20260709 12:27:14.693624 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:14.732338 19603 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:14.748059 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:14.766392 20775 consensus_peers.cc:597] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:14.767369 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:14.798288 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:14.800714 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:14.804322 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.841005 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:14.886162 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:15.147673 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:15.153939 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 108167us
W20260709 12:27:15.177840 20775 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:15.180792 20775 consensus_peers.cc:597] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:15.182101 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.188576 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.201839 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.221388 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:15.230020 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.256356 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.287362 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:15.288033 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.320542 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.387627 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.415742 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.534832 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.592118 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.635677 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.647320 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.701700 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.732832 19603 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:15.760628 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:15.837556 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.850234 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.065825 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1d40 after lost signal to thread 19606
W20260709 12:27:16.067145 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1140 after lost signal to thread 20667
W20260709 12:27:16.068658 20905 debug-util.cc:398] Leaking SignalData structure 0x7b080026f780 after lost signal to thread 20834
W20260709 12:27:16.069916 20905 debug-util.cc:398] Leaking SignalData structure 0x7b08003fdf80 after lost signal to thread 20908
W20260709 12:27:16.072386 20905 debug-util.cc:398] Leaking SignalData structure 0x7b0800145ae0 after lost signal to thread 21459
W20260709 12:27:16.091257 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.124662 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.213320 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:16.254988 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.307960 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:16.394497 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.424140 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:17.638356  1559 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:17.639083  1559 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.642177  1559 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:17.649837 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08000a9200 after lost signal to thread 19606
W20260709 12:27:17.649825 19603 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 20756)
I20260709 12:27:17.675369  1566 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:17.676090  1566 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.678184  1566 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:17.696360  1551 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:17.697126  1551 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.714160  1573 raft_consensus.cc:493] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:17.714955  1573 raft_consensus.cc:515] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.717590  1573 leader_election.cc:290] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:17.718247  1573 raft_consensus.cc:493] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:17.718761  1573 raft_consensus.cc:515] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.720801 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 43 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:17.721207  1573 leader_election.cc:290] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:17.724429 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7040eacc899473abc05a913812e82a4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:17.727748  1551 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:17.707149  1559 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:17.749878 20622 maintenance_manager.cc:419] P 9f77a9352772433bb5809bf72fd4f0ee: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033657
I20260709 12:27:17.750079  1559 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.753593  1559 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:17.763429  1563 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:17.764205  1563 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.767783  1563 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:17.769059 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.005s	sys 0.017s
W20260709 12:27:17.769420 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.006s	sys 0.018s
I20260709 12:27:17.773655 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:17.774302 21433 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 42.
I20260709 12:27:17.779899 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:17.780706  1565 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:17.781057 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:17.781368  1565 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.784852 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:17.805704  1532 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader pre-election won for term 43
I20260709 12:27:17.806542  1532 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:17.807077  1532 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:17.758944  1568 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:17.828416  1532 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:17.864217 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08000aeda0 after lost signal to thread 20667
W20260709 12:27:17.866104 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08003bb8e0 after lost signal to thread 20834
W20260709 12:27:17.867775 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08003baec0 after lost signal to thread 20908
W20260709 12:27:17.869361 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08000cc1e0 after lost signal to thread 21459
I20260709 12:27:17.821244  1565 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:17.882166  1532 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:17.834537  1566 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:17.904318  1532 raft_consensus.cc:493] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:17.904870  1532 raft_consensus.cc:515] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:17.822160  1568 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.954485  1532 leader_election.cc:290] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:17.903595  1566 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.968050 20875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 28 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:17.988138 20875 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 39.
I20260709 12:27:17.994270  1593 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:17.994470  1570 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:17.994967  1593 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:17.995137  1570 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.986579  1565 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:17.990620  1567 raft_consensus.cc:493] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.029368  1532 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:27:18.031579 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 113241us
I20260709 12:27:18.030763  1532 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.035053  1565 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.036037  1594 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.036520  1532 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.037122  1587 raft_consensus.cc:493] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.036609  1594 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.037688  1587 raft_consensus.cc:515] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:17.994298  1583 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.040884  1589 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.041136  1594 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.041745  1584 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.042145  1587 leader_election.cc:290] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.042672  1572 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.044453  1575 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.045958  1570 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.046648  1560 raft_consensus.cc:493] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.047053  1567 raft_consensus.cc:515] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.047778  1568 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.048045  1533 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.051787  1551 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.048741  1593 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.033188  1566 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.028502  1563 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.058624  1576 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.058918  1532 raft_consensus.cc:493] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.059325  1577 raft_consensus.cc:493] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.060416  1580 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.058735  1585 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.061375  1586 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.054944  1583 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.057001  1589 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.061681 20883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.063337 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.091555  1572 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.096020  1584 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.091679  1560 raft_consensus.cc:515] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.092929  1605 raft_consensus.cc:493] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.102819  1587 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.095283  1575 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.103425  1568 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.131883  1532 raft_consensus.cc:515] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.120326  1588 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.105558  1551 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.109107  1570 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.127980  1592 raft_consensus.cc:493] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.122732  1567 leader_election.cc:290] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.177862  1601 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.128443  1579 raft_consensus.cc:493] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.149892  1562 raft_consensus.cc:493] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.150272  1566 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.104039  1533 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.187152  1532 leader_election.cc:290] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.129437  1563 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.178188  1600 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
W20260709 12:27:18.199615 20621 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.402s	user 0.006s	sys 0.017s
I20260709 12:27:18.164009  1588 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.146130  1576 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.148589  1580 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.205509  1572 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.207139  1551 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.164269  1587 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.191769  1562 raft_consensus.cc:515] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.208243  1583 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.194635  1566 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.208696  1591 raft_consensus.cc:493] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.145362  1586 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.235608  1591 raft_consensus.cc:515] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.237285  1586 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.237768  1591 leader_election.cc:290] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.195673  1579 raft_consensus.cc:515] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.210991  1600 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.239732  1588 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.240111  1579 leader_election.cc:290] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.209517  1606 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.210749 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.241024 20878 raft_consensus.cc:2468] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 40.
I20260709 12:27:18.240898  1606 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.211601 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626209a224524080801f3a9ad7307fbe" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.242209 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3046bc775e64931b4a1ff963cccf0e2" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 43 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:18.211248 21399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 82206.5us
I20260709 12:27:18.242522 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.165786  1568 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.182343  1570 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.243860 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.244165 21399 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:18.244729  1568 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.205528  1560 leader_election.cc:290] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.227419 20806 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 42. Current term is 43. Ops: []
I20260709 12:27:18.229084  1575 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.229663  1595 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.230183  1596 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.164489  1577 raft_consensus.cc:515] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.253705  1595 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.255493  1597 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.230728  1587 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.232053  1533 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.150250  1605 raft_consensus.cc:515] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.266646  1551 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.267060  1604 raft_consensus.cc:493] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.267359  1551 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.267642  1604 raft_consensus.cc:515] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.269507  1551 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.269646  1604 leader_election.cc:290] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.270319  1598 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.232424  1562 leader_election.cc:290] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.199321  1573 raft_consensus.cc:493] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.273356  1603 raft_consensus.cc:493] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.273427  1573 raft_consensus.cc:515] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.274048  1603 raft_consensus.cc:515] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.241448  1600 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.275079  1572 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.192020  1567 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.275645  1572 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.276154  1567 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.276712  1566 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.189162  1601 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.278139  1572 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.228884  1594 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.278489  1567 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.278875  1594 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.279659  1611 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.280200  1611 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.280633  1563 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.281041  1594 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.281189  1563 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.278867  1601 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.281713  1563 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.282413  1611 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.223225  1532 raft_consensus.cc:493] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.279682  1584 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.251350  1596 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.283437  1532 raft_consensus.cc:515] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.284893 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.254886  1577 leader_election.cc:290] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.258176 20884 raft_consensus.cc:2468] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:27:18.285418  1532 leader_election.cc:290] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.285589 21426 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:27:18.288462 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4415902e5b30489691f67f4a635305b4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.289070 21431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec190d02960a4e88af0807817b789e6c" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.289599 21428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.290573 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.291168 20884 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 33.
I20260709 12:27:18.292012 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.292819 21435 raft_consensus.cc:2468] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:27:18.256079  1597 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.295444  1599 raft_consensus.cc:493] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.296027  1599 raft_consensus.cc:515] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.297469 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.298122 20884 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 30.
I20260709 12:27:18.298223  1599 leader_election.cc:290] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.299631  1563 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.301563  1603 leader_election.cc:290] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.301620 20884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.173162  1585 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.255897  1595 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.245321  1589 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.250108  1570 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.271004  1598 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.304926  1585 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.182303  1592 raft_consensus.cc:515] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.306759  1598 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.307044  1592 leader_election.cc:290] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.246500 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:18.285063  1596 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.252350  1588 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:27:18.307837 21427 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:18.308039  1588 raft_consensus.cc:493] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.267886  1605 leader_election.cc:290] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.308578  1588 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:18.283284  1565 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.275938  1573 leader_election.cc:290] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.302307  1606 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.311046  1599 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.311525  1605 raft_consensus.cc:493] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.311740  1532 raft_consensus.cc:493] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.311020 20621 maintenance_manager.cc:643] P 9f77a9352772433bb5809bf72fd4f0ee: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.560s	user 0.041s	sys 0.025s Metrics: {"bytes_written":7953,"delete_count":0,"lbm_write_time_us":1007,"lbm_writes_lt_1ms":7,"reinsert_count":0,"thread_start_us":327282,"threads_started":1,"update_count":308,"wal-append.queue_time_us":358793}
I20260709 12:27:18.311069  1616 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.314554 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.320626 20622 maintenance_manager.cc:419] P 9f77a9352772433bb5809bf72fd4f0ee: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.345512
I20260709 12:27:18.321298 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:18.321637 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d64dd1339c4abfb4ef05669402e801" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.323295  1605 raft_consensus.cc:515] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.323743 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.325519  1605 leader_election.cc:290] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.315512  1532 raft_consensus.cc:515] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:18.326547  1580 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.319772 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.318068 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.315002  1597 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.328274  1532 leader_election.cc:290] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.329267  1576 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:18.320220  1599 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.332495 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a32323ce10e140bb895085f97c97fa46" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 43 candidate_status { last_received { term: 42 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.333266 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.333528  1599 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.334852 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986267de598f431fb25840d9d741436d" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.336408 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.337273 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be08f375c22d436fa6e99ac7bd7f6c93" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.338227 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be2fc0a60b1472a9f537a9cd3bcc133" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.339942 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5374e2302374917828294b0a4e3313a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.340143 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.335041 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 43.
I20260709 12:27:18.340680 20882 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:27:18.341657 20776 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:18.342442 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.342592 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c00fffbb02ff4bc2a974d8a5b179a97d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 41 candidate_status { last_received { term: 40 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.346581 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 43 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.347409 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a458432a57843e4af30e5d9af5dfcfe" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 45 candidate_status { last_received { term: 44 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.345703 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3894130cfc2549b5a8c99d056675ef94" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 43 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.346355 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13cfeabebc54415b87ab9ce1d8a4caab" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.352057  1588 raft_consensus.cc:515] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.353237 21433 raft_consensus.cc:1240] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 39. Current term is 40. Ops: []
I20260709 12:27:18.353175 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:18.355974 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ea542665e419c8a63392e3014efa7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 27 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.356631 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.358659 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.360497  1597 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:27:18.361048  1597 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.361541  1597 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:18.362114 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.371487 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.372769 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.376603  1597 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.375149 21399 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:18.355581  1616 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:18.383168 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 132383us
I20260709 12:27:18.385998  1570 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:18.386451  1570 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:18.387167  1570 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:18.387936  1588 leader_election.cc:290] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.392573  1597 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:18.393050  1616 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.393695  1570 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Starting leader election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.395043 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4e3bf73fae46119cf926d5af0782a6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 44 candidate_status { last_received { term: 43 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.395586 21432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f381d2ab1764a739c2d1b3d41e65181" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.396260 21426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.396759 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.396904 21426 raft_consensus.cc:2468] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:27:18.398193 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.398802 21427 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:27:18.404165  1570 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:18.439126 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 97151.9us
I20260709 12:27:18.443569 20774 leader_election.cc:304] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:18.439345 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 88068.5us
I20260709 12:27:18.447468 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82740a2c094456a91cf0e411dd72159" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.453655  1599 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:27:18.454607  1599 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.459556  1599 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:18.455746  1532 raft_consensus.cc:2749] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader pre-election lost for term 44. Reason: could not achieve majority
I20260709 12:27:18.445066 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 28 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:18.440089 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 98375.7us
I20260709 12:27:18.458914 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 28 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:18.469466 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87775c62d4bc48d3b1831b4bdeff2870" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.484756 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:18.499415 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.506500 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.507922 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fe576036c04c149240cd6107285eca" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 45 candidate_status { last_received { term: 44 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.508567 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48bec319402467881543ac686fb8f0c" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.509967 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36a8df2d02f44ca965324882aba3ef2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 18 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.511725 20804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.512614 20804 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 38.
I20260709 12:27:18.507953 20805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.514837 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335e9cdc3f6745ab9b44f08b6e8b2118" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 42 candidate_status { last_received { term: 41 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.519680 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.520869 20801 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:27:18.533075 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:27:18.536377 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 72205.1us
W20260709 12:27:18.554401 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 50958us
I20260709 12:27:18.567855 20802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.569381 20806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.570032 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62396122903848d4a49e6d637c5c8b49" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.571550 21433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.572594 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3efcc9a1f14042de975fa0b662cad24d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 42 candidate_status { last_received { term: 41 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:18.574630 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 190766us
I20260709 12:27:18.576331 21432 raft_consensus.cc:1240] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 42. Current term is 43. Ops: []
I20260709 12:27:18.577925 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.571609 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f15b5f1ac1444b63bad60a89fa9f8a29" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.587610 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.588634 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37c593b286274096a8c431979866be8a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.618177 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f517f5d84b3437eb2fd2ae144fc6380" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.619870 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:18.620010 20775 leader_election.cc:336] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.620645 20775 leader_election.cc:304] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.621433 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:18.622080 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:18.622195  1532 raft_consensus.cc:2749] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:18.622972 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.624624 20776 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:27:18.625741  1532 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader election won for term 43
W20260709 12:27:18.625964 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:18.626119 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ef54d60e4646b68d43519ad3fffa6d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.626315  1532 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 43 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:18.628293 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9c17d2fc98f481597ce7b97e8f98c43" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.627866  1532 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 42.27, Last appended by leader: 27, Current term: 43, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:18.626865 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:18.630463 20775 leader_election.cc:336] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.631036 20775 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.631705 20775 leader_election.cc:336] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.632436 20775 leader_election.cc:336] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.632988 20775 leader_election.cc:336] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.632822  1616 consensus_queue.cc:1059] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 39.28, Next index: 29, Last known committed idx: 28, Time since last communication: 0.001s
I20260709 12:27:18.633952 20775 leader_election.cc:304] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.635077  1596 consensus_queue.cc:1059] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 42.27, Next index: 28, Last known committed idx: 27, Time since last communication: 0.001s
I20260709 12:27:18.639153  1605 raft_consensus.cc:3055] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:27:18.639827  1605 raft_consensus.cc:740] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:18.641346  1605 consensus_queue.cc:260] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 39.28, Last appended by leader: 28, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.643594  1573 raft_consensus.cc:2749] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:18.639251  1576 raft_consensus.cc:3055] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Stepping down as leader of term 42
I20260709 12:27:18.645442  1576 raft_consensus.cc:740] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:18.646970  1576 consensus_queue.cc:260] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 42.27, Last appended by leader: 27, Current term: 42, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:18.650792 19603 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 20756)
I20260709 12:27:18.652472 20774 leader_election.cc:304] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.651648  1605 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:18.653555  1580 raft_consensus.cc:2749] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Leader pre-election lost for term 43. Reason: could not achieve majority
I20260709 12:27:18.654210 20774 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.655157 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.655172  1580 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:18.658262 20776 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:18.659265  1580 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:27:18.659693  1580 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:18.660153  1580 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:27:18.660619  1576 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:27:18.663592 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.668380 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.675623 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd857e30c3240c5a4fdd11bc0602c00" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:18.678273 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.686966 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70cdcdfe95804895979397c927fae46f" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.694528 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff0a54292b924069aae5a0cd6c1366d1" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.695672 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:18.699625 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 66661.2us
I20260709 12:27:18.699597  1599 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:18.700508 20775 leader_election.cc:336] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.716809 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:18.714818 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 60728.1us
I20260709 12:27:18.732281 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:18.732535 20774 leader_election.cc:304] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.733520  1605 raft_consensus.cc:2749] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:18.735139  1599 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.713572 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:18.736871 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.738071  1576 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:27:18.738760  1576 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.739420  1576 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:18.738020 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a4e4474d2fd45438665993211ae862a" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.738672 20849 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:18.743984 20775 leader_election.cc:304] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.742904  1580 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.748802 20774 leader_election.cc:304] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.750301 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3199c6fc2e3441849ac7dd78244a3a18" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:18.751495 20775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 51054.6us
W20260709 12:27:18.752152 20775 leader_election.cc:336] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.745966  1599 raft_consensus.cc:2749] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:27:18.754189 20775 leader_election.cc:336] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.754278  1575 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:27:18.754804 20775 leader_election.cc:336] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.754818  1575 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.755438  1575 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:18.755479 20775 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.756247 20775 leader_election.cc:336] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.756814 20775 leader_election.cc:336] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.761559 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5c70f45aef146f4bcb47f1498221b99" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:18.761646 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e78bb14a7646c58f2f948b2d33b2cf" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
W20260709 12:27:18.764631 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:18.765300 21400 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.765710 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.769658 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46401ecb9b443118647ee361081a01e" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 28 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:18.771245 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:18.778523  1551 raft_consensus.cc:2749] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:18.782883  1588 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:18.786574  1580 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.788255 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
W20260709 12:27:18.792212 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 47254.4us
I20260709 12:27:18.806015 21427 raft_consensus.cc:1240] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 33. Current term is 34. Ops: []
I20260709 12:27:18.807598  1599 consensus_queue.cc:1059] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:27:18.808689  1551 raft_consensus.cc:3055] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:27:18.809067  1551 raft_consensus.cc:740] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:18.810155  1551 consensus_queue.cc:260] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.814430 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.815622  1551 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:18.825774  1605 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:18.831616  1605 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:18.847406  1605 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:18.835780 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:18.851341 20849 leader_election.cc:336] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.851504 20848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.852805 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:18.851861 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.863086 20810 raft_consensus.cc:2468] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
I20260709 12:27:18.869694  1596 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:18.872349 20850 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.873533 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.874838  1592 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:27:18.875280  1606 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:18.877120 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.878032  1606 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:18.883575 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 72456.1us
I20260709 12:27:18.888908  1605 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.891894  1575 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.892462 21435 raft_consensus.cc:1240] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Rejecting Update request from peer 657bc62d20374e539b23641715f22745 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:18.893246  1576 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.892904 20774 leader_election.cc:304] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:27:18.897053 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 45724.4us
W20260709 12:27:18.900161 20775 leader_election.cc:336] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.909582 20775 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:18.909155 20849 leader_election.cc:336] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.911451 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.919255  1606 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:18.920125 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:18.922091 20774 leader_election.cc:304] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.923650  1623 raft_consensus.cc:2749] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader pre-election lost for term 44. Reason: could not achieve majority
I20260709 12:27:18.923045  1622 consensus_queue.cc:1059] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.002s
I20260709 12:27:18.925467  1622 raft_consensus.cc:3055] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:18.925868  1622 raft_consensus.cc:740] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:18.928265 20774 leader_election.cc:304] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.929157  1573 raft_consensus.cc:2749] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Leader pre-election lost for term 44. Reason: could not achieve majority
I20260709 12:27:18.929756 20774 leader_election.cc:304] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.929862  1573 raft_consensus.cc:2749] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:18.930721  1573 raft_consensus.cc:2749] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:18.934700  1575 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:18.936717 20849 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.919917  1551 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:18.926847  1622 consensus_queue.cc:260] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 30.23, Last appended by leader: 23, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:18.920380  1605 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.940980  1576 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:18.941964  1622 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:18.943882 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:18.944692 20849 leader_election.cc:336] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.944029 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:18.945302 20849 leader_election.cc:336] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:18.945851 20849 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.946286 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:18.946537 20775 leader_election.cc:336] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.953363 20775 leader_election.cc:304] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.948653 20774 leader_election.cc:304] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.952787 20850 leader_election.cc:304] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:18.961174 20775 leader_election.cc:336] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.964036 20775 leader_election.cc:304] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.965785  1605 raft_consensus.cc:2749] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:18.965816 21400 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.961612  1532 raft_consensus.cc:2749] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:27:18.967779  1606 raft_consensus.cc:2749] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:18.962329 20774 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:18.962913  1623 raft_consensus.cc:2749] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:18.967018  1589 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:18.969485  1532 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:18.972196 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942b8c65bd1e42a99f971b099d0e19e6" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.972954 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:18.973376 20775 leader_election.cc:336] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.973639 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1303aa66bdb4474b6bcb604461e74a5" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.973984 20775 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:18.974911  1623 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:18.974885 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.975807 20881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae342ffbd6894fb1aa23aa48f4d2dd1b" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:18.978587 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:18.983592  1576 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:27:18.984535 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:18.985791 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:27:18.983584 20849 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.986411 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.987622 20625 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 42 to 43, leader changed from 116023b4c0c94bfea737fc238bda3838 (127.19.36.195) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 43 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:18.989773 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.990720  1576 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:18.991889 20849 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.992324 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.996815  1576 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:18.997911 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
W20260709 12:27:18.998262 20849 leader_election.cc:336] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:18.998735 20849 leader_election.cc:304] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:18.998920  1576 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:19.001350 20850 leader_election.cc:304] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:19.007764  1575 raft_consensus.cc:2749] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:19.008702 20849 leader_election.cc:336] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.009192 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.009742 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.015355  1606 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:19.015375  1589 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:19.018426 20849 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.018909 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.019160 20774 leader_election.cc:304] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:19.022617  1622 raft_consensus.cc:2749] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:19.022073 20774 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:19.026678  1576 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:19.015601 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.027293  1622 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:19.028039  1589 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:19.028340 20774 leader_election.cc:304] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:27:19.029114 20775 consensus_peers.cc:597] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:19.029676  1622 raft_consensus.cc:2749] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:19.030014  1576 raft_consensus.cc:2749] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:19.028061 20849 leader_election.cc:336] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.036082 20850 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.029615 21400 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.037994  1589 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:19.040308 20774 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 40 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:19.040899 21400 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.044190 20850 leader_election.cc:304] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:19.044766  1576 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:19.044890  1622 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader election lost for term 40. Reason: could not achieve majority
I20260709 12:27:19.045666  1576 raft_consensus.cc:2749] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:19.049860  1589 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:19.051626 20876 raft_consensus.cc:1240] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 32. Current term is 33. Ops: []
I20260709 12:27:19.052652 20774 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:19.052968 20775 consensus_peers.cc:597] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:19.051218 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.054044  1589 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:19.055372  1622 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:27:19.059082  1589 consensus_queue.cc:1059] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 32.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:27:19.060276  1588 raft_consensus.cc:3055] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:27:19.060638  1588 raft_consensus.cc:740] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:19.061746  1588 consensus_queue.cc:260] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 32.25, Last appended by leader: 25, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:19.067517  1588 raft_consensus.cc:3060] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:19.093298 20810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:19.093803 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:19.058580 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.101714  1589 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:19.108980 20801 raft_consensus.cc:1240] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 35. Current term is 36. Ops: []
I20260709 12:27:19.111615 20774 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb
W20260709 12:27:19.112164 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 53737.2us
I20260709 12:27:19.114136 21400 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.114709  1532 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:27:19.125272  1589 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:27:19.127297 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.129519 21400 leader_election.cc:304] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:27:19.133980  1589 raft_consensus.cc:2804] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:27:19.134480  1589 raft_consensus.cc:697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:19.129871  1575 consensus_queue.cc:1059] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 35.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:27:19.137089  1565 raft_consensus.cc:3055] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:27:19.137486  1565 raft_consensus.cc:740] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:19.138447  1565 consensus_queue.cc:260] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:19.139525  1565 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:27:19.130137  1588 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:27:19.135504  1589 consensus_queue.cc:237] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 39.28, Last appended by leader: 28, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:19.163370 21400 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.165279 20876 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 42 index: 27. Preceding OpId from leader: term: 43 index: 28. (index mismatch)
W20260709 12:27:19.162680 20849 leader_election.cc:336] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.169993 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.171298  1622 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20260709 12:27:19.171783  1588 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:19.170046 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.175484  1606 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:19.191278 20801 raft_consensus.cc:1240] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Rejecting Update request from peer 116023b4c0c94bfea737fc238bda3838 for earlier term 39. Current term is 40. Ops: []
W20260709 12:27:19.171732 21400 leader_election.cc:336] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:19.195485 21400 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:19.200723 21400 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:19.201329 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:19.205106 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:19.205736 21400 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:19.206861 21400 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.207378 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.208415  1589 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:19.209837 21400 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.210268 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:19.210999  1589 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:19.212412 21400 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.212877 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.213673  1589 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:19.215124 21400 leader_election.cc:336] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.215831 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
W20260709 12:27:19.205775 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.216745  1589 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:19.217463 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:19.218139 21400 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.218796 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:19.220701  1589 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
W20260709 12:27:19.222002 21400 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.222446 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.223850  1589 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:19.225122 21400 leader_election.cc:336] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.226243  1565 consensus_queue.cc:1059] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: INVALID_TERM, Last received: 39.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:27:19.227056 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:19.227463  1606 raft_consensus.cc:3055] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Stepping down as leader of term 39
I20260709 12:27:19.227887  1606 raft_consensus.cc:740] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:19.228011  1589 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:19.229130 21400 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.228796  1606 consensus_queue.cc:260] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 39.25, Last appended by leader: 25, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:19.229655 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.230464  1589 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:27:19.231633 21400 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:19.232118 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.232924  1589 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:19.233538  1606 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:19.239084 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b31661191c14b44ac964555853900f1" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:19.249101 21427 raft_consensus.cc:2468] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:27:19.250195 21435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:19.264725 20876 raft_consensus.cc:1240] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Rejecting Update request from peer 50c1ab13351d4427b836fbb449c6d7bb for earlier term 38. Current term is 39. Ops: []
I20260709 12:27:19.266521  1589 consensus_queue.cc:1059] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: INVALID_TERM, Last received: 38.26, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20260709 12:27:19.267552  1589 raft_consensus.cc:3055] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:27:19.267885  1589 raft_consensus.cc:740] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:19.269124  1589 consensus_queue.cc:260] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 38.26, Last appended by leader: 26, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:19.271065  1589 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:19.261036 21429 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 42 index: 27. Preceding OpId from leader: term: 43 index: 28. (index mismatch)
I20260709 12:27:19.277251  1599 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20260709 12:27:19.312842 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:19.315526 20625 catalog_manager.cc:5697] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 39 to 40, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 40 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:19.323447 20850 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:27:19.325742 20850 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:19.327515  1575 raft_consensus.cc:2804] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:27:19.327876  1565 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:27:19.328047  1575 raft_consensus.cc:697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:19.329046  1575 consensus_queue.cc:237] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 32.25, Last appended by leader: 25, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:19.334823 20621 maintenance_manager.cc:643] P 9f77a9352772433bb5809bf72fd4f0ee: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 1.001s	user 0.101s	sys 0.019s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":28413,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":40764,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":241,"lbm_read_time_us":38209,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":26,"lbm_write_time_us":1754,"lbm_writes_lt_1ms":12,"peak_mem_usage":311597,"reinsert_count":0,"thread_start_us":41064,"threads_started":1,"update_count":308}
W20260709 12:27:19.341928 20775 consensus_peers.cc:597] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:19.360361 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:19.374053 20625 catalog_manager.cc:5697] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 32 to 33, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 33 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:19.428694 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.451886 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.464279 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.470244 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.545159 20807 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 39 index: 28. Preceding OpId from leader: term: 40 index: 29. (index mismatch)
I20260709 12:27:19.576571  1597 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.001s
W20260709 12:27:19.580613 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.600471 20775 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:19.608685 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:19.642459 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.647615 20775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111) [suppressed 334 similar messages]
W20260709 12:27:19.651664 19603 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 20756)
W20260709 12:27:19.656073 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.681902 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.689091 20882 raft_consensus.cc:3060] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:27:19.717628 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.698956 20882 raft_consensus.cc:1275] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 39 index: 28. Preceding OpId from leader: term: 40 index: 29. (index mismatch)
I20260709 12:27:19.719709  1597 consensus_queue.cc:1048] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20260709 12:27:19.744683 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:19.745101 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:19.762979 21427 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 32 index: 25. Preceding OpId from leader: term: 33 index: 26. (index mismatch)
I20260709 12:27:19.766396 20801 raft_consensus.cc:1275] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 32 index: 25. Preceding OpId from leader: term: 33 index: 26. (index mismatch)
I20260709 12:27:19.781838  1606 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:27:19.809620  1639 consensus_queue.cc:1048] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
W20260709 12:27:19.874514 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.912109 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.918283 20775 consensus_peers.cc:597] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:20.086885  1605 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.087630  1605 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.090790 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:20.094732 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:20.095551 21427 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 34.
I20260709 12:27:20.096995 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:27:20.098846  1622 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:27:20.099359  1622 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.100005  1622 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:27:20.121335  1605 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.153625  1605 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.154305  1605 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.158864 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:20.159719 20882 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 30.
I20260709 12:27:20.161265 20774 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 
I20260709 12:27:20.163084  1623 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:20.163602  1623 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.163998  1623 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:20.174578  1623 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.178180  1623 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.181663  1605 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:20.182966 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 56844.2us
I20260709 12:27:20.191799  1622 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.195540  1622 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:20.215004 20774 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:45387, user_credentials={real_user=slave}} blocked reactor thread for 53972us
I20260709 12:27:20.219416 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:20.224190 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:20.225003 20882 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Advancing to term 32
W20260709 12:27:20.224390 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 83581.5us
W20260709 12:27:20.235922 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:20.235957 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 73333.9us
I20260709 12:27:20.240664 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:20.241308 21427 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:27:20.236726 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.248505 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:20.262212 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:20.262964 21429 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:27:20.268514 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:20.274947 21429 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 35.
I20260709 12:27:20.277016 21427 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:27:20.278254 20776 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:27:20.279107  1622 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:27:20.279922  1622 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:20.280853  1622 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.285411 20776 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 
W20260709 12:27:20.286353 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:20.286767  1645 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:20.288923  1645 raft_consensus.cc:697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:20.289853  1645 consensus_queue.cc:237] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 30.23, Last appended by leader: 23, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.290294 20882 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 32.
I20260709 12:27:20.313077 20625 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 33 to 35. New cstate: current_term: 35 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:20.376292 20625 catalog_manager.cc:5697] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:20.397994 20775 consensus_peers.cc:597] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.441622 20775 consensus_peers.cc:597] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:20.453521  1576 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.454098  1576 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.456410  1576 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.457618 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:20.459882 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:20.460629 20801 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:27:20.466195 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:20.469139  1639 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:27:20.469553  1639 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.469878  1639 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:27:20.478374  1576 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:20.478987  1576 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.481068  1576 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.487834 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:20.489176 21427 raft_consensus.cc:2368] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:27:20.491955 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:20.492156 20850 leader_election.cc:400] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:27:20.492678 20850 leader_election.cc:403] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:20.493626  1576 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 35
W20260709 12:27:20.494064 20775 consensus_peers.cc:597] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:20.526007  1576 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339) with higher term. Message: Invalid argument: T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 for earlier term 34. Current term is 35.
I20260709 12:27:20.532665  1639 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.549535  1639 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.550933 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:20.552721 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:20.575453 20801 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:27:20.582867 20801 raft_consensus.cc:2468] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:27:20.594029 20849 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:20.595216  1639 raft_consensus.cc:2804] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:27:20.595757  1639 raft_consensus.cc:697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:20.596648  1639 consensus_queue.cc:237] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.617074  1576 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.617594  1576 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.619755  1576 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.621035 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:20.624387 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:20.625036 20801 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 40.
I20260709 12:27:20.629427 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:20.631933  1576 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:27:20.632404  1576 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.632786  1576 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:27:20.642961  1646 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:20.643832  1646 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.647225  1646 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.647241 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:20.648067 21427 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:27:20.649050 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:20.649803 20876 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 39: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:27:20.651027 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
W20260709 12:27:20.652141 19603 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 20756)
I20260709 12:27:20.652477  1622 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:27:20.652894  1622 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 50c1ab13351d4427b836fbb449c6d7bb)
I20260709 12:27:20.653241  1622 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:27:20.667856  1622 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.670135  1622 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Requested vote from peers 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.671294 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:20.674647 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "657bc62d20374e539b23641715f22745" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:20.675499 20882 raft_consensus.cc:2393] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 657bc62d20374e539b23641715f22745 in current term 39: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:27:20.680016 21434 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657bc62d20374e539b23641715f22745 in term 39.
I20260709 12:27:20.681418 20776 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745; no voters: 116023b4c0c94bfea737fc238bda3838
I20260709 12:27:20.683457  1645 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:27:20.683959  1645 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Becoming Leader. State: Replica: 657bc62d20374e539b23641715f22745, State: Running, Role: LEADER
I20260709 12:27:20.684494 20625 catalog_manager.cc:5697] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 35 to 37. New cstate: current_term: 37 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:20.684991  1645 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 38.26, Last appended by leader: 26, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:20.686605 20775 consensus_peers.cc:597] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:20.694455  1576 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.697873 21434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:20.700474  1576 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:20.701758 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 41 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745"
I20260709 12:27:20.702404 20807 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:27:20.710392 20807 raft_consensus.cc:2468] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 41.
I20260709 12:27:20.717689 20849 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745; no voters: 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:20.719708  1639 raft_consensus.cc:2804] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:27:20.721714  1639 raft_consensus.cc:697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:27:20.722461 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:20.722743  1639 consensus_queue.cc:237] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 39.25, Last appended by leader: 25, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.733901 20625 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 reported cstate change: term changed from 38 to 39, leader changed from 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196) to 657bc62d20374e539b23641715f22745 (127.19.36.194). New cstate: current_term: 39 leader_uuid: "657bc62d20374e539b23641715f22745" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:20.753218 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:20.783640 20882 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:27:20.785161  1645 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:27:20.792918  1597 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:20.793557  1597 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.794683  1589 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:20.793951 20625 catalog_manager.cc:5697] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 39 to 41. New cstate: current_term: 41 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:20.795256  1589 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:20.796793  1597 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:20.797333  1589 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:20.798488 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:20.798871 20801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:20.799156 20807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5e3c520e78241879a779a912c66a4ba" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "657bc62d20374e539b23641715f22745" is_pre_election: true
I20260709 12:27:20.799856 20801 raft_consensus.cc:2368] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 40. Current term is 41.
I20260709 12:27:20.799923 20807 raft_consensus.cc:2368] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37.
I20260709 12:27:20.801278 21400 leader_election.cc:400] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 40. Current term is 41.
I20260709 12:27:20.801568 21400 leader_election.cc:403] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:20.807643 20876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981a4d0731e940b79eccfb33d1928fb4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 25 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:20.810163 21400 leader_election.cc:400] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37.
I20260709 12:27:20.810491 21400 leader_election.cc:403] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:20.811044  1644 raft_consensus.cc:3060] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 41
I20260709 12:27:20.820504  1589 raft_consensus.cc:3060] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 37
I20260709 12:27:20.837545 21434 raft_consensus.cc:1275] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 30 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:27:20.839224  1646 consensus_queue.cc:1048] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:27:20.843344 21427 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:27:20.849161  1645 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:27:20.859915 20882 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:27:20.864724  1646 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:27:20.894070  1644 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election lost for term 40. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 40. Current term is 41.
I20260709 12:27:20.920243  1589 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 36. Reason: Vote denied by peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085) with higher term. Message: Invalid argument: T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb for earlier term 36. Current term is 37.
W20260709 12:27:20.933740 20775 consensus_peers.cc:597] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.937376 20775 consensus_peers.cc:597] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.957409 20775 consensus_peers.cc:597] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.983942 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:21.008631 20776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 46062.9us
W20260709 12:27:21.011345 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:21.012006 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:21.040953 20775 consensus_peers.cc:597] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:21.048045 21427 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:27:21.051026  1639 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:27:21.079998 20801 raft_consensus.cc:1275] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:27:21.082899  1639 consensus_queue.cc:1048] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
W20260709 12:27:21.124848 20775 consensus_peers.cc:597] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:21.125452 20775 consensus_peers.cc:597] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:21.202792 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:21.256114 20882 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 26. Preceding OpId from leader: term: 39 index: 27. (index mismatch)
I20260709 12:27:21.257737  1663 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20260709 12:27:21.262956 21429 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 25. Preceding OpId from leader: term: 41 index: 26. (index mismatch)
I20260709 12:27:21.264919  1575 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260709 12:27:21.309785 21427 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Refusing update from remote peer 657bc62d20374e539b23641715f22745: Log matching property violated. Preceding OpId in replica: term: 38 index: 26. Preceding OpId from leader: term: 39 index: 27. (index mismatch)
W20260709 12:27:21.313920 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:21.327992 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:21.359887  1665 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.001s
I20260709 12:27:21.406891 20801 raft_consensus.cc:1275] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 39 index: 25. Preceding OpId from leader: term: 41 index: 26. (index mismatch)
I20260709 12:27:21.561842  1639 consensus_queue.cc:1048] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
W20260709 12:27:21.618598 20775 consensus_peers.cc:597] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:21.619099 20775 consensus_peers.cc:597] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:21.619629 20775 consensus_peers.cc:597] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:21.658921 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:21.671649 20850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:44339, user_credentials={real_user=slave}} blocked reactor thread for 71669.2us
W20260709 12:27:21.674340 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:21.683436 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:21.693511 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:21.825078 20775 consensus_peers.cc:597] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:21.867417 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:21.909578 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
W20260709 12:27:21.976534 20775 consensus_peers.cc:597] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:21.989331 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.107494 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.113490 20775 consensus_peers.cc:597] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:22.118860 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.135313 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.153774 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:22.165961 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
W20260709 12:27:22.184954 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.207326 20849 consensus_peers.cc:597] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.211977 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.212600 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:22.213069 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:22.250430 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.256044 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.270385 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.281754 21400 consensus_peers.cc:597] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:22.304919 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:22.306160 19603 tablet_replica.cc:333] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.307252 19603 raft_consensus.cc:2243] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.308419 19603 raft_consensus.cc:2272] T ad57e68a6a114847ac252579307658f7 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.310467 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:22.311818 19603 tablet_replica.cc:333] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.312963 19603 raft_consensus.cc:2243] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 43 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.314061 19603 raft_consensus.cc:2272] T 2b098f0a68194f18be24dc4f3742a4ab P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.317364 19603 tablet_replica.cc:333] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.318357 19603 raft_consensus.cc:2243] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.319059 19603 raft_consensus.cc:2272] T 3894130cfc2549b5a8c99d056675ef94 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.322150 19603 tablet_replica.cc:333] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.322963 19603 raft_consensus.cc:2243] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.323511 19603 raft_consensus.cc:2272] T 0b31661191c14b44ac964555853900f1 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.326067 19603 tablet_replica.cc:333] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.326864 19603 raft_consensus.cc:2243] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.327414 19603 raft_consensus.cc:2272] T d5e3c520e78241879a779a912c66a4ba P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.329923 19603 tablet_replica.cc:333] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.330794 19603 raft_consensus.cc:2243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.331844 19603 raft_consensus.cc:2272] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.334358 19603 tablet_replica.cc:333] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.335331 19603 raft_consensus.cc:2243] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.336517 19603 raft_consensus.cc:2272] T 54cd23b6f74a477a99461098dd4f236d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.339226 19603 tablet_replica.cc:333] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.339949 19603 raft_consensus.cc:2243] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.341727 19603 raft_consensus.cc:2272] T 87775c62d4bc48d3b1831b4bdeff2870 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.344533 19603 tablet_replica.cc:333] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.345299 19603 raft_consensus.cc:2243] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.346405 19603 raft_consensus.cc:2272] T 3efcc9a1f14042de975fa0b662cad24d P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.349085 19603 tablet_replica.cc:333] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.349836 19603 raft_consensus.cc:2243] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.350492 19603 raft_consensus.cc:2272] T 942b8c65bd1e42a99f971b099d0e19e6 P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.353097 19603 tablet_replica.cc:333] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.353888 19603 raft_consensus.cc:2243] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.354593 19603 raft_consensus.cc:2272] T c3046bc775e64931b4a1ff963cccf0e2 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.357394 19603 tablet_replica.cc:333] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.358225 19603 raft_consensus.cc:2243] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.358956 19603 raft_consensus.cc:2272] T a9c17d2fc98f481597ce7b97e8f98c43 P 657bc62d20374e539b23641715f22745 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.362088 19603 tablet_replica.cc:333] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.362852 19603 raft_consensus.cc:2243] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.363423 19603 raft_consensus.cc:2272] T 8f381d2ab1764a739c2d1b3d41e65181 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.367810 19603 tablet_replica.cc:333] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.368615 19603 raft_consensus.cc:2243] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.369836 19603 raft_consensus.cc:2272] T c6fe576036c04c149240cd6107285eca P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.372529 19603 tablet_replica.cc:333] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.374013 19603 raft_consensus.cc:2243] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.375114 19603 raft_consensus.cc:2272] T d5c70f45aef146f4bcb47f1498221b99 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.377875 19603 tablet_replica.cc:333] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.378847 19603 raft_consensus.cc:2243] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.380560 19603 raft_consensus.cc:2272] T 72e60e5ddf96445ab5e284dbeed45c81 P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.383162 19603 tablet_replica.cc:333] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745: stopping tablet replica
W20260709 12:27:22.383458 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:22.383967 19603 raft_consensus.cc:2243] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.384626 19603 raft_consensus.cc:2272] T 6dc725b7c88e47f693e3a16879798246 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.387328 19603 tablet_replica.cc:333] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.388119 19603 raft_consensus.cc:2243] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.388651 19603 raft_consensus.cc:2272] T 13cfeabebc54415b87ab9ce1d8a4caab P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.391438 19603 tablet_replica.cc:333] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.392244 19603 raft_consensus.cc:2243] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.393276 19603 raft_consensus.cc:2272] T e36a8df2d02f44ca965324882aba3ef2 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.395754 19603 tablet_replica.cc:333] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.396512 19603 raft_consensus.cc:2243] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.397147 19603 raft_consensus.cc:2272] T 986267de598f431fb25840d9d741436d P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.399879 19603 tablet_replica.cc:333] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.400735 19603 raft_consensus.cc:2243] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.401829 19603 raft_consensus.cc:2272] T a48bec319402467881543ac686fb8f0c P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.404621 19603 tablet_replica.cc:333] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.405433 19603 raft_consensus.cc:2243] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.406196 19603 raft_consensus.cc:2272] T 37c593b286274096a8c431979866be8a P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.409454 19603 tablet_replica.cc:333] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.410216 19603 raft_consensus.cc:2243] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.411125 19603 raft_consensus.cc:2272] T b7040eacc899473abc05a913812e82a4 P 657bc62d20374e539b23641715f22745 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.414176 19603 tablet_replica.cc:333] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.415084 19603 raft_consensus.cc:2243] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:22.415151 21400 consensus_peers.cc:597] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:22.416360 19603 raft_consensus.cc:2272] T 62396122903848d4a49e6d637c5c8b49 P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.419866 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800141280 after lost signal to thread 19606
W20260709 12:27:22.539945 20831 debug-util.cc:398] Leaking SignalData structure 0x7b08000826c0 after lost signal to thread 20667
W20260709 12:27:22.540072 20849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:34333, user_credentials={real_user=slave}} blocked reactor thread for 121975us
W20260709 12:27:22.541383 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800158ee0 after lost signal to thread 20834
W20260709 12:27:22.542582 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800140700 after lost signal to thread 20908
W20260709 12:27:22.543857 20831 debug-util.cc:398] Leaking SignalData structure 0x7b0800401d20 after lost signal to thread 21459
I20260709 12:27:22.548765 19603 tablet_replica.cc:333] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.549623 19603 raft_consensus.cc:2243] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.564458 19603 raft_consensus.cc:2272] T 981a4d0731e940b79eccfb33d1928fb4 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.568418 19603 tablet_replica.cc:333] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.569291 19603 raft_consensus.cc:2243] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.570403 19603 raft_consensus.cc:2272] T c5ef54d60e4646b68d43519ad3fffa6d P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.573278 19603 tablet_replica.cc:333] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.574186 19603 raft_consensus.cc:2243] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.582418 19603 raft_consensus.cc:2272] T 761ea542665e419c8a63392e3014efa7 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.600271 20619 catalog_manager.cc:4755] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1047135f71834dbab6b869eb3c081b81: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:22.635339 19603 tablet_replica.cc:333] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.649118 19603 raft_consensus.cc:2243] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.649885 19603 raft_consensus.cc:2272] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.654170 19603 tablet_replica.cc:333] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.654901 19603 raft_consensus.cc:2243] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.655757 19603 raft_consensus.cc:2272] T 1047135f71834dbab6b869eb3c081b81 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.677276 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.677975 21400 consensus_peers.cc:597] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.678525 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:22.679203 19603 tablet_replica.cc:333] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745: stopping tablet replica
W20260709 12:27:22.679643 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:22.680068 19603 raft_consensus.cc:2243] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.680714 19603 raft_consensus.cc:2272] T a32323ce10e140bb895085f97c97fa46 P 657bc62d20374e539b23641715f22745 [term 42 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.683530 19603 tablet_replica.cc:333] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745: stopping tablet replica
W20260709 12:27:22.687664 21400 consensus_peers.cc:597] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.688282 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.702064 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:22.702947 19603 raft_consensus.cc:2243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.709760 19603 raft_consensus.cc:2272] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.717679 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:22.721715 19603 tablet_replica.cc:333] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.722589 19603 raft_consensus.cc:2243] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.723291 19603 raft_consensus.cc:2272] T 4415902e5b30489691f67f4a635305b4 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.726015 19603 tablet_replica.cc:333] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.726742 19603 raft_consensus.cc:2243] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.728137 19603 raft_consensus.cc:2272] T 0a4e4474d2fd45438665993211ae862a P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.738612 19603 tablet_replica.cc:333] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.739351 19603 raft_consensus.cc:2243] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 39 LEADER]: Raft consensus shutting down.
W20260709 12:27:22.739331 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.751490 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:22.751564 19603 raft_consensus.cc:2272] T 626209a224524080801f3a9ad7307fbe P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.750809 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.758085 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.758692 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.760510 20849 consensus_peers.cc:597] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.761039 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.761577 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.762068 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.763998 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:22.767059 19603 tablet_replica.cc:333] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.767959 19603 raft_consensus.cc:2243] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.769826 19603 raft_consensus.cc:2272] T c1303aa66bdb4474b6bcb604461e74a5 P 657bc62d20374e539b23641715f22745 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.778167 19603 tablet_replica.cc:333] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.778963 19603 raft_consensus.cc:2243] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.779983 19603 raft_consensus.cc:2272] T 70cdcdfe95804895979397c927fae46f P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.782598 19603 tablet_replica.cc:333] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.783388 19603 raft_consensus.cc:2243] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.784035 19603 raft_consensus.cc:2272] T ec190d02960a4e88af0807817b789e6c P 657bc62d20374e539b23641715f22745 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.786748 19603 tablet_replica.cc:333] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745: stopping tablet replica
W20260709 12:27:22.795811 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.797263 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.798108 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.802847 20849 consensus_peers.cc:597] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:22.803481 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:22.804711 19603 raft_consensus.cc:2243] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.819592 19603 raft_consensus.cc:2272] T 75e78bb14a7646c58f2f948b2d33b2cf P 657bc62d20374e539b23641715f22745 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.822667 19603 tablet_replica.cc:333] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.823488 19603 raft_consensus.cc:2243] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.824565 19603 raft_consensus.cc:2272] T ff0a54292b924069aae5a0cd6c1366d1 P 657bc62d20374e539b23641715f22745 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.827168 19603 tablet_replica.cc:333] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.829766 19603 raft_consensus.cc:2243] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.830456 19603 raft_consensus.cc:2272] T 6a458432a57843e4af30e5d9af5dfcfe P 657bc62d20374e539b23641715f22745 [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.833418 19603 tablet_replica.cc:333] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.834157 19603 raft_consensus.cc:2243] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.834985 19603 raft_consensus.cc:2272] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 657bc62d20374e539b23641715f22745 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.837814 19603 tablet_replica.cc:333] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.838550 19603 raft_consensus.cc:2243] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.840150 19603 raft_consensus.cc:2272] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.843099 19603 tablet_replica.cc:333] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.843891 19603 raft_consensus.cc:2243] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.844436 19603 raft_consensus.cc:2272] T d46401ecb9b443118647ee361081a01e P 657bc62d20374e539b23641715f22745 [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.847119 19603 tablet_replica.cc:333] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.847847 19603 raft_consensus.cc:2243] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.849009 19603 raft_consensus.cc:2272] T f61efc9c6d5f43ac84cfe1def41de8a0 P 657bc62d20374e539b23641715f22745 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.854614 19603 tablet_replica.cc:333] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.855329 19603 raft_consensus.cc:2243] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.856043 19603 raft_consensus.cc:2272] T 38d64dd1339c4abfb4ef05669402e801 P 657bc62d20374e539b23641715f22745 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.858788 19603 tablet_replica.cc:333] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.859474 19603 raft_consensus.cc:2243] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.860797 19603 raft_consensus.cc:2272] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 657bc62d20374e539b23641715f22745 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.863476 19603 tablet_replica.cc:333] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.864137 19603 raft_consensus.cc:2243] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.864704 19603 raft_consensus.cc:2272] T ba4e3bf73fae46119cf926d5af0782a6 P 657bc62d20374e539b23641715f22745 [term 43 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.870600 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:22.870224 19603 tablet_replica.cc:333] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745: stopping tablet replica
I20260709 12:27:22.871734 19603 raft_consensus.cc:2243] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.872782 19603 raft_consensus.cc:2272] T 3199c6fc2e3441849ac7dd78244a3a18 P 657bc62d20374e539b23641715f22745 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:23.167068 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:23.189978 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:23.268747  1685 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.269358  1685 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.271677  1685 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:23.275720 20878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 44 candidate_status { last_received { term: 43 index: 28 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
W20260709 12:27:23.281577 21400 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 44 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.282196 21400 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.283645  1685 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader pre-election lost for term 44. Reason: could not achieve majority
W20260709 12:27:23.308223 20849 consensus_peers.cc:597] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:23.471385 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:23.479154  1683 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.479818  1683 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.483551  1685 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.484124  1685 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.486151  1685 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:23.488504 20849 leader_election.cc:336] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.490415 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:23.496343  1683 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:23.504931 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:23.506742 21400 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.507462 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.509397  1685 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:23.511425 20850 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.512593  1683 raft_consensus.cc:2749] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:23.532372  1683 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.533033  1683 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.533449  1682 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.534152  1682 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:23.536803  1682 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:23.545804  1683 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:23.549063  1685 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.549932  1685 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.557854 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:23.560245  1685 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:23.564527 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:23.575424 20849 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:23.577172 20849 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.577658 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.578459  1683 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:23.580391  1685 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.581089  1685 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.586488 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:23.587172 20882 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 35.
W20260709 12:27:23.588189 20849 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:23.588702 20849 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:23.589083 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:23.591398  1682 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:23.592193  1644 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:27:23.592715  1644 raft_consensus.cc:493] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.593127  1644 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:27:23.595391 20849 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.603516  1683 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.604238  1683 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:23.606873  1683 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:23.614593  1682 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.620709  1682 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:23.621204 20849 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:23.622212 20849 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:23.622653 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.619733  1685 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:23.624265  1683 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:23.625312  1682 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:23.626273 21400 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.630052 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.630908  1685 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:23.629170  1644 raft_consensus.cc:515] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.637933 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ccae903e8484f8d8c3af847e2c4c3d4" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:23.638577 20882 raft_consensus.cc:3060] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:27:23.639228 20849 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:23.646551 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:23.649204 20882 raft_consensus.cc:2468] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 36.
I20260709 12:27:23.653463 21399 leader_election.cc:304] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:23.654500  1685 raft_consensus.cc:2804] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:27:23.656984  1685 raft_consensus.cc:697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:23.665881  1685 consensus_queue.cc:237] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.651973  1682 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.671639 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:23.672359 21429 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 33.
I20260709 12:27:23.671612  1682 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:23.656554  1644 leader_election.cc:290] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:23.678220  1682 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:23.679158  1644 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.679837  1644 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.682240  1644 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:23.684131  1681 raft_consensus.cc:493] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.684726  1681 raft_consensus.cc:515] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:23.686929  1681 leader_election.cc:290] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:23.690080 21400 leader_election.cc:336] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:23.695068 20849 leader_election.cc:336] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:23.697113 20849 leader_election.cc:336] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.697670 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:23.698431 21400 leader_election.cc:336] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.697816 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:27:23.699258  1682 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:23.699965  1682 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:27:23.700340  1682 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.700695  1682 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:23.703480  1683 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.704068  1683 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.706152  1683 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:23.704044 21400 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:23.707172 21400 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.707566 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.708225  1685 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:23.712438 20627 catalog_manager.cc:5697] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 35 to 36, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 36 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:23.716194 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:23.716853 21429 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 36.
I20260709 12:27:23.718528 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:23.719410  1681 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:27:23.719802  1681 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.720176  1681 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:27:23.726181  1690 raft_consensus.cc:493] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.726984  1683 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.727609  1683 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:23.730014  1683 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:23.730624  1690 raft_consensus.cc:515] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:23.732939  1690 leader_election.cc:290] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:23.733878 20849 leader_election.cc:336] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.743407  1644 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.732656  1681 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.755389  1644 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.756780  1681 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:23.757889 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54cd23b6f74a477a99461098dd4f236d" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:23.758514 21429 raft_consensus.cc:3060] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:27:23.768126  1685 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.768960  1685 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.774338 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e60e5ddf96445ab5e284dbeed45c81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 39 candidate_status { last_received { term: 38 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:23.775049 20882 raft_consensus.cc:2468] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 38.
I20260709 12:27:23.779160 21429 raft_consensus.cc:2468] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 37.
I20260709 12:27:23.779780  1685 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:23.780922  1644 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:23.724326  1682 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:23.781950 20849 leader_election.cc:336] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:23.785945 20849 leader_election.cc:304] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.786754  1681 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.786741  1682 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:23.787828  1690 raft_consensus.cc:2749] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:23.787629  1681 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.789836  1681 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:23.805279 20850 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:23.806082  1681 raft_consensus.cc:2804] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:27:23.806617  1681 raft_consensus.cc:697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
W20260709 12:27:23.807592 21400 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.807549  1681 consensus_queue.cc:237] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 36.26, Last appended by leader: 26, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:23.810359 20849 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.812328 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 44 candidate_status { last_received { term: 43 index: 28 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:23.812681 21400 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:23.813040 21427 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 43.
I20260709 12:27:23.813164 21400 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.810304 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad57e68a6a114847ac252579307658f7" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:23.815115 21429 raft_consensus.cc:3060] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:23.821171  1690 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.822026 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:23.821849  1690 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:23.822496 20849 leader_election.cc:336] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:23.823135 20849 leader_election.cc:336] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:23.823450 21400 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:23.825479  1644 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:23.823374  1683 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Leader pre-election won for term 44
I20260709 12:27:23.826653  1683 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.827052  1683 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Advancing to term 44
W20260709 12:27:23.828833 21400 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.829260 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:23.829664 20849 leader_election.cc:336] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.830159 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:23.831080 20849 leader_election.cc:336] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.831583  1682 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:23.831607 20849 leader_election.cc:304] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.833751  1682 raft_consensus.cc:2749] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:23.834630  1644 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:23.833171 20849 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:23.836441 20849 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:23.836984 20849 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.840584 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
W20260709 12:27:23.855943 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:23.857574 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:27:23.857407  1690 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:23.863179 21429 raft_consensus.cc:2468] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 34.
W20260709 12:27:23.870275 20849 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:23.873446 20849 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 50c1ab13351d4427b836fbb449c6d7bb
I20260709 12:27:23.873461 20850 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:27:23.874722  1690 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:23.875242  1682 raft_consensus.cc:2804] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:27:23.875698  1683 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.876681  1682 raft_consensus.cc:697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 34 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:23.878139  1683 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:23.880148 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b098f0a68194f18be24dc4f3742a4ab" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 44 candidate_status { last_received { term: 43 index: 28 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:23.880900 21429 raft_consensus.cc:3060] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Advancing to term 44
I20260709 12:27:23.880052  1682 consensus_queue.cc:237] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.24, Last appended by leader: 24, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:23.885470 20849 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.897825 20627 catalog_manager.cc:5697] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 36 to 37, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 37 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:23.901228  1681 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:23.901870  1681 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:23.906220 21427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:23.907586  1681 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
W20260709 12:27:23.912174 20849 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.914716 21429 raft_consensus.cc:2468] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 44 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 44.
I20260709 12:27:23.915865 20850 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 44 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:27:23.917174  1695 raft_consensus.cc:2804] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader election won for term 44
I20260709 12:27:23.922458  1695 raft_consensus.cc:697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 44 LEADER]: Becoming Leader. State: Replica: 116023b4c0c94bfea737fc238bda3838, State: Running, Role: LEADER
I20260709 12:27:23.924026  1695 consensus_queue.cc:237] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 43.28, Last appended by leader: 28, Current term: 44, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:23.922983 20849 leader_election.cc:336] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:23.927837 20849 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:23.928714  1683 raft_consensus.cc:2749] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:23.934623 20627 catalog_manager.cc:5697] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 33 to 34, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 34 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:23.960071 20627 catalog_manager.cc:5697] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 reported cstate change: term changed from 43 to 44, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 116023b4c0c94bfea737fc238bda3838 (127.19.36.195). New cstate: current_term: 44 leader_uuid: "116023b4c0c94bfea737fc238bda3838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:23.976769 20849 consensus_peers.cc:597] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:23.977120 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:24.104929  1686 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.105558  1686 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.106966  1671 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.107771  1686 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:24.107527  1671 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.108224  1685 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.108750  1685 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.109891  1671 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:24.110814  1685 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:24.112103 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:24.112712 20882 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 39.
I20260709 12:27:24.113911 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:24.114710  1671 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:27:24.115073  1671 raft_consensus.cc:493] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.115458  1671 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:24.121841  1695 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.122409  1695 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.126178  1695 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:24.138437  1686 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.138841  1695 raft_consensus.cc:493] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.139103  1686 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.145358  1686 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:24.148098  1695 raft_consensus.cc:515] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:24.153195  1686 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.153776  1686 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:24.154181 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:24.155946  1686 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:24.131750  1671 raft_consensus.cc:515] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.159142 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:24.163149 20882 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 32.
I20260709 12:27:24.164167 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1668b10d46ef4bfeaa2dc7ccbad48f81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:24.158316  1695 leader_election.cc:290] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:24.168690  1686 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.169245  1686 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.171504  1686 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:24.161854  1671 leader_election.cc:290] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:24.162592 20878 raft_consensus.cc:1275] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
W20260709 12:27:24.174998 20849 consensus_peers.cc:597] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:24.175895 20849 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.164793 20880 raft_consensus.cc:3060] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:27:24.164788 21399 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:24.172660 21429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb" is_pre_election: true
I20260709 12:27:24.177628  1685 consensus_queue.cc:1048] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:27:24.177796 21429 raft_consensus.cc:2468] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116023b4c0c94bfea737fc238bda3838 in term 32.
I20260709 12:27:24.172683  1686 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.180142  1686 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.185240 20882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b73d72c3ae4b17abbf64b777ddb8c2" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838" is_pre_election: true
I20260709 12:27:24.185897 20882 raft_consensus.cc:2468] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 34.
I20260709 12:27:24.187135  1686 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:24.185089  1671 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:24.187963  1671 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.188361  1671 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:24.189849 20880 raft_consensus.cc:2468] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c1ab13351d4427b836fbb449c6d7bb in term 40.
I20260709 12:27:24.210418 21399 leader_election.cc:304] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 
I20260709 12:27:24.211284  1686 raft_consensus.cc:2804] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:27:24.211774  1686 raft_consensus.cc:697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Becoming Leader. State: Replica: 50c1ab13351d4427b836fbb449c6d7bb, State: Running, Role: LEADER
I20260709 12:27:24.212677  1686 consensus_queue.cc:237] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 39.27, Last appended by leader: 27, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:24.196426 21400 leader_election.cc:336] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:24.220268 21400 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:24.220960 21400 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:24.221480 21400 consensus_peers.cc:597] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:24.228885 20849 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:24.245997  1685 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.251500 20850 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 657bc62d20374e539b23641715f22745
I20260709 12:27:24.252796  1681 raft_consensus.cc:2804] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:24.253306  1681 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.253826  1681 raft_consensus.cc:3060] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:24.272570 20849 leader_election.cc:336] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.274701 20849 leader_election.cc:304] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.275633  1695 raft_consensus.cc:2749] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:24.280795  1671 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting leader election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.283687  1685 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.286026  1685 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:24.292802 21400 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:24.303582 20880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "50c1ab13351d4427b836fbb449c6d7bb" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "116023b4c0c94bfea737fc238bda3838"
I20260709 12:27:24.307384 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:24.308563 21400 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:24.308974 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.309590  1671 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:24.312414  1686 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:24.313154  1686 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:24.354707 21400 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:24.355302 21400 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:24.355974 21400 leader_election.cc:336] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.360940  1681 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.362231 20880 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 50c1ab13351d4427b836fbb449c6d7bb in current term 33: Already voted for candidate 116023b4c0c94bfea737fc238bda3838 in this term.
I20260709 12:27:24.383611  1681 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Requested vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339)
I20260709 12:27:24.394692  1685 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:24.395334  1685 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:24.398267  1685 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:24.405040 21429 raft_consensus.cc:1275] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 36 index: 26. Preceding OpId from leader: term: 37 index: 27. (index mismatch)
I20260709 12:27:24.416514 21430 raft_consensus.cc:1275] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:27:24.419857 21430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047135f71834dbab6b869eb3c081b81" candidate_uuid: "116023b4c0c94bfea737fc238bda3838" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "50c1ab13351d4427b836fbb449c6d7bb"
I20260709 12:27:24.420627 21430 raft_consensus.cc:2393] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 116023b4c0c94bfea737fc238bda3838 in current term 33: Already voted for candidate 50c1ab13351d4427b836fbb449c6d7bb in this term.
I20260709 12:27:24.426280 21430 raft_consensus.cc:1275] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 44 FOLLOWER]: Refusing update from remote peer 116023b4c0c94bfea737fc238bda3838: Log matching property violated. Preceding OpId in replica: term: 43 index: 28. Preceding OpId from leader: term: 44 index: 29. (index mismatch)
W20260709 12:27:24.428548 21400 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:24.428987 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.435351  1685 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:24.437518 21400 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:24.438407 21400 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:24.438921 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:24.441004 21400 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.441409 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:24.485081 20849 consensus_peers.cc:597] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:24.485806 20849 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.487950  1685 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
W20260709 12:27:24.500742 21400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:43085, user_credentials={real_user=slave}} blocked reactor thread for 59750us
W20260709 12:27:24.501267 21400 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.501677 21400 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838, 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.502580  1671 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:24.610713 20849 consensus_peers.cc:597] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:24.620694 21400 leader_election.cc:336] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:24.622387 21400 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.622810 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.627589  1682 consensus_queue.cc:1048] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.001s
W20260709 12:27:24.629475 21400 leader_election.cc:336] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:24.629760  1644 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:27:24.629982 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.630909  1685 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:24.641997  1681 consensus_queue.cc:1048] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:27:24.642405 20850 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 50c1ab13351d4427b836fbb449c6d7bb, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.643401  1681 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:27:24.648268  1681 consensus_queue.cc:1048] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20260709 12:27:24.652844 20849 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111) [suppressed 542 similar messages]
W20260709 12:27:24.653674 21400 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:24.655874 21400 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:24.655541 20627 catalog_manager.cc:5697] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb reported cstate change: term changed from 39 to 40, leader changed from 657bc62d20374e539b23641715f22745 (127.19.36.194) to 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196). New cstate: current_term: 40 leader_uuid: "50c1ab13351d4427b836fbb449c6d7bb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:24.653730 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:24.656383 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:24.657230  1671 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:27:24.657861 20849 consensus_peers.cc:597] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.658308 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.658391 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:24.658891 20849 consensus_peers.cc:597] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:24.668741 21400 consensus_peers.cc:597] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.669492 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:27:24.715232 20880 raft_consensus.cc:1275] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Refusing update from remote peer 50c1ab13351d4427b836fbb449c6d7bb: Log matching property violated. Preceding OpId in replica: term: 39 index: 27. Preceding OpId from leader: term: 40 index: 28. (index mismatch)
W20260709 12:27:24.721261 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:24.723090 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:27:24.727300  1686 consensus_queue.cc:1048] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20260709 12:27:24.746515 21400 consensus_peers.cc:597] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:24.805576 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:24.830852 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.877983 20849 consensus_peers.cc:597] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.901444 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:24.905277 20849 consensus_peers.cc:597] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.921252 21400 consensus_peers.cc:597] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.938768 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:24.978168 20849 consensus_peers.cc:597] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:24.996264 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:25.002656 20849 consensus_peers.cc:597] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.038038 21400 consensus_peers.cc:597] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.104245 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:25.134064 21400 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:25.157151 20849 consensus_peers.cc:597] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.169553 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:25.174659 20849 consensus_peers.cc:597] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.175237 20849 consensus_peers.cc:597] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.187898 20849 consensus_peers.cc:597] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.195122 20849 consensus_peers.cc:597] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.207149 20849 consensus_peers.cc:597] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:25.223410 20849 consensus_peers.cc:597] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:25.251091 20849 consensus_peers.cc:597] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:25.264696  1681 raft_consensus.cc:493] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:25.265235  1681 raft_consensus.cc:515] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:25.267028  1681 leader_election.cc:290] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:25.272419 20849 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:25.272990 20849 leader_election.cc:336] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:25.273381 20849 leader_election.cc:304] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:25.273990  1681 raft_consensus.cc:2749] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
W20260709 12:27:25.280157 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:25.290105  1671 raft_consensus.cc:493] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:25.290680  1671 raft_consensus.cc:515] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:25.292476  1671 leader_election.cc:290] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:25.298061 21400 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:25.331799 20849 consensus_peers.cc:597] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:25.334949 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
W20260709 12:27:25.347733 20849 consensus_peers.cc:597] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:27:25.356003  1695 raft_consensus.cc:493] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:25.356516  1695 raft_consensus.cc:515] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:25.358453  1695 leader_election.cc:290] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:25.363413 20849 leader_election.cc:336] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:25.365248 20849 leader_election.cc:336] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:25.365648 20849 leader_election.cc:304] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:25.366309  1695 raft_consensus.cc:2749] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
W20260709 12:27:25.387102 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:25.389881 21400 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:25.406608 20849 consensus_peers.cc:597] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:25.451603 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
W20260709 12:27:25.480330 21399 consensus_peers.cc:597] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:25.509094 21399 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:25.515659 21399 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.516443  1695 raft_consensus.cc:493] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:25.517004  1695 raft_consensus.cc:515] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:25.519181  1695 leader_election.cc:290] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:25.524034 20849 leader_election.cc:336] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:25.524782 20849 leader_election.cc:336] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:25.525163 20849 leader_election.cc:304] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:25.525928  1695 raft_consensus.cc:2749] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:25.532996 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:25.534124 19603 tablet_replica.cc:333] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.535313 19603 raft_consensus.cc:2243] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.536161 19603 raft_consensus.cc:2272] T 8f3454624b7c40c6934257c77f2174de P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.538507 19603 tablet_replica.cc:333] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.539086 19603 raft_consensus.cc:2243] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 43 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.540038 19603 raft_consensus.cc:2272] T 3894130cfc2549b5a8c99d056675ef94 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.542071 19603 tablet_replica.cc:333] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.542855 19603 raft_consensus.cc:2243] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.544051 19603 raft_consensus.cc:2272] T ad57e68a6a114847ac252579307658f7 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.547626 19603 tablet_replica.cc:333] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
W20260709 12:27:25.547684 20849 consensus_peers.cc:597] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:25.548530 19603 raft_consensus.cc:2243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.549295 19603 raft_consensus.cc:2272] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.551914 19603 tablet_replica.cc:333] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.552634 19603 raft_consensus.cc:2243] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Raft consensus shutting down.
W20260709 12:27:25.552649 21399 consensus_peers.cc:597] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.553552 19603 raft_consensus.cc:2272] T 0b31661191c14b44ac964555853900f1 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.555624 19603 tablet_replica.cc:333] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.556236 19603 raft_consensus.cc:2243] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 44 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.557168 19603 raft_consensus.cc:2272] T 2b098f0a68194f18be24dc4f3742a4ab P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.557196  1695 raft_consensus.cc:493] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:25.557811  1695 raft_consensus.cc:515] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } }
I20260709 12:27:25.560492 19603 tablet_replica.cc:333] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.561415 19603 raft_consensus.cc:2243] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.562538 19603 raft_consensus.cc:2272] T 335e9cdc3f6745ab9b44f08b6e8b2118 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.565923 19603 tablet_replica.cc:333] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.566677 19603 raft_consensus.cc:2243] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.568747 19603 raft_consensus.cc:2272] T 54cd23b6f74a477a99461098dd4f236d P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.572137 19603 tablet_replica.cc:333] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
W20260709 12:27:25.572284 20849 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:25.561414  1695 leader_election.cc:290] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:25.572974 19603 raft_consensus.cc:2243] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.573941 19603 raft_consensus.cc:2272] T 13cfeabebc54415b87ab9ce1d8a4caab P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.575541 20849 leader_election.cc:336] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:25.576067 20849 leader_election.cc:304] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116023b4c0c94bfea737fc238bda3838; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:25.576723 19603 tablet_replica.cc:333] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.576776  1695 raft_consensus.cc:2749] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:25.578545 19603 raft_consensus.cc:2243] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.579241 19603 raft_consensus.cc:2272] T 87775c62d4bc48d3b1831b4bdeff2870 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.582070 19603 tablet_replica.cc:333] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.582860 19603 raft_consensus.cc:2243] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Raft consensus shutting down.
W20260709 12:27:25.583940 21399 consensus_peers.cc:597] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.584151 19603 raft_consensus.cc:2272] T 942b8c65bd1e42a99f971b099d0e19e6 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.586941 19603 tablet_replica.cc:333] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.587816 19603 raft_consensus.cc:2243] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.588464 19603 raft_consensus.cc:2272] T c3046bc775e64931b4a1ff963cccf0e2 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.591138 19603 tablet_replica.cc:333] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.591921 19603 raft_consensus.cc:2243] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.593331 19603 raft_consensus.cc:2272] T c00fffbb02ff4bc2a974d8a5b179a97d P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.595160 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:25.596856 19603 tablet_replica.cc:333] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.597694 19603 raft_consensus.cc:2243] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.598292 19603 raft_consensus.cc:2272] T 210839e60f364f8cab9f53ff440c461d P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.600965 19603 tablet_replica.cc:333] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.601748 19603 raft_consensus.cc:2243] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.602319 19603 raft_consensus.cc:2272] T 7cd857e30c3240c5a4fdd11bc0602c00 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.605201 19603 tablet_replica.cc:333] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.605963 19603 raft_consensus.cc:2243] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.606626 19603 raft_consensus.cc:2272] T 8f381d2ab1764a739c2d1b3d41e65181 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.609298 19603 tablet_replica.cc:333] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
W20260709 12:27:25.609330 21456 debug-util.cc:398] Leaking SignalData structure 0x7b08001c88c0 after lost signal to thread 19606
I20260709 12:27:25.610814 19603 raft_consensus.cc:2243] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:25.611004 21456 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad3e0 after lost signal to thread 20667
I20260709 12:27:25.611596 19603 raft_consensus.cc:2272] T a9c17d2fc98f481597ce7b97e8f98c43 P 116023b4c0c94bfea737fc238bda3838 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.612187 21456 debug-util.cc:398] Leaking SignalData structure 0x7b08003034e0 after lost signal to thread 20908
W20260709 12:27:25.613211 21456 debug-util.cc:398] Leaking SignalData structure 0x7b0800224c40 after lost signal to thread 21459
I20260709 12:27:25.614591 19603 tablet_replica.cc:333] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.615602 19603 raft_consensus.cc:2243] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.616556 19603 raft_consensus.cc:2272] T 72e60e5ddf96445ab5e284dbeed45c81 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.619994 19603 tablet_replica.cc:333] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.620855 19603 raft_consensus.cc:2243] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.621970 19603 raft_consensus.cc:2272] T 6dc725b7c88e47f693e3a16879798246 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.624895 19603 tablet_replica.cc:333] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.625725 19603 raft_consensus.cc:2243] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.626300 19603 raft_consensus.cc:2272] T c6fe576036c04c149240cd6107285eca P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.629046 19603 tablet_replica.cc:333] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.629829 19603 raft_consensus.cc:2243] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.630446 19603 raft_consensus.cc:2272] T c5ef54d60e4646b68d43519ad3fffa6d P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.633317 19603 tablet_replica.cc:333] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.634114 19603 raft_consensus.cc:2243] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.634912 19603 raft_consensus.cc:2272] T 761ea542665e419c8a63392e3014efa7 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.637718 19603 tablet_replica.cc:333] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.638550 19603 raft_consensus.cc:2243] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.639101 19603 raft_consensus.cc:2272] T 1047135f71834dbab6b869eb3c081b81 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.640753 21399 consensus_peers.cc:597] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.641800 19603 tablet_replica.cc:333] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.642746 19603 raft_consensus.cc:2243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.643513 19603 raft_consensus.cc:2272] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.650823 21399 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.652812 19603 tablet_replica.cc:333] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.654145 19603 raft_consensus.cc:2243] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 42 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.655292 19603 raft_consensus.cc:2272] T a32323ce10e140bb895085f97c97fa46 P 116023b4c0c94bfea737fc238bda3838 [term 42 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.657904 19603 tablet_replica.cc:333] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.658677 19603 raft_consensus.cc:2243] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.659250 19603 raft_consensus.cc:2272] T d46401ecb9b443118647ee361081a01e P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.660517 21399 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.662040 19603 tablet_replica.cc:333] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.662894 19603 raft_consensus.cc:2243] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.663633 19603 raft_consensus.cc:2272] T 26ced453445642b09bda798a2963748b P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.666376 19603 tablet_replica.cc:333] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.667157 19603 raft_consensus.cc:2243] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.667759 19603 raft_consensus.cc:2272] T e36a8df2d02f44ca965324882aba3ef2 P 116023b4c0c94bfea737fc238bda3838 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.670341 19603 tablet_replica.cc:333] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
W20260709 12:27:25.670507 21399 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.671228 19603 raft_consensus.cc:2243] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.672372 19603 raft_consensus.cc:2272] T 986267de598f431fb25840d9d741436d P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.680231 19603 tablet_replica.cc:333] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
W20260709 12:27:25.681048 21399 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.681041 19603 raft_consensus.cc:2243] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.682633 19603 raft_consensus.cc:2272] T be08f375c22d436fa6e99ac7bd7f6c93 P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.685577 19603 tablet_replica.cc:333] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.686344 19603 raft_consensus.cc:2243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.687011 19603 raft_consensus.cc:2272] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.689807 19603 tablet_replica.cc:333] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.690547 19603 raft_consensus.cc:2243] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.691606 19603 raft_consensus.cc:2272] T 38d64dd1339c4abfb4ef05669402e801 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.694188 19603 tablet_replica.cc:333] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.694924 19603 raft_consensus.cc:2243] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.696017 19603 raft_consensus.cc:2272] T 1be2fc0a60b1472a9f537a9cd3bcc133 P 116023b4c0c94bfea737fc238bda3838 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.698550 19603 tablet_replica.cc:333] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.699255 19603 raft_consensus.cc:2243] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.699949 19603 raft_consensus.cc:2272] T a48bec319402467881543ac686fb8f0c P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.701119  1671 raft_consensus.cc:493] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:25.701757  1671 raft_consensus.cc:515] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 24 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:25.702550 19603 tablet_replica.cc:333] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.703454 19603 raft_consensus.cc:2243] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 44 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.704404 19603 raft_consensus.cc:2272] T 6a458432a57843e4af30e5d9af5dfcfe P 116023b4c0c94bfea737fc238bda3838 [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.707161 19603 tablet_replica.cc:333] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.709067 19603 raft_consensus.cc:2243] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.709743 19603 raft_consensus.cc:2272] T 0f517f5d84b3437eb2fd2ae144fc6380 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.704754  1671 leader_election.cc:290] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:25.713022 21400 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:25.714531 19603 tablet_replica.cc:333] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.715345 19603 raft_consensus.cc:2243] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.716264 19603 raft_consensus.cc:2272] T 981a4d0731e940b79eccfb33d1928fb4 P 116023b4c0c94bfea737fc238bda3838 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.719867 19603 tablet_replica.cc:333] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.720641 19603 raft_consensus.cc:2243] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.721663 19603 raft_consensus.cc:2272] T 62396122903848d4a49e6d637c5c8b49 P 116023b4c0c94bfea737fc238bda3838 [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.724246 19603 tablet_replica.cc:333] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.724972 19603 raft_consensus.cc:2243] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.726439 19603 raft_consensus.cc:2272] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.729387 19603 tablet_replica.cc:333] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.730105 19603 raft_consensus.cc:2243] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.730917 19603 raft_consensus.cc:2272] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 116023b4c0c94bfea737fc238bda3838 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.733695 19603 tablet_replica.cc:333] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.734447 19603 raft_consensus.cc:2243] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.735586 19603 raft_consensus.cc:2272] T 37c593b286274096a8c431979866be8a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.739959 21399 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:25.740334 19603 tablet_replica.cc:333] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.742542 21399 leader_election.cc:304] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:25.743439  1671 raft_consensus.cc:2749] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:27:25.744338 19603 raft_consensus.cc:2243] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.746310 19603 raft_consensus.cc:2272] T 70cdcdfe95804895979397c927fae46f P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.749011 19603 tablet_replica.cc:333] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.749677 19603 raft_consensus.cc:2243] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 39 LEADER]: Raft consensus shutting down.
W20260709 12:27:25.750494 21399 consensus_peers.cc:597] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.750866 19603 raft_consensus.cc:2272] T c1303aa66bdb4474b6bcb604461e74a5 P 116023b4c0c94bfea737fc238bda3838 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.756775 19603 tablet_replica.cc:333] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.757530 19603 raft_consensus.cc:2243] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.758814 19603 raft_consensus.cc:2272] T f15b5f1ac1444b63bad60a89fa9f8a29 P 116023b4c0c94bfea737fc238bda3838 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.765319 19603 tablet_replica.cc:333] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.766129 19603 raft_consensus.cc:2243] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.767307 19603 raft_consensus.cc:2272] T d5374e2302374917828294b0a4e3313a P 116023b4c0c94bfea737fc238bda3838 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.770038 19603 tablet_replica.cc:333] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.771178 19603 raft_consensus.cc:2243] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.772390 19603 raft_consensus.cc:2272] T d5e3c520e78241879a779a912c66a4ba P 116023b4c0c94bfea737fc238bda3838 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.775588 19603 tablet_replica.cc:333] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
W20260709 12:27:25.775995 21399 consensus_peers.cc:597] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.776404 19603 raft_consensus.cc:2243] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:25.777235 19603 raft_consensus.cc:2272] T ba4e3bf73fae46119cf926d5af0782a6 P 116023b4c0c94bfea737fc238bda3838 [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:25.781754 19603 tablet_replica.cc:333] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838: stopping tablet replica
I20260709 12:27:25.782510 19603 raft_consensus.cc:2243] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:25.783710 19603 raft_consensus.cc:2272] T d82740a2c094456a91cf0e411dd72159 P 116023b4c0c94bfea737fc238bda3838 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:25.804219 21399 consensus_peers.cc:597] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.814785  1671 raft_consensus.cc:493] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:25.815359  1671 raft_consensus.cc:515] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:25.817471  1671 leader_election.cc:290] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:25.823621 21400 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:25.918330 21399 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:25.929343 21399 consensus_peers.cc:597] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Couldn't send request to peer 116023b4c0c94bfea737fc238bda3838. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:25.983749 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:27:26.127789  1716 raft_consensus.cc:493] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:26.128381  1716 raft_consensus.cc:515] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.130409  1716 leader_election.cc:290] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:26.148594 21400 consensus_peers.cc:597] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:26.149125 21400 consensus_peers.cc:597] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:26.149535 21400 consensus_peers.cc:597] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:26.149945 21400 consensus_peers.cc:597] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:26.150349 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:26.150936 21400 leader_election.cc:336] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.151379 21400 leader_election.cc:304] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.152245  1671 raft_consensus.cc:2749] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
W20260709 12:27:26.153614 21400 leader_election.cc:336] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:26.154441 21400 leader_election.cc:336] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.154805 21400 leader_election.cc:304] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.155579  1671 raft_consensus.cc:2749] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:26.157536 21400 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:26.170292 21400 leader_election.cc:336] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.170763 21400 leader_election.cc:304] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.171536  1716 raft_consensus.cc:2749] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:26.194329  1671 raft_consensus.cc:493] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:26.194902  1671 raft_consensus.cc:515] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.196997  1671 leader_election.cc:290] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:26.206029 21400 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:26.208437 21400 leader_election.cc:336] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.208925 21400 leader_election.cc:304] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.209709  1671 raft_consensus.cc:2749] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:27:26.242931  1671 raft_consensus.cc:493] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:26.243503  1671 raft_consensus.cc:515] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.245492  1671 leader_election.cc:290] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:26.266952 21400 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:26.267752 21400 leader_election.cc:336] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:26.268155 21400 leader_election.cc:304] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.268885  1671 raft_consensus.cc:2749] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:26.333786 21400 consensus_peers.cc:597] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:26.356230 21400 consensus_peers.cc:597] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:26.367444  1671 raft_consensus.cc:493] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:26.368007  1671 raft_consensus.cc:515] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.370584  1671 leader_election.cc:290] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:26.374975  1671 raft_consensus.cc:493] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:26.375440  1671 raft_consensus.cc:515] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.377305  1671 leader_election.cc:290] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:26.377926 21399 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:26.379347 21400 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.379839 21400 leader_election.cc:304] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.380479  1671 raft_consensus.cc:2749] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:26.382845 21400 leader_election.cc:336] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:26.388270 21400 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:26.388971 21400 leader_election.cc:336] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.389394 21400 leader_election.cc:304] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.390105  1671 raft_consensus.cc:2749] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:26.660776  1671 raft_consensus.cc:493] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 44 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.661345  1671 raft_consensus.cc:515] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 44 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.663180  1671 leader_election.cc:290] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 45 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.668092 21400 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
W20260709 12:27:26.670728 21399 leader_election.cc:336] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 45 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:27:26.671147 21399 leader_election.cc:304] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 45 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.671914  1715 raft_consensus.cc:2749] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 44 FOLLOWER]: Leader pre-election lost for term 45. Reason: could not achieve majority
I20260709 12:27:26.708806  1715 raft_consensus.cc:493] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:26.709359  1715 raft_consensus.cc:515] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.711455  1715 leader_election.cc:290] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
W20260709 12:27:26.716190 21400 leader_election.cc:336] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:26.716707 21400 leader_election.cc:336] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.717023 21400 leader_election.cc:304] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.717856  1715 raft_consensus.cc:2749] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:26.728763  1715 raft_consensus.cc:493] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.729303  1715 raft_consensus.cc:515] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.733512  1715 leader_election.cc:290] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.736505 21399 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:26.737965 21400 consensus_peers.cc:597] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:26.740479 21400 leader_election.cc:336] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.743121  1716 raft_consensus.cc:493] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 657bc62d20374e539b23641715f22745)
I20260709 12:27:26.743682  1716 raft_consensus.cc:515] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.740896 21400 leader_election.cc:304] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.745719  1716 leader_election.cc:290] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085)
I20260709 12:27:26.746388  1671 raft_consensus.cc:2749] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:27:26.753728 21400 consensus_peers.cc:597] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:26.754268 21400 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:26.755285 21400 leader_election.cc:336] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.755739 21400 leader_election.cc:304] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.756467  1671 raft_consensus.cc:2749] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:27:26.779920  1671 raft_consensus.cc:493] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.780586  1671 raft_consensus.cc:515] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.783149  1671 leader_election.cc:290] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:26.787322  1671 raft_consensus.cc:493] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.787963  1671 raft_consensus.cc:515] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.790315  1671 leader_election.cc:290] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.790561 21399 leader_election.cc:336] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:26.791074 21400 leader_election.cc:336] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.791555 21400 leader_election.cc:304] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
W20260709 12:27:26.797112 21399 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:27:26.808687  1715 raft_consensus.cc:493] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.809254  1715 raft_consensus.cc:515] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.811794  1715 leader_election.cc:290] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:26.812525  1671 raft_consensus.cc:2749] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:27:26.819331  1671 raft_consensus.cc:493] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.823917  1671 raft_consensus.cc:515] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.825663  1715 raft_consensus.cc:493] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:27:26.828996 21399 leader_election.cc:336] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:27:26.829084  1715 raft_consensus.cc:515] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.826403  1716 raft_consensus.cc:493] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.830176  1716 raft_consensus.cc:515] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.831552  1715 leader_election.cc:290] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.828051 21400 leader_election.cc:336] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:26.832245 21400 leader_election.cc:304] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:26.832923  1716 leader_election.cc:290] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
I20260709 12:27:26.833050  1715 raft_consensus.cc:2749] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:26.836979  1671 leader_election.cc:290] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.844357 21399 leader_election.cc:336] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:26.844946 21399 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:26.845417 21399 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:26.846452 21400 consensus_peers.cc:597] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Couldn't send request to peer 657bc62d20374e539b23641715f22745. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:26.846819 21400 leader_election.cc:336] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.847117 21400 leader_election.cc:304] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.847852  1671 raft_consensus.cc:2749] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:26.849000 21400 leader_election.cc:336] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:26.849364 21400 leader_election.cc:304] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:26.849942  1671 raft_consensus.cc:2749] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:26.851826 21400 leader_election.cc:336] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.852263 21400 leader_election.cc:304] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.852898  1671 raft_consensus.cc:2749] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:26.853098 21400 leader_election.cc:336] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.853518 21400 leader_election.cc:304] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:26.854084  1671 raft_consensus.cc:2749] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:27:26.874059  1671 raft_consensus.cc:493] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.874605  1671 raft_consensus.cc:515] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.881119  1671 leader_election.cc:290] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.883616 21400 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:26.887435 21400 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:26.887981 21400 leader_election.cc:304] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:26.888540 21399 leader_election.cc:336] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:27:26.889021  1671 raft_consensus.cc:2749] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:26.931720  1671 raft_consensus.cc:493] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.932359  1671 raft_consensus.cc:515] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.934508  1671 leader_election.cc:290] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.941632 21399 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:27:26.946745  1671 raft_consensus.cc:493] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:26.947424  1671 raft_consensus.cc:515] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:26.949411  1671 leader_election.cc:290] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:26.952564 21400 leader_election.cc:336] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:26.952996 21400 leader_election.cc:304] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:26.953668  1671 raft_consensus.cc:2749] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
W20260709 12:27:26.956055 21399 leader_election.cc:336] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:26.956681 21400 leader_election.cc:336] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:26.957090 21400 leader_election.cc:304] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:26.957813  1671 raft_consensus.cc:2749] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:27.083062 21400 consensus_peers.cc:597] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:27.139077  1671 raft_consensus.cc:493] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:27.139678  1671 raft_consensus.cc:515] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:27.141696  1671 leader_election.cc:290] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:27.151140 21399 leader_election.cc:336] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:27:27.152137  1671 raft_consensus.cc:493] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
W20260709 12:27:27.152277 21400 consensus_peers.cc:597] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:27.152735  1671 raft_consensus.cc:515] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } attrs { replace: true } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
W20260709 12:27:27.153254 21400 leader_election.cc:336] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:27.153908 21400 leader_election.cc:304] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 657bc62d20374e539b23641715f22745
I20260709 12:27:27.154727  1715 raft_consensus.cc:2749] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:27.155992  1671 leader_election.cc:290] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:27.164315 21400 leader_election.cc:336] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:27.165366 21400 leader_election.cc:336] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:27.165843 21400 leader_election.cc:304] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 49323f66286f4798b268f216bb62b3be, 657bc62d20374e539b23641715f22745
W20260709 12:27:27.166461 21399 leader_election.cc:336] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:27:27.167726  1715 raft_consensus.cc:2749] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:27.241040  1671 raft_consensus.cc:493] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 116023b4c0c94bfea737fc238bda3838)
I20260709 12:27:27.241580  1671 raft_consensus.cc:515] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49323f66286f4798b268f216bb62b3be" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 34333 } } peers { permanent_uuid: "657bc62d20374e539b23641715f22745" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 43085 } } peers { permanent_uuid: "116023b4c0c94bfea737fc238bda3838" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 45387 } attrs { replace: true } } peers { permanent_uuid: "50c1ab13351d4427b836fbb449c6d7bb" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 44339 } attrs { promote: false } }
I20260709 12:27:27.244078  1671 leader_election.cc:290] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333), 657bc62d20374e539b23641715f22745 (127.19.36.194:43085), 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387)
W20260709 12:27:27.249728 21399 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
W20260709 12:27:27.252534 21400 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
I20260709 12:27:27.253003 21400 leader_election.cc:304] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 50c1ab13351d4427b836fbb449c6d7bb; no voters: 116023b4c0c94bfea737fc238bda3838, 49323f66286f4798b268f216bb62b3be
I20260709 12:27:27.253677  1671 raft_consensus.cc:2749] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:27.253863 21400 leader_election.cc:336] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 657bc62d20374e539b23641715f22745 (127.19.36.194:43085): Network error: Client connection negotiation failed: client connection to 127.19.36.194:43085: connect: Connection refused (error 111)
I20260709 12:27:27.279794 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
I20260709 12:27:27.381157 19603 tablet_server.cc:179] TabletServer@127.19.36.196:0 shutting down...
I20260709 12:27:27.441607 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:27.446488 19603 tablet_replica.cc:333] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.447499 19603 raft_consensus.cc:2243] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.448283 19603 raft_consensus.cc:2272] T c0b73d72c3ae4b17abbf64b777ddb8c2 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.452080 19603 tablet_replica.cc:333] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.452920 19603 raft_consensus.cc:2243] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.453614 19603 raft_consensus.cc:2272] T ae342ffbd6894fb1aa23aa48f4d2dd1b P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.456457 19603 tablet_replica.cc:333] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.457206 19603 raft_consensus.cc:2243] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.457818 19603 raft_consensus.cc:2272] T 37c593b286274096a8c431979866be8a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.460558 19603 tablet_replica.cc:333] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.461414 19603 raft_consensus.cc:2243] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.462685 19603 raft_consensus.cc:2272] T 7ccae903e8484f8d8c3af847e2c4c3d4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.465390 19603 tablet_replica.cc:333] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.466264 19603 raft_consensus.cc:2243] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.466807 19603 raft_consensus.cc:2272] T 1047135f71834dbab6b869eb3c081b81 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.469676 19603 tablet_replica.cc:333] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.470450 19603 raft_consensus.cc:2243] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.471781 19603 raft_consensus.cc:2272] T a9c17d2fc98f481597ce7b97e8f98c43 P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.474349 19603 tablet_replica.cc:333] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.475135 19603 raft_consensus.cc:2243] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.476227 19603 raft_consensus.cc:2272] T f15b5f1ac1444b63bad60a89fa9f8a29 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.478997 19603 tablet_replica.cc:333] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.479794 19603 raft_consensus.cc:2243] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.480381 19603 raft_consensus.cc:2272] T d5c70f45aef146f4bcb47f1498221b99 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.483070 19603 tablet_replica.cc:333] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.483915 19603 raft_consensus.cc:2243] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.484941 19603 raft_consensus.cc:2272] T b7040eacc899473abc05a913812e82a4 P 50c1ab13351d4427b836fbb449c6d7bb [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.487617 19603 tablet_replica.cc:333] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.488391 19603 raft_consensus.cc:2243] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.489696 19603 raft_consensus.cc:2272] T 62396122903848d4a49e6d637c5c8b49 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.492614 19603 tablet_replica.cc:333] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.493404 19603 raft_consensus.cc:2243] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.493953 19603 raft_consensus.cc:2272] T 981a4d0731e940b79eccfb33d1928fb4 P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.496584 19603 tablet_replica.cc:333] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.497318 19603 raft_consensus.cc:2243] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.497889 19603 raft_consensus.cc:2272] T ff0a54292b924069aae5a0cd6c1366d1 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.500427 19603 tablet_replica.cc:333] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.501168 19603 raft_consensus.cc:2243] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.502172 19603 raft_consensus.cc:2272] T 0f517f5d84b3437eb2fd2ae144fc6380 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.504848 19603 tablet_replica.cc:333] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.505568 19603 raft_consensus.cc:2243] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.506153 19603 raft_consensus.cc:2272] T c00fffbb02ff4bc2a974d8a5b179a97d P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.508742 19603 tablet_replica.cc:333] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.509460 19603 raft_consensus.cc:2243] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.510494 19603 raft_consensus.cc:2272] T ec190d02960a4e88af0807817b789e6c P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.513296 19603 tablet_replica.cc:333] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.514161 19603 raft_consensus.cc:2243] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.515275 19603 raft_consensus.cc:2272] T 8f3454624b7c40c6934257c77f2174de P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.517956 19603 tablet_replica.cc:333] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.518694 19603 raft_consensus.cc:2243] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.519255 19603 raft_consensus.cc:2272] T 3199c6fc2e3441849ac7dd78244a3a18 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.521883 19603 tablet_replica.cc:333] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.522642 19603 raft_consensus.cc:2243] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.523674 19603 raft_consensus.cc:2272] T c3046bc775e64931b4a1ff963cccf0e2 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.526177 19603 tablet_replica.cc:333] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.526947 19603 raft_consensus.cc:2243] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.527562 19603 raft_consensus.cc:2272] T ad57e68a6a114847ac252579307658f7 P 50c1ab13351d4427b836fbb449c6d7bb [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.530061 19603 tablet_replica.cc:333] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.530746 19603 raft_consensus.cc:2243] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.531317 19603 raft_consensus.cc:2272] T be08f375c22d436fa6e99ac7bd7f6c93 P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.533948 19603 tablet_replica.cc:333] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.534729 19603 raft_consensus.cc:2243] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.535434 19603 raft_consensus.cc:2272] T 70cdcdfe95804895979397c927fae46f P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.538057 19603 tablet_replica.cc:333] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.538833 19603 raft_consensus.cc:2243] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.540354 19603 raft_consensus.cc:2272] T 0b31661191c14b44ac964555853900f1 P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.542960 19603 tablet_replica.cc:333] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.544559 19603 raft_consensus.cc:2243] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.545266 19603 raft_consensus.cc:2272] T 75e78bb14a7646c58f2f948b2d33b2cf P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.547880 19603 tablet_replica.cc:333] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.548799 19603 raft_consensus.cc:2243] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 44 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.549261 19603 raft_consensus.cc:2272] T 2b098f0a68194f18be24dc4f3742a4ab P 50c1ab13351d4427b836fbb449c6d7bb [term 44 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.551285 19603 tablet_replica.cc:333] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.551822 19603 raft_consensus.cc:2243] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.552347 19603 raft_consensus.cc:2272] T d5374e2302374917828294b0a4e3313a P 50c1ab13351d4427b836fbb449c6d7bb [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.554222 19603 tablet_replica.cc:333] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.554767 19603 raft_consensus.cc:2243] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.555611 19603 raft_consensus.cc:2272] T 210839e60f364f8cab9f53ff440c461d P 50c1ab13351d4427b836fbb449c6d7bb [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.557588 19603 tablet_replica.cc:333] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.558283 19603 raft_consensus.cc:2243] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.559343 19603 raft_consensus.cc:2272] T 8f381d2ab1764a739c2d1b3d41e65181 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.562013 19603 tablet_replica.cc:333] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.562608 19603 raft_consensus.cc:2243] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:27.563112 21400 consensus_peers.cc:597] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb -> Peer 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): Couldn't send request to peer 49323f66286f4798b268f216bb62b3be. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:27.563180 19603 raft_consensus.cc:2272] T 54cd23b6f74a477a99461098dd4f236d P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.565503 19603 tablet_replica.cc:333] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.566080 19603 raft_consensus.cc:2243] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.566715 19603 raft_consensus.cc:2272] T 0a4e4474d2fd45438665993211ae862a P 50c1ab13351d4427b836fbb449c6d7bb [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.569331 19603 tablet_replica.cc:333] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.569950 19603 raft_consensus.cc:2243] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.570667 19603 raft_consensus.cc:2272] T f61efc9c6d5f43ac84cfe1def41de8a0 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.572695 19603 tablet_replica.cc:333] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.573247 19603 raft_consensus.cc:2243] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.574131 19603 raft_consensus.cc:2272] T 1668b10d46ef4bfeaa2dc7ccbad48f81 P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.576045 19603 tablet_replica.cc:333] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.576649 19603 raft_consensus.cc:2243] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.577667 19603 raft_consensus.cc:2272] T 4415902e5b30489691f67f4a635305b4 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.579761 19603 tablet_replica.cc:333] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.580322 19603 raft_consensus.cc:2243] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.580809 19603 raft_consensus.cc:2272] T 72e60e5ddf96445ab5e284dbeed45c81 P 50c1ab13351d4427b836fbb449c6d7bb [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.583731 19603 tablet_replica.cc:333] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.584399 19603 raft_consensus.cc:2243] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.585407 19603 raft_consensus.cc:2272] T 7cd857e30c3240c5a4fdd11bc0602c00 P 50c1ab13351d4427b836fbb449c6d7bb [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.587510 19603 tablet_replica.cc:333] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.588035 19603 raft_consensus.cc:2243] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.588541 19603 raft_consensus.cc:2272] T 626209a224524080801f3a9ad7307fbe P 50c1ab13351d4427b836fbb449c6d7bb [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.590523 19603 tablet_replica.cc:333] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.591053 19603 raft_consensus.cc:2243] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.591555 19603 raft_consensus.cc:2272] T d82740a2c094456a91cf0e411dd72159 P 50c1ab13351d4427b836fbb449c6d7bb [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.593611 19603 tablet_replica.cc:333] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.594127 19603 raft_consensus.cc:2243] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.594566 19603 raft_consensus.cc:2272] T 3efcc9a1f14042de975fa0b662cad24d P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.596455 19603 tablet_replica.cc:333] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.596973 19603 raft_consensus.cc:2243] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.597780 19603 raft_consensus.cc:2272] T 6a8b742e1fe24e6d82f1a1498f17a5fb P 50c1ab13351d4427b836fbb449c6d7bb [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.599810 19603 tablet_replica.cc:333] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.600347 19603 raft_consensus.cc:2243] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.601130 19603 raft_consensus.cc:2272] T d46401ecb9b443118647ee361081a01e P 50c1ab13351d4427b836fbb449c6d7bb [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.602941 19603 tablet_replica.cc:333] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.603499 19603 raft_consensus.cc:2243] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 43 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.604382 19603 raft_consensus.cc:2272] T 26ced453445642b09bda798a2963748b P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.606283 19603 tablet_replica.cc:333] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.606757 19603 raft_consensus.cc:2243] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:27.607298 19603 raft_consensus.cc:2272] T d5e3c520e78241879a779a912c66a4ba P 50c1ab13351d4427b836fbb449c6d7bb [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:27.609076 19603 tablet_replica.cc:333] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb: stopping tablet replica
I20260709 12:27:27.609524 19603 raft_consensus.cc:2243] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 LEADER]: Raft consensus shutting down.
I20260709 12:27:27.610404 19603 raft_consensus.cc:2272] T ba4e3bf73fae46119cf926d5af0782a6 P 50c1ab13351d4427b836fbb449c6d7bb [term 43 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:28.221993 19603 tablet_server.cc:196] TabletServer@127.19.36.196:0 shutdown complete.
I20260709 12:27:28.291222 19603 master.cc:562] Master@127.19.36.254:46039 shutting down...
W20260709 12:27:28.331136 20681 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:27:28.331491 20681 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:28.331707 20681 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:33.872082 20620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.196:44339: connect: Connection refused (error 111) [suppressed 332 similar messages]
W20260709 12:27:33.875711 20620 catalog_manager.cc:4755] TS 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): ChangeConfig:REMOVE_PEER RPC failed for tablet a9c17d2fc98f481597ce7b97e8f98c43: Network error: Client connection negotiation failed: client connection to 127.19.36.196:44339: connect: Connection refused (error 111)
W20260709 12:27:38.991693 20619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:27:38.995527 20619 catalog_manager.cc:4755] TS 49323f66286f4798b268f216bb62b3be (127.19.36.193:34333): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1047135f71834dbab6b869eb3c081b81: Network error: Client connection negotiation failed: client connection to 127.19.36.193:34333: connect: Connection refused (error 111)
W20260709 12:28:07.814267 20620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.196:44339: connect: Connection refused (error 111)
W20260709 12:28:07.818157 20620 catalog_manager.cc:4755] TS 50c1ab13351d4427b836fbb449c6d7bb (127.19.36.196:44339): ChangeConfig:REMOVE_PEER RPC failed for tablet 62396122903848d4a49e6d637c5c8b49: Network error: Client connection negotiation failed: client connection to 127.19.36.196:44339: connect: Connection refused (error 111)
W20260709 12:28:12.313237 20618 catalog_manager.cc:4755] TS 116023b4c0c94bfea737fc238bda3838 (127.19.36.195:45387): ChangeConfig:REMOVE_PEER RPC failed for tablet 37c593b286274096a8c431979866be8a: Network error: Client connection negotiation failed: client connection to 127.19.36.195:45387: connect: Connection refused (error 111)
I20260709 12:28:19.813539 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:19.814735 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.815069 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9f77a9352772433bb5809bf72fd4f0ee: stopping tablet replica
I20260709 12:28:20.086386 19603 master.cc:584] Master@127.19.36.254:46039 shutdown complete.
I20260709 12:28:20.138767 19603 test_util.cc:182] -----------------------------------------------
I20260709 12:28:20.138984 19603 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599599738032-19603-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (441037 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:28:20.142346 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:40093
I20260709 12:28:20.143545 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:20.149533  1735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.149628  1736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.150458  1738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:20.151054 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:20.155901 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:20.156087 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:20.156198 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600100156188 us; error 0 us; skew 500 ppm
I20260709 12:28:20.156684 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:20.159096 19603 webserver.cc:533] Webserver started at http://127.19.36.254:45389/ using document root <none> and password file <none>
I20260709 12:28:20.159631 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:20.159782 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:20.160000 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:20.161222 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "07aa32703cc844cda506fcb5d3109a62"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.165759 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:28:20.169114  1743 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.169847 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:28:20.170145 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "07aa32703cc844cda506fcb5d3109a62"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.170418 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:20.185209 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:20.186462 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:20.226946 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:40093
I20260709 12:28:20.227062  1794 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:40093 every 8 connection(s)
I20260709 12:28:20.231400  1795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.244024  1795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62: Bootstrap starting.
I20260709 12:28:20.249441  1795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:20.254037  1795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62: No bootstrap required, opened a new log
I20260709 12:28:20.256542  1795 raft_consensus.cc:359] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07aa32703cc844cda506fcb5d3109a62" member_type: VOTER }
I20260709 12:28:20.257063  1795 raft_consensus.cc:385] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:20.257318  1795 raft_consensus.cc:740] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07aa32703cc844cda506fcb5d3109a62, State: Initialized, Role: FOLLOWER
I20260709 12:28:20.257974  1795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07aa32703cc844cda506fcb5d3109a62" member_type: VOTER }
I20260709 12:28:20.258483  1795 raft_consensus.cc:399] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:20.258745  1795 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:20.259075  1795 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:20.264393  1795 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07aa32703cc844cda506fcb5d3109a62" member_type: VOTER }
I20260709 12:28:20.264964  1795 leader_election.cc:304] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07aa32703cc844cda506fcb5d3109a62; no voters: 
I20260709 12:28:20.266233  1795 leader_election.cc:290] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:20.266654  1798 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:20.268141  1798 raft_consensus.cc:697] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 1 LEADER]: Becoming Leader. State: Replica: 07aa32703cc844cda506fcb5d3109a62, State: Running, Role: LEADER
I20260709 12:28:20.268800  1798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07aa32703cc844cda506fcb5d3109a62" member_type: VOTER }
I20260709 12:28:20.269503  1795 sys_catalog.cc:565] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:20.275481  1800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07aa32703cc844cda506fcb5d3109a62. Latest consensus state: current_term: 1 leader_uuid: "07aa32703cc844cda506fcb5d3109a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07aa32703cc844cda506fcb5d3109a62" member_type: VOTER } }
I20260709 12:28:20.275640  1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07aa32703cc844cda506fcb5d3109a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07aa32703cc844cda506fcb5d3109a62" member_type: VOTER } }
I20260709 12:28:20.276233  1800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:20.276252  1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:20.278648  1806 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:20.284111  1806 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:20.288443 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:20.294731  1806 catalog_manager.cc:1383] Generated new cluster ID: 719eef93df1141959090f7e5b5acd2cb
I20260709 12:28:20.294999  1806 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:20.320712  1806 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:20.322185  1806 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:20.334780  1806 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62: Generated new TSK 0
I20260709 12:28:20.335376  1806 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:20.355684 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:20.362267  1816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.363119  1817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.366024  1819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:20.367444 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:20.370190 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:20.370432 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:20.370607 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600100370584 us; error 0 us; skew 500 ppm
I20260709 12:28:20.371249 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:20.373798 19603 webserver.cc:533] Webserver started at http://127.19.36.193:37467/ using document root <none> and password file <none>
I20260709 12:28:20.374287 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:20.374492 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:20.374773 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:20.376036 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "8c59f6f4b90d460e832bcff010c18400"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.380555 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:28:20.384048  1824 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.384794 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:28:20.385105 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "8c59f6f4b90d460e832bcff010c18400"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.385401 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:20.400676 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:20.401871 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:20.403687 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:20.406033 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:20.406252 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.406494 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:20.406689 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.449141 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:35551
I20260709 12:28:20.449245  1886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:35551 every 8 connection(s)
I20260709 12:28:20.454511 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:20.463532  1892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.465324  1891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.468775  1894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:20.470147 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:20.474154 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:20.474375 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:20.474565 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600100474530 us; error 0 us; skew 500 ppm
I20260709 12:28:20.475343 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:20.477372  1887 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40093
I20260709 12:28:20.477807  1887 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:20.478440 19603 webserver.cc:533] Webserver started at http://127.19.36.194:34627/ using document root <none> and password file <none>
I20260709 12:28:20.478958 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:20.478909  1887 heartbeater.cc:507] Master 127.19.36.254:40093 requested a full tablet report, sending...
I20260709 12:28:20.479437 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:20.479773 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:20.481287 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "3e95870e8ff04dc7ab920b8917898eee"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.481829  1760 ts_manager.cc:194] Registered new tserver with Master: 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:20.484472  1760 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40494
I20260709 12:28:20.487630 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:28:20.491106  1899 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.491974 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:28:20.492264 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "3e95870e8ff04dc7ab920b8917898eee"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.492530 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:20.519541 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:20.520812 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:20.522336 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:20.524708 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:20.524927 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.525177 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:20.525345 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.569414 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:39535
I20260709 12:28:20.569542  1961 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:39535 every 8 connection(s)
I20260709 12:28:20.574699 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:20.582720  1965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.585486  1966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.589207  1968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:20.589747  1962 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40093
I20260709 12:28:20.590252  1962 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:20.590263 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:20.591238  1962 heartbeater.cc:507] Master 127.19.36.254:40093 requested a full tablet report, sending...
I20260709 12:28:20.591390 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:20.591732 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:20.591917 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600100591884 us; error 0 us; skew 500 ppm
I20260709 12:28:20.592700 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:20.593436  1760 ts_manager.cc:194] Registered new tserver with Master: 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:20.595419 19603 webserver.cc:533] Webserver started at http://127.19.36.195:33765/ using document root <none> and password file <none>
I20260709 12:28:20.595465  1760 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40510
I20260709 12:28:20.596047 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:20.596284 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:20.596661 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:20.597947 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "8e43e9e956f14e52897d2efc3f361630"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.602522 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 12:28:20.605975  1973 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.606758 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:28:20.607086 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "8e43e9e956f14e52897d2efc3f361630"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.607443 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:20.620651 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:20.621850 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:20.623417 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:20.625557 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:20.625780 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.626019 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:20.626190 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.666994 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:39583
I20260709 12:28:20.667097  2035 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:39583 every 8 connection(s)
I20260709 12:28:20.671844 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:20.681159  2039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:20.681608  2040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:20.683123 19603 server_base.cc:1061] running on GCE node
W20260709 12:28:20.684505  2042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:20.685720 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:28:20.685833  2036 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40093
W20260709 12:28:20.686049 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:20.686328  2036 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:20.686338 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600100686314 us; error 0 us; skew 500 ppm
I20260709 12:28:20.687335 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:20.687305  2036 heartbeater.cc:507] Master 127.19.36.254:40093 requested a full tablet report, sending...
I20260709 12:28:20.689757  1760 ts_manager.cc:194] Registered new tserver with Master: 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:20.690304 19603 webserver.cc:533] Webserver started at http://127.19.36.196:41707/ using document root <none> and password file <none>
I20260709 12:28:20.690864 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:20.691092 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:20.691412 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:20.691562  1760 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40524
I20260709 12:28:20.692858 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-3-root/instance:
uuid: "5edb06eaed8e41868299eb62ade6eef0"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.697335 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:28:20.700589  2047 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.701349 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260709 12:28:20.701658 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-3-root
uuid: "5edb06eaed8e41868299eb62ade6eef0"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:20.701947 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:20.716151 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:20.717418 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:20.718874 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:20.721187 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:20.721426 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.721668 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:20.721843 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:20.763679 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.196:37785
I20260709 12:28:20.763777  2109 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.196:37785 every 8 connection(s)
I20260709 12:28:20.778196  2110 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40093
I20260709 12:28:20.778609  2110 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:20.779500  2110 heartbeater.cc:507] Master 127.19.36.254:40093 requested a full tablet report, sending...
I20260709 12:28:20.781349  1760 ts_manager.cc:194] Registered new tserver with Master: 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:20.782043 19603 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.014869026s
I20260709 12:28:20.782931  1760 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40526
I20260709 12:28:20.807389  1760 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40530:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:28:21.044062  2001 tablet_service.cc:1511] Processing CreateTablet for tablet eb6c6f10db0b41ae8960133a8573f0b5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:21.045640  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6c6f10db0b41ae8960133a8573f0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.048739  2000 tablet_service.cc:1511] Processing CreateTablet for tablet e4414b0202fb4cd58bc7e1b4249efe0a (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:21.048977  1852 tablet_service.cc:1511] Processing CreateTablet for tablet eb6c6f10db0b41ae8960133a8573f0b5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:21.050371  2000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4414b0202fb4cd58bc7e1b4249efe0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.050853  1852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6c6f10db0b41ae8960133a8573f0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.057951  1851 tablet_service.cc:1511] Processing CreateTablet for tablet 2c6aef41b0274acb8e13d886aa250805 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:21.059926  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c6aef41b0274acb8e13d886aa250805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.067636  1850 tablet_service.cc:1511] Processing CreateTablet for tablet e4414b0202fb4cd58bc7e1b4249efe0a (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:21.069132  1850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4414b0202fb4cd58bc7e1b4249efe0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.072656  1999 tablet_service.cc:1511] Processing CreateTablet for tablet ed23ad21c29f479f8dc03b1a48fd9061 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:21.074261  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed23ad21c29f479f8dc03b1a48fd9061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.077057  1849 tablet_service.cc:1511] Processing CreateTablet for tablet ee564988f3774725bff3cd527794c6a1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:21.078536  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee564988f3774725bff3cd527794c6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.082659  1998 tablet_service.cc:1511] Processing CreateTablet for tablet ee564988f3774725bff3cd527794c6a1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:21.084574  1998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee564988f3774725bff3cd527794c6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.092447  1848 tablet_service.cc:1511] Processing CreateTablet for tablet 8a38771542bf449280983d8f709895dc (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:21.094538  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a38771542bf449280983d8f709895dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.109458  1997 tablet_service.cc:1511] Processing CreateTablet for tablet dfd2a42648874aef8ca5935e76f879c5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:21.111685  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd2a42648874aef8ca5935e76f879c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.114326  2075 tablet_service.cc:1511] Processing CreateTablet for tablet 2c6aef41b0274acb8e13d886aa250805 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:21.115744  2075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c6aef41b0274acb8e13d886aa250805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.119462  2074 tablet_service.cc:1511] Processing CreateTablet for tablet e4414b0202fb4cd58bc7e1b4249efe0a (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:21.120918  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4414b0202fb4cd58bc7e1b4249efe0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.124084  2073 tablet_service.cc:1511] Processing CreateTablet for tablet ed23ad21c29f479f8dc03b1a48fd9061 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:21.125550  2073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed23ad21c29f479f8dc03b1a48fd9061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.128156  2072 tablet_service.cc:1511] Processing CreateTablet for tablet 8a38771542bf449280983d8f709895dc (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:21.129628  2072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a38771542bf449280983d8f709895dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.129303  2071 tablet_service.cc:1511] Processing CreateTablet for tablet dfd2a42648874aef8ca5935e76f879c5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:21.130645  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd2a42648874aef8ca5935e76f879c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.132184  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 985c695826744eb693cfa90b8a3f3691 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:21.133481  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985c695826744eb693cfa90b8a3f3691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.134397  1927 tablet_service.cc:1511] Processing CreateTablet for tablet eb6c6f10db0b41ae8960133a8573f0b5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:21.135766  1927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6c6f10db0b41ae8960133a8573f0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.138063  1926 tablet_service.cc:1511] Processing CreateTablet for tablet 2c6aef41b0274acb8e13d886aa250805 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:21.140735  1925 tablet_service.cc:1511] Processing CreateTablet for tablet ed23ad21c29f479f8dc03b1a48fd9061 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:21.143309  1924 tablet_service.cc:1511] Processing CreateTablet for tablet ee564988f3774725bff3cd527794c6a1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:21.145936  1923 tablet_service.cc:1511] Processing CreateTablet for tablet 8a38771542bf449280983d8f709895dc (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:21.148656  1922 tablet_service.cc:1511] Processing CreateTablet for tablet 985c695826744eb693cfa90b8a3f3691 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:21.128157  1847 tablet_service.cc:1511] Processing CreateTablet for tablet dfd2a42648874aef8ca5935e76f879c5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:21.151125  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd2a42648874aef8ca5935e76f879c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.151851  1926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c6aef41b0274acb8e13d886aa250805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.163527  1922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985c695826744eb693cfa90b8a3f3691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.171159  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a38771542bf449280983d8f709895dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.176014  1924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee564988f3774725bff3cd527794c6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.177259  1925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed23ad21c29f479f8dc03b1a48fd9061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.197630  2129 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:21.199000  2070 tablet_service.cc:1511] Processing CreateTablet for tablet 985c695826744eb693cfa90b8a3f3691 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:21.204183  2070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985c695826744eb693cfa90b8a3f3691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.204159  1921 tablet_service.cc:1511] Processing CreateTablet for tablet 466fe5ff431c463a84050bd3e265a246 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:21.205673  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 466fe5ff431c463a84050bd3e265a246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.207471  1849 tablet_service.cc:1511] Processing CreateTablet for tablet 466fe5ff431c463a84050bd3e265a246 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:21.208881  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 466fe5ff431c463a84050bd3e265a246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.255733  2130 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:21.204160  2000 tablet_service.cc:1511] Processing CreateTablet for tablet 466fe5ff431c463a84050bd3e265a246 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:21.264732  2000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 466fe5ff431c463a84050bd3e265a246. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:21.287884  1745 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.36.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 66634.8us
W20260709 12:28:21.289240  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:21.313796  2129 tablet_bootstrap.cc:654] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.329672  2071 tablet_service.cc:1511] Processing CreateTablet for tablet 891bc4cc61ee4451a2cb82d28286d5f8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:21.331674  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 891bc4cc61ee4451a2cb82d28286d5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.331879  2131 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:21.339792  1997 tablet_service.cc:1511] Processing CreateTablet for tablet 11795a035a1e482d88901e8aea8ee185 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:21.341264  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11795a035a1e482d88901e8aea8ee185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.340994  2073 tablet_service.cc:1511] Processing CreateTablet for tablet 11795a035a1e482d88901e8aea8ee185 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:21.348439  2073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11795a035a1e482d88901e8aea8ee185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.348204  1849 tablet_service.cc:1511] Processing CreateTablet for tablet 891bc4cc61ee4451a2cb82d28286d5f8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:21.349540  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 891bc4cc61ee4451a2cb82d28286d5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.360015  1923 tablet_service.cc:1511] Processing CreateTablet for tablet 891bc4cc61ee4451a2cb82d28286d5f8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:21.361470  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 891bc4cc61ee4451a2cb82d28286d5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.362187  2131 tablet_bootstrap.cc:654] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.366863  2129 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:21.368800  2129 ts_tablet_manager.cc:1403] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.172s	user 0.018s	sys 0.005s
I20260709 12:28:21.372061  2133 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:21.373787  2129 raft_consensus.cc:359] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:21.374665  2129 raft_consensus.cc:385] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.375015  2129 raft_consensus.cc:740] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.381664  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8ed38ed6a4a8cb9ad7301aae1035d (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:21.381819  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 11795a035a1e482d88901e8aea8ee185 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:21.383023  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8ed38ed6a4a8cb9ad7301aae1035d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.385229  2071 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8ed38ed6a4a8cb9ad7301aae1035d (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:21.387872  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8ed38ed6a4a8cb9ad7301aae1035d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.390898  2130 tablet_bootstrap.cc:654] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.385897  2129 consensus_queue.cc:260] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:21.386392  1851 tablet_service.cc:1511] Processing CreateTablet for tablet 29f59bee90544d80a4b32bfcd1116d4a (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:21.403566  2131 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:21.406726  1999 tablet_service.cc:1511] Processing CreateTablet for tablet 29f59bee90544d80a4b32bfcd1116d4a (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:21.408035  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f59bee90544d80a4b32bfcd1116d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.410020  2074 tablet_service.cc:1511] Processing CreateTablet for tablet e2645174b413446fbf4c9423465821be (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:21.404547  1921 tablet_service.cc:1511] Processing CreateTablet for tablet 29f59bee90544d80a4b32bfcd1116d4a (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:21.411314  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2645174b413446fbf4c9423465821be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.386348  1925 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8ed38ed6a4a8cb9ad7301aae1035d (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:21.412613  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f59bee90544d80a4b32bfcd1116d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.405743  2131 ts_tablet_manager.cc:1403] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.074s	user 0.020s	sys 0.009s
I20260709 12:28:21.420840  2131 raft_consensus.cc:359] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.421732  2131 raft_consensus.cc:385] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.421840  1848 tablet_service.cc:1511] Processing CreateTablet for tablet e2645174b413446fbf4c9423465821be (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:21.422231  2131 raft_consensus.cc:740] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.424095  1925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8ed38ed6a4a8cb9ad7301aae1035d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.423689  2131 consensus_queue.cc:260] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.383155  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11795a035a1e482d88901e8aea8ee185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.428516  2036 heartbeater.cc:499] Master 127.19.36.254:40093 was elected leader, sending a full tablet report...
I20260709 12:28:21.431710  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 2c7707b908f046398ca5ee4defefe55f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:21.436308  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7707b908f046398ca5ee4defefe55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.439565  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2645174b413446fbf4c9423465821be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.443513  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f59bee90544d80a4b32bfcd1116d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.431655  1927 tablet_service.cc:1511] Processing CreateTablet for tablet e2645174b413446fbf4c9423465821be (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:21.448010  1927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2645174b413446fbf4c9423465821be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.448513  2129 ts_tablet_manager.cc:1434] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 12:28:21.449599  2129 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:21.460897  2133 tablet_bootstrap.cc:654] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.463622  1850 tablet_service.cc:1511] Processing CreateTablet for tablet 2c7707b908f046398ca5ee4defefe55f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:21.464996  1850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7707b908f046398ca5ee4defefe55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.472740  2072 tablet_service.cc:1511] Processing CreateTablet for tablet 2c7707b908f046398ca5ee4defefe55f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:21.474406  2072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7707b908f046398ca5ee4defefe55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.489503  1887 heartbeater.cc:499] Master 127.19.36.254:40093 was elected leader, sending a full tablet report...
I20260709 12:28:21.495615  2129 tablet_bootstrap.cc:654] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.500620  2130 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:21.501209  2130 ts_tablet_manager.cc:1403] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.246s	user 0.011s	sys 0.006s
I20260709 12:28:21.510138  1921 tablet_service.cc:1511] Processing CreateTablet for tablet 812870f8eb264d3caf48fefa3d70cfa1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:21.511565  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 812870f8eb264d3caf48fefa3d70cfa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.512293  1999 tablet_service.cc:1511] Processing CreateTablet for tablet 812870f8eb264d3caf48fefa3d70cfa1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:21.517813  2130 raft_consensus.cc:359] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:21.519343  2130 raft_consensus.cc:385] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.519769  2130 raft_consensus.cc:740] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.520660  2130 consensus_queue.cc:260] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:21.518937  2131 ts_tablet_manager.cc:1434] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.103s	user 0.008s	sys 0.034s
I20260709 12:28:21.523321  1962 heartbeater.cc:499] Master 127.19.36.254:40093 was elected leader, sending a full tablet report...
I20260709 12:28:21.524093  2071 tablet_service.cc:1511] Processing CreateTablet for tablet 812870f8eb264d3caf48fefa3d70cfa1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:21.526171  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 812870f8eb264d3caf48fefa3d70cfa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.532842  2131 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:21.525960  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 812870f8eb264d3caf48fefa3d70cfa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.555677  1848 tablet_service.cc:1511] Processing CreateTablet for tablet bbf8686b20194e3981d420491c825046 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:21.557718  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf8686b20194e3981d420491c825046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.563481  2136 raft_consensus.cc:493] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.571051  2133 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:21.571626  2133 ts_tablet_manager.cc:1403] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.200s	user 0.013s	sys 0.005s
I20260709 12:28:21.574604  2133 raft_consensus.cc:359] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.577024  2133 raft_consensus.cc:385] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.577381  2133 raft_consensus.cc:740] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.578202  2133 consensus_queue.cc:260] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
W20260709 12:28:21.576380  1963 tablet.cc:2406] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:21.576644  2129 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:21.566454  2136 raft_consensus.cc:515] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.582710  2129 ts_tablet_manager.cc:1403] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.133s	user 0.024s	sys 0.018s
I20260709 12:28:21.585146  2131 tablet_bootstrap.cc:654] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.590544  1925 tablet_service.cc:1511] Processing CreateTablet for tablet bbf8686b20194e3981d420491c825046 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:21.592018  1925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf8686b20194e3981d420491c825046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.600085  2136 leader_election.cc:290] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:21.600307  2110 heartbeater.cc:499] Master 127.19.36.254:40093 was elected leader, sending a full tablet report...
I20260709 12:28:21.601112  2130 ts_tablet_manager.cc:1434] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.028s
I20260709 12:28:21.604002  2133 ts_tablet_manager.cc:1434] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.008s
I20260709 12:28:21.612229  2133 tablet_bootstrap.cc:492] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:21.607533  2074 tablet_service.cc:1511] Processing CreateTablet for tablet 4773e370acbd4490afddcc7393f8be04 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:21.613260  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4773e370acbd4490afddcc7393f8be04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.608175  1997 tablet_service.cc:1511] Processing CreateTablet for tablet bbf8686b20194e3981d420491c825046 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:21.615967  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf8686b20194e3981d420491c825046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.601171  2129 raft_consensus.cc:359] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.619386  2129 raft_consensus.cc:385] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.619758  2129 raft_consensus.cc:740] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.620599  2129 consensus_queue.cc:260] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.622838  2129 ts_tablet_manager.cc:1434] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.038s	user 0.000s	sys 0.004s
I20260709 12:28:21.625717  2130 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:21.626083  1851 tablet_service.cc:1511] Processing CreateTablet for tablet 4773e370acbd4490afddcc7393f8be04 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:21.627919  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4773e370acbd4490afddcc7393f8be04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.629066  2129 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:21.650352  1921 tablet_service.cc:1511] Processing CreateTablet for tablet 4773e370acbd4490afddcc7393f8be04 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:21.651831  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4773e370acbd4490afddcc7393f8be04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.635757  2130 tablet_bootstrap.cc:654] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.651634  2000 tablet_service.cc:1511] Processing CreateTablet for tablet b1b4fd397d24490f8e5ad99e1a217e66 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:21.664258  2000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b4fd397d24490f8e5ad99e1a217e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.642518  2133 tablet_bootstrap.cc:654] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.698129  2133 tablet_bootstrap.cc:492] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:21.698673  2133 ts_tablet_manager.cc:1403] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.001s
I20260709 12:28:21.698796  2135 raft_consensus.cc:493] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.699563  2135 raft_consensus.cc:515] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.702605  2135 leader_election.cc:290] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:21.706920  2130 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:21.707744  2130 ts_tablet_manager.cc:1403] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.004s
I20260709 12:28:21.702379  2133 raft_consensus.cc:359] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:21.708617  2133 raft_consensus.cc:385] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.704217  2131 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:21.709026  2133 raft_consensus.cc:740] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.709579  2131 ts_tablet_manager.cc:1403] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.177s	user 0.019s	sys 0.022s
I20260709 12:28:21.710011  2133 consensus_queue.cc:260] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:21.716265  2133 ts_tablet_manager.cc:1434] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:28:21.717653  2133 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:21.713049  2130 raft_consensus.cc:359] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.657123  2129 tablet_bootstrap.cc:654] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.721258  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:21.727499  2130 raft_consensus.cc:385] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.728645  2130 raft_consensus.cc:740] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
W20260709 12:28:21.728807  1901 leader_election.cc:343] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.729763  2130 consensus_queue.cc:260] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.732363  2130 ts_tablet_manager.cc:1434] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:28:21.733414  2130 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:21.742600  2130 tablet_bootstrap.cc:654] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.744962  2129 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:21.745513  2129 ts_tablet_manager.cc:1403] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.117s	user 0.004s	sys 0.011s
I20260709 12:28:21.748027  2131 raft_consensus.cc:359] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.748813  2131 raft_consensus.cc:385] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.749092  2131 raft_consensus.cc:740] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.751852  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6c6f10db0b41ae8960133a8573f0b5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:21.752524  1862 raft_consensus.cc:2468] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:21.754076  1975 leader_election.cc:304] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:21.754969  2135 raft_consensus.cc:2804] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.755333  2135 raft_consensus.cc:493] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.755651  2135 raft_consensus.cc:3060] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.757159  2131 consensus_queue.cc:260] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.759500  2131 ts_tablet_manager.cc:1434] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:28:21.759387  2074 tablet_service.cc:1511] Processing CreateTablet for tablet b1b4fd397d24490f8e5ad99e1a217e66 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:21.760638  2131 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:21.760918  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b4fd397d24490f8e5ad99e1a217e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.764056  2136 raft_consensus.cc:493] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.764554  2136 raft_consensus.cc:515] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.764724  2072 tablet_service.cc:1511] Processing CreateTablet for tablet 74e46e6bd8574e31824d103146fc9830 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:21.765966  2072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e46e6bd8574e31824d103146fc9830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.765761  1851 tablet_service.cc:1511] Processing CreateTablet for tablet b1b4fd397d24490f8e5ad99e1a217e66 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:21.766923  2136 leader_election.cc:290] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:21.771286  2133 tablet_bootstrap.cc:654] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.773904  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6c6f10db0b41ae8960133a8573f0b5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:21.768191  1927 tablet_service.cc:1511] Processing CreateTablet for tablet 74e46e6bd8574e31824d103146fc9830 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:21.773810  1848 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5ff55130ab4cbf8e84dc429552a7e8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:21.783805  1927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e46e6bd8574e31824d103146fc9830. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:21.784116  1975 leader_election.cc:343] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.778384  2131 tablet_bootstrap.cc:654] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.775945  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c6aef41b0274acb8e13d886aa250805" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:21.785712  2135 raft_consensus.cc:515] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
W20260709 12:28:21.787587  1901 leader_election.cc:343] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.767017  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b4fd397d24490f8e5ad99e1a217e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.789362  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6c6f10db0b41ae8960133a8573f0b5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:21.789585  1921 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5ff55130ab4cbf8e84dc429552a7e8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:21.789971  1862 raft_consensus.cc:3060] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.790954  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5ff55130ab4cbf8e84dc429552a7e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.791896  2129 raft_consensus.cc:359] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.793491  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5ff55130ab4cbf8e84dc429552a7e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.795835  2129 raft_consensus.cc:385] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.797353  2129 raft_consensus.cc:740] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.800232  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6c6f10db0b41ae8960133a8573f0b5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:21.807588  1862 raft_consensus.cc:2468] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:21.808024  2135 leader_election.cc:290] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:21.812146  1975 leader_election.cc:304] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:21.816416  2071 tablet_service.cc:1511] Processing CreateTablet for tablet 880a5a05cb024c17b64d7ef2621311c5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:21.817137  1999 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5ff55130ab4cbf8e84dc429552a7e8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:21.820187  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5ff55130ab4cbf8e84dc429552a7e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.823541  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 880a5a05cb024c17b64d7ef2621311c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.826081  2129 consensus_queue.cc:260] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.829069  2129 ts_tablet_manager.cc:1434] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 12:28:21.834712  2129 tablet_bootstrap.cc:492] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:21.836359  1997 tablet_service.cc:1511] Processing CreateTablet for tablet 74e46e6bd8574e31824d103146fc9830 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:21.845028  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 880a5a05cb024c17b64d7ef2621311c5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:21.846427  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 880a5a05cb024c17b64d7ef2621311c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.852828  1925 tablet_service.cc:1511] Processing CreateTablet for tablet 880a5a05cb024c17b64d7ef2621311c5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:21.854558  1925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 880a5a05cb024c17b64d7ef2621311c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.830744  2135 raft_consensus.cc:2804] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.863796  2131 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:21.864359  2131 ts_tablet_manager.cc:1403] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.104s	user 0.021s	sys 0.018s
I20260709 12:28:21.868775  2130 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:21.869336  2130 ts_tablet_manager.cc:1403] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.136s	user 0.016s	sys 0.003s
I20260709 12:28:21.869277  2131 raft_consensus.cc:359] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.870383  2131 raft_consensus.cc:385] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.870759  2131 raft_consensus.cc:740] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.871832  2131 consensus_queue.cc:260] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.872592  2000 tablet_service.cc:1511] Processing CreateTablet for tablet 2e8e346d58d44c1dbcf74cd388a21904 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:21.872890  2130 raft_consensus.cc:359] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.873649  2130 raft_consensus.cc:385] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.873952  2130 raft_consensus.cc:740] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.874842  2130 consensus_queue.cc:260] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.877172  2130 ts_tablet_manager.cc:1434] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:28:21.878011  2160 raft_consensus.cc:493] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.886376  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c6aef41b0274acb8e13d886aa250805" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:21.887178  2085 raft_consensus.cc:2468] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
W20260709 12:28:21.887930  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 75543.6us
W20260709 12:28:21.889897  1975 leader_election.cc:343] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.889942  1903 leader_election.cc:304] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:21.891263  2136 raft_consensus.cc:2804] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.891680  2136 raft_consensus.cc:493] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.892071  2136 raft_consensus.cc:3060] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.892076  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:21.884704  2160 raft_consensus.cc:515] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:21.890960  1850 tablet_service.cc:1511] Processing CreateTablet for tablet 2e8e346d58d44c1dbcf74cd388a21904 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:21.894902  1850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e8e346d58d44c1dbcf74cd388a21904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.898207  2131 ts_tablet_manager.cc:1434] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.005s
I20260709 12:28:21.899710  2160 leader_election.cc:290] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:21.900871  2133 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:21.901382  2133 ts_tablet_manager.cc:1403] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.184s	user 0.010s	sys 0.028s
I20260709 12:28:21.904807  2133 raft_consensus.cc:359] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:21.905545  2133 raft_consensus.cc:385] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.905826  2133 raft_consensus.cc:740] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.906730  2133 consensus_queue.cc:260] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:21.909034  2133 ts_tablet_manager.cc:1434] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:28:21.910063  2133 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:21.916491  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:21.878181  2130 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:21.916461  2136 raft_consensus.cc:515] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.919166  1923 tablet_service.cc:1511] Processing CreateTablet for tablet 78d17658b0174e638c0041193b99bf82 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:21.920064  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c6aef41b0274acb8e13d886aa250805" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:21.920553  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d17658b0174e638c0041193b99bf82. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:21.921504  1901 leader_election.cc:343] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.888154  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e46e6bd8574e31824d103146fc9830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.922912  2131 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:21.910816  2072 tablet_service.cc:1511] Processing CreateTablet for tablet 2e8e346d58d44c1dbcf74cd388a21904 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:21.924242  2140 raft_consensus.cc:493] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:21.924646  1901 leader_election.cc:343] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.925166  1901 leader_election.cc:304] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:21.925537  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c6aef41b0274acb8e13d886aa250805" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:21.926193  2085 raft_consensus.cc:3060] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.926676  2141 raft_consensus.cc:2749] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:21.924343  1977 leader_election.cc:343] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.924983  2140 raft_consensus.cc:515] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:21.928678  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:21.929302  1862 raft_consensus.cc:2468] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:21.929621  2000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e8e346d58d44c1dbcf74cd388a21904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.930019  2140 leader_election.cc:290] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:21.930465  2135 raft_consensus.cc:697] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:21.931469  2135 consensus_queue.cc:237] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.931818  1975 leader_election.cc:304] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:21.924213  2072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e8e346d58d44c1dbcf74cd388a21904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.935595  2161 raft_consensus.cc:2804] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.936020  2161 raft_consensus.cc:493] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.936368  2161 raft_consensus.cc:3060] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.956621  2073 tablet_service.cc:1511] Processing CreateTablet for tablet 78d17658b0174e638c0041193b99bf82 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:21.958132  2073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d17658b0174e638c0041193b99bf82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.959442  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 78d17658b0174e638c0041193b99bf82 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:21.960862  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d17658b0174e638c0041193b99bf82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.971529  2085 raft_consensus.cc:2468] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:21.973366  2136 leader_election.cc:290] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:21.973974  1903 leader_election.cc:304] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:21.976583  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 979e0700017a488aa7012c1d228deeb1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:21.977880  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979e0700017a488aa7012c1d228deeb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.979012  2129 tablet_bootstrap.cc:654] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.979764  2136 raft_consensus.cc:2804] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.981045  2136 raft_consensus.cc:697] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:21.981884  2136 consensus_queue.cc:237] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:21.984297  1927 tablet_service.cc:1511] Processing CreateTablet for tablet 979e0700017a488aa7012c1d228deeb1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:21.985621  1927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979e0700017a488aa7012c1d228deeb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.986303  1998 tablet_service.cc:1511] Processing CreateTablet for tablet 979e0700017a488aa7012c1d228deeb1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:21.988754  2133 tablet_bootstrap.cc:654] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.991389  2131 tablet_bootstrap.cc:654] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.991740  1998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979e0700017a488aa7012c1d228deeb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:21.996312  2074 tablet_service.cc:1511] Processing CreateTablet for tablet e922d7ff58764a529d119fa6491f1768 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:21.997645  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e922d7ff58764a529d119fa6491f1768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.005908  2130 tablet_bootstrap.cc:654] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.002861  2161 raft_consensus.cc:515] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.015290  2129 tablet_bootstrap.cc:492] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:22.015880  2129 ts_tablet_manager.cc:1403] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.181s	user 0.014s	sys 0.001s
I20260709 12:28:22.016937  2142 raft_consensus.cc:493] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.021857  2161 leader_election.cc:290] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:22.030810  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:22.031148  1921 tablet_service.cc:1511] Processing CreateTablet for tablet e922d7ff58764a529d119fa6491f1768 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:22.032528  1848 tablet_service.cc:1511] Processing CreateTablet for tablet e922d7ff58764a529d119fa6491f1768 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:22.032675  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:22.033483  1862 raft_consensus.cc:3060] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.033844  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e922d7ff58764a529d119fa6491f1768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.017447  2142 raft_consensus.cc:515] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.041419  2142 leader_election.cc:290] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
W20260709 12:28:22.048254  1977 leader_election.cc:343] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.043356  2136 raft_consensus.cc:493] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.053922  1999 tablet_service.cc:1511] Processing CreateTablet for tablet 172b29319c78428499e095bf5e76b74e (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:22.054363  2136 raft_consensus.cc:515] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:22.057157  2136 leader_election.cc:290] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:22.066736  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 172b29319c78428499e095bf5e76b74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.068197  2070 tablet_service.cc:1511] Processing CreateTablet for tablet 172b29319c78428499e095bf5e76b74e (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:22.069569  2070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 172b29319c78428499e095bf5e76b74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.069778  2133 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:22.070288  2133 ts_tablet_manager.cc:1403] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.160s	user 0.011s	sys 0.006s
I20260709 12:28:22.071424  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e922d7ff58764a529d119fa6491f1768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.072957  1757 catalog_manager.cc:5697] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:22.073937  2133 raft_consensus.cc:359] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.074749  2133 raft_consensus.cc:385] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.075032  2133 raft_consensus.cc:740] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.096454  1862 raft_consensus.cc:2468] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:22.097947  1975 leader_election.cc:304] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:22.099177  2135 raft_consensus.cc:2804] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:22.099735  2135 raft_consensus.cc:697] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:22.100576  2135 consensus_queue.cc:237] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.111878  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 172b29319c78428499e095bf5e76b74e (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:22.113291  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 172b29319c78428499e095bf5e76b74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.113801  1923 tablet_service.cc:1511] Processing CreateTablet for tablet f2bbe2dde38842ab846f9c2d8a260918 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:22.114473  1997 tablet_service.cc:1511] Processing CreateTablet for tablet f2bbe2dde38842ab846f9c2d8a260918 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:22.115486  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2bbe2dde38842ab846f9c2d8a260918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.115839  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2bbe2dde38842ab846f9c2d8a260918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.119499  2129 raft_consensus.cc:359] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.120363  2129 raft_consensus.cc:385] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.120718  2129 raft_consensus.cc:740] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.127964  2129 consensus_queue.cc:260] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.129036  2133 consensus_queue.cc:260] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.131376  2071 tablet_service.cc:1511] Processing CreateTablet for tablet f2bbe2dde38842ab846f9c2d8a260918 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:22.147097  2131 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:22.147715  2131 ts_tablet_manager.cc:1403] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.225s	user 0.013s	sys 0.023s
I20260709 12:28:22.149365  2129 ts_tablet_manager.cc:1434] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.133s	user 0.004s	sys 0.004s
I20260709 12:28:22.150468  2129 tablet_bootstrap.cc:492] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:22.150959  2131 raft_consensus.cc:359] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.152607  2133 ts_tablet_manager.cc:1434] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.082s	user 0.004s	sys 0.004s
I20260709 12:28:22.152791  1850 tablet_service.cc:1511] Processing CreateTablet for tablet baf92d63594b4e3081ad94cac31690d1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:22.153741  2133 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:22.154309  1850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baf92d63594b4e3081ad94cac31690d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.154659  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:22.155356  1937 raft_consensus.cc:2468] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:22.152556  2131 raft_consensus.cc:385] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.167516  2131 raft_consensus.cc:740] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.168345  2131 consensus_queue.cc:260] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.168607  1999 tablet_service.cc:1511] Processing CreateTablet for tablet baf92d63594b4e3081ad94cac31690d1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:22.169893  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baf92d63594b4e3081ad94cac31690d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.158512  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2bbe2dde38842ab846f9c2d8a260918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.158527  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:22.166545  1927 tablet_service.cc:1511] Processing CreateTablet for tablet baf92d63594b4e3081ad94cac31690d1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:22.171473  2011 raft_consensus.cc:2468] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:22.172040  1927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baf92d63594b4e3081ad94cac31690d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.175586  1826 leader_election.cc:304] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:22.178076  2140 raft_consensus.cc:2804] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:22.178465  2140 raft_consensus.cc:493] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.178828  2140 raft_consensus.cc:3060] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.179847  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:22.180752  2011 raft_consensus.cc:2468] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:22.183166  1901 leader_election.cc:304] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:22.184892  1760 catalog_manager.cc:5697] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.187024  2136 raft_consensus.cc:2804] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:22.187500  2136 raft_consensus.cc:493] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.187834  2136 raft_consensus.cc:3060] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.191490  2131 ts_tablet_manager.cc:1434] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.026s	user 0.001s	sys 0.008s
I20260709 12:28:22.200927  2131 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:22.200575  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 8d082f4dbaeb47959b209411cd69957b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:22.202013  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d082f4dbaeb47959b209411cd69957b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.191432  2140 raft_consensus.cc:515] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:22.206631  1925 tablet_service.cc:1511] Processing CreateTablet for tablet 8d082f4dbaeb47959b209411cd69957b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:22.206542  2070 tablet_service.cc:1511] Processing CreateTablet for tablet 8d082f4dbaeb47959b209411cd69957b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:22.208083  2070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d082f4dbaeb47959b209411cd69957b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.208086  1925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d082f4dbaeb47959b209411cd69957b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.211295  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:22.211414  2136 raft_consensus.cc:515] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:22.213474  2136 leader_election.cc:290] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:22.209300  2133 tablet_bootstrap.cc:654] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.215983  1862 raft_consensus.cc:2393] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 8c59f6f4b90d460e832bcff010c18400 in this term.
I20260709 12:28:22.218616  2136 raft_consensus.cc:493] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.218583  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:22.218993  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:22.219462  2011 raft_consensus.cc:3060] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.219907  1861 raft_consensus.cc:2393] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 8c59f6f4b90d460e832bcff010c18400 in this term.
I20260709 12:28:22.219432  2136 raft_consensus.cc:515] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.226140  2130 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:22.226683  2130 ts_tablet_manager.cc:1403] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.349s	user 0.019s	sys 0.008s
I20260709 12:28:22.230077  2130 raft_consensus.cc:359] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.230787  2130 raft_consensus.cc:385] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.231534  2131 tablet_bootstrap.cc:654] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.234635  2011 raft_consensus.cc:2468] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:22.236207  1997 tablet_service.cc:1511] Processing CreateTablet for tablet f6241be73e134c25909f8054db9f85dc (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:22.237545  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6241be73e134c25909f8054db9f85dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.242126  2129 tablet_bootstrap.cc:654] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.246342  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a38771542bf449280983d8f709895dc" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:22.247738  1903 leader_election.cc:343] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.248660  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed23ad21c29f479f8dc03b1a48fd9061" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:22.248811  2074 tablet_service.cc:1511] Processing CreateTablet for tablet f6241be73e134c25909f8054db9f85dc (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:22.250674  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6241be73e134c25909f8054db9f85dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.249364  2010 raft_consensus.cc:2468] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:22.255096  1850 tablet_service.cc:1511] Processing CreateTablet for tablet f6241be73e134c25909f8054db9f85dc (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:22.255577  1901 leader_election.cc:304] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:22.256451  1850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6241be73e134c25909f8054db9f85dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.256795  2141 raft_consensus.cc:2804] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:22.263661  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:22.264566  2010 raft_consensus.cc:2393] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:22.251015  2140 leader_election.cc:290] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:22.263442  1925 tablet_service.cc:1511] Processing CreateTablet for tablet dcdf235113b34d5f8237b06b7662479f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:22.275075  2141 raft_consensus.cc:697] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:22.276083  2141 consensus_queue.cc:237] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:22.277284  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee564988f3774725bff3cd527794c6a1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:22.278427  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed23ad21c29f479f8dc03b1a48fd9061" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:22.279013  2049 leader_election.cc:304] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:22.280640  2073 tablet_service.cc:1511] Processing CreateTablet for tablet dcdf235113b34d5f8237b06b7662479f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:22.281380  1998 tablet_service.cc:1511] Processing CreateTablet for tablet dcdf235113b34d5f8237b06b7662479f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:22.282029  2073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdf235113b34d5f8237b06b7662479f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.282501  2142 raft_consensus.cc:2804] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:22.282927  2142 raft_consensus.cc:493] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.283156  1998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdf235113b34d5f8237b06b7662479f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.283408  2142 raft_consensus.cc:3060] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.284718  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a38771542bf449280983d8f709895dc" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:22.284888  1826 leader_election.cc:304] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:22.286654  2140 raft_consensus.cc:2749] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:28:22.287879  2049 leader_election.cc:343] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.231093  2130 raft_consensus.cc:740] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.293121  2130 consensus_queue.cc:260] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.294328  1861 raft_consensus.cc:2468] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:22.295810  2130 ts_tablet_manager.cc:1434] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.000s
I20260709 12:28:22.296775  1901 leader_election.cc:304] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:22.298307  2179 raft_consensus.cc:2804] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:22.298695  2179 raft_consensus.cc:493] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.298965  1848 tablet_service.cc:1511] Processing CreateTablet for tablet 70ee2ecd6eef424bb1bf355492553d8e (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:22.300990  2133 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:22.301522  2133 ts_tablet_manager.cc:1403] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.148s	user 0.007s	sys 0.007s
I20260709 12:28:22.302258  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 70ee2ecd6eef424bb1bf355492553d8e (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:22.302410  1923 tablet_service.cc:1511] Processing CreateTablet for tablet 70ee2ecd6eef424bb1bf355492553d8e (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:22.303611  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ee2ecd6eef424bb1bf355492553d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.305325  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ee2ecd6eef424bb1bf355492553d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.306164  2142 raft_consensus.cc:515] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.306684  2131 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:22.307256  2131 ts_tablet_manager.cc:1403] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.107s	user 0.007s	sys 0.013s
I20260709 12:28:22.269464  2136 leader_election.cc:290] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:22.312286  2130 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:22.312001  2131 raft_consensus.cc:359] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.312800  2131 raft_consensus.cc:385] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.313131  2131 raft_consensus.cc:740] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
W20260709 12:28:22.316417  1975 consensus_peers.cc:597] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.317217  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ee2ecd6eef424bb1bf355492553d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.316854  2071 tablet_service.cc:1511] Processing CreateTablet for tablet d17fb96cd6f94ffa9824eb3868a5fe92 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:22.319196  1851 tablet_service.cc:1511] Processing CreateTablet for tablet d17fb96cd6f94ffa9824eb3868a5fe92 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:22.322207  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d17fb96cd6f94ffa9824eb3868a5fe92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.327864  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d17fb96cd6f94ffa9824eb3868a5fe92. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:22.329890  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:22.332125  2072 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4f181266414f459ea67d333f044205 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:22.333452  2072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4f181266414f459ea67d333f044205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.336272  1921 tablet_service.cc:1511] Processing CreateTablet for tablet d17fb96cd6f94ffa9824eb3868a5fe92 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:22.299026  2179 raft_consensus.cc:3060] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.338120  1999 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4f181266414f459ea67d333f044205 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:22.339692  1925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdf235113b34d5f8237b06b7662479f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.340528  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d17fb96cd6f94ffa9824eb3868a5fe92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.345134  2129 tablet_bootstrap.cc:492] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:22.345707  2129 ts_tablet_manager.cc:1403] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.195s	user 0.009s	sys 0.003s
I20260709 12:28:22.347842  2131 consensus_queue.cc:260] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.348126  1849 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4f181266414f459ea67d333f044205 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:22.349504  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4f181266414f459ea67d333f044205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.350546  2131 ts_tablet_manager.cc:1434] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.000s
I20260709 12:28:22.339761  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4f181266414f459ea67d333f044205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.368659  2179 raft_consensus.cc:515] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.376221  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a38771542bf449280983d8f709895dc" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
W20260709 12:28:22.377564  1903 leader_election.cc:343] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.378660  2131 tablet_bootstrap.cc:492] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:22.382269  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a38771542bf449280983d8f709895dc" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:22.382987  1861 raft_consensus.cc:3060] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.388540  1927 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2a8fdea8624729ae4b5191c85fb520 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:22.389019  2133 raft_consensus.cc:359] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.391734  2133 raft_consensus.cc:385] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.390453  1927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2a8fdea8624729ae4b5191c85fb520. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.404737  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2a8fdea8624729ae4b5191c85fb520 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:22.406248  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2a8fdea8624729ae4b5191c85fb520. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.408877  1852 tablet_service.cc:1511] Processing CreateTablet for tablet 179c67e242914e64a56c8d1f829cdef5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:22.410243  1852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179c67e242914e64a56c8d1f829cdef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.410468  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed23ad21c29f479f8dc03b1a48fd9061" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:22.410364  1922 tablet_service.cc:1511] Processing CreateTablet for tablet 179c67e242914e64a56c8d1f829cdef5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 12:28:22.413697  2049 leader_election.cc:343] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.414357  2070 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2a8fdea8624729ae4b5191c85fb520 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:22.415783  2070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2a8fdea8624729ae4b5191c85fb520. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.414399  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed23ad21c29f479f8dc03b1a48fd9061" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:22.417840  1861 raft_consensus.cc:2468] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:22.418229  2010 raft_consensus.cc:3060] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.395265  2133 raft_consensus.cc:740] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.420884  2133 consensus_queue.cc:260] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.424415  2136 consensus_queue.cc:1048] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:22.394738  2142 leader_election.cc:290] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:22.426913  1901 leader_election.cc:304] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:22.427703  2142 raft_consensus.cc:493] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.428221  2142 raft_consensus.cc:515] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.428997  2136 raft_consensus.cc:2804] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:22.430513  2142 leader_election.cc:290] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:22.437271  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfd2a42648874aef8ca5935e76f879c5" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
W20260709 12:28:22.445067  2049 leader_election.cc:343] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.395303  2179 leader_election.cc:290] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:22.449999  2130 tablet_bootstrap.cc:654] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.474996  1760 catalog_manager.cc:5697] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:22.474586  1922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179c67e242914e64a56c8d1f829cdef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.486311  2131 tablet_bootstrap.cc:654] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.485401  2136 raft_consensus.cc:697] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:22.496294  2010 raft_consensus.cc:2468] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:22.497807  2049 leader_election.cc:304] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee
I20260709 12:28:22.499047  2142 raft_consensus.cc:2804] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:22.499881  2142 raft_consensus.cc:697] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:22.488986  2136 consensus_queue.cc:237] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.500757  2142 consensus_queue.cc:237] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.533165  2075 tablet_service.cc:1511] Processing CreateTablet for tablet 3a6b8329d4924908b3a364b840d691fe (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:22.528606  2182 raft_consensus.cc:493] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.550249  2129 raft_consensus.cc:359] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.551148  2129 raft_consensus.cc:385] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.551499  2129 raft_consensus.cc:740] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.552210  2129 consensus_queue.cc:260] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.552862  1997 tablet_service.cc:1511] Processing CreateTablet for tablet 179c67e242914e64a56c8d1f829cdef5 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:22.554339  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179c67e242914e64a56c8d1f829cdef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.568156  2075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a6b8329d4924908b3a364b840d691fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.569226  2084 raft_consensus.cc:3060] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.573648  1852 tablet_service.cc:1511] Processing CreateTablet for tablet 3a6b8329d4924908b3a364b840d691fe (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:22.575004  1852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a6b8329d4924908b3a364b840d691fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.576050  2159 consensus_queue.cc:1048] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:22.578837  2130 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:22.578675  2159 consensus_queue.cc:1048] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.579365  2130 ts_tablet_manager.cc:1403] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.267s	user 0.018s	sys 0.006s
I20260709 12:28:22.562233  2182 raft_consensus.cc:515] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.581018  2001 tablet_service.cc:1511] Processing CreateTablet for tablet cf093503636049fbb48b9a304d578c38 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:22.581827  2182 leader_election.cc:290] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:22.572854  1923 tablet_service.cc:1511] Processing CreateTablet for tablet 3a6b8329d4924908b3a364b840d691fe (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:22.582363  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf093503636049fbb48b9a304d578c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.583122  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a6b8329d4924908b3a364b840d691fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.593972  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:22.596675  1760 catalog_manager.cc:5697] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:22.612022  2049 leader_election.cc:343] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.612402  2129 ts_tablet_manager.cc:1434] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.266s	user 0.007s	sys 0.003s
I20260709 12:28:22.613560  2129 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:22.626348  1924 tablet_service.cc:1511] Processing CreateTablet for tablet c5ff50d36854429790ff6c78da6f2c8b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:22.592435  2133 ts_tablet_manager.cc:1434] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.291s	user 0.011s	sys 0.016s
I20260709 12:28:22.629065  2133 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:22.628739  1851 tablet_service.cc:1511] Processing CreateTablet for tablet cf093503636049fbb48b9a304d578c38 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:22.630086  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf093503636049fbb48b9a304d578c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.622927  2182 raft_consensus.cc:493] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:22.642364  1901 consensus_peers.cc:597] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.645658  1924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ff50d36854429790ff6c78da6f2c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.641013  2182 raft_consensus.cc:515] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.661777  2159 consensus_queue.cc:1048] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.666293  2131 tablet_bootstrap.cc:492] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:22.666855  2131 ts_tablet_manager.cc:1403] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.299s	user 0.014s	sys 0.014s
I20260709 12:28:22.641167  2130 raft_consensus.cc:359] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.658665  1999 tablet_service.cc:1511] Processing CreateTablet for tablet c5ff50d36854429790ff6c78da6f2c8b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:22.682478  2133 tablet_bootstrap.cc:654] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.676607  2070 tablet_service.cc:1511] Processing CreateTablet for tablet cf093503636049fbb48b9a304d578c38 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:22.684053  2070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf093503636049fbb48b9a304d578c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.686712  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:22.681524  1849 tablet_service.cc:1511] Processing CreateTablet for tablet edae6c4094614367a5bd04271ce7b99c (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:22.689255  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edae6c4094614367a5bd04271ce7b99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.689412  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfd2a42648874aef8ca5935e76f879c5" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:22.689304  2131 raft_consensus.cc:359] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.690301  2131 raft_consensus.cc:385] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.690644  2131 raft_consensus.cc:740] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
W20260709 12:28:22.691278  2049 leader_election.cc:343] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.691511  2131 consensus_queue.cc:260] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.691809  2049 leader_election.cc:304] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:22.693004  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ff50d36854429790ff6c78da6f2c8b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:22.693077  2049 leader_election.cc:343] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.693668  2049 leader_election.cc:304] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:22.694335  2131 ts_tablet_manager.cc:1434] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.003s
I20260709 12:28:22.694593  2183 raft_consensus.cc:2749] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:22.695338  2183 raft_consensus.cc:2749] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:22.695571  2131 tablet_bootstrap.cc:492] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:22.699707  2073 tablet_service.cc:1511] Processing CreateTablet for tablet c5ff50d36854429790ff6c78da6f2c8b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:22.700479  2129 tablet_bootstrap.cc:654] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.701085  2073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ff50d36854429790ff6c78da6f2c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.671403  2130 raft_consensus.cc:385] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.707679  2130 raft_consensus.cc:740] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.707224  2184 raft_consensus.cc:493] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.708626  2184 raft_consensus.cc:515] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.710987  2184 leader_election.cc:290] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:22.712123  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6c6f10db0b41ae8960133a8573f0b5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:22.713913  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6c6f10db0b41ae8960133a8573f0b5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:22.715787  1901 leader_election.cc:304] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:22.715248  2131 tablet_bootstrap.cc:654] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.728941  2182 leader_election.cc:290] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:22.731262  2184 raft_consensus.cc:2749] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:22.732203  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:22.743818  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:22.732398  2130 consensus_queue.cc:260] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.750262  2075 tablet_service.cc:1511] Processing CreateTablet for tablet db6a2096401b4cec9b542b6eb99f705b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:22.751718  2075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db6a2096401b4cec9b542b6eb99f705b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.751549  2049 leader_election.cc:304] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:22.755982  1924 tablet_service.cc:1511] Processing CreateTablet for tablet edae6c4094614367a5bd04271ce7b99c (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:22.757833  1924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edae6c4094614367a5bd04271ce7b99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.757438  1759 catalog_manager.cc:5697] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.750681  2130 ts_tablet_manager.cc:1434] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.171s	user 0.007s	sys 0.001s
I20260709 12:28:22.760111  2130 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:22.762173  1852 tablet_service.cc:1511] Processing CreateTablet for tablet db6a2096401b4cec9b542b6eb99f705b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:22.762791  1923 tablet_service.cc:1511] Processing CreateTablet for tablet db6a2096401b4cec9b542b6eb99f705b (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:22.763558  1852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db6a2096401b4cec9b542b6eb99f705b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.764061  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db6a2096401b4cec9b542b6eb99f705b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.768878  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 32982b9d743f4ceb80f17292b281fd6f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:22.770081  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32982b9d743f4ceb80f17292b281fd6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.772301  1997 tablet_service.cc:1511] Processing CreateTablet for tablet edae6c4094614367a5bd04271ce7b99c (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:22.782611  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edae6c4094614367a5bd04271ce7b99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.786469  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 32982b9d743f4ceb80f17292b281fd6f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:22.787853  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32982b9d743f4ceb80f17292b281fd6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.788134  2071 tablet_service.cc:1511] Processing CreateTablet for tablet 32982b9d743f4ceb80f17292b281fd6f (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:22.789398  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32982b9d743f4ceb80f17292b281fd6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.796653  2133 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:22.796916  2184 consensus_queue.cc:1048] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.797223  2133 ts_tablet_manager.cc:1403] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.168s	user 0.008s	sys 0.014s
I20260709 12:28:22.802151  2133 raft_consensus.cc:359] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.802891  2133 raft_consensus.cc:385] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.803223  2133 raft_consensus.cc:740] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.805712  2133 consensus_queue.cc:260] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:22.809921  2133 ts_tablet_manager.cc:1434] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:28:22.809165  1760 catalog_manager.cc:5697] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.811008  2133 tablet_bootstrap.cc:492] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:22.843346  2131 tablet_bootstrap.cc:492] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:22.843978  2131 ts_tablet_manager.cc:1403] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.149s	user 0.025s	sys 0.022s
I20260709 12:28:22.847745  2130 tablet_bootstrap.cc:654] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.850401  1924 tablet_service.cc:1511] Processing CreateTablet for tablet ff2bca7c619a4c36bb0cb4bdf08c8e78 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:22.851804  1924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2bca7c619a4c36bb0cb4bdf08c8e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.852255  2073 tablet_service.cc:1511] Processing CreateTablet for tablet ff2bca7c619a4c36bb0cb4bdf08c8e78 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:22.853626  2073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2bca7c619a4c36bb0cb4bdf08c8e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.857486  2001 tablet_service.cc:1511] Processing CreateTablet for tablet ff2bca7c619a4c36bb0cb4bdf08c8e78 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:22.858685  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2bca7c619a4c36bb0cb4bdf08c8e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.859050  2184 raft_consensus.cc:493] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.860481  2184 raft_consensus.cc:515] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.862792  2184 leader_election.cc:290] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:22.848369  2131 raft_consensus.cc:359] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.870364  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed23ad21c29f479f8dc03b1a48fd9061" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:22.870623  2131 raft_consensus.cc:385] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.871061  2131 raft_consensus.cc:740] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.871294  2010 raft_consensus.cc:2393] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:22.872061  2131 consensus_queue.cc:260] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.872382  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed23ad21c29f479f8dc03b1a48fd9061" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:22.879858  2133 tablet_bootstrap.cc:654] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.873580  2161 raft_consensus.cc:493] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.881502  2161 raft_consensus.cc:515] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.883664  2161 leader_election.cc:290] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:22.884820  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:22.885685  2129 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:22.885030  2131 ts_tablet_manager.cc:1434] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 12:28:22.886193  2129 ts_tablet_manager.cc:1403] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.273s	user 0.012s	sys 0.008s
W20260709 12:28:22.886392  1977 leader_election.cc:343] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:22.887735  1903 leader_election.cc:304] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:22.888649  2136 raft_consensus.cc:3060] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.889601  1851 tablet_service.cc:1511] Processing CreateTablet for tablet 0a8c550169674d72af0524338f68fbee (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:22.890431  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:22.891048  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a8c550169674d72af0524338f68fbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.891119  1937 raft_consensus.cc:2468] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:22.891834  2129 raft_consensus.cc:359] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.892516  1975 leader_election.cc:304] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:22.899127  2161 raft_consensus.cc:2804] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:22.899598  2161 raft_consensus.cc:493] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.900039  2161 raft_consensus.cc:3060] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.902022  2136 raft_consensus.cc:2749] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:22.892549  2129 raft_consensus.cc:385] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:22.903106  2129 raft_consensus.cc:740] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:22.904093  2129 consensus_queue.cc:260] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:22.905179  2131 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:22.906407  2129 ts_tablet_manager.cc:1434] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.005s
I20260709 12:28:22.907493  2182 raft_consensus.cc:2749] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:22.926888  2161 raft_consensus.cc:515] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:22.927817  2131 tablet_bootstrap.cc:654] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.941069  2129 tablet_bootstrap.cc:492] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:22.957777  1937 raft_consensus.cc:3060] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:22.962878  2205 raft_consensus.cc:493] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.967990  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 6146e39d5a724f05b9b3cbc7a8145d98 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:22.967981  1849 tablet_service.cc:1511] Processing CreateTablet for tablet 91363564e93f4297b4e825de5b7e5176 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:22.970183  1852 tablet_service.cc:1511] Processing CreateTablet for tablet 4631b580b30e4f57b7a7a09b2452dc75 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:22.973219  1922 tablet_service.cc:1511] Processing CreateTablet for tablet 0a8c550169674d72af0524338f68fbee (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:22.998703  1922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a8c550169674d72af0524338f68fbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.976795  2075 tablet_service.cc:1511] Processing CreateTablet for tablet 91363564e93f4297b4e825de5b7e5176 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:22.999303  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:22.999950  1936 raft_consensus.cc:3060] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.000156  2075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91363564e93f4297b4e825de5b7e5176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.001329  2129 tablet_bootstrap.cc:654] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:22.977089  2070 tablet_service.cc:1511] Processing CreateTablet for tablet 5ebf212c059342559ae4781db6ba8e83 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:23.002854  2070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ebf212c059342559ae4781db6ba8e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:22.977739  2140 raft_consensus.cc:493] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.005795  2140 raft_consensus.cc:515] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.008316  2140 leader_election.cc:290] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:22.978343  2141 consensus_queue.cc:1048] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:28:22.989400  2188 consensus_queue.cc:1048] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:22.985131  1923 tablet_service.cc:1511] Processing CreateTablet for tablet 6146e39d5a724f05b9b3cbc7a8145d98 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:22.994925  2182 consensus_queue.cc:1048] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:22.987206  1977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:37785, user_credentials={real_user=slave}} blocked reactor thread for 78917us
I20260709 12:28:22.997504  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146e39d5a724f05b9b3cbc7a8145d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.009833  2071 tablet_service.cc:1511] Processing CreateTablet for tablet 6146e39d5a724f05b9b3cbc7a8145d98 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:23.018759  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c6aef41b0274acb8e13d886aa250805" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:23.019264  2072 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0af4a569f84ef9841ad0c788c5d6ca (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:23.026986  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146e39d5a724f05b9b3cbc7a8145d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.030411  2072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0af4a569f84ef9841ad0c788c5d6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.043444  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91363564e93f4297b4e825de5b7e5176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.043237  2074 tablet_service.cc:1511] Processing CreateTablet for tablet b385e95426fd4870b36475731ab7d0b8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:23.044766  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b385e95426fd4870b36475731ab7d0b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.044529  1848 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0af4a569f84ef9841ad0c788c5d6ca (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:23.045781  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0af4a569f84ef9841ad0c788c5d6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.048161  1924 tablet_service.cc:1511] Processing CreateTablet for tablet 5ebf212c059342559ae4781db6ba8e83 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:23.049407  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146e39d5a724f05b9b3cbc7a8145d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.052788  2001 tablet_service.cc:1511] Processing CreateTablet for tablet 0a8c550169674d72af0524338f68fbee (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:23.053561  1999 tablet_service.cc:1511] Processing CreateTablet for tablet 5ebf212c059342559ae4781db6ba8e83 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:23.054906  1997 tablet_service.cc:1511] Processing CreateTablet for tablet 91363564e93f4297b4e825de5b7e5176 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:23.056821  1998 tablet_service.cc:1511] Processing CreateTablet for tablet 4631b580b30e4f57b7a7a09b2452dc75 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:23.053895  2001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a8c550169674d72af0524338f68fbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.058928  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91363564e93f4297b4e825de5b7e5176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.059800  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ebf212c059342559ae4781db6ba8e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.065665  2000 tablet_service.cc:1511] Processing CreateTablet for tablet b385e95426fd4870b36475731ab7d0b8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:23.067378  2161 leader_election.cc:290] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:23.069396  2131 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:23.069871  2131 ts_tablet_manager.cc:1403] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.165s	user 0.026s	sys 0.010s
I20260709 12:28:23.004401  2205 raft_consensus.cc:515] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.061235  2130 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:23.078569  1998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4631b580b30e4f57b7a7a09b2452dc75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.079939  1925 tablet_service.cc:1511] Processing CreateTablet for tablet 4631b580b30e4f57b7a7a09b2452dc75 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:23.085666  2000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b385e95426fd4870b36475731ab7d0b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.088032  2130 ts_tablet_manager.cc:1403] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.328s	user 0.021s	sys 0.008s
I20260709 12:28:23.080647  1924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ebf212c059342559ae4781db6ba8e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.080906  1852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4631b580b30e4f57b7a7a09b2452dc75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.089418  1925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4631b580b30e4f57b7a7a09b2452dc75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.091487  1921 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0af4a569f84ef9841ad0c788c5d6ca (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:23.092780  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0af4a569f84ef9841ad0c788c5d6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.093454  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
W20260709 12:28:23.099450  1977 leader_election.cc:343] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.100411  2182 consensus_queue.cc:1048] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:23.112345  2130 raft_consensus.cc:359] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.113168  2130 raft_consensus.cc:385] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.113466  2130 raft_consensus.cc:740] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.125008  2133 tablet_bootstrap.cc:492] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:23.125576  2133 ts_tablet_manager.cc:1403] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.315s	user 0.005s	sys 0.012s
I20260709 12:28:23.132421  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfd2a42648874aef8ca5935e76f879c5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:23.131817  1862 raft_consensus.cc:3060] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.135462  2131 raft_consensus.cc:359] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.136200  2131 raft_consensus.cc:385] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.136464  2131 raft_consensus.cc:740] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.136969  2131 consensus_queue.cc:260] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.076776  1936 raft_consensus.cc:2468] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:23.133962  2205 leader_election.cc:290] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:23.114609  2130 consensus_queue.cc:260] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
W20260709 12:28:23.143440  1975 leader_election.cc:343] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.145560  2133 raft_consensus.cc:359] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.176043  2130 ts_tablet_manager.cc:1434] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.001s
I20260709 12:28:23.182849  2085 raft_consensus.cc:3060] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.164317  2133 raft_consensus.cc:385] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.191916  2131 ts_tablet_manager.cc:1434] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.122s	user 0.008s	sys 0.002s
I20260709 12:28:23.193020  2131 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:23.194108  2133 raft_consensus.cc:740] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.203809  2133 consensus_queue.cc:260] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.206944  1999 tablet_service.cc:1511] Processing CreateTablet for tablet 0b15f3e29c654279b3811d5ed11c31a2 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:23.208253  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b15f3e29c654279b3811d5ed11c31a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.216151  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfd2a42648874aef8ca5935e76f879c5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:23.216915  2084 raft_consensus.cc:2468] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:23.220243  1922 tablet_service.cc:1511] Processing CreateTablet for tablet 0b15f3e29c654279b3811d5ed11c31a2 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:23.231353  1977 leader_election.cc:304] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:23.234066  2205 raft_consensus.cc:2804] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:23.234438  2205 raft_consensus.cc:493] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.234766  2205 raft_consensus.cc:3060] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.233103  2133 ts_tablet_manager.cc:1434] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.107s	user 0.005s	sys 0.006s
I20260709 12:28:23.234773  1849 tablet_service.cc:1511] Processing CreateTablet for tablet d9e0977a3ed1476f871be3d3a77ae4b2 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:23.183609  1851 tablet_service.cc:1511] Processing CreateTablet for tablet b385e95426fd4870b36475731ab7d0b8 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:23.238101  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9e0977a3ed1476f871be3d3a77ae4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.231655  1975 leader_election.cc:304] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:23.241575  2161 raft_consensus.cc:2804] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:23.242058  2161 raft_consensus.cc:697] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:23.242882  2161 consensus_queue.cc:237] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.244753  1851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b385e95426fd4870b36475731ab7d0b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.246559  2133 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:23.247975  2072 tablet_service.cc:1511] Processing CreateTablet for tablet 0b15f3e29c654279b3811d5ed11c31a2 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:23.249397  2072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b15f3e29c654279b3811d5ed11c31a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.251641  2130 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:23.254388  2133 tablet_bootstrap.cc:654] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.256044  2205 raft_consensus.cc:515] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.258996  2083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfd2a42648874aef8ca5935e76f879c5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:23.259601  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c6aef41b0274acb8e13d886aa250805" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:23.259828  2083 raft_consensus.cc:3060] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.260830  1922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b15f3e29c654279b3811d5ed11c31a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.269646  1828 leader_election.cc:304] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:23.270975  2140 raft_consensus.cc:493] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.271510  2140 raft_consensus.cc:515] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.273761  2140 leader_election.cc:290] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:23.274825  2083 raft_consensus.cc:2468] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:23.276580  1977 leader_election.cc:304] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:23.276580  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:23.277565  1936 raft_consensus.cc:2468] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:23.277664  2208 raft_consensus.cc:2804] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:23.278264  2208 raft_consensus.cc:697] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:23.279456  1826 leader_election.cc:304] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:23.279260  2208 consensus_queue.cc:237] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.284235  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:23.282137  2075 tablet_service.cc:1511] Processing CreateTablet for tablet 90834683513744559ef8f7efe530f459 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:23.286044  2075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90834683513744559ef8f7efe530f459. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:23.286003  1826 leader_election.cc:343] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.286621  1921 tablet_service.cc:1511] Processing CreateTablet for tablet d9e0977a3ed1476f871be3d3a77ae4b2 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:23.287508  2205 leader_election.cc:290] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:23.294414  2131 tablet_bootstrap.cc:654] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.310288  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9e0977a3ed1476f871be3d3a77ae4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.321310  2071 tablet_service.cc:1511] Processing CreateTablet for tablet 736e3203de274189acfaef187bfea762 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:23.322595  2071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 736e3203de274189acfaef187bfea762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.325119  2129 tablet_bootstrap.cc:492] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:23.325914  2129 ts_tablet_manager.cc:1403] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.417s	user 0.029s	sys 0.009s
I20260709 12:28:23.316814  2070 tablet_service.cc:1511] Processing CreateTablet for tablet 5128bcc1e7cb49009f2b376e8fb5e125 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:23.346480  2070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5128bcc1e7cb49009f2b376e8fb5e125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.346225  1999 tablet_service.cc:1511] Processing CreateTablet for tablet d9e0977a3ed1476f871be3d3a77ae4b2 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:23.347378  1999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9e0977a3ed1476f871be3d3a77ae4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.347260  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfd2a42648874aef8ca5935e76f879c5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
W20260709 12:28:23.349370  1975 leader_election.cc:343] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.356047  2140 raft_consensus.cc:2749] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:23.361166  2073 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca90853e7a245c090a24e9f2b6b88e1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:23.362807  2073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca90853e7a245c090a24e9f2b6b88e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.363538  1998 tablet_service.cc:1511] Processing CreateTablet for tablet 5128bcc1e7cb49009f2b376e8fb5e125 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:23.364856  1998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5128bcc1e7cb49009f2b376e8fb5e125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.368003  1852 tablet_service.cc:1511] Processing CreateTablet for tablet 5128bcc1e7cb49009f2b376e8fb5e125 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:23.368207  1849 tablet_service.cc:1511] Processing CreateTablet for tablet 90834683513744559ef8f7efe530f459 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:23.369289  1852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5128bcc1e7cb49009f2b376e8fb5e125. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:23.332978  1826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 47014.4us
I20260709 12:28:23.371977  1927 tablet_service.cc:1511] Processing CreateTablet for tablet 736e3203de274189acfaef187bfea762 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:23.373587  1927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 736e3203de274189acfaef187bfea762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.373777  2000 tablet_service.cc:1511] Processing CreateTablet for tablet 736e3203de274189acfaef187bfea762 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:23.375505  1997 tablet_service.cc:1511] Processing CreateTablet for tablet ac43f6f37d474da58e0a962b25fe9235 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:23.375790  1926 tablet_service.cc:1511] Processing CreateTablet for tablet ac43f6f37d474da58e0a962b25fe9235 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:23.376601  1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac43f6f37d474da58e0a962b25fe9235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.377532  1926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac43f6f37d474da58e0a962b25fe9235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.379920  2000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 736e3203de274189acfaef187bfea762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.372004  1847 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca90853e7a245c090a24e9f2b6b88e1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:23.385002  1847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca90853e7a245c090a24e9f2b6b88e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.368228  1848 tablet_service.cc:1511] Processing CreateTablet for tablet ac43f6f37d474da58e0a962b25fe9235 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:23.388614  1848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac43f6f37d474da58e0a962b25fe9235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.398334  1923 tablet_service.cc:1511] Processing CreateTablet for tablet 90834683513744559ef8f7efe530f459 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:23.400002  1923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90834683513744559ef8f7efe530f459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.471149  1849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90834683513744559ef8f7efe530f459. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:23.489831  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:23.488209  2219 raft_consensus.cc:2804] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:23.515934  2129 raft_consensus.cc:359] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.516433  2184 raft_consensus.cc:493] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.516753  2129 raft_consensus.cc:385] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.517086  2129 raft_consensus.cc:740] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.516935  2184 raft_consensus.cc:515] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.517866  2129 consensus_queue.cc:260] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.511929  2219 raft_consensus.cc:493] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.519901  2219 raft_consensus.cc:3060] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.525512  2133 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:23.526012  2133 ts_tablet_manager.cc:1403] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.280s	user 0.018s	sys 0.030s
I20260709 12:28:23.608637  2133 raft_consensus.cc:359] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.609412  2133 raft_consensus.cc:385] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.609674  2133 raft_consensus.cc:740] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.610342  2133 consensus_queue.cc:260] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.617456  2074 tablet_service.cc:1511] Processing CreateTablet for tablet bf128bd1b4b546dab8e2c0504d06cb87 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:23.618750  2074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf128bd1b4b546dab8e2c0504d06cb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.623596  2184 leader_election.cc:290] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:23.624409  2130 tablet_bootstrap.cc:654] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.624785  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
W20260709 12:28:23.626573  1901 leader_election.cc:343] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.631469  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:23.636797  2133 ts_tablet_manager.cc:1434] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.111s	user 0.021s	sys 0.003s
W20260709 12:28:23.638429  1901 leader_election.cc:343] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.638245  2219 raft_consensus.cc:515] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.638908  1901 leader_election.cc:304] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:23.639066  2083 raft_consensus.cc:3060] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.642055  2184 raft_consensus.cc:2749] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:23.647764  2208 consensus_queue.cc:1048] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:23.650666  1759 catalog_manager.cc:5697] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.652966  1759 catalog_manager.cc:5697] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.657096  2219 leader_election.cc:290] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
W20260709 12:28:23.600314  1745 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.36.194:39535, user_credentials={real_user=slave}} blocked reactor thread for 125107us
I20260709 12:28:23.665807  2133 tablet_bootstrap.cc:492] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:23.673940  2129 ts_tablet_manager.cc:1434] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.348s	user 0.010s	sys 0.006s
W20260709 12:28:23.676549  1975 consensus_peers.cc:597] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.683139  1998 tablet_service.cc:1511] Processing CreateTablet for tablet bf128bd1b4b546dab8e2c0504d06cb87 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:23.684347  2182 raft_consensus.cc:493] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.684552  1998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf128bd1b4b546dab8e2c0504d06cb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.684917  2182 raft_consensus.cc:515] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.685653  1921 tablet_service.cc:1511] Processing CreateTablet for tablet 5ca90853e7a245c090a24e9f2b6b88e1 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:23.686890  1921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca90853e7a245c090a24e9f2b6b88e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.686697  2141 consensus_queue.cc:1048] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:23.692449  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:23.692675  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:23.693110  1937 raft_consensus.cc:3060] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:23.694856  1826 leader_election.cc:343] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.675101  2129 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:23.697692  2133 tablet_bootstrap.cc:654] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.708703  2182 leader_election.cc:290] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:23.713660  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a38771542bf449280983d8f709895dc" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:23.713755  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a38771542bf449280983d8f709895dc" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:23.716675  2049 leader_election.cc:304] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:23.722532  1937 raft_consensus.cc:2468] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:23.737268  2131 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:23.737896  2131 ts_tablet_manager.cc:1403] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.545s	user 0.027s	sys 0.022s
I20260709 12:28:23.741122  2131 raft_consensus.cc:359] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.741914  2131 raft_consensus.cc:385] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.742267  2131 raft_consensus.cc:740] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.743574  2131 consensus_queue.cc:260] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.748677  2131 ts_tablet_manager.cc:1434] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:28:23.749819  2131 tablet_bootstrap.cc:492] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
W20260709 12:28:23.743350  1745 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.36.194:39535, user_credentials={real_user=slave}} blocked reactor thread for 45920.5us
I20260709 12:28:23.754194  2129 tablet_bootstrap.cc:654] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.757062  2131 tablet_bootstrap.cc:654] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.748817  1826 leader_election.cc:304] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:23.762864  2219 raft_consensus.cc:2804] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:23.763537  2219 raft_consensus.cc:697] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:23.764629  2219 consensus_queue.cc:237] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.762873  2130 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:23.767680  2130 ts_tablet_manager.cc:1403] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.536s	user 0.020s	sys 0.024s
I20260709 12:28:23.766674  1852 tablet_service.cc:1511] Processing CreateTablet for tablet bf128bd1b4b546dab8e2c0504d06cb87 (DEFAULT_TABLE table=test-workload [id=268bfc86f58341e796d15d933905ca87]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:23.774290  1852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf128bd1b4b546dab8e2c0504d06cb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.784121  2130 raft_consensus.cc:359] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.790400  2182 raft_consensus.cc:2749] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:23.792713  2130 raft_consensus.cc:385] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.793192  2130 raft_consensus.cc:740] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.795753  2130 consensus_queue.cc:260] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.804219  2161 consensus_queue.cc:1048] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:23.806145  2129 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:23.806700  2129 ts_tablet_manager.cc:1403] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.007s
I20260709 12:28:23.809989  2130 ts_tablet_manager.cc:1434] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.002s
I20260709 12:28:23.812076  2130 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:23.812486  2129 raft_consensus.cc:359] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.819509  2129 raft_consensus.cc:385] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.819909  2129 raft_consensus.cc:740] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.762171  2133 tablet_bootstrap.cc:492] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:23.820955  2133 ts_tablet_manager.cc:1403] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.157s	user 0.012s	sys 0.014s
I20260709 12:28:23.820734  2129 consensus_queue.cc:260] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.824786  2129 ts_tablet_manager.cc:1434] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:28:23.826534  2129 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:23.832408  2133 raft_consensus.cc:359] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.833102  2133 raft_consensus.cc:385] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.833377  2133 raft_consensus.cc:740] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.834743  2130 tablet_bootstrap.cc:654] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.842873  2129 tablet_bootstrap.cc:654] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.870033  2133 consensus_queue.cc:260] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.876742  2133 ts_tablet_manager.cc:1434] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 12:28:23.877802  2133 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:23.880692  2131 tablet_bootstrap.cc:492] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:23.881237  2131 ts_tablet_manager.cc:1403] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.132s	user 0.052s	sys 0.039s
I20260709 12:28:23.883442  2130 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:23.882226  1758 catalog_manager.cc:5697] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.883989  2130 ts_tablet_manager.cc:1403] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.072s	user 0.004s	sys 0.008s
I20260709 12:28:23.885177  2131 raft_consensus.cc:359] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.885936  2131 raft_consensus.cc:385] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.886211  2131 raft_consensus.cc:740] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.887265  2130 raft_consensus.cc:359] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.887982  2130 raft_consensus.cc:385] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.901970  2130 raft_consensus.cc:740] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.888252  2131 consensus_queue.cc:260] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.907027  2130 consensus_queue.cc:260] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.907410  2161 consensus_queue.cc:1048] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:23.909837  2130 ts_tablet_manager.cc:1434] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:28:23.914979  2130 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:23.900048  2129 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:23.915593  2129 ts_tablet_manager.cc:1403] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.090s	user 0.005s	sys 0.007s
I20260709 12:28:23.919914  2129 raft_consensus.cc:359] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.920620  2129 raft_consensus.cc:385] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.920892  2129 raft_consensus.cc:740] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.921581  2129 consensus_queue.cc:260] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.922914  2130 tablet_bootstrap.cc:654] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.901582  2133 tablet_bootstrap.cc:654] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.925187  2131 ts_tablet_manager.cc:1434] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.003s
I20260709 12:28:23.926224  2131 tablet_bootstrap.cc:492] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:23.932189  2131 tablet_bootstrap.cc:654] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.933738  2182 raft_consensus.cc:493] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.934267  2182 raft_consensus.cc:515] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.936497  2182 leader_election.cc:290] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:23.938793  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:23.940083  2049 leader_election.cc:343] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.941890  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
W20260709 12:28:23.952041  2049 leader_election.cc:343] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:23.952728  2049 leader_election.cc:304] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:23.953415  2131 tablet_bootstrap.cc:492] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:23.953742  2182 raft_consensus.cc:2749] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:23.953976  2131 ts_tablet_manager.cc:1403] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.028s	user 0.004s	sys 0.004s
I20260709 12:28:23.957243  2131 raft_consensus.cc:359] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.957955  2131 raft_consensus.cc:385] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.958240  2131 raft_consensus.cc:740] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.959009  2131 consensus_queue.cc:260] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.961102  2129 ts_tablet_manager.cc:1434] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 12:28:23.964509  2131 ts_tablet_manager.cc:1434] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:23.965596  2131 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:23.968619  2133 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:23.969148  2133 ts_tablet_manager.cc:1403] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.092s	user 0.006s	sys 0.007s
I20260709 12:28:23.969920  2209 raft_consensus.cc:493] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.970422  2209 raft_consensus.cc:515] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:23.971741  2131 tablet_bootstrap.cc:654] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.972786  2209 leader_election.cc:290] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:23.974712  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ed38ed6a4a8cb9ad7301aae1035d" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:23.975348  2130 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:23.975396  1937 raft_consensus.cc:2468] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:23.976006  2130 ts_tablet_manager.cc:1403] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.061s	user 0.039s	sys 0.006s
I20260709 12:28:23.976895  1975 leader_election.cc:304] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:23.977787  2205 raft_consensus.cc:2804] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:23.978152  2205 raft_consensus.cc:493] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.978487  2205 raft_consensus.cc:3060] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.978916  2133 raft_consensus.cc:359] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.979668  2133 raft_consensus.cc:385] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.979935  2133 raft_consensus.cc:740] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.981493  2130 raft_consensus.cc:359] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.982182  2130 raft_consensus.cc:385] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.982474  2130 raft_consensus.cc:740] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.975831  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ed38ed6a4a8cb9ad7301aae1035d" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:23.983441  2084 raft_consensus.cc:2468] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:23.983400  2133 consensus_queue.cc:260] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:23.984845  2131 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:23.985368  2131 ts_tablet_manager.cc:1403] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.003s
I20260709 12:28:23.985566  2133 ts_tablet_manager.cc:1434] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:28:23.986595  2133 tablet_bootstrap.cc:492] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:23.987622  2130 consensus_queue.cc:260] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:23.989123  2131 raft_consensus.cc:359] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.989751  2131 raft_consensus.cc:385] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.990000  2131 raft_consensus.cc:740] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.990651  2130 ts_tablet_manager.cc:1434] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:28:23.990757  2131 consensus_queue.cc:260] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:23.991671  2130 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:23.993044  2131 ts_tablet_manager.cc:1434] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:24.003458  2131 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:24.004666  2133 tablet_bootstrap.cc:654] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.009657  2131 tablet_bootstrap.cc:654] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.010560  2208 raft_consensus.cc:493] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.011281  2208 raft_consensus.cc:515] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.013950  2208 leader_election.cc:290] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:24.016386  2133 tablet_bootstrap.cc:492] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:24.016902  2133 ts_tablet_manager.cc:1403] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.003s
I20260709 12:28:24.017740  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:24.020627  1975 leader_election.cc:343] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.021715  2133 raft_consensus.cc:359] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.023820  2133 raft_consensus.cc:385] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.024101  2133 raft_consensus.cc:740] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.025310  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:23.998147  2130 tablet_bootstrap.cc:654] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.996361  2205 raft_consensus.cc:515] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.024977  2133 consensus_queue.cc:260] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.026727  2129 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:24.030015  1937 raft_consensus.cc:2468] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:24.032224  2131 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:24.032925  2131 ts_tablet_manager.cc:1403] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.007s
I20260709 12:28:24.035830  2131 raft_consensus.cc:359] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.036495  2131 raft_consensus.cc:385] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.036741  2131 raft_consensus.cc:740] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.037472  2131 consensus_queue.cc:260] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.039839  2131 ts_tablet_manager.cc:1434] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:28:24.040753  2131 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:24.036900  2133 ts_tablet_manager.cc:1434] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 12:28:24.043356  2133 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:24.046617  2131 tablet_bootstrap.cc:654] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.055450  2133 tablet_bootstrap.cc:654] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.054383  1975 leader_election.cc:304] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:24.064810  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ed38ed6a4a8cb9ad7301aae1035d" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:24.065768  1937 raft_consensus.cc:3060] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.068070  2161 raft_consensus.cc:2804] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.068674  2161 raft_consensus.cc:493] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.069115  2161 raft_consensus.cc:3060] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.071880  2205 leader_election.cc:290] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:24.074532  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ed38ed6a4a8cb9ad7301aae1035d" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:24.075471  2085 raft_consensus.cc:3060] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.095424  2219 raft_consensus.cc:493] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.096011  2219 raft_consensus.cc:515] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.098204  2219 leader_election.cc:290] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:24.100324  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.101042  2084 raft_consensus.cc:2468] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:24.102380  1828 leader_election.cc:304] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:24.104411  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:24.105134  1935 raft_consensus.cc:2468] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:24.106880  2219 raft_consensus.cc:2804] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.107331  2219 raft_consensus.cc:493] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.107667  2219 raft_consensus.cc:3060] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.111447  1860 raft_consensus.cc:3060] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.115316  2219 raft_consensus.cc:515] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.118212  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:24.118800  2084 raft_consensus.cc:3060] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.126770  2084 raft_consensus.cc:2468] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:24.128027  1828 leader_election.cc:304] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:24.128834  2225 raft_consensus.cc:2804] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.137183  2085 raft_consensus.cc:2468] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:24.138772  1977 leader_election.cc:304] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:24.139961  2205 raft_consensus.cc:2804] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.140822  2205 raft_consensus.cc:697] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:24.141741  1937 raft_consensus.cc:2468] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:24.142903  2129 tablet_bootstrap.cc:654] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.143065  2205 consensus_queue.cc:237] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.147483  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:24.148099  1935 raft_consensus.cc:3060] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.145857  2219 leader_election.cc:290] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:24.145758  2225 raft_consensus.cc:697] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:24.145736  2161 raft_consensus.cc:515] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.155110  2225 consensus_queue.cc:237] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.159029  2130 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:24.159729  2130 ts_tablet_manager.cc:1403] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.168s	user 0.019s	sys 0.062s
I20260709 12:28:24.160993  1935 raft_consensus.cc:2468] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:24.166901  2129 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:24.166597  2130 raft_consensus.cc:359] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.167320  2130 raft_consensus.cc:385] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.167413  2129 ts_tablet_manager.cc:1403] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.005s
I20260709 12:28:24.167623  2130 raft_consensus.cc:740] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.168931  2130 consensus_queue.cc:260] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.171801  2161 leader_election.cc:290] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:24.173393  2130 ts_tablet_manager.cc:1434] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:28:24.175029  2131 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:24.175558  2131 ts_tablet_manager.cc:1403] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.135s	user 0.048s	sys 0.044s
I20260709 12:28:24.176970  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:24.177397  2129 raft_consensus.cc:359] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.178494  1758 catalog_manager.cc:5697] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.181649  2129 raft_consensus.cc:385] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.182029  2129 raft_consensus.cc:740] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
W20260709 12:28:24.183029  1975 leader_election.cc:343] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.185228  2133 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:24.185778  2133 ts_tablet_manager.cc:1403] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.143s	user 0.013s	sys 0.002s
I20260709 12:28:24.170300  2219 raft_consensus.cc:493] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.186712  2219 raft_consensus.cc:515] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.189244  2133 raft_consensus.cc:359] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.190476  2133 raft_consensus.cc:385] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.190791  2133 raft_consensus.cc:740] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.191565  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:24.194501  2219 leader_election.cc:290] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:24.190021  2130 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:24.196012  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
W20260709 12:28:24.197760  1826 leader_election.cc:343] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.191820  2133 consensus_queue.cc:260] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.180895  2131 raft_consensus.cc:359] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.194690  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.199088  2131 raft_consensus.cc:385] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.198959  2182 raft_consensus.cc:493] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.200138  2182 raft_consensus.cc:515] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.201105  2141 raft_consensus.cc:493] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.201622  2141 raft_consensus.cc:515] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.203222  2182 leader_election.cc:290] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:24.194130  1935 raft_consensus.cc:3060] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.204885  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
W20260709 12:28:24.206444  2049 leader_election.cc:343] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.207695  2141 leader_election.cc:290] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:24.208606  2085 raft_consensus.cc:2468] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:24.209806  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "812870f8eb264d3caf48fefa3d70cfa1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.207901  1758 catalog_manager.cc:5697] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.210618  2084 raft_consensus.cc:2468] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:24.212200  1903 leader_election.cc:304] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:24.209803  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "812870f8eb264d3caf48fefa3d70cfa1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:24.213680  1828 leader_election.cc:304] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 8e43e9e956f14e52897d2efc3f361630
W20260709 12:28:24.214427  1901 leader_election.cc:343] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.216747  2141 raft_consensus.cc:2804] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.182888  2129 consensus_queue.cc:260] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.217182  2141 raft_consensus.cc:493] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.217597  2141 raft_consensus.cc:3060] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.221527  2130 tablet_bootstrap.cc:654] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.221954  2219 raft_consensus.cc:2804] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.222509  2219 raft_consensus.cc:493] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.229038  2219 raft_consensus.cc:3060] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.229379  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.199664  2131 raft_consensus.cc:740] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.222551  2010 raft_consensus.cc:3060] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.230124  1935 raft_consensus.cc:2468] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:24.231630  1975 leader_election.cc:304] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:24.235376  2161 raft_consensus.cc:2804] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.225492  2129 ts_tablet_manager.cc:1434] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.003s
I20260709 12:28:24.236088  2161 raft_consensus.cc:697] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:24.236629  2129 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:24.228206  2141 raft_consensus.cc:515] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.238409  2133 ts_tablet_manager.cc:1434] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 12:28:24.239439  2133 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:24.239687  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "812870f8eb264d3caf48fefa3d70cfa1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:24.240077  2161 consensus_queue.cc:237] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
W20260709 12:28:24.241186  1901 leader_election.cc:343] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.243472  2131 consensus_queue.cc:260] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.246268  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "812870f8eb264d3caf48fefa3d70cfa1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:24.246945  2084 raft_consensus.cc:3060] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.249395  2129 tablet_bootstrap.cc:654] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.247056  2141 leader_election.cc:290] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:24.254904  2133 tablet_bootstrap.cc:654] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.253926  2141 raft_consensus.cc:493] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.256619  2219 raft_consensus.cc:515] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.269706  1860 raft_consensus.cc:2393] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 1: Already voted for candidate 8c59f6f4b90d460e832bcff010c18400 in this term.
I20260709 12:28:24.271867  2049 leader_election.cc:304] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:24.272861  2130 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:24.272960  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:24.273471  2130 ts_tablet_manager.cc:1403] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.006s
I20260709 12:28:24.273535  2182 raft_consensus.cc:3060] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.273610  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
W20260709 12:28:24.274514  1826 leader_election.cc:343] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.275394  2133 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:24.275869  2133 ts_tablet_manager.cc:1403] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.005s
I20260709 12:28:24.276747  2130 raft_consensus.cc:359] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.277422  2130 raft_consensus.cc:385] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.277696  2130 raft_consensus.cc:740] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.278584  2130 consensus_queue.cc:260] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.262511  2141 raft_consensus.cc:515] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.281438  2130 ts_tablet_manager.cc:1434] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:24.281719  2182 raft_consensus.cc:2749] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:24.271343  2219 leader_election.cc:290] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:24.285567  2130 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:24.296663  2131 ts_tablet_manager.cc:1434] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.121s	user 0.013s	sys 0.000s
I20260709 12:28:24.298139  2131 tablet_bootstrap.cc:492] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:24.301913  2133 raft_consensus.cc:359] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.304090  2133 raft_consensus.cc:385] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.295867  2161 raft_consensus.cc:493] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.305426  2161 raft_consensus.cc:515] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.304503  2133 raft_consensus.cc:740] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.309954  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.317942  2133 consensus_queue.cc:260] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.310963  2219 raft_consensus.cc:493] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.324608  2219 raft_consensus.cc:515] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.328024  2225 consensus_queue.cc:1048] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:24.329071  2219 leader_election.cc:290] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:24.329487  2083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.330154  2083 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:24.331168  1828 leader_election.cc:304] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:24.334110  2085 raft_consensus.cc:2468] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:24.321194  2084 raft_consensus.cc:2468] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:24.339690  2141 leader_election.cc:290] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:24.299882  2263 raft_consensus.cc:493] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.343063  2263 raft_consensus.cc:515] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.345296  2263 leader_election.cc:290] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:24.348237  2225 raft_consensus.cc:2804] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.348819  2225 raft_consensus.cc:493] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.349217  2225 raft_consensus.cc:3060] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.358356  2219 consensus_queue.cc:1048] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.361461  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:24.362267  2010 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:24.375669  2133 ts_tablet_manager.cc:1434] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.000s
I20260709 12:28:24.378336  2133 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:24.381567  2129 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:24.384018  2129 ts_tablet_manager.cc:1403] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.148s	user 0.034s	sys 0.046s
I20260709 12:28:24.391754  1828 leader_election.cc:304] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:24.392329  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:24.392717  2219 raft_consensus.cc:2804] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.393273  2219 raft_consensus.cc:697] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:24.394269  1975 leader_election.cc:304] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:24.394150  2219 consensus_queue.cc:237] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.384459  2161 leader_election.cc:290] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:24.399694  1903 leader_election.cc:304] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:24.402316  2266 raft_consensus.cc:2804] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.404663  2266 raft_consensus.cc:697] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:24.405845  2266 consensus_queue.cc:237] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.388991  1758 catalog_manager.cc:5697] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.408815  2209 raft_consensus.cc:2749] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:24.409196  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2645174b413446fbf4c9423465821be" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.409466  2133 tablet_bootstrap.cc:654] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.409921  1860 raft_consensus.cc:2468] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:24.413151  2129 raft_consensus.cc:359] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.414068  2129 raft_consensus.cc:385] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.413754  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.414657  2084 raft_consensus.cc:2468] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:24.415966  1903 leader_election.cc:304] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:24.416747  2129 raft_consensus.cc:740] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.417476  2129 consensus_queue.cc:260] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.420048  2129 ts_tablet_manager.cc:1434] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
I20260709 12:28:24.421039  2129 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:24.421164  2130 tablet_bootstrap.cc:654] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.422180  2182 raft_consensus.cc:493] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.417441  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.422607  2264 raft_consensus.cc:2804] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.428382  2131 tablet_bootstrap.cc:654] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.431710  2264 raft_consensus.cc:493] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.436895  2264 raft_consensus.cc:3060] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.432960  2182 raft_consensus.cc:515] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.440894  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2645174b413446fbf4c9423465821be" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.441088  2182 leader_election.cc:290] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:24.441604  1862 raft_consensus.cc:2468] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:24.442919  2049 leader_election.cc:304] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:24.447283  2182 raft_consensus.cc:2804] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.446902  1760 catalog_manager.cc:5697] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.447875  2182 raft_consensus.cc:493] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.448527  2182 raft_consensus.cc:3060] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.448028  2264 raft_consensus.cc:515] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.450505  2264 leader_election.cc:290] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:24.451375  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:24.452024  2084 raft_consensus.cc:3060] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.453639  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2645174b413446fbf4c9423465821be" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.457784  2182 raft_consensus.cc:515] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.459779  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:24.460755  2084 raft_consensus.cc:2468] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:24.460690  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2645174b413446fbf4c9423465821be" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:24.460114  2182 leader_election.cc:290] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:24.461555  1860 raft_consensus.cc:3060] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.468979  1903 leader_election.cc:304] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:24.469274  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2645174b413446fbf4c9423465821be" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:24.469849  1937 raft_consensus.cc:3060] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.470232  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2645174b413446fbf4c9423465821be" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:24.468377  2264 raft_consensus.cc:493] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.475317  2266 raft_consensus.cc:2804] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.475911  2266 raft_consensus.cc:697] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:24.475540  2264 raft_consensus.cc:515] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.477718  2133 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:24.478412  2133 ts_tablet_manager.cc:1403] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.101s	user 0.027s	sys 0.011s
I20260709 12:28:24.482283  2264 leader_election.cc:290] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:24.483058  2266 consensus_queue.cc:237] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.474862  2085 raft_consensus.cc:2393] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:24.497994  2133 raft_consensus.cc:359] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.498747  2133 raft_consensus.cc:385] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.499022  2133 raft_consensus.cc:740] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.488145  2225 raft_consensus.cc:515] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.487972  1901 leader_election.cc:304] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:24.508693  2133 consensus_queue.cc:260] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.511902  1937 raft_consensus.cc:2468] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:24.519162  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
W20260709 12:28:24.522946  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:24.524444  1860 raft_consensus.cc:2468] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:24.525687  2010 raft_consensus.cc:3060] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.529219  2225 leader_election.cc:290] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:24.533839  2049 leader_election.cc:304] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:24.535599  2133 ts_tablet_manager.cc:1434] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.002s
I20260709 12:28:24.528204  2264 raft_consensus.cc:2764] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:24.538210  2279 raft_consensus.cc:2804] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.538910  2279 raft_consensus.cc:697] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:24.539153  2129 tablet_bootstrap.cc:654] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.539801  2279 consensus_queue.cc:237] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
W20260709 12:28:24.527699  1901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 53358.6us
W20260709 12:28:24.543839  1901 leader_election.cc:343] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.544467  2131 tablet_bootstrap.cc:492] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:24.545131  2131 ts_tablet_manager.cc:1403] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.247s	user 0.017s	sys 0.015s
I20260709 12:28:24.549278  2010 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:24.549911  2131 raft_consensus.cc:359] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.550508  1826 leader_election.cc:304] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:24.550779  2131 raft_consensus.cc:385] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.551254  2131 raft_consensus.cc:740] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.551678  2277 raft_consensus.cc:2804] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.552239  2277 raft_consensus.cc:697] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:24.552112  2131 consensus_queue.cc:260] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.553267  2277 consensus_queue.cc:237] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.554301  2130 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:24.555506  2130 ts_tablet_manager.cc:1403] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.270s	user 0.023s	sys 0.017s
I20260709 12:28:24.555568  2133 tablet_bootstrap.cc:492] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:24.554298  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:24.561867  2084 raft_consensus.cc:3060] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.567345  2130 raft_consensus.cc:359] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.568120  2130 raft_consensus.cc:385] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.568390  2130 raft_consensus.cc:740] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.569054  2130 consensus_queue.cc:260] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.569520  2133 tablet_bootstrap.cc:654] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.574942  2130 ts_tablet_manager.cc:1434] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:28:24.577960  2131 ts_tablet_manager.cc:1434] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.001s
I20260709 12:28:24.578858  2131 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
W20260709 12:28:24.563632  1901 leader_election.cc:343] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.582773  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.571300  2225 consensus_queue.cc:1048] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:28:24.588112  2130 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:24.588514  1901 leader_election.cc:343] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.591353  2161 consensus_queue.cc:1048] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:28:24.602770  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:24.602393  2084 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:24.610283  2279 raft_consensus.cc:493] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.615631  2279 raft_consensus.cc:515] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.618278  2279 leader_election.cc:290] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:24.619783  2129 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:24.620285  2129 ts_tablet_manager.cc:1403] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.199s	user 0.026s	sys 0.003s
I20260709 12:28:24.621891  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:24.623108  2129 raft_consensus.cc:359] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
W20260709 12:28:24.623430  2049 leader_election.cc:343] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.624001  2129 raft_consensus.cc:385] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.624364  2129 raft_consensus.cc:740] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.624521  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.625238  2129 consensus_queue.cc:260] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.623412  2131 tablet_bootstrap.cc:654] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:24.626004  2049 leader_election.cc:343] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.627285  2130 tablet_bootstrap.cc:654] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.627372  2049 leader_election.cc:304] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
W20260709 12:28:24.628666  1901 leader_election.cc:343] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.629803  2279 raft_consensus.cc:2749] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:24.631268  2129 ts_tablet_manager.cc:1434] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:28:24.633558  2129 tablet_bootstrap.cc:492] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:24.633352  2161 consensus_queue.cc:1048] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.635265  1901 leader_election.cc:304] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:24.636080  1756 catalog_manager.cc:5697] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.638156  2267 raft_consensus.cc:2749] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:24.652772  2277 consensus_queue.cc:1048] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:24.656601  2133 tablet_bootstrap.cc:492] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:24.657119  2133 ts_tablet_manager.cc:1403] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.102s	user 0.007s	sys 0.007s
I20260709 12:28:24.666451  1759 catalog_manager.cc:5697] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.667675  2133 raft_consensus.cc:359] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.668442  2133 raft_consensus.cc:385] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.668718  2133 raft_consensus.cc:740] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.669454  2133 consensus_queue.cc:260] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.669946  1759 catalog_manager.cc:5697] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.671913  2133 ts_tablet_manager.cc:1434] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:28:24.684098  2133 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:24.676199  1758 catalog_manager.cc:5697] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.677323  2129 tablet_bootstrap.cc:654] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.695474  2133 tablet_bootstrap.cc:654] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.722070  2208 consensus_queue.cc:1048] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.724964  2131 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:24.725524  2131 ts_tablet_manager.cc:1403] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.147s	user 0.014s	sys 0.003s
I20260709 12:28:24.732365  2133 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:24.732941  2133 ts_tablet_manager.cc:1403] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.049s	user 0.005s	sys 0.012s
I20260709 12:28:24.738880  2131 raft_consensus.cc:359] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.739662  2131 raft_consensus.cc:385] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.739954  2131 raft_consensus.cc:740] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.739820  2133 raft_consensus.cc:359] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.740724  2133 raft_consensus.cc:385] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.740710  2131 consensus_queue.cc:260] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.741202  2133 raft_consensus.cc:740] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.742017  2133 consensus_queue.cc:260] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.742944  2131 ts_tablet_manager.cc:1434] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:28:24.743906  2131 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:24.744285  2133 ts_tablet_manager.cc:1434] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:28:24.745159  2133 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:24.752405  2133 tablet_bootstrap.cc:654] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.753295  2279 raft_consensus.cc:493] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.753976  2279 raft_consensus.cc:515] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.756439  2279 leader_election.cc:290] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:24.757675  2131 tablet_bootstrap.cc:654] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.758850  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:24.759694  1937 raft_consensus.cc:2468] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:24.761023  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:24.763314  2049 leader_election.cc:304] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
W20260709 12:28:24.764276  2049 leader_election.cc:343] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.764407  2129 tablet_bootstrap.cc:492] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:24.765290  2129 ts_tablet_manager.cc:1403] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.001s
I20260709 12:28:24.765501  2279 raft_consensus.cc:2804] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.766038  2279 raft_consensus.cc:493] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.766466  2279 raft_consensus.cc:3060] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.769938  2129 raft_consensus.cc:359] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.782579  2129 raft_consensus.cc:385] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.784286  2267 raft_consensus.cc:493] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.784992  2267 raft_consensus.cc:515] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.789340  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ff55130ab4cbf8e84dc429552a7e8" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:24.790833  1901 leader_election.cc:343] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.791169  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ff55130ab4cbf8e84dc429552a7e8" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:24.782981  2129 raft_consensus.cc:740] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.795884  2129 consensus_queue.cc:260] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
W20260709 12:28:24.803632  1901 leader_election.cc:343] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.804317  1901 leader_election.cc:304] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:24.806016  2264 raft_consensus.cc:2749] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:24.814476  2130 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:24.815223  2130 ts_tablet_manager.cc:1403] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.227s	user 0.029s	sys 0.019s
I20260709 12:28:24.816109  2129 ts_tablet_manager.cc:1434] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 12:28:24.817073  2129 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:24.793582  2267 leader_election.cc:290] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:24.818898  2130 raft_consensus.cc:359] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.819927  2130 raft_consensus.cc:385] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.820156  2130 raft_consensus.cc:740] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.820956  2130 consensus_queue.cc:260] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.827941  2130 ts_tablet_manager.cc:1434] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:28:24.832096  2266 consensus_queue.cc:1048] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.832674  2130 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:24.824522  2279 raft_consensus.cc:515] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:24.840034  2205 raft_consensus.cc:493] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.843178  2205 raft_consensus.cc:515] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:24.844475  1860 raft_consensus.cc:3060] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.845556  2205 leader_election.cc:290] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:24.847556  2133 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:24.848178  2133 ts_tablet_manager.cc:1403] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.103s	user 0.011s	sys 0.002s
I20260709 12:28:24.846895  2010 raft_consensus.cc:3060] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.852248  2131 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:24.852784  2131 ts_tablet_manager.cc:1403] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.109s	user 0.015s	sys 0.027s
I20260709 12:28:24.852692  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.854070  2083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.854828  2083 raft_consensus.cc:2393] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e43e9e956f14e52897d2efc3f361630 in current term 1: Already voted for candidate 8c59f6f4b90d460e832bcff010c18400 in this term.
I20260709 12:28:24.855742  2131 raft_consensus.cc:359] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.856420  2131 raft_consensus.cc:385] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.856685  2131 raft_consensus.cc:740] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.856784  1977 leader_election.cc:304] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:24.857378  2131 consensus_queue.cc:260] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.863422  2131 ts_tablet_manager.cc:1434] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:28:24.864346  2131 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:24.871567  2129 tablet_bootstrap.cc:654] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.876439  2130 tablet_bootstrap.cc:654] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.879581  2133 raft_consensus.cc:359] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.880327  2133 raft_consensus.cc:385] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.880612  2133 raft_consensus.cc:740] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.881611  2279 leader_election.cc:290] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:24.884123  2205 raft_consensus.cc:2749] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:24.887001  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:24.887842  1937 raft_consensus.cc:3060] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.892577  2131 tablet_bootstrap.cc:654] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.894989  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
W20260709 12:28:24.896474  2049 leader_election.cc:343] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:24.901870  2294 log.cc:927] Time spent T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:28:24.915287  2219 raft_consensus.cc:493] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.918012  2219 raft_consensus.cc:515] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.926610  2011 raft_consensus.cc:3060] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.929186  2133 consensus_queue.cc:260] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.936077  2133 ts_tablet_manager.cc:1434] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.087s	user 0.009s	sys 0.002s
I20260709 12:28:24.936467  2263 consensus_queue.cc:1048] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.936740  2225 consensus_queue.cc:1048] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.937057  2133 tablet_bootstrap.cc:492] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:24.939007  2130 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:24.942266  2130 ts_tablet_manager.cc:1403] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.110s	user 0.017s	sys 0.026s
I20260709 12:28:24.945142  2130 raft_consensus.cc:359] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.945827  2130 raft_consensus.cc:385] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.946079  2130 raft_consensus.cc:740] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.946777  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:24.947449  1935 raft_consensus.cc:2468] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:24.948261  2219 leader_election.cc:290] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:24.948472  2009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:24.949506  1826 leader_election.cc:304] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
W20260709 12:28:24.951264  1826 leader_election.cc:343] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.946765  2130 consensus_queue.cc:260] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.958777  2129 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:24.959292  2129 ts_tablet_manager.cc:1403] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.142s	user 0.019s	sys 0.011s
I20260709 12:28:24.961196  2225 consensus_queue.cc:1048] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:24.961927  2129 raft_consensus.cc:359] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.962632  2129 raft_consensus.cc:385] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.962894  2129 raft_consensus.cc:740] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.954980  2219 raft_consensus.cc:2804] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.963506  2219 raft_consensus.cc:493] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.963915  2219 raft_consensus.cc:3060] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.964033  2129 consensus_queue.cc:260] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.968812  2129 ts_tablet_manager.cc:1434] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:24.969980  2129 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:24.969679  2277 raft_consensus.cc:493] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.973755  2277 raft_consensus.cc:515] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:24.977190  2129 tablet_bootstrap.cc:654] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.977517  2277 leader_election.cc:290] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:24.980111  2130 ts_tablet_manager.cc:1434] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:28:24.981415  2263 raft_consensus.cc:493] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.981912  2263 raft_consensus.cc:515] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:24.982573  2133 tablet_bootstrap.cc:654] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.984021  2263 leader_election.cc:290] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:24.986460  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:24.986716  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:24.984555  2263 consensus_queue.cc:1048] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:28:24.992803  1901 leader_election.cc:343] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:24.995457  2131 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:24.995990  2131 ts_tablet_manager.cc:1403] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.026s
I20260709 12:28:24.996702  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:24.997417  2084 raft_consensus.cc:2468] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:24.998451  2130 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:24.999861  1903 leader_election.cc:304] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:24.999994  2208 consensus_queue.cc:1048] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.001528  2266 raft_consensus.cc:2804] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.001909  2266 raft_consensus.cc:493] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.002255  2266 raft_consensus.cc:3060] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.009850  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f59bee90544d80a4b32bfcd1116d4a" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:25.012800  1826 leader_election.cc:304] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:24.998371  1937 raft_consensus.cc:2468] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:25.021221  2208 raft_consensus.cc:493] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.021824  2208 raft_consensus.cc:515] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.021252  2273 consensus_queue.cc:1048] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:25.022854  2131 raft_consensus.cc:359] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.023626  2131 raft_consensus.cc:385] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.023901  2131 raft_consensus.cc:740] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.025787  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.025983  2208 leader_election.cc:290] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.026496  2082 raft_consensus.cc:2468] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:25.027841  1861 raft_consensus.cc:3060] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.030556  2131 consensus_queue.cc:260] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.043028  1977 leader_election.cc:304] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.028488  2049 leader_election.cc:304] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:25.056950  2133 tablet_bootstrap.cc:492] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:25.057686  2133 ts_tablet_manager.cc:1403] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.121s	user 0.018s	sys 0.006s
I20260709 12:28:25.059504  2130 tablet_bootstrap.cc:654] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.059734  2131 ts_tablet_manager.cc:1434] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 12:28:25.063741  2131 tablet_bootstrap.cc:492] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:25.066838  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:25.069058  1975 leader_election.cc:343] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.065335  2263 consensus_queue.cc:1048] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:28:25.049911  2161 raft_consensus.cc:2804] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.071628  2161 raft_consensus.cc:493] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.071996  2161 raft_consensus.cc:3060] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.048714  2279 raft_consensus.cc:2804] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.075995  2279 raft_consensus.cc:697] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:25.077180  2279 consensus_queue.cc:237] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.086042  2266 raft_consensus.cc:515] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.107841  2182 consensus_queue.cc:1048] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.109412  2131 tablet_bootstrap.cc:654] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.111128  2182 consensus_queue.cc:1048] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.061748  2219 raft_consensus.cc:515] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.114089  2254 raft_consensus.cc:2749] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:25.115432  2129 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.117837  2129 ts_tablet_manager.cc:1403] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.148s	user 0.011s	sys 0.049s
I20260709 12:28:25.106424  2133 raft_consensus.cc:359] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.118386  2133 raft_consensus.cc:385] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.118671  2133 raft_consensus.cc:740] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.120383  2133 consensus_queue.cc:260] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.122247  2133 ts_tablet_manager.cc:1434] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 12:28:25.148715  2219 leader_election.cc:290] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:25.152697  2083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:25.152842  2133 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:25.167469  2083 raft_consensus.cc:3060] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.170734  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:25.171425  1935 raft_consensus.cc:3060] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.172613  2266 leader_election.cc:290] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.176460  2083 raft_consensus.cc:2468] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:25.177410  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:25.176648  2129 raft_consensus.cc:359] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.187352  2129 raft_consensus.cc:385] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.187824  2129 raft_consensus.cc:740] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.190317  2129 consensus_queue.cc:260] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.205168  2129 ts_tablet_manager.cc:1434] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 12:28:25.192052  2010 raft_consensus.cc:3060] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.206218  2129 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.230433  2279 raft_consensus.cc:493] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.233534  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:25.233842  2279 raft_consensus.cc:515] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.236311  2130 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:25.236969  2130 ts_tablet_manager.cc:1403] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.256s	user 0.023s	sys 0.014s
I20260709 12:28:25.237601  2279 leader_election.cc:290] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.240546  2161 raft_consensus.cc:515] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.244151  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:25.244863  2084 raft_consensus.cc:3060] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.227326  1935 raft_consensus.cc:2468] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:25.255692  2161 leader_election.cc:290] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
W20260709 12:28:25.254089  2049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 90915.9us
I20260709 12:28:25.268385  2130 raft_consensus.cc:359] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.272205  2130 raft_consensus.cc:385] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.272758  2130 raft_consensus.cc:740] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.273419  2129 tablet_bootstrap.cc:654] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:25.267431  1901 leader_election.cc:343] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:25.275218  2309 log.cc:927] Time spent T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:28:25.278813  1826 leader_election.cc:304] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:25.279632  2219 raft_consensus.cc:2804] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.280153  2219 raft_consensus.cc:697] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:25.280992  2219 consensus_queue.cc:237] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.285451  2133 tablet_bootstrap.cc:654] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.286463  2010 raft_consensus.cc:2468] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:25.290910  2084 raft_consensus.cc:2468] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.291934  2225 consensus_queue.cc:1048] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.291697  2130 consensus_queue.cc:260] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.292917  1903 leader_election.cc:304] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:25.293495  2131 tablet_bootstrap.cc:492] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:25.295490  2131 ts_tablet_manager.cc:1403] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.235s	user 0.007s	sys 0.012s
I20260709 12:28:25.296041  1977 leader_election.cc:304] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
W20260709 12:28:25.270841  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:39535, user_credentials={real_user=slave}} blocked reactor thread for 65537.8us
I20260709 12:28:25.297425  2266 raft_consensus.cc:2804] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.298471  2316 raft_consensus.cc:2804] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.298585  2266 raft_consensus.cc:697] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:25.299147  2316 raft_consensus.cc:697] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:25.300103  2316 consensus_queue.cc:237] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.300181  2266 consensus_queue.cc:237] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.302150  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:25.302768  1860 raft_consensus.cc:3060] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.294059  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:25.306082  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.309844  2131 raft_consensus.cc:359] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.310514  2131 raft_consensus.cc:385] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.310784  2131 raft_consensus.cc:740] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.318209  2130 ts_tablet_manager.cc:1434] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.005s
I20260709 12:28:25.319213  2130 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:25.318984  2131 consensus_queue.cc:260] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.319761  2133 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:25.320410  2133 ts_tablet_manager.cc:1403] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.168s	user 0.012s	sys 0.002s
I20260709 12:28:25.324345  1759 catalog_manager.cc:5697] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:25.324165  2133 raft_consensus.cc:359] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.329195  2133 raft_consensus.cc:385] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.330113  2133 raft_consensus.cc:740] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.332947  1860 raft_consensus.cc:2468] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.334654  2131 ts_tablet_manager.cc:1434] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
W20260709 12:28:25.329986  2049 leader_election.cc:343] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.335753  2131 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:25.334228  1756 catalog_manager.cc:5697] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:25.339761  2133 consensus_queue.cc:260] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.347033  2133 ts_tablet_manager.cc:1434] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:28:25.348022  2133 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:25.350925  2130 tablet_bootstrap.cc:654] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.350548  1758 catalog_manager.cc:5697] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:25.353981  2133 tablet_bootstrap.cc:654] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.356732  2131 tablet_bootstrap.cc:654] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:25.345386  2049 leader_election.cc:343] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.361794  2049 leader_election.cc:304] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:25.362789  2313 raft_consensus.cc:2749] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:25.369251  2130 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:25.369987  2130 ts_tablet_manager.cc:1403] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.000s
I20260709 12:28:25.382023  2129 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.382614  2129 ts_tablet_manager.cc:1403] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.177s	user 0.017s	sys 0.006s
I20260709 12:28:25.381800  2130 raft_consensus.cc:359] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.385407  2182 raft_consensus.cc:493] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.388294  2182 raft_consensus.cc:515] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.389586  2263 raft_consensus.cc:493] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.390087  2263 raft_consensus.cc:515] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.390483  2266 raft_consensus.cc:493] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.391070  2266 raft_consensus.cc:515] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.392163  2263 leader_election.cc:290] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:25.393194  2266 leader_election.cc:290] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:25.394503  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.395326  2182 leader_election.cc:290] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:25.387787  2133 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:25.396168  2133 ts_tablet_manager.cc:1403] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.005s
I20260709 12:28:25.397425  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:25.397645  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
W20260709 12:28:25.399945  1901 leader_election.cc:343] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.400655  2133 raft_consensus.cc:359] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.400666  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:25.401686  2130 raft_consensus.cc:385] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.401999  2130 raft_consensus.cc:740] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
W20260709 12:28:25.402377  1901 leader_election.cc:343] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.402706  2130 consensus_queue.cc:260] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
W20260709 12:28:25.403123  1901 leader_election.cc:343] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.402931  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.403770  2082 raft_consensus.cc:2468] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
W20260709 12:28:25.403862  2049 leader_election.cc:343] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.404917  2130 ts_tablet_manager.cc:1434] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:28:25.405607  1903 leader_election.cc:304] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:25.405886  2130 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:25.406793  2266 raft_consensus.cc:2804] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.401167  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:25.407248  2266 raft_consensus.cc:493] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.405936  1760 catalog_manager.cc:5697] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:25.407692  2266 raft_consensus.cc:3060] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.407823  1901 leader_election.cc:304] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:25.408629  2267 raft_consensus.cc:2749] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:25.412396  2129 raft_consensus.cc:359] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.413120  2129 raft_consensus.cc:385] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.413362  2130 tablet_bootstrap.cc:654] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.413389  2129 raft_consensus.cc:740] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.414481  2129 consensus_queue.cc:260] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.416561  2129 ts_tablet_manager.cc:1434] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:28:25.418124  2129 tablet_bootstrap.cc:492] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.424598  2129 tablet_bootstrap.cc:654] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.427098  2131 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:25.427699  2131 ts_tablet_manager.cc:1403] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.092s	user 0.010s	sys 0.007s
I20260709 12:28:25.423750  2266 raft_consensus.cc:515] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.430229  1937 raft_consensus.cc:2393] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 1: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:25.401376  2133 raft_consensus.cc:385] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.430897  2133 raft_consensus.cc:740] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.431751  2133 consensus_queue.cc:260] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.433655  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:25.434265  2082 raft_consensus.cc:3060] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.432597  2266 leader_election.cc:290] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:25.434484  2133 ts_tablet_manager.cc:1434] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 12:28:25.435995  2133 tablet_bootstrap.cc:492] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:25.437695  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:25.441270  2131 raft_consensus.cc:359] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.441974  2131 raft_consensus.cc:385] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.442245  2131 raft_consensus.cc:740] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.442957  2131 consensus_queue.cc:260] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
W20260709 12:28:25.453820  1901 leader_election.cc:343] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.443625  2049 leader_election.cc:304] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:25.457566  2131 ts_tablet_manager.cc:1434] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 12:28:25.458956  2131 tablet_bootstrap.cc:492] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:25.474754  2082 raft_consensus.cc:2468] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:25.475572  2182 raft_consensus.cc:2749] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:25.489513  1903 leader_election.cc:304] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:25.490779  2266 raft_consensus.cc:2804] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.491300  2266 raft_consensus.cc:697] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:25.492285  2129 tablet_bootstrap.cc:492] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.492123  2266 consensus_queue.cc:237] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.492815  2129 ts_tablet_manager.cc:1403] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.075s	user 0.024s	sys 0.015s
I20260709 12:28:25.499840  2131 tablet_bootstrap.cc:654] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.506901  2129 raft_consensus.cc:359] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.507650  2129 raft_consensus.cc:385] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.507894  2129 raft_consensus.cc:740] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.508669  2129 consensus_queue.cc:260] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.509027  2130 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:25.509907  2130 ts_tablet_manager.cc:1403] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.104s	user 0.043s	sys 0.028s
I20260709 12:28:25.511258  2129 ts_tablet_manager.cc:1434] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:28:25.513551  2182 consensus_queue.cc:1048] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:25.514240  2129 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.519485  2130 raft_consensus.cc:359] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.520538  2130 raft_consensus.cc:385] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.520880  2130 raft_consensus.cc:740] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.521561  2129 tablet_bootstrap.cc:654] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.537043  2129 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.537546  2129 ts_tablet_manager.cc:1403] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:28:25.540823  2129 raft_consensus.cc:359] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.541486  2129 raft_consensus.cc:385] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.541992  2129 raft_consensus.cc:740] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.542038  2316 raft_consensus.cc:493] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.542528  2316 raft_consensus.cc:515] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.542739  2129 consensus_queue.cc:260] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.544955  2316 leader_election.cc:290] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:25.545172  2129 ts_tablet_manager.cc:1434] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:25.546141  2129 tablet_bootstrap.cc:492] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.547243  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ff55130ab4cbf8e84dc429552a7e8" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.547341  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ff55130ab4cbf8e84dc429552a7e8" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:25.547786  2130 consensus_queue.cc:260] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.548117  1935 raft_consensus.cc:2468] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:25.548063  1860 raft_consensus.cc:2468] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:25.552848  1975 leader_election.cc:304] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.554039  2182 raft_consensus.cc:493] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.554605  2316 raft_consensus.cc:2804] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.555130  2316 raft_consensus.cc:493] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.554718  2182 raft_consensus.cc:515] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.555541  2129 tablet_bootstrap.cc:654] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.555611  2316 raft_consensus.cc:3060] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.558279  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e922d7ff58764a529d119fa6491f1768" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.559801  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e922d7ff58764a529d119fa6491f1768" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
W20260709 12:28:25.561666  2049 leader_election.cc:343] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:25.562554  2049 leader_election.cc:343] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.562929  2049 leader_election.cc:304] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:25.563649  2328 raft_consensus.cc:2749] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:25.564462  2182 leader_election.cc:290] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:25.566478  2133 tablet_bootstrap.cc:654] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:25.580642  2049 consensus_peers.cc:597] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:25.572535  2130 ts_tablet_manager.cc:1434] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.002s
I20260709 12:28:25.575865  2129 tablet_bootstrap.cc:492] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.577936  2316 raft_consensus.cc:515] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.586351  2129 ts_tablet_manager.cc:1403] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.000s
I20260709 12:28:25.589310  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ff55130ab4cbf8e84dc429552a7e8" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:25.589955  1860 raft_consensus.cc:3060] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.592749  2130 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:25.593478  2129 raft_consensus.cc:359] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.594203  2129 raft_consensus.cc:385] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.594470  2129 raft_consensus.cc:740] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.588682  2316 leader_election.cc:290] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:25.595674  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ff55130ab4cbf8e84dc429552a7e8" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:25.595113  2129 consensus_queue.cc:260] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.596365  1937 raft_consensus.cc:3060] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.597847  2129 ts_tablet_manager.cc:1434] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:25.598803  2129 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.582299  2131 tablet_bootstrap.cc:492] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:25.600248  2131 ts_tablet_manager.cc:1403] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.142s	user 0.007s	sys 0.007s
I20260709 12:28:25.610150  1937 raft_consensus.cc:2468] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.610745  1860 raft_consensus.cc:2468] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.611440  1975 leader_election.cc:304] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.612253  2316 raft_consensus.cc:2804] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.612725  2316 raft_consensus.cc:697] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:25.613437  2316 consensus_queue.cc:237] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.620764  2315 raft_consensus.cc:493] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.621296  2315 raft_consensus.cc:515] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.623505  2129 tablet_bootstrap.cc:654] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.623591  2315 leader_election.cc:290] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:25.626634  2131 raft_consensus.cc:359] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.628270  2131 raft_consensus.cc:385] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.628350  2133 tablet_bootstrap.cc:492] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:25.628571  2131 raft_consensus.cc:740] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.628865  2133 ts_tablet_manager.cc:1403] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.193s	user 0.011s	sys 0.008s
I20260709 12:28:25.630115  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.632318  2130 tablet_bootstrap.cc:654] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:25.632283  1975 leader_election.cc:343] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.630926  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:25.633545  1937 raft_consensus.cc:2468] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:25.633224  1758 catalog_manager.cc:5697] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:25.637495  1975 leader_election.cc:304] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:25.638662  2315 raft_consensus.cc:2804] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.639063  2315 raft_consensus.cc:493] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.639230  2133 raft_consensus.cc:359] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.639595  2315 raft_consensus.cc:3060] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.639452  2131 consensus_queue.cc:260] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.642009  2131 ts_tablet_manager.cc:1434] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.001s
I20260709 12:28:25.639974  2133 raft_consensus.cc:385] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.643132  2131 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:25.643143  2133 raft_consensus.cc:740] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.644140  2133 consensus_queue.cc:260] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.658237  2315 raft_consensus.cc:515] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.661737  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:25.661517  1758 catalog_manager.cc:5697] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:25.662833  2129 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.663501  2129 ts_tablet_manager.cc:1403] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.065s	user 0.019s	sys 0.000s
W20260709 12:28:25.663422  1975 leader_election.cc:343] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.665822  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:25.666486  1937 raft_consensus.cc:3060] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.666895  2131 tablet_bootstrap.cc:654] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.666496  2129 raft_consensus.cc:359] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.667393  2129 raft_consensus.cc:385] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.667680  2129 raft_consensus.cc:740] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.668279  2129 consensus_queue.cc:260] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.670885  2129 ts_tablet_manager.cc:1434] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:25.671895  2129 tablet_bootstrap.cc:492] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.672704  2316 raft_consensus.cc:493] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.673202  2316 raft_consensus.cc:515] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.675401  2316 leader_election.cc:290] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
W20260709 12:28:25.662771  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:25.677412  2133 ts_tablet_manager.cc:1434] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:28:25.681015  2130 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:25.681512  2130 ts_tablet_manager.cc:1403] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.001s
I20260709 12:28:25.682595  2129 tablet_bootstrap.cc:654] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.677430  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.686437  2082 raft_consensus.cc:2393] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e43e9e956f14e52897d2efc3f361630 in current term 1: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:25.689502  2130 raft_consensus.cc:359] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.690734  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d17658b0174e638c0041193b99bf82" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:25.690446  2130 raft_consensus.cc:385] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.699662  1977 leader_election.cc:304] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:25.704303  2130 raft_consensus.cc:740] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.705332  2316 raft_consensus.cc:3060] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.707720  2130 consensus_queue.cc:260] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.709561  2130 ts_tablet_manager.cc:1434] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.003s
I20260709 12:28:25.710618  2130 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:25.705827  2209 raft_consensus.cc:493] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.712118  2209 raft_consensus.cc:515] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.712606  2219 consensus_queue.cc:1048] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:25.714283  2209 leader_election.cc:290] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.716223  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.716871  1860 raft_consensus.cc:2468] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:25.718082  1975 leader_election.cc:304] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.718883  2209 raft_consensus.cc:2804] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.719283  2209 raft_consensus.cc:493] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.719494  2316 raft_consensus.cc:2749] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:25.719659  2209 raft_consensus.cc:3060] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.720650  2316 raft_consensus.cc:493] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.721094  2316 raft_consensus.cc:515] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.723878  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:25.725939  2316 leader_election.cc:290] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:25.727432  2209 raft_consensus.cc:515] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.727862  2131 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:25.728396  2131 ts_tablet_manager.cc:1403] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.000s
I20260709 12:28:25.730801  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:25.731447  1860 raft_consensus.cc:3060] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.731303  2131 raft_consensus.cc:359] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.732000  2131 raft_consensus.cc:385] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.732288  2131 raft_consensus.cc:740] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.733127  2131 consensus_queue.cc:260] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.735997  2131 ts_tablet_manager.cc:1434] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:25.736901  2131 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:25.738355  1860 raft_consensus.cc:2468] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.739539  1975 leader_election.cc:304] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.741209  2266 raft_consensus.cc:493] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.741699  2266 raft_consensus.cc:515] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.744977  2266 leader_election.cc:290] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.745823  2129 tablet_bootstrap.cc:492] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.746305  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e922d7ff58764a529d119fa6491f1768" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.746282  2129 ts_tablet_manager.cc:1403] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.075s	user 0.015s	sys 0.013s
W20260709 12:28:25.748126  1901 leader_election.cc:343] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.749577  2129 raft_consensus.cc:359] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.750242  2129 raft_consensus.cc:385] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.750504  2129 raft_consensus.cc:740] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.751117  2129 consensus_queue.cc:260] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.746482  2316 raft_consensus.cc:2804] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.752209  2219 consensus_queue.cc:1048] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.753181  2130 tablet_bootstrap.cc:654] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.756038  2209 leader_election.cc:290] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.757083  2131 tablet_bootstrap.cc:654] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.757447  2315 leader_election.cc:290] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:25.761588  2130 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:25.761492  1937 raft_consensus.cc:2468] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.763259  2316 raft_consensus.cc:697] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:25.762233  2130 ts_tablet_manager.cc:1403] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.001s
I20260709 12:28:25.764420  2316 consensus_queue.cc:237] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.764914  1860 raft_consensus.cc:3060] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.766613  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e922d7ff58764a529d119fa6491f1768" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.767441  2080 raft_consensus.cc:2468] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:25.763163  1975 leader_election.cc:304] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:25.768405  2208 raft_consensus.cc:2804] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.768324  2130 raft_consensus.cc:359] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.769126  2130 raft_consensus.cc:385] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.769075  2208 raft_consensus.cc:697] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:25.769598  2130 raft_consensus.cc:740] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.769544  1903 leader_election.cc:304] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:25.770197  2129 ts_tablet_manager.cc:1434] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:28:25.770553  2130 consensus_queue.cc:260] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.771265  2129 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.770303  2208 consensus_queue.cc:237] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.771618  2267 consensus_queue.cc:1048] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.772697  2130 ts_tablet_manager.cc:1434] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:25.773188  2267 raft_consensus.cc:2804] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.773590  2130 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:25.774127  2081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:25.778725  2081 raft_consensus.cc:3060] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.780556  2129 tablet_bootstrap.cc:654] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.774323  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.785096  2131 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:25.785714  2131 ts_tablet_manager.cc:1403] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 12:28:25.792325  1977 leader_election.cc:304] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:25.793314  2316 raft_consensus.cc:2749] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:25.773643  2267 raft_consensus.cc:493] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.794075  2267 raft_consensus.cc:3060] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.796692  2131 raft_consensus.cc:359] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.797664  2131 raft_consensus.cc:385] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.798000  2131 raft_consensus.cc:740] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.802125  2130 tablet_bootstrap.cc:654] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.776685  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.810045  2131 consensus_queue.cc:260] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.813153  2131 ts_tablet_manager.cc:1434] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 12:28:25.814440  2131 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:25.818692  2130 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:25.819445  2130 ts_tablet_manager.cc:1403] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.003s
I20260709 12:28:25.822243  2208 raft_consensus.cc:493] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.822806  2208 raft_consensus.cc:515] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.823089  2130 raft_consensus.cc:359] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.823815  2130 raft_consensus.cc:385] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.824084  2130 raft_consensus.cc:740] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.824694  2130 consensus_queue.cc:260] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.825619  2208 leader_election.cc:290] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.827267  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172b29319c78428499e095bf5e76b74e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.828088  1862 raft_consensus.cc:2468] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:25.828199  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172b29319c78428499e095bf5e76b74e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.829276  2084 raft_consensus.cc:2468] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:25.829767  1975 leader_election.cc:304] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.830193  2266 consensus_queue.cc:1048] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:25.830801  2208 raft_consensus.cc:2804] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.831228  2208 raft_consensus.cc:493] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.831636  2208 raft_consensus.cc:3060] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.835479  2131 tablet_bootstrap.cc:654] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.838939  2208 raft_consensus.cc:515] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.842864  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172b29319c78428499e095bf5e76b74e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:25.843374  2130 ts_tablet_manager.cc:1434] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:28:25.843461  1862 raft_consensus.cc:3060] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.844442  2130 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:25.850296  2266 raft_consensus.cc:493] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.851636  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172b29319c78428499e095bf5e76b74e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:25.852329  2085 raft_consensus.cc:3060] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.852358  1862 raft_consensus.cc:2468] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.853780  1975 leader_election.cc:304] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.854784  2316 consensus_queue.cc:1048] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.856324  2129 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.856501  2316 raft_consensus.cc:2804] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:25.856832  2129 ts_tablet_manager.cc:1403] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.086s	user 0.027s	sys 0.019s
I20260709 12:28:25.850847  2266 raft_consensus.cc:515] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:25.858597  2130 tablet_bootstrap.cc:654] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.858695  2208 leader_election.cc:290] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.860432  2129 raft_consensus.cc:359] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.861063  2316 raft_consensus.cc:697] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:25.862084  2316 consensus_queue.cc:237] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:25.863618  2131 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:25.864116  2131 ts_tablet_manager.cc:1403] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 12:28:25.861068  2129 raft_consensus.cc:385] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.865891  2129 raft_consensus.cc:740] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.866290  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:25.866600  2129 consensus_queue.cc:260] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
W20260709 12:28:25.867902  1901 leader_election.cc:343] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.866900  2131 raft_consensus.cc:359] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.871965  2131 raft_consensus.cc:385] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.872340  2131 raft_consensus.cc:740] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.876024  2131 consensus_queue.cc:260] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.880266  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:25.882548  1903 leader_election.cc:343] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:25.883455  1903 leader_election.cc:304] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:25.883248  1756 catalog_manager.cc:5697] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:25.885167  2129 ts_tablet_manager.cc:1434] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:28:25.885346  1756 catalog_manager.cc:5697] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:25.887240  2133 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:25.887866  2362 raft_consensus.cc:2749] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:25.888816  2129 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.898847  2131 ts_tablet_manager.cc:1434] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.002s
I20260709 12:28:25.899930  2131 tablet_bootstrap.cc:492] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:25.900074  2129 tablet_bootstrap.cc:654] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:25.909178  2345 log.cc:927] Time spent T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:28:25.915001  2315 consensus_queue.cc:1048] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:25.919557  2266 leader_election.cc:290] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:25.923736  2131 tablet_bootstrap.cc:654] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.929423  2182 raft_consensus.cc:493] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.930137  2182 raft_consensus.cc:515] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.930495  2081 raft_consensus.cc:2468] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:25.932646  2182 leader_election.cc:290] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:25.934406  2362 raft_consensus.cc:493] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.934820  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:25.935213  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:25.934959  2362 raft_consensus.cc:515] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.935920  2010 raft_consensus.cc:2468] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:25.936654  1937 raft_consensus.cc:2468] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:25.937449  2362 leader_election.cc:290] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:25.938267  2049 leader_election.cc:304] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:25.940001  2182 raft_consensus.cc:2804] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.939893  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:25.940603  2182 raft_consensus.cc:493] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.940943  2010 raft_consensus.cc:2468] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:25.941705  2182 raft_consensus.cc:3060] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.943068  1901 leader_election.cc:304] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:25.944700  2362 raft_consensus.cc:2804] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:25.945217  2362 raft_consensus.cc:493] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:25.945660  2362 raft_consensus.cc:3060] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.945847  2129 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:25.946476  2129 ts_tablet_manager.cc:1403] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.006s
I20260709 12:28:25.949016  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:25.949468  2129 raft_consensus.cc:359] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.950145  2129 raft_consensus.cc:385] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.950409  2129 raft_consensus.cc:740] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.951074  2129 consensus_queue.cc:260] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:25.953426  2129 ts_tablet_manager.cc:1434] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:25.954447  2129 tablet_bootstrap.cc:492] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:25.954834  2182 raft_consensus.cc:515] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.956053  2080 raft_consensus.cc:2393] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:25.958606  2131 tablet_bootstrap.cc:492] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:25.959105  2131 ts_tablet_manager.cc:1403] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.000s
I20260709 12:28:25.960362  2129 tablet_bootstrap.cc:654] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:25.961983  2131 raft_consensus.cc:359] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.962692  2131 raft_consensus.cc:385] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:25.968501  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:25.972400  2182 leader_election.cc:290] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:25.973301  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:25.974257  2010 raft_consensus.cc:3060] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:25.963001  2131 raft_consensus.cc:740] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:25.977465  2131 consensus_queue.cc:260] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:25.980382  2131 ts_tablet_manager.cc:1434] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:28:25.981622  2131 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.010130  1758 catalog_manager.cc:5697] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:26.046213  2011 raft_consensus.cc:3060] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.051451  2085 raft_consensus.cc:2468] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:26.018409  2361 raft_consensus.cc:493] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.019093  2266 raft_consensus.cc:493] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.064217  2129 tablet_bootstrap.cc:492] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:26.064735  2129 ts_tablet_manager.cc:1403] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.110s	user 0.018s	sys 0.011s
I20260709 12:28:26.053208  2362 raft_consensus.cc:515] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.068146  2219 raft_consensus.cc:493] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.071343  2219 raft_consensus.cc:515] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.073606  2219 leader_election.cc:290] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:26.074684  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:26.076627  1935 raft_consensus.cc:2393] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:26.057137  2267 raft_consensus.cc:515] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.058851  2266 raft_consensus.cc:515] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.059262  2361 raft_consensus.cc:515] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.090891  1937 raft_consensus.cc:2393] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 1: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:26.098253  2361 leader_election.cc:290] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:26.111533  2266 leader_election.cc:290] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:26.111624  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e922d7ff58764a529d119fa6491f1768" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:26.112363  2080 raft_consensus.cc:3060] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.113758  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.114763  2131 tablet_bootstrap.cc:654] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.121064  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baf92d63594b4e3081ad94cac31690d1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:26.122778  1826 leader_election.cc:304] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:26.122921  2316 raft_consensus.cc:493] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.117254  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baf92d63594b4e3081ad94cac31690d1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.123513  2316 raft_consensus.cc:515] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.124991  2263 consensus_queue.cc:1048] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:26.125349  2008 raft_consensus.cc:2468] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:26.126223  2267 leader_election.cc:290] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:26.126698  2316 leader_election.cc:290] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:26.130527  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:26.128535  2362 leader_election.cc:290] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:26.131266  1937 raft_consensus.cc:2468] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:26.132687  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:26.134203  1975 leader_election.cc:343] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.135022  1975 leader_election.cc:304] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:26.136958  2161 raft_consensus.cc:2804] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:26.139402  2219 raft_consensus.cc:3060] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.141064  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:26.143112  2129 raft_consensus.cc:359] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.143877  2129 raft_consensus.cc:385] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.144171  2129 raft_consensus.cc:740] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.147037  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:26.148128  2085 raft_consensus.cc:2393] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:26.150766  2129 consensus_queue.cc:260] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.141759  2161 raft_consensus.cc:493] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.151765  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.152019  2161 raft_consensus.cc:3060] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.153455  2130 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:26.153950  2130 ts_tablet_manager.cc:1403] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.310s	user 0.038s	sys 0.064s
I20260709 12:28:26.154785  2080 raft_consensus.cc:2468] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:26.156723  2133 tablet_bootstrap.cc:654] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.156648  2130 raft_consensus.cc:359] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.159993  2130 raft_consensus.cc:385] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.160281  2130 raft_consensus.cc:740] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.160686  2129 ts_tablet_manager.cc:1434] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 12:28:26.162360  2129 tablet_bootstrap.cc:492] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:26.163556  1901 leader_election.cc:304] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:26.164431  2267 raft_consensus.cc:2804] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:26.164810  2267 raft_consensus.cc:493] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.165184  2267 raft_consensus.cc:3060] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.165977  2010 raft_consensus.cc:2468] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:26.165071  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e922d7ff58764a529d119fa6491f1768" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:26.160959  2130 consensus_queue.cc:260] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.174451  2130 ts_tablet_manager.cc:1434] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:28:26.175681  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2bbe2dde38842ab846f9c2d8a260918" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:26.176442  2010 raft_consensus.cc:2393] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
W20260709 12:28:26.176532  1901 leader_election.cc:343] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.173367  2362 consensus_queue.cc:1048] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:28:26.177763  1901 leader_election.cc:343] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.173393  1862 raft_consensus.cc:3060] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.180968  2130 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:26.182016  2129 tablet_bootstrap.cc:654] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.172446  2049 leader_election.cc:304] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee
I20260709 12:28:26.183377  2131 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:26.183859  2131 ts_tablet_manager.cc:1403] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.203s	user 0.025s	sys 0.003s
I20260709 12:28:26.184538  2182 raft_consensus.cc:2804] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.185326  2316 consensus_queue.cc:1048] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:26.185577  1903 leader_election.cc:304] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:26.186407  2263 raft_consensus.cc:2804] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.186870  2263 raft_consensus.cc:697] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:26.187781  2263 consensus_queue.cc:237] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.203112  2130 tablet_bootstrap.cc:654] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.205868  2182 raft_consensus.cc:697] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:26.209390  2131 raft_consensus.cc:359] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.211040  2131 raft_consensus.cc:385] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.211357  2131 raft_consensus.cc:740] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.221275  2131 consensus_queue.cc:260] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.222285  2316 consensus_queue.cc:1048] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:26.224097  2131 ts_tablet_manager.cc:1434] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 12:28:26.225137  2131 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.227351  2182 consensus_queue.cc:237] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.229132  1862 raft_consensus.cc:2468] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:26.230278  2131 tablet_bootstrap.cc:654] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.235968  2133 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:26.236589  2133 ts_tablet_manager.cc:1403] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.350s	user 0.012s	sys 0.010s
I20260709 12:28:26.248543  2133 raft_consensus.cc:359] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.249267  2133 raft_consensus.cc:385] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.249547  2133 raft_consensus.cc:740] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.250958  2133 consensus_queue.cc:260] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.255136  2267 raft_consensus.cc:515] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.256697  1901 leader_election.cc:304] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:26.259502  2362 raft_consensus.cc:2749] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.266958  2133 ts_tablet_manager.cc:1434] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:28:26.267985  2133 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:26.271901  2219 raft_consensus.cc:2749] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.273391  2161 raft_consensus.cc:515] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.275204  2133 tablet_bootstrap.cc:654] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.275525  2008 raft_consensus.cc:2393] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 1: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:26.295593  2267 leader_election.cc:290] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:26.296523  2161 leader_election.cc:290] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:26.305366  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baf92d63594b4e3081ad94cac31690d1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
W20260709 12:28:26.306874  1901 leader_election.cc:343] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.316723  2131 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:26.317286  2131 ts_tablet_manager.cc:1403] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.015s
I20260709 12:28:26.320007  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baf92d63594b4e3081ad94cac31690d1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:26.320667  2010 raft_consensus.cc:3060] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.320389  2131 raft_consensus.cc:359] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.321121  2131 raft_consensus.cc:385] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.321403  2131 raft_consensus.cc:740] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.322130  2131 consensus_queue.cc:260] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.323661  1901 leader_election.cc:304] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:26.324294  2131 ts_tablet_manager.cc:1434] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:26.325233  2131 tablet_bootstrap.cc:492] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.325171  2362 raft_consensus.cc:3060] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.329105  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:26.331914  2131 tablet_bootstrap.cc:654] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.333709  2316 consensus_queue.cc:1048] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:26.336248  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:26.355712  2130 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:26.356411  2130 ts_tablet_manager.cc:1403] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.181s	user 0.017s	sys 0.011s
I20260709 12:28:26.360672  2130 raft_consensus.cc:359] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.362141  2130 raft_consensus.cc:385] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.362677  2130 raft_consensus.cc:740] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.364125  2267 raft_consensus.cc:493] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.363994  2130 consensus_queue.cc:260] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.391330  2130 ts_tablet_manager.cc:1434] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 12:28:26.378136  2267 raft_consensus.cc:515] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.404299  2129 tablet_bootstrap.cc:492] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:26.404922  2129 ts_tablet_manager.cc:1403] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.243s	user 0.026s	sys 0.018s
I20260709 12:28:26.405030  2133 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:26.405684  2133 ts_tablet_manager.cc:1403] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.138s	user 0.010s	sys 0.008s
I20260709 12:28:26.410723  2129 raft_consensus.cc:359] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.411871  2129 raft_consensus.cc:385] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.411643  2315 consensus_queue.cc:1048] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:26.413915  2129 raft_consensus.cc:740] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.416877  2010 raft_consensus.cc:2468] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:26.419612  2267 leader_election.cc:290] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:26.421509  2129 consensus_queue.cc:260] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.424659  2266 raft_consensus.cc:493] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.425298  2266 raft_consensus.cc:515] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.429399  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179c67e242914e64a56c8d1f829cdef5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:26.434032  1901 leader_election.cc:343] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.438791  2266 leader_election.cc:290] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:26.442574  2219 raft_consensus.cc:493] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.443099  2219 raft_consensus.cc:515] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.445438  2219 leader_election.cc:290] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:26.445456  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2a8fdea8624729ae4b5191c85fb520" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:26.447752  1903 leader_election.cc:343] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.453258  2129 ts_tablet_manager.cc:1434] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 12:28:26.457830  2129 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:26.460074  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179c67e242914e64a56c8d1f829cdef5" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.436462  2133 raft_consensus.cc:359] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.461086  2133 raft_consensus.cc:385] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.461356  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2a8fdea8624729ae4b5191c85fb520" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.462143  2007 raft_consensus.cc:2468] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
W20260709 12:28:26.462811  1975 leader_election.cc:343] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.459123  1756 catalog_manager.cc:5697] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:26.464241  2372 consensus_queue.cc:1048] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:26.473018  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baf92d63594b4e3081ad94cac31690d1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:26.474005  1937 raft_consensus.cc:2393] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:26.474731  2131 tablet_bootstrap.cc:492] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:26.475277  2131 ts_tablet_manager.cc:1403] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.150s	user 0.052s	sys 0.085s
I20260709 12:28:26.476016  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baf92d63594b4e3081ad94cac31690d1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.476835  2007 raft_consensus.cc:2393] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:26.478072  2131 raft_consensus.cc:359] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.478736  2131 raft_consensus.cc:385] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.478977  2131 raft_consensus.cc:740] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.479781  2131 consensus_queue.cc:260] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.461503  2133 raft_consensus.cc:740] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.480700  1826 leader_election.cc:304] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:26.482363  2131 ts_tablet_manager.cc:1434] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:26.483412  2131 tablet_bootstrap.cc:492] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.481860  2133 consensus_queue.cc:260] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.486562  2133 ts_tablet_manager.cc:1434] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.002s
I20260709 12:28:26.487524  2133 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:26.489104  2161 raft_consensus.cc:493] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.491039  2131 tablet_bootstrap.cc:654] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.490473  2161 raft_consensus.cc:515] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.494192  2161 leader_election.cc:290] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:26.495849  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:26.498279  1977 leader_election.cc:343] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.502868  2129 tablet_bootstrap.cc:654] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.504936  2130 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:26.505527  2133 tablet_bootstrap.cc:654] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.509104  1756 catalog_manager.cc:5697] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:26.519451  2362 raft_consensus.cc:2749] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.511448  2372 consensus_queue.cc:1048] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:26.515334  2129 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:26.520538  2129 ts_tablet_manager.cc:1403] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.005s
I20260709 12:28:26.521759  2254 raft_consensus.cc:3060] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.523375  2129 raft_consensus.cc:359] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.524067  2129 raft_consensus.cc:385] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.524317  2129 raft_consensus.cc:740] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.524905  2129 consensus_queue.cc:260] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.527697  2131 tablet_bootstrap.cc:492] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:26.528149  2131 ts_tablet_manager.cc:1403] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.002s
I20260709 12:28:26.530792  2131 raft_consensus.cc:359] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.531531  2131 raft_consensus.cc:385] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:28:26.531622  1826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 51002.6us
I20260709 12:28:26.531960  2131 raft_consensus.cc:740] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.532732  2131 consensus_queue.cc:260] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.531011  1936 raft_consensus.cc:2468] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:26.534713  2131 ts_tablet_manager.cc:1434] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:26.535307  1901 leader_election.cc:304] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:26.536185  2129 ts_tablet_manager.cc:1434] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
W20260709 12:28:26.536532  1901 leader_election.cc:343] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.537143  2129 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:26.537683  2267 raft_consensus.cc:2804] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.538352  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:26.538427  2267 raft_consensus.cc:697] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:26.536988  1901 leader_election.cc:304] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:26.539845  2267 consensus_queue.cc:237] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.540426  1901 leader_election.cc:304] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:26.543802  2131 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.530913  2254 raft_consensus.cc:2749] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.539057  1935 raft_consensus.cc:2468] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:26.549059  2129 tablet_bootstrap.cc:654] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.550638  2361 raft_consensus.cc:2749] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.551393  2361 raft_consensus.cc:2804] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:26.551714  2361 raft_consensus.cc:493] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.552038  2361 raft_consensus.cc:3060] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.553598  2131 tablet_bootstrap.cc:654] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.555358  2263 raft_consensus.cc:493] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.555847  2263 raft_consensus.cc:515] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.558204  2263 leader_election.cc:290] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:26.560603  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:26.563510  1903 leader_election.cc:343] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.563807  2131 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:26.564286  2131 ts_tablet_manager.cc:1403] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.006s
I20260709 12:28:26.566830  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:26.568612  1901 leader_election.cc:343] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.569072  1901 leader_election.cc:304] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:26.569850  2266 raft_consensus.cc:2749] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.578533  2129 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:26.579216  2129 ts_tablet_manager.cc:1403] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.042s	user 0.006s	sys 0.006s
W20260709 12:28:26.582150  1975 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.19.36.254:40093, user_credentials={real_user=slave}} blocked reactor thread for 44068.9us
I20260709 12:28:26.581883  1760 catalog_manager.cc:5697] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:26.587934  2131 raft_consensus.cc:359] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.588739  2131 raft_consensus.cc:385] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.589042  2131 raft_consensus.cc:740] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.589578  2130 tablet_bootstrap.cc:654] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.589938  2131 consensus_queue.cc:260] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.582512  2129 raft_consensus.cc:359] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.591871  2129 raft_consensus.cc:385] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.588376  1975 leader_election.cc:304] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:26.592190  2129 raft_consensus.cc:740] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.593407  2129 consensus_queue.cc:260] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.593649  1975 leader_election.cc:304] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:26.595765  2129 ts_tablet_manager.cc:1434] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.004s
I20260709 12:28:26.596179  2315 consensus_queue.cc:1048] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:26.597051  2161 raft_consensus.cc:2804] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:26.597461  2161 raft_consensus.cc:493] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.597787  2161 raft_consensus.cc:3060] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.598268  2131 ts_tablet_manager.cc:1434] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.002s
I20260709 12:28:26.599270  2131 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.599531  2376 raft_consensus.cc:493] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.596709  2129 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:26.599992  2376 raft_consensus.cc:515] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.602221  2376 leader_election.cc:290] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:26.604368  2315 consensus_queue.cc:1048] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:26.604655  1858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:26.606892  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:26.607571  1936 raft_consensus.cc:2468] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:26.607933  2131 tablet_bootstrap.cc:654] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.613152  2049 leader_election.cc:304] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:26.614459  2129 tablet_bootstrap.cc:654] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.616398  2376 raft_consensus.cc:2804] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:26.616801  2376 raft_consensus.cc:493] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:26.614514  2049 leader_election.cc:343] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.621016  2130 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:26.621527  2130 ts_tablet_manager.cc:1403] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.229s	user 0.008s	sys 0.003s
I20260709 12:28:26.592878  2372 raft_consensus.cc:2804] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.622442  2372 raft_consensus.cc:697] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:26.618515  2376 raft_consensus.cc:3060] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.623577  2182 raft_consensus.cc:493] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.623330  2372 consensus_queue.cc:237] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.624143  2182 raft_consensus.cc:515] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.626461  2182 leader_election.cc:290] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:26.627107  2161 raft_consensus.cc:515] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.627345  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6241be73e134c25909f8054db9f85dc" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.628095  2010 raft_consensus.cc:2468] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:26.628706  2130 raft_consensus.cc:359] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.629524  2130 raft_consensus.cc:385] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.629851  2130 raft_consensus.cc:740] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.592327  2361 raft_consensus.cc:515] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.630535  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:26.631517  1936 raft_consensus.cc:3060] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.633337  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2a8fdea8624729ae4b5191c85fb520" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:26.634739  2182 consensus_queue.cc:1048] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:26.635221  2010 raft_consensus.cc:3060] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.636035  2362 consensus_queue.cc:1048] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:26.636366  2049 leader_election.cc:304] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:26.637202  2182 raft_consensus.cc:2804] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:26.636952  2130 consensus_queue.cc:260] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.638322  2182 raft_consensus.cc:493] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.641769  2131 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:26.643493  2131 ts_tablet_manager.cc:1403] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.010s
I20260709 12:28:26.644292  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:26.645383  2182 raft_consensus.cc:3060] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.647612  2376 raft_consensus.cc:515] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.649884  2376 leader_election.cc:290] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:26.653174  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:26.654758  2130 ts_tablet_manager.cc:1434] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:28:26.660317  2133 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:26.638274  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6241be73e134c25909f8054db9f85dc" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:26.661212  2133 ts_tablet_manager.cc:1403] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.174s	user 0.021s	sys 0.022s
I20260709 12:28:26.662964  2130 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:26.668730  2131 raft_consensus.cc:359] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.669770  2131 raft_consensus.cc:385] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.670095  2131 raft_consensus.cc:740] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.671777  2362 raft_consensus.cc:493] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.671450  1861 raft_consensus.cc:2468] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:26.671030  2131 consensus_queue.cc:260] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.639315  2361 leader_election.cc:290] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
W20260709 12:28:26.699656  2049 leader_election.cc:343] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.672330  2362 raft_consensus.cc:515] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.700456  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:26.701234  1934 raft_consensus.cc:3060] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.639077  2161 leader_election.cc:290] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
W20260709 12:28:26.702847  1977 leader_election.cc:343] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:26.720245  1977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:37785, user_credentials={real_user=slave}} blocked reactor thread for 44908.6us
I20260709 12:28:26.702986  2010 raft_consensus.cc:2468] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:26.711905  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2a8fdea8624729ae4b5191c85fb520" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:26.733153  2131 ts_tablet_manager.cc:1434] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.089s	user 0.005s	sys 0.002s
I20260709 12:28:26.723217  2362 leader_election.cc:290] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:26.735613  2131 tablet_bootstrap.cc:492] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.739230  1936 raft_consensus.cc:2468] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:26.743993  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ff50d36854429790ff6c78da6f2c8b" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:26.746141  1903 leader_election.cc:343] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.748759  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ff50d36854429790ff6c78da6f2c8b" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.749146  1975 leader_election.cc:304] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:26.753917  2161 raft_consensus.cc:2804] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.754441  2161 raft_consensus.cc:697] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:26.755275  2161 consensus_queue.cc:237] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.755769  1934 raft_consensus.cc:2468] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:26.761725  2133 raft_consensus.cc:359] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.762506  2133 raft_consensus.cc:385] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.762778  2133 raft_consensus.cc:740] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.768895  2133 consensus_queue.cc:260] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.771554  2133 ts_tablet_manager.cc:1434] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.109s	user 0.003s	sys 0.003s
I20260709 12:28:26.772769  2133 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
W20260709 12:28:26.775032  1901 leader_election.cc:343] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.775595  1901 leader_election.cc:304] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:26.776746  1901 leader_election.cc:304] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:26.778846  2355 raft_consensus.cc:2749] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.779148  2373 consensus_queue.cc:1048] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:26.779609  2355 raft_consensus.cc:2804] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.779949  2049 leader_election.cc:304] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:26.780121  2355 raft_consensus.cc:697] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:26.780817  2373 raft_consensus.cc:2804] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.781091  2355 consensus_queue.cc:237] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.781698  2373 raft_consensus.cc:697] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
W20260709 12:28:26.783428  1903 leader_election.cc:343] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.746870  2182 raft_consensus.cc:515] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.784595  2373 consensus_queue.cc:237] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.786276  2182 leader_election.cc:290] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:26.797933  2133 tablet_bootstrap.cc:654] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.805469  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6241be73e134c25909f8054db9f85dc" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:26.805421  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6241be73e134c25909f8054db9f85dc" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:26.806263  1862 raft_consensus.cc:3060] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.806670  2007 raft_consensus.cc:3060] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.813557  1760 catalog_manager.cc:5697] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:26.823518  1862 raft_consensus.cc:2468] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:26.828765  2049 leader_election.cc:304] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:26.829494  2129 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:26.830124  2129 ts_tablet_manager.cc:1403] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.234s	user 0.005s	sys 0.025s
I20260709 12:28:26.831712  2376 raft_consensus.cc:2804] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.833622  2376 raft_consensus.cc:697] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:26.836565  2376 consensus_queue.cc:237] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.839710  2131 tablet_bootstrap.cc:654] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.846089  2130 tablet_bootstrap.cc:654] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.876905  2355 raft_consensus.cc:493] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.873562  2007 raft_consensus.cc:2468] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:26.891885  2355 raft_consensus.cc:515] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
W20260709 12:28:26.913157  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:26.926688  2129 raft_consensus.cc:359] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.924006  2263 raft_consensus.cc:493] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.929773  2129 raft_consensus.cc:385] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.935377  2263 raft_consensus.cc:515] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.939043  2133 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:26.939615  2133 ts_tablet_manager.cc:1403] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.167s	user 0.007s	sys 0.027s
I20260709 12:28:26.931512  2129 raft_consensus.cc:740] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.942024  1760 catalog_manager.cc:5697] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:26.945191  2263 leader_election.cc:290] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:26.944531  2129 consensus_queue.cc:260] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.950225  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:26.950630  2416 consensus_queue.cc:1048] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:26.951121  2409 log.cc:927] Time spent T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:28:26.955443  2129 ts_tablet_manager.cc:1434] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.125s	user 0.005s	sys 0.004s
I20260709 12:28:26.957034  2129 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:26.957636  2355 leader_election.cc:290] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
W20260709 12:28:26.955931  1903 leader_election.cc:343] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.958163  2133 raft_consensus.cc:359] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.958847  2133 raft_consensus.cc:385] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.959121  2133 raft_consensus.cc:740] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.959841  2133 consensus_queue.cc:260] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:26.961580  1858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:26.964033  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:26.965845  2131 tablet_bootstrap.cc:492] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:26.966095  2133 ts_tablet_manager.cc:1434] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:28:26.966334  2131 ts_tablet_manager.cc:1403] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.232s	user 0.015s	sys 0.003s
I20260709 12:28:26.967126  2133 tablet_bootstrap.cc:492] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:26.969157  2131 raft_consensus.cc:359] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.969816  2131 raft_consensus.cc:385] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.969990  2372 raft_consensus.cc:493] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.970109  2131 raft_consensus.cc:740] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.970537  2372 raft_consensus.cc:515] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:26.972338  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:26.972678  2372 leader_election.cc:290] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:26.973471  2129 tablet_bootstrap.cc:654] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:26.973995  1901 leader_election.cc:343] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.974516  1901 leader_election.cc:304] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:26.974159  2130 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:26.975224  2416 raft_consensus.cc:2749] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:26.975543  2130 ts_tablet_manager.cc:1403] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.318s	user 0.018s	sys 0.002s
I20260709 12:28:26.970916  2131 consensus_queue.cc:260] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:26.975809  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:26.976404  2080 raft_consensus.cc:2468] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:26.978628  1858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:26.980228  1975 leader_election.cc:343] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.979595  1759 catalog_manager.cc:5697] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:26.967242  2412 consensus_queue.cc:1048] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:28:26.982093  2130 raft_consensus.cc:359] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
W20260709 12:28:26.983058  1901 leader_election.cc:343] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:26.984155  1901 leader_election.cc:343] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:26.984547  1901 leader_election.cc:304] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:26.985659  1977 leader_election.cc:304] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:26.986101  2130 raft_consensus.cc:385] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.986424  2130 raft_consensus.cc:740] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.987098  2130 consensus_queue.cc:260] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.989059  2131 ts_tablet_manager.cc:1434] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:28:26.989675  2130 ts_tablet_manager.cc:1434] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:28:26.991792  2131 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:26.991101  1756 catalog_manager.cc:5697] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:26.991092  2129 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:26.992584  2133 tablet_bootstrap.cc:654] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.993000  2129 ts_tablet_manager.cc:1403] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.002s
I20260709 12:28:26.992811  1756 catalog_manager.cc:5697] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:26.995127  2130 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:26.996248  2316 raft_consensus.cc:2804] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:26.996162  2129 raft_consensus.cc:359] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:26.996737  2316 raft_consensus.cc:493] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.996929  2129 raft_consensus.cc:385] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.997165  2316 raft_consensus.cc:3060] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.997282  2129 raft_consensus.cc:740] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.998227  2129 consensus_queue.cc:260] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.000605  2129 ts_tablet_manager.cc:1434] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:28:27.001497  2129 tablet_bootstrap.cc:492] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:27.002540  2130 tablet_bootstrap.cc:654] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.007722  2129 tablet_bootstrap.cc:654] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.009917  2412 raft_consensus.cc:2749] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.023661  2416 consensus_queue.cc:1048] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.028409  2131 tablet_bootstrap.cc:654] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.037869  2129 tablet_bootstrap.cc:492] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:27.038475  2129 ts_tablet_manager.cc:1403] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.016s
I20260709 12:28:27.049587  2129 raft_consensus.cc:359] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.051355  2129 raft_consensus.cc:385] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.051623  2129 raft_consensus.cc:740] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.052350  2129 consensus_queue.cc:260] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.054406  2129 ts_tablet_manager.cc:1434] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 12:28:27.055365  2129 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:27.061550  2129 tablet_bootstrap.cc:654] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.081699  2254 raft_consensus.cc:493] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.082394  2254 raft_consensus.cc:515] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.084585  2254 leader_election.cc:290] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:27.087100  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:27.089859  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.090664  1936 raft_consensus.cc:2393] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:27.092358  1826 leader_election.cc:304] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:27.093328  2254 raft_consensus.cc:3060] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.100282  2131 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:27.100811  2131 ts_tablet_manager.cc:1403] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.111s	user 0.025s	sys 0.016s
I20260709 12:28:27.068768  2316 raft_consensus.cc:515] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.110589  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:27.111220  2084 raft_consensus.cc:3060] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.112604  2254 raft_consensus.cc:2749] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.118232  2084 raft_consensus.cc:2468] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:27.119395  1977 leader_election.cc:304] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:27.120558  2161 raft_consensus.cc:2804] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.121203  2161 raft_consensus.cc:697] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:27.122318  2161 consensus_queue.cc:237] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.123064  2316 leader_election.cc:290] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:27.110605  2131 raft_consensus.cc:359] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.128086  2131 raft_consensus.cc:385] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.128397  2131 raft_consensus.cc:740] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.129683  2130 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:27.130198  2130 ts_tablet_manager.cc:1403] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.135s	user 0.007s	sys 0.025s
I20260709 12:28:27.130102  2131 consensus_queue.cc:260] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.131573  2131 ts_tablet_manager.cc:1434] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:28:27.132318  2131 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:27.132606  2133 tablet_bootstrap.cc:492] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:27.133109  2133 ts_tablet_manager.cc:1403] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.166s	user 0.027s	sys 0.012s
I20260709 12:28:27.135900  2133 raft_consensus.cc:359] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.136570  2133 raft_consensus.cc:385] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.136819  2133 raft_consensus.cc:740] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.137501  2133 consensus_queue.cc:260] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.137929  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
W20260709 12:28:27.140792  1975 leader_election.cc:343] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.140995  2130 raft_consensus.cc:359] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.141637  2130 raft_consensus.cc:385] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.141884  2130 raft_consensus.cc:740] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.142714  2130 consensus_queue.cc:260] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.148248  2130 ts_tablet_manager.cc:1434] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:28:27.148458  2129 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:27.148909  2129 ts_tablet_manager.cc:1403] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.094s	user 0.018s	sys 0.015s
I20260709 12:28:27.149140  2133 ts_tablet_manager.cc:1434] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:28:27.150106  2133 tablet_bootstrap.cc:492] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:27.151909  2130 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:27.158272  2130 tablet_bootstrap.cc:654] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.158967  2376 raft_consensus.cc:493] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.159508  2376 raft_consensus.cc:515] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.161644  2376 leader_election.cc:290] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:27.162792  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:27.162830  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.163880  1936 raft_consensus.cc:2393] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 1: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:27.165112  2129 raft_consensus.cc:359] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.165868  2129 raft_consensus.cc:385] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.166139  2129 raft_consensus.cc:740] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.166894  2129 consensus_queue.cc:260] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.168262  2133 tablet_bootstrap.cc:654] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.168094  2049 leader_election.cc:304] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:27.169735  2376 raft_consensus.cc:3060] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.170338  2129 ts_tablet_manager.cc:1434] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 12:28:27.170555  2131 tablet_bootstrap.cc:654] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.171486  2129 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:27.177071  2376 raft_consensus.cc:2749] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.177656  2129 tablet_bootstrap.cc:654] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.192703  2130 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:27.193277  2130 ts_tablet_manager.cc:1403] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.042s	user 0.006s	sys 0.004s
I20260709 12:28:27.194940  2316 consensus_queue.cc:1048] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.195974  2130 raft_consensus.cc:359] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.195840  2129 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:27.196707  2130 raft_consensus.cc:385] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.196934  2129 ts_tablet_manager.cc:1403] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.005s
I20260709 12:28:27.197016  2130 raft_consensus.cc:740] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.198025  2130 consensus_queue.cc:260] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.201447  2133 tablet_bootstrap.cc:492] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:27.202080  2133 ts_tablet_manager.cc:1403] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.000s
I20260709 12:28:27.202863  1760 catalog_manager.cc:5697] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.205268  2133 raft_consensus.cc:359] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.206682  2133 raft_consensus.cc:385] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.206943  2133 raft_consensus.cc:740] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.207945  2131 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:27.199877  2129 raft_consensus.cc:359] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.208415  2131 ts_tablet_manager.cc:1403] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.076s	user 0.006s	sys 0.007s
I20260709 12:28:27.200477  2130 ts_tablet_manager.cc:1434] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:28:27.209139  2133 consensus_queue.cc:260] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.208460  2129 raft_consensus.cc:385] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.210361  2129 raft_consensus.cc:740] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.210592  2130 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:27.211256  2131 raft_consensus.cc:359] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.211006  2129 consensus_queue.cc:260] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.212632  2131 raft_consensus.cc:385] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.212973  2131 raft_consensus.cc:740] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.213798  2131 consensus_queue.cc:260] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.215939  2131 ts_tablet_manager.cc:1434] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:28:27.216488  2133 ts_tablet_manager.cc:1434] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:28:27.217429  2133 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:27.218426  2316 consensus_queue.cc:1048] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.227419  2131 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:27.230166  1861 raft_consensus.cc:3060] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.235630  2129 ts_tablet_manager.cc:1434] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:28:27.240788  2376 consensus_queue.cc:1048] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:27.241585  2316 raft_consensus.cc:493] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.242107  2316 raft_consensus.cc:515] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.243139  2161 consensus_queue.cc:1048] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.244210  2316 leader_election.cc:290] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:27.248430  2407 consensus_queue.cc:1048] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.249625  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32982b9d743f4ceb80f17292b281fd6f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:27.250895  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32982b9d743f4ceb80f17292b281fd6f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:27.252746  1977 leader_election.cc:343] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.254467  2133 tablet_bootstrap.cc:654] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.255443  2131 tablet_bootstrap.cc:654] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.260488  2254 raft_consensus.cc:493] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.260977  2254 raft_consensus.cc:515] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.262072  2130 tablet_bootstrap.cc:654] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:27.263553  1975 leader_election.cc:343] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.263103  2254 leader_election.cc:290] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:27.264463  1975 leader_election.cc:304] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:27.264707  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.265368  1935 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:27.265897  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:27.266762  1826 leader_election.cc:304] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:27.268266  2254 raft_consensus.cc:2804] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.268654  2254 raft_consensus.cc:493] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.268983  2254 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.265911  2161 raft_consensus.cc:2749] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.270160  2084 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 0.
I20260709 12:28:27.267062  2407 raft_consensus.cc:493] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.271708  2372 consensus_queue.cc:1048] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.275153  2254 raft_consensus.cc:515] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.271700  2407 raft_consensus.cc:515] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.278815  2254 leader_election.cc:290] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:27.280225  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:27.280634  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2a8fdea8624729ae4b5191c85fb520" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.281064  1935 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.281186  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:27.282195  2084 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.282982  2130 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:27.283599  2130 ts_tablet_manager.cc:1403] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.002s
I20260709 12:28:27.286644  2407 leader_election.cc:290] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:27.287595  2130 raft_consensus.cc:359] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.289179  2130 raft_consensus.cc:385] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.289805  2130 raft_consensus.cc:740] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.290851  2130 consensus_queue.cc:260] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.292703  1935 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:27.294077  1826 leader_election.cc:304] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:27.295207  2254 raft_consensus.cc:2804] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.295876  2254 raft_consensus.cc:697] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:27.296162  2130 ts_tablet_manager.cc:1434] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:28:27.297036  2254 consensus_queue.cc:237] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.299150  2130 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:27.308235  2129 tablet_bootstrap.cc:492] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:27.323882  2131 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:27.325946  2131 ts_tablet_manager.cc:1403] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.099s	user 0.004s	sys 0.008s
I20260709 12:28:27.331550  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2a8fdea8624729ae4b5191c85fb520" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:27.332746  2007 raft_consensus.cc:2393] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 1: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:27.334524  2130 tablet_bootstrap.cc:654] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.335294  2049 leader_election.cc:304] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:27.338827  2316 raft_consensus.cc:493] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.339545  2316 raft_consensus.cc:515] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.340075  2407 raft_consensus.cc:3060] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.342273  2316 leader_election.cc:290] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:27.343814  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179c67e242914e64a56c8d1f829cdef5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.344547  1934 raft_consensus.cc:2468] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:27.346215  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179c67e242914e64a56c8d1f829cdef5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:27.346868  1862 raft_consensus.cc:2468] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:27.348206  1975 leader_election.cc:304] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:27.350283  2209 raft_consensus.cc:2804] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.350656  2209 raft_consensus.cc:493] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.350987  2209 raft_consensus.cc:3060] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.381794  2316 raft_consensus.cc:493] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.382402  2316 raft_consensus.cc:515] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.384532  2316 leader_election.cc:290] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:27.388288  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:27.388859  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:27.390235  1977 leader_election.cc:343] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.391040  2416 consensus_queue.cc:1048] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:27.398728  2416 raft_consensus.cc:493] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.399345  2416 raft_consensus.cc:515] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.401765  2373 consensus_queue.cc:1048] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:27.405108  1975 leader_election.cc:343] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.405725  1975 leader_election.cc:304] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:27.406145  2414 raft_consensus.cc:493] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.410959  2316 raft_consensus.cc:2749] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.410301  2416 leader_election.cc:290] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:27.423108  2414 raft_consensus.cc:515] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.410601  2209 raft_consensus.cc:515] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.421659  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:27.414765  2376 consensus_queue.cc:1048] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:28:27.435369  2133 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:27.423676  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:27.436888  2084 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 1.
I20260709 12:28:27.437995  2209 leader_election.cc:290] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:27.439095  2129 tablet_bootstrap.cc:654] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.447949  2133 ts_tablet_manager.cc:1403] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.231s	user 0.051s	sys 0.024s
I20260709 12:28:27.453991  2414 leader_election.cc:290] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:27.456431  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:27.454988  2133 raft_consensus.cc:359] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.460465  2133 raft_consensus.cc:385] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.469882  2133 raft_consensus.cc:740] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
W20260709 12:28:27.480718  1901 leader_election.cc:343] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.471515  2208 raft_consensus.cc:493] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.479701  2133 consensus_queue.cc:260] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.476503  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179c67e242914e64a56c8d1f829cdef5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:27.484668  1937 raft_consensus.cc:3060] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.486635  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179c67e242914e64a56c8d1f829cdef5" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:27.487601  1862 raft_consensus.cc:3060] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:27.491468  1901 leader_election.cc:343] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.501286  1760 catalog_manager.cc:5697] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.507360  2372 raft_consensus.cc:493] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.516997  2372 raft_consensus.cc:515] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.492030  2208 raft_consensus.cc:515] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.519075  2372 leader_election.cc:290] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:27.534909  2131 raft_consensus.cc:359] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.535899  2131 raft_consensus.cc:385] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.536162  2131 raft_consensus.cc:740] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.530205  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:27.541766  2133 ts_tablet_manager.cc:1434] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.091s	user 0.010s	sys 0.019s
W20260709 12:28:27.539680  2436 log.cc:927] Time spent T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:28:27.546438  2131 consensus_queue.cc:260] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.549180  1932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.549651  2133 tablet_bootstrap.cc:492] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:27.549875  1932 raft_consensus.cc:2468] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:27.555243  1858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:27.556656  1975 leader_election.cc:343] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.557644  1975 leader_election.cc:304] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:27.558979  2316 raft_consensus.cc:2804] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.559368  2316 raft_consensus.cc:493] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:27.559532  1901 leader_election.cc:343] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.547096  2414 raft_consensus.cc:493] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.560247  1901 leader_election.cc:304] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:27.560983  2414 raft_consensus.cc:515] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.562785  2208 leader_election.cc:290] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:27.559741  2316 raft_consensus.cc:3060] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.575438  2412 raft_consensus.cc:2749] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:27.538980  1903 leader_election.cc:343] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.586637  2131 ts_tablet_manager.cc:1434] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.258s	user 0.008s	sys 0.000s
I20260709 12:28:27.588143  2131 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:27.580493  2414 leader_election.cc:290] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:27.611619  2081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ff50d36854429790ff6c78da6f2c8b" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:27.601979  1903 leader_election.cc:304] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:27.628751  2414 raft_consensus.cc:2749] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:27.628983  1903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.196:37785, user_credentials={real_user=slave}} blocked reactor thread for 94963.9us
I20260709 12:28:27.631573  2131 tablet_bootstrap.cc:654] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.636490  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db6a2096401b4cec9b542b6eb99f705b" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:27.640436  1937 raft_consensus.cc:2468] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:27.644558  2407 raft_consensus.cc:2749] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:27.650779  1977 leader_election.cc:343] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.653962  2316 raft_consensus.cc:515] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.657788  1862 raft_consensus.cc:2468] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:27.662415  2372 consensus_queue.cc:1048] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:27.664885  2316 leader_election.cc:290] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:27.672722  2129 tablet_bootstrap.cc:492] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:27.673369  2129 ts_tablet_manager.cc:1403] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.365s	user 0.022s	sys 0.028s
I20260709 12:28:27.677008  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db6a2096401b4cec9b542b6eb99f705b" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:27.679338  2416 consensus_queue.cc:1048] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:27.683058  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 123839us
W20260709 12:28:27.689834  1901 leader_election.cc:343] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.710970  2129 raft_consensus.cc:359] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.713605  2129 raft_consensus.cc:385] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.713398  1975 leader_election.cc:304] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:27.713954  2129 raft_consensus.cc:740] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.705137  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ff50d36854429790ff6c78da6f2c8b" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.716063  2129 consensus_queue.cc:260] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:27.715363  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:27.716491  2208 raft_consensus.cc:2804] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:27.728171  1903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:37785, user_credentials={real_user=slave}} blocked reactor thread for 48848.7us
I20260709 12:28:27.726644  1935 raft_consensus.cc:2468] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:27.744932  2130 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:27.745474  2130 ts_tablet_manager.cc:1403] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.447s	user 0.026s	sys 0.064s
I20260709 12:28:27.745760  2131 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:27.746265  2131 ts_tablet_manager.cc:1403] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.158s	user 0.016s	sys 0.003s
I20260709 12:28:27.733630  2208 raft_consensus.cc:697] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:27.734117  1934 raft_consensus.cc:3060] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.766287  2133 tablet_bootstrap.cc:654] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:27.767912  1903 leader_election.cc:343] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.768523  1903 leader_election.cc:304] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:27.769363  2355 raft_consensus.cc:2749] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.773254  2130 raft_consensus.cc:359] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.780390  2130 raft_consensus.cc:385] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.780858  2130 raft_consensus.cc:740] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.787251  1934 raft_consensus.cc:2468] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:27.799113  2130 consensus_queue.cc:260] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.811573  2131 raft_consensus.cc:359] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.812599  2131 raft_consensus.cc:385] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.812945  2131 raft_consensus.cc:740] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.815724  2131 consensus_queue.cc:260] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.817936  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:27.779961  2208 consensus_queue.cc:237] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.829875  2378 raft_consensus.cc:493] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.830518  2378 raft_consensus.cc:515] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.832618  2378 leader_election.cc:290] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:27.835079  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:27.835633  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:27.836407  2254 consensus_queue.cc:1048] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.836561  1935 raft_consensus.cc:2468] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:27.839649  2049 leader_election.cc:304] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
W20260709 12:28:27.840891  2049 leader_election.cc:343] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.842778  2376 raft_consensus.cc:2804] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.843446  2376 raft_consensus.cc:493] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.843920  2376 raft_consensus.cc:3060] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.845314  2219 consensus_queue.cc:1048] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.864002  2416 raft_consensus.cc:493] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.864693  2416 raft_consensus.cc:515] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.867610  2416 leader_election.cc:290] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:27.873589  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:27.875589  1901 leader_election.cc:343] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.878310  2219 raft_consensus.cc:493] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.879172  2219 raft_consensus.cc:515] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:27.880501  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:27.884392  2219 leader_election.cc:290] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:27.885915  2130 ts_tablet_manager.cc:1434] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.140s	user 0.010s	sys 0.000s
I20260709 12:28:27.898723  2131 ts_tablet_manager.cc:1434] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.152s	user 0.009s	sys 0.001s
I20260709 12:28:27.900404  2131 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:27.912379  2130 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:27.922431  2129 ts_tablet_manager.cc:1434] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.249s	user 0.016s	sys 0.031s
I20260709 12:28:27.926339  2129 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:27.938355  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:27.928032  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:27.946259  1975 consensus_peers.cc:597] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.947271  1975 leader_election.cc:343] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.965888  1975 leader_election.cc:304] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:27.967446  1975 leader_election.cc:304] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:27.967063  2458 raft_consensus.cc:2804] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:28:27.944201  1903 leader_election.cc:343] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.970628  2458 raft_consensus.cc:493] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.974568  1903 leader_election.cc:304] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:27.975973  2355 raft_consensus.cc:2749] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.973572  2457 raft_consensus.cc:2804] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.977233  2457 raft_consensus.cc:697] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:27.978163  2457 consensus_queue.cc:237] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:27.973973  2458 raft_consensus.cc:3060] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.986176  2130 tablet_bootstrap.cc:654] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.987483  2131 tablet_bootstrap.cc:654] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.989769  2133 tablet_bootstrap.cc:492] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:27.990319  2133 ts_tablet_manager.cc:1403] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.441s	user 0.025s	sys 0.006s
I20260709 12:28:27.967698  2376 raft_consensus.cc:515] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:27.996470  2376 leader_election.cc:290] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:28.002336  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:28.002975  1934 raft_consensus.cc:3060] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.004544  1826 leader_election.cc:304] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:28.005817  2445 raft_consensus.cc:2749] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.999594  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:28.008997  2133 raft_consensus.cc:359] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
W20260709 12:28:28.010911  2049 leader_election.cc:343] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.015123  2133 raft_consensus.cc:385] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.016952  2133 raft_consensus.cc:740] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.001076  2458 raft_consensus.cc:515] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.017680  2133 consensus_queue.cc:260] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.024792  2133 ts_tablet_manager.cc:1434] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:28:28.037758  2133 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:28.041738  2458 leader_election.cc:290] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:28.026506  2129 tablet_bootstrap.cc:654] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.044248  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ff50d36854429790ff6c78da6f2c8b" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:28.044970  2082 raft_consensus.cc:3060] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.051923  2133 tablet_bootstrap.cc:654] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.033658  1934 raft_consensus.cc:2468] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:28.068197  2049 leader_election.cc:304] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:28.069813  2378 raft_consensus.cc:2804] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.070322  2378 raft_consensus.cc:697] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:28.071172  2378 consensus_queue.cc:237] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.075133  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ff50d36854429790ff6c78da6f2c8b" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:28.075380  2414 raft_consensus.cc:493] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.075845  1934 raft_consensus.cc:3060] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.075865  2414 raft_consensus.cc:515] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.079718  2414 leader_election.cc:290] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.081755  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0af4a569f84ef9841ad0c788c5d6ca" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:28.083690  1901 leader_election.cc:343] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.089103  1758 catalog_manager.cc:5697] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.112493  2082 raft_consensus.cc:2468] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:28.126194  1977 leader_election.cc:304] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:28.105253  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0af4a569f84ef9841ad0c788c5d6ca" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:28.130797  1903 leader_election.cc:343] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.131310  1903 leader_election.cc:304] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:28.132303  2361 raft_consensus.cc:2749] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.133790  2459 raft_consensus.cc:2804] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.134598  2459 raft_consensus.cc:697] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:28.136468  2458 raft_consensus.cc:493] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.154635  1934 raft_consensus.cc:2468] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
W20260709 12:28:28.153818  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:28.154906  2458 raft_consensus.cc:515] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.158134  2458 leader_election.cc:290] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:28.159286  2081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:28.160733  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:28.161396  2131 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:28.161419  1935 raft_consensus.cc:2468] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:28.162057  2131 ts_tablet_manager.cc:1403] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.262s	user 0.015s	sys 0.009s
W20260709 12:28:28.162246  1977 leader_election.cc:343] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.162755  2129 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:28.163273  2129 ts_tablet_manager.cc:1403] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.237s	user 0.006s	sys 0.011s
I20260709 12:28:28.163879  1975 leader_election.cc:304] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:28.160992  2459 consensus_queue.cc:237] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.165032  2458 raft_consensus.cc:2804] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:28.165414  2458 raft_consensus.cc:493] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.165762  2458 raft_consensus.cc:3060] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.165848  2129 raft_consensus.cc:359] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.166575  2129 raft_consensus.cc:385] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.166878  2129 raft_consensus.cc:740] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.167567  2129 consensus_queue.cc:260] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.169086  2129 ts_tablet_manager.cc:1434] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:28.169968  2129 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:28.163858  2130 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:28.171411  2130 ts_tablet_manager.cc:1403] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.284s	user 0.011s	sys 0.005s
I20260709 12:28:28.171169  2131 raft_consensus.cc:359] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.173434  2131 raft_consensus.cc:385] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.174454  2131 raft_consensus.cc:740] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.180972  1862 raft_consensus.cc:3060] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.183914  2129 tablet_bootstrap.cc:654] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.185866  2458 raft_consensus.cc:515] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.188724  2131 consensus_queue.cc:260] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.185150  2130 raft_consensus.cc:359] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.189864  2130 raft_consensus.cc:385] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.190196  2130 raft_consensus.cc:740] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.211750  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:28.212462  1934 raft_consensus.cc:3060] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.212797  2458 leader_election.cc:290] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:28.212375  2130 consensus_queue.cc:260] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.215291  2130 ts_tablet_manager.cc:1434] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:28:28.216251  2130 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:28.211790  2081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
W20260709 12:28:28.218446  1977 leader_election.cc:343] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.222347  2131 ts_tablet_manager.cc:1434] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.003s
I20260709 12:28:28.224749  2131 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:28.225139  1934 raft_consensus.cc:2468] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:28.226320  1975 leader_election.cc:304] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:28.227025  1760 catalog_manager.cc:5697] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.230779  2131 tablet_bootstrap.cc:654] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.232606  2458 raft_consensus.cc:2804] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.233163  2458 raft_consensus.cc:697] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:28.235739  2458 consensus_queue.cc:237] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.240742  2130 tablet_bootstrap.cc:654] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.241221  2129 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:28.243937  2133 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:28.244442  2133 ts_tablet_manager.cc:1403] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.207s	user 0.009s	sys 0.010s
I20260709 12:28:28.246524  2129 ts_tablet_manager.cc:1403] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.004s
I20260709 12:28:28.247428  2133 raft_consensus.cc:359] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.248167  2133 raft_consensus.cc:385] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.248435  2133 raft_consensus.cc:740] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.249090  2133 consensus_queue.cc:260] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.249631  2129 raft_consensus.cc:359] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.250660  2129 raft_consensus.cc:385] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.251854  2133 ts_tablet_manager.cc:1434] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:28.253445  2129 raft_consensus.cc:740] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.254335  2133 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:28.254227  2458 consensus_queue.cc:1048] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:28.254271  2129 consensus_queue.cc:260] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.261086  2129 ts_tablet_manager.cc:1434] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:28:28.262768  2129 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:28.261233  2133 tablet_bootstrap.cc:654] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.269512  2131 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:28.266181  2130 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:28.271870  2130 ts_tablet_manager.cc:1403] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.056s	user 0.007s	sys 0.005s
I20260709 12:28:28.273021  2378 raft_consensus.cc:493] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.273517  2378 raft_consensus.cc:515] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.274523  2130 raft_consensus.cc:359] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.275214  2130 raft_consensus.cc:385] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.275466  2130 raft_consensus.cc:740] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.275606  2378 leader_election.cc:290] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.276193  2130 consensus_queue.cc:260] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.277403  2161 consensus_queue.cc:1048] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.271263  2131 ts_tablet_manager.cc:1403] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.000s
I20260709 12:28:28.283730  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.284933  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.285588  2008 raft_consensus.cc:2468] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:28.286159  2129 tablet_bootstrap.cc:654] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.288367  2049 leader_election.cc:304] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:28.289152  2130 ts_tablet_manager.cc:1434] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
W20260709 12:28:28.289178  2049 leader_election.cc:343] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.292253  2130 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:28.292768  2378 raft_consensus.cc:2804] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:28.293222  2378 raft_consensus.cc:493] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.293932  2378 raft_consensus.cc:3060] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.302491  2378 raft_consensus.cc:515] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.302371  2131 raft_consensus.cc:359] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.304095  2131 raft_consensus.cc:385] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.304499  2131 raft_consensus.cc:740] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.305397  2131 consensus_queue.cc:260] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.308619  2131 ts_tablet_manager.cc:1434] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:28:28.309605  2131 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:28.309690  2130 tablet_bootstrap.cc:654] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.313458  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:28.314093  2008 raft_consensus.cc:3060] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.316200  2131 tablet_bootstrap.cc:654] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.317302  1756 catalog_manager.cc:5697] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.320379  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
W20260709 12:28:28.324826  2049 leader_election.cc:343] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.324661  1756 catalog_manager.cc:5697] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.319424  2378 leader_election.cc:290] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.327247  1756 catalog_manager.cc:5697] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.326561  2008 raft_consensus.cc:2468] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:28.332084  2049 leader_election.cc:304] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:28.332861  2376 raft_consensus.cc:2804] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.333340  2376 raft_consensus.cc:697] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:28.336519  2376 consensus_queue.cc:237] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.360886  2133 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:28.361558  2133 ts_tablet_manager.cc:1403] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.107s	user 0.007s	sys 0.012s
I20260709 12:28:28.363085  1757 catalog_manager.cc:5697] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.365165  2129 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:28.365691  2129 ts_tablet_manager.cc:1403] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.013s
I20260709 12:28:28.368781  2129 raft_consensus.cc:359] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.369442  2129 raft_consensus.cc:385] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.369735  2129 raft_consensus.cc:740] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.370363  2129 consensus_queue.cc:260] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.371310  2161 raft_consensus.cc:493] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.371755  2161 raft_consensus.cc:515] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.373673  2161 leader_election.cc:290] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.373674  2133 raft_consensus.cc:359] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.374552  2133 raft_consensus.cc:385] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.374837  2133 raft_consensus.cc:740] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.375525  2133 consensus_queue.cc:260] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.376187  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.377820  2133 ts_tablet_manager.cc:1434] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 12:28:28.378888  2133 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:28.379093  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:28.381047  2131 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:28.381712  2131 ts_tablet_manager.cc:1403] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.072s	user 0.007s	sys 0.011s
I20260709 12:28:28.382017  2161 raft_consensus.cc:493] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:28.381007  1975 leader_election.cc:343] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.382658  2161 raft_consensus.cc:515] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.378054  2129 ts_tablet_manager.cc:1434] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
W20260709 12:28:28.384680  1977 leader_election.cc:343] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.384802  2161 leader_election.cc:290] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:28.385128  1977 leader_election.cc:304] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:28.385326  2129 tablet_bootstrap.cc:492] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:28.385210  2133 tablet_bootstrap.cc:654] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.388975  2459 raft_consensus.cc:2749] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.388839  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:28.389665  1934 raft_consensus.cc:2468] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:28.390491  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.390939  1975 leader_election.cc:304] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:28.392064  2459 raft_consensus.cc:2804] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:28:28.392143  1975 leader_election.cc:343] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.392320  2131 raft_consensus.cc:359] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.393170  2131 raft_consensus.cc:385] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.390654  2130 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:28.396907  2130 ts_tablet_manager.cc:1403] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.107s	user 0.015s	sys 0.007s
I20260709 12:28:28.392499  2459 raft_consensus.cc:493] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.399582  2459 raft_consensus.cc:3060] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.399606  2130 raft_consensus.cc:359] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.400616  2130 raft_consensus.cc:385] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.400906  2130 raft_consensus.cc:740] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.401593  2130 consensus_queue.cc:260] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.403112  2129 tablet_bootstrap.cc:654] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.404711  2130 ts_tablet_manager.cc:1434] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:28.405723  2130 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:28.399639  2131 raft_consensus.cc:740] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.406632  2131 consensus_queue.cc:260] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.408450  2131 ts_tablet_manager.cc:1434] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:28:28.409955  2131 tablet_bootstrap.cc:492] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:28.415640  2131 tablet_bootstrap.cc:654] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:28.420433  1975 consensus_peers.cc:597] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:28.425588  2459 raft_consensus.cc:515] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.431490  2459 leader_election.cc:290] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:28.432587  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:28.433235  1937 raft_consensus.cc:3060] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.438791  2133 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:28.439361  2133 ts_tablet_manager.cc:1403] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.004s
I20260709 12:28:28.440218  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
W20260709 12:28:28.441431  1975 leader_election.cc:343] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.442720  2133 raft_consensus.cc:359] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.443431  2133 raft_consensus.cc:385] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.443969  1937 raft_consensus.cc:2468] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:28.443709  2133 raft_consensus.cc:740] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.445288  1975 leader_election.cc:304] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:28.448709  2133 consensus_queue.cc:260] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.449581  2459 raft_consensus.cc:2804] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.450225  2129 tablet_bootstrap.cc:492] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:28.450176  2459 raft_consensus.cc:697] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:28.450812  2129 ts_tablet_manager.cc:1403] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.005s
I20260709 12:28:28.451387  2459 consensus_queue.cc:237] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.452986  2416 raft_consensus.cc:493] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.453470  2416 raft_consensus.cc:515] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.453640  2129 raft_consensus.cc:359] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.454339  2129 raft_consensus.cc:385] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.454627  2129 raft_consensus.cc:740] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.455015  2133 ts_tablet_manager.cc:1434] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:28:28.456151  2133 tablet_bootstrap.cc:492] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:28.455827  2129 consensus_queue.cc:260] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.457013  2416 leader_election.cc:290] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:28.457806  2129 ts_tablet_manager.cc:1434] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:28.458777  2129 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:28.459970  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4631b580b30e4f57b7a7a09b2452dc75" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.460397  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4631b580b30e4f57b7a7a09b2452dc75" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.460428  2416 raft_consensus.cc:493] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.461649  2130 tablet_bootstrap.cc:654] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:28.461337  1901 leader_election.cc:343] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:28.462421  1901 leader_election.cc:343] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.462173  2416 raft_consensus.cc:515] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.462846  1901 leader_election.cc:304] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:28.463570  2355 raft_consensus.cc:2749] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.465885  2416 leader_election.cc:290] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:28.466174  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:28.467893  1901 leader_election.cc:343] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.468343  2133 tablet_bootstrap.cc:654] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.469007  2376 raft_consensus.cc:493] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.469513  2376 raft_consensus.cc:515] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.472219  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.472903  2376 leader_election.cc:290] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:28.474931  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db6a2096401b4cec9b542b6eb99f705b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
W20260709 12:28:28.475179  1901 leader_election.cc:343] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.475682  1901 leader_election.cc:304] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:28.475755  1937 raft_consensus.cc:2468] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:28.476505  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db6a2096401b4cec9b542b6eb99f705b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.477365  1862 raft_consensus.cc:2468] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:28.478293  2355 raft_consensus.cc:2749] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.480696  2049 leader_election.cc:304] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:28.482985  2131 tablet_bootstrap.cc:492] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:28.483980  2254 raft_consensus.cc:493] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.484486  2254 raft_consensus.cc:515] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.485056  2449 raft_consensus.cc:2804] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:28.485481  2449 raft_consensus.cc:493] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.485818  2449 raft_consensus.cc:3060] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.490406  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:28.491290  1934 raft_consensus.cc:2393] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:28.493289  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a6b8329d4924908b3a364b840d691fe" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:28.494542  2209 consensus_queue.cc:1048] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.495756  2449 raft_consensus.cc:515] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.496518  1828 leader_election.cc:304] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:28.498312  2445 raft_consensus.cc:3060] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.498716  2129 tablet_bootstrap.cc:654] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.499823  2449 leader_election.cc:290] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:28.505685  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db6a2096401b4cec9b542b6eb99f705b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:28.506585  1934 raft_consensus.cc:3060] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.511289  2130 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:28.511818  2130 ts_tablet_manager.cc:1403] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.106s	user 0.012s	sys 0.000s
I20260709 12:28:28.512028  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db6a2096401b4cec9b542b6eb99f705b" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:28.512622  1862 raft_consensus.cc:3060] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.514557  2130 raft_consensus.cc:359] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.515275  2130 raft_consensus.cc:385] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.515532  2130 raft_consensus.cc:740] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.518144  2131 ts_tablet_manager.cc:1403] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.108s	user 0.012s	sys 0.001s
I20260709 12:28:28.519500  1756 catalog_manager.cc:5697] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.521016  2131 raft_consensus.cc:359] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.521788  2131 raft_consensus.cc:385] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.522047  2131 raft_consensus.cc:740] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.522740  2254 leader_election.cc:290] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:28.523739  2131 consensus_queue.cc:260] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.516201  2130 consensus_queue.cc:260] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.528827  2446 raft_consensus.cc:493] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.529467  2446 raft_consensus.cc:515] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.532570  2130 ts_tablet_manager.cc:1434] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:28:28.537712  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.538895  2008 raft_consensus.cc:2393] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:28.541203  2446 leader_election.cc:290] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:28.543365  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf093503636049fbb48b9a304d578c38" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:28.544716  1862 raft_consensus.cc:2468] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:28.545786  2445 raft_consensus.cc:2749] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.546245  2049 leader_election.cc:304] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:28.547835  2449 raft_consensus.cc:2804] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.548463  2449 raft_consensus.cc:697] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:28.548820  1934 raft_consensus.cc:2468] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:28.548138  2131 ts_tablet_manager.cc:1434] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:28:28.549414  2449 consensus_queue.cc:237] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.550904  2131 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:28.554169  2130 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:28.555661  1828 leader_election.cc:304] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:28.555558  2376 consensus_queue.cc:1048] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.558996  2131 tablet_bootstrap.cc:654] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.559743  2133 tablet_bootstrap.cc:492] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:28.560276  2133 ts_tablet_manager.cc:1403] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.104s	user 0.025s	sys 0.048s
I20260709 12:28:28.561203  2129 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:28.561303  2445 raft_consensus.cc:3060] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.561721  2129 ts_tablet_manager.cc:1403] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.103s	user 0.008s	sys 0.004s
I20260709 12:28:28.565831  2416 raft_consensus.cc:493] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.565615  2133 raft_consensus.cc:359] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.566344  2133 raft_consensus.cc:385] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.566299  2416 raft_consensus.cc:515] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.567284  2129 raft_consensus.cc:359] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.567965  2129 raft_consensus.cc:385] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.568284  2129 raft_consensus.cc:740] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.568849  2416 leader_election.cc:290] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.569689  2081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:28.570871  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.568951  2129 consensus_queue.cc:260] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
W20260709 12:28:28.571441  1903 leader_election.cc:343] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:28.573534  1901 leader_election.cc:343] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.573967  1901 leader_election.cc:304] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:28.575354  2412 raft_consensus.cc:2749] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.566805  2133 raft_consensus.cc:740] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.578747  2133 consensus_queue.cc:260] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.601939  2456 raft_consensus.cc:493] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.602461  2456 raft_consensus.cc:515] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.609669  2209 consensus_queue.cc:1048] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:28.611409  2378 raft_consensus.cc:493] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.611945  2378 raft_consensus.cc:515] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.649537  2130 tablet_bootstrap.cc:654] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.650499  2445 raft_consensus.cc:2749] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.651618  2378 leader_election.cc:290] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:28.652647  2129 ts_tablet_manager.cc:1434] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.000s
I20260709 12:28:28.626777  1760 catalog_manager.cc:5697] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.666715  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:28.667655  1937 raft_consensus.cc:2393] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 1: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:28.653703  2129 tablet_bootstrap.cc:492] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:28.656493  2133 ts_tablet_manager.cc:1434] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.096s	user 0.012s	sys 0.017s
I20260709 12:28:28.673135  2133 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:28.657518  2504 raft_consensus.cc:493] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.675493  2504 raft_consensus.cc:515] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.677341  2504 leader_election.cc:290] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.659154  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebf212c059342559ae4781db6ba8e83" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.682320  2008 raft_consensus.cc:2468] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
W20260709 12:28:28.661885  2049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 47504.6us
I20260709 12:28:28.683317  2456 leader_election.cc:290] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:28.684424  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2bca7c619a4c36bb0cb4bdf08c8e78" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.685251  2130 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:28.690541  2130 ts_tablet_manager.cc:1403] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.137s	user 0.013s	sys 0.000s
I20260709 12:28:28.687747  2133 tablet_bootstrap.cc:654] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.692224  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32982b9d743f4ceb80f17292b281fd6f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.693171  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32982b9d743f4ceb80f17292b281fd6f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.697911  2129 tablet_bootstrap.cc:654] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.699111  1861 raft_consensus.cc:2468] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:28.700371  1901 leader_election.cc:304] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:28.693424  2130 raft_consensus.cc:359] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.701509  2130 raft_consensus.cc:385] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.701581  2416 raft_consensus.cc:2804] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:28.702262  2416 raft_consensus.cc:493] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.702591  2416 raft_consensus.cc:3060] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.708750  2161 raft_consensus.cc:493] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.709421  2161 raft_consensus.cc:515] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.697304  2131 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:28.710706  2131 ts_tablet_manager.cc:1403] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.160s	user 0.017s	sys 0.007s
I20260709 12:28:28.711839  2161 leader_election.cc:290] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.713805  2131 raft_consensus.cc:359] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.716763  2131 raft_consensus.cc:385] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.719331  2131 raft_consensus.cc:740] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.713806  2081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b385e95426fd4870b36475731ab7d0b8" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:28.701812  2130 raft_consensus.cc:740] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
W20260709 12:28:28.723407  1977 leader_election.cc:343] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.721690  2131 consensus_queue.cc:260] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.718492  2416 raft_consensus.cc:515] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.729741  2416 leader_election.cc:290] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:28.738159  2008 raft_consensus.cc:2468] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:28.746730  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b385e95426fd4870b36475731ab7d0b8" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:28.751660  2081 raft_consensus.cc:3060] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.753489  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebf212c059342559ae4781db6ba8e83" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:28.756982  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebf212c059342559ae4781db6ba8e83" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:28.819515  2315 consensus_queue.cc:1048] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:28.842567  1975 leader_election.cc:343] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.843254  1975 leader_election.cc:304] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:28.848336  2315 raft_consensus.cc:2749] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.866117  2133 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:28.866554  2131 ts_tablet_manager.cc:1434] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.155s	user 0.017s	sys 0.023s
I20260709 12:28:28.866668  2133 ts_tablet_manager.cc:1403] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.194s	user 0.023s	sys 0.021s
I20260709 12:28:28.867592  2131 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:28.869544  2133 raft_consensus.cc:359] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.870236  2133 raft_consensus.cc:385] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.870501  2133 raft_consensus.cc:740] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.865747  2130 consensus_queue.cc:260] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.871997  2446 raft_consensus.cc:493] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.872466  2446 raft_consensus.cc:515] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.871150  2133 consensus_queue.cc:260] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.875033  2129 tablet_bootstrap.cc:492] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:28.875653  2129 ts_tablet_manager.cc:1403] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.222s	user 0.021s	sys 0.025s
I20260709 12:28:28.877713  2446 leader_election.cc:290] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
W20260709 12:28:28.878561  1903 leader_election.cc:343] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.878382  2129 raft_consensus.cc:359] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.879107  2129 raft_consensus.cc:385] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.879446  2129 raft_consensus.cc:740] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.880270  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:28.880121  2129 consensus_queue.cc:260] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.881815  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:28.882434  2133 ts_tablet_manager.cc:1434] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
W20260709 12:28:28.882349  1903 leader_election.cc:343] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:28.883880  2133 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:28.885304  2049 leader_election.cc:304] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
W20260709 12:28:28.887506  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 44999.6us
I20260709 12:28:28.888269  2130 ts_tablet_manager.cc:1434] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.197s	user 0.026s	sys 0.023s
I20260709 12:28:28.889550  2129 ts_tablet_manager.cc:1434] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:28:28.889380  1826 leader_election.cc:304] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:28.890480  2446 raft_consensus.cc:2749] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.896170  2133 tablet_bootstrap.cc:654] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.896721  2130 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:28.896639  1862 raft_consensus.cc:3060] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.899032  2315 consensus_queue.cc:1048] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.899389  2129 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:28.899842  2505 raft_consensus.cc:2804] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:28.900235  2505 raft_consensus.cc:493] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.900566  2505 raft_consensus.cc:3060] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.900673  2049 leader_election.cc:304] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:28.905901  2129 tablet_bootstrap.cc:654] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.906313  2376 consensus_queue.cc:1048] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.907398  2504 raft_consensus.cc:2749] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:28.912094  2378 consensus_queue.cc:1048] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:28.914604  2131 tablet_bootstrap.cc:654] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.917454  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebf212c059342559ae4781db6ba8e83" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:28.918058  2008 raft_consensus.cc:3060] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.919430  2133 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:28.919921  2133 ts_tablet_manager.cc:1403] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.005s
I20260709 12:28:28.925168  2008 raft_consensus.cc:2468] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:28.925868  2133 raft_consensus.cc:359] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.926568  2133 raft_consensus.cc:385] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.926833  2133 raft_consensus.cc:740] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.927482  2133 consensus_queue.cc:260] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.929961  2378 consensus_queue.cc:1048] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.934212  2129 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:28.934775  2129 ts_tablet_manager.cc:1403] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.007s
I20260709 12:28:28.938688  2129 raft_consensus.cc:359] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.941663  2129 raft_consensus.cc:385] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.942077  2129 raft_consensus.cc:740] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.929263  2133 ts_tablet_manager.cc:1434] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:28.943858  2133 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:28.947474  2129 consensus_queue.cc:260] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.947674  2505 raft_consensus.cc:515] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.956189  2133 tablet_bootstrap.cc:654] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.956846  2505 leader_election.cc:290] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:28.959462  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32982b9d743f4ceb80f17292b281fd6f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:28.960306  1861 raft_consensus.cc:3060] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.962947  2129 ts_tablet_manager.cc:1434] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 12:28:28.963948  2129 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:28.964373  2130 tablet_bootstrap.cc:654] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.967146  1861 raft_consensus.cc:2468] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:28.969784  2129 tablet_bootstrap.cc:654] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:28.970284  2454 consensus_queue.cc:1048] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.968611  1901 leader_election.cc:304] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:28.971662  2456 raft_consensus.cc:2804] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.972136  2456 raft_consensus.cc:697] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:28.972992  2456 consensus_queue.cc:237] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:28.974215  2131 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:28.974696  2131 ts_tablet_manager.cc:1403] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.107s	user 0.002s	sys 0.008s
I20260709 12:28:28.975636  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32982b9d743f4ceb80f17292b281fd6f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:28.976246  2010 raft_consensus.cc:3060] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:28.979597  2412 raft_consensus.cc:493] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.980218  2412 raft_consensus.cc:515] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.982616  2412 leader_election.cc:290] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:28.986411  2131 raft_consensus.cc:359] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.987365  2131 raft_consensus.cc:385] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:28.987690  2131 raft_consensus.cc:740] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:28.988560  2131 consensus_queue.cc:260] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:28.992234  2010 raft_consensus.cc:2468] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:28.995059  2049 leader_election.cc:304] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:28.995247  2130 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:28.995836  2130 ts_tablet_manager.cc:1403] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.099s	user 0.009s	sys 0.002s
I20260709 12:28:28.996121  2378 raft_consensus.cc:2804] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:28.996812  2378 raft_consensus.cc:697] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:28.996029  2131 ts_tablet_manager.cc:1434] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:28:28.997660  2378 consensus_queue.cc:237] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:28.999002  2130 raft_consensus.cc:359] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:28.999783  2130 raft_consensus.cc:385] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.000070  2130 raft_consensus.cc:740] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.000702  2130 consensus_queue.cc:260] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.005774  2130 ts_tablet_manager.cc:1434] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:29.001168  2131 tablet_bootstrap.cc:492] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:29.007431  2130 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:29.081277  2459 raft_consensus.cc:493] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.084497  2521 raft_consensus.cc:493] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.084153  2459 raft_consensus.cc:515] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.086167  2521 raft_consensus.cc:515] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.090512  2521 leader_election.cc:290] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:29.091244  2459 leader_election.cc:290] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
W20260709 12:28:29.111438  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:39535, user_credentials={real_user=slave}} blocked reactor thread for 141281us
I20260709 12:28:29.116127  2459 consensus_queue.cc:1048] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:29.118880  1901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 150215us
I20260709 12:28:29.125648  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:29.141636  2129 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:29.142871  2129 ts_tablet_manager.cc:1403] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.179s	user 0.008s	sys 0.016s
I20260709 12:28:29.143749  2378 consensus_queue.cc:1048] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:29.162197  2131 tablet_bootstrap.cc:654] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.164212  2129 raft_consensus.cc:359] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.165167  2129 raft_consensus.cc:385] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.165534  2129 raft_consensus.cc:740] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.176915  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4631b580b30e4f57b7a7a09b2452dc75" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:29.177906  1937 raft_consensus.cc:2468] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:29.178718  1858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:29.179314  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4631b580b30e4f57b7a7a09b2452dc75" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:29.188472  2133 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:29.188638  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:29.189649  1937 raft_consensus.cc:2468] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:29.174813  2129 consensus_queue.cc:260] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.205564  2129 ts_tablet_manager.cc:1434] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 12:28:29.207968  2129 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:29.191104  2130 tablet_bootstrap.cc:654] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.189654  2133 ts_tablet_manager.cc:1403] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.246s	user 0.016s	sys 0.012s
I20260709 12:28:29.217545  2133 raft_consensus.cc:359] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.220747  2133 raft_consensus.cc:385] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.221016  2133 raft_consensus.cc:740] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.218204  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:29.237016  1756 catalog_manager.cc:5697] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:29.243201  2446 raft_consensus.cc:493] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.243873  2446 raft_consensus.cc:515] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.248711  2446 leader_election.cc:290] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:29.250607  1935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:29.253159  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:29.255568  1826 leader_election.cc:304] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:29.231511  2008 raft_consensus.cc:2468] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 0.
I20260709 12:28:29.257277  2446 raft_consensus.cc:2749] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:29.274564  2133 consensus_queue.cc:260] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
W20260709 12:28:29.232097  1901 leader_election.cc:343] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:29.285950  1901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 66222us
I20260709 12:28:29.295529  2376 raft_consensus.cc:493] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.295861  2133 ts_tablet_manager.cc:1434] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 12:28:29.296252  2376 raft_consensus.cc:515] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.297436  2133 tablet_bootstrap.cc:492] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:29.303135  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:29.299129  2376 leader_election.cc:290] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:29.304464  1862 raft_consensus.cc:3060] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:29.306849  2049 leader_election.cc:343] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.313419  1901 leader_election.cc:304] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:29.315609  2416 raft_consensus.cc:2804] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:29.316058  2416 raft_consensus.cc:493] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.316459  2416 raft_consensus.cc:3060] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.325559  1975 leader_election.cc:304] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:29.326512  2315 raft_consensus.cc:2804] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:29.326944  2315 raft_consensus.cc:493] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.328359  2315 raft_consensus.cc:3060] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.327759  2416 raft_consensus.cc:515] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.327136  2449 raft_consensus.cc:493] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.331310  2449 raft_consensus.cc:515] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.333864  2449 leader_election.cc:290] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:29.334674  2416 leader_election.cc:290] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:29.335335  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:29.340644  2376 consensus_queue.cc:1048] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:29.343310  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0af4a569f84ef9841ad0c788c5d6ca" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:29.344252  1936 raft_consensus.cc:2468] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:29.347865  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0af4a569f84ef9841ad0c788c5d6ca" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:29.348675  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:29.339156  1934 raft_consensus.cc:2468] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
W20260709 12:28:29.351315  1901 leader_election.cc:343] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.352100  2049 leader_election.cc:304] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:29.353425  2376 raft_consensus.cc:2804] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:29.353797  2376 raft_consensus.cc:493] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.354107  1975 leader_election.cc:304] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:29.354159  2376 raft_consensus.cc:3060] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.356801  2130 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:29.357245  2129 tablet_bootstrap.cc:654] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.357494  2130 ts_tablet_manager.cc:1403] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.350s	user 0.029s	sys 0.031s
I20260709 12:28:29.355125  2457 raft_consensus.cc:2804] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:29.358186  2457 raft_consensus.cc:493] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.358579  2457 raft_consensus.cc:3060] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.371344  2049 leader_election.cc:304] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
W20260709 12:28:29.375516  2049 leader_election.cc:343] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.377249  2449 raft_consensus.cc:2804] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:29.380039  2449 raft_consensus.cc:493] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.380441  2449 raft_consensus.cc:3060] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.398478  2457 raft_consensus.cc:515] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.417425  2131 tablet_bootstrap.cc:492] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:29.419662  2131 ts_tablet_manager.cc:1403] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.419s	user 0.013s	sys 0.007s
I20260709 12:28:29.413736  2130 raft_consensus.cc:359] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.420660  2130 raft_consensus.cc:385] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.420939  2130 raft_consensus.cc:740] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.421762  2130 consensus_queue.cc:260] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.403168  2457 leader_election.cc:290] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
W20260709 12:28:29.425364  1903 consensus_peers.cc:597] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Couldn't send request to peer 5edb06eaed8e41868299eb62ade6eef0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:29.426324  2129 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:29.426725  2131 raft_consensus.cc:359] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.427680  2131 raft_consensus.cc:385] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.427986  2131 raft_consensus.cc:740] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.426903  2129 ts_tablet_manager.cc:1403] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.219s	user 0.012s	sys 0.017s
W20260709 12:28:29.430153  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:39535, user_credentials={real_user=slave}} blocked reactor thread for 74045.3us
I20260709 12:28:29.422514  2315 raft_consensus.cc:515] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.437716  2131 consensus_queue.cc:260] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.434442  2129 raft_consensus.cc:359] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.438946  2129 raft_consensus.cc:385] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.439369  2129 raft_consensus.cc:740] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.442757  2355 raft_consensus.cc:493] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:29.443462  1975 leader_election.cc:343] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:29.444803  1975 leader_election.cc:343] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.446969  2129 consensus_queue.cc:260] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.443326  2355 raft_consensus.cc:515] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.449748  2133 tablet_bootstrap.cc:654] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.451368  2355 leader_election.cc:290] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:29.455047  2378 consensus_queue.cc:1048] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:29.455586  2130 ts_tablet_manager.cc:1434] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.002s
I20260709 12:28:29.455175  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca90853e7a245c090a24e9f2b6b88e1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:29.457044  2315 leader_election.cc:290] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
W20260709 12:28:29.457907  1901 leader_election.cc:343] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.455137  2449 raft_consensus.cc:515] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.460716  2449 leader_election.cc:290] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:29.462829  1862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:29.463928  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
W20260709 12:28:29.464486  2049 leader_election.cc:343] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.464556  1936 raft_consensus.cc:3060] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.457820  2130 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:29.468007  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca90853e7a245c090a24e9f2b6b88e1" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:29.469334  1903 leader_election.cc:343] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.469820  1903 leader_election.cc:304] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:29.471088  2416 raft_consensus.cc:2749] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:29.467947  2131 ts_tablet_manager.cc:1434] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.001s
W20260709 12:28:29.484934  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:29.485940  2533 consensus_queue.cc:1048] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:28:29.496045  1936 raft_consensus.cc:2468] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:29.497462  2049 leader_election.cc:304] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:29.498175  2376 raft_consensus.cc:515] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.500983  2449 raft_consensus.cc:2804] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:29.502947  2449 raft_consensus.cc:697] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:29.504062  2449 consensus_queue.cc:237] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.507445  2129 ts_tablet_manager.cc:1434] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.003s
I20260709 12:28:29.508641  2129 tablet_bootstrap.cc:492] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:29.508566  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:29.510660  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4631b580b30e4f57b7a7a09b2452dc75" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:29.512369  2315 raft_consensus.cc:493] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.512411  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:29.513275  2007 raft_consensus.cc:3060] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.513150  2315 raft_consensus.cc:515] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.517158  2315 leader_election.cc:290] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:29.524658  2133 tablet_bootstrap.cc:492] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:29.526597  2131 tablet_bootstrap.cc:492] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee: Bootstrap starting.
I20260709 12:28:29.525243  2133 ts_tablet_manager.cc:1403] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.228s	user 0.022s	sys 0.017s
I20260709 12:28:29.527321  2081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b15f3e29c654279b3811d5ed11c31a2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
W20260709 12:28:29.528797  1977 leader_election.cc:343] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.529209  2007 raft_consensus.cc:2468] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
W20260709 12:28:29.532322  1975 leader_election.cc:343] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:29.536125  1975 leader_election.cc:343] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.538015  1901 leader_election.cc:304] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:29.539166  2456 raft_consensus.cc:2804] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:29.539451  2376 leader_election.cc:290] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:29.539744  2456 raft_consensus.cc:697] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:29.540778  2456 consensus_queue.cc:237] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.543262  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0af4a569f84ef9841ad0c788c5d6ca" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:29.545712  2131 tablet_bootstrap.cc:654] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.547683  1859 raft_consensus.cc:3060] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.548296  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0af4a569f84ef9841ad0c788c5d6ca" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:29.548887  1936 raft_consensus.cc:3060] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.555888  2133 raft_consensus.cc:359] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.558220  2133 raft_consensus.cc:385] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.558559  2133 raft_consensus.cc:740] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.565430  2133 consensus_queue.cc:260] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.569344  1936 raft_consensus.cc:2468] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:29.569988  2505 raft_consensus.cc:493] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.570588  2505 raft_consensus.cc:515] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.564307  2416 consensus_queue.cc:1048] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:28:29.572630  2133 ts_tablet_manager.cc:1434] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 12:28:29.572726  2505 leader_election.cc:290] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:29.573854  2133 tablet_bootstrap.cc:492] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:29.575766  2049 leader_election.cc:304] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:29.577999  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:29.578877  2011 raft_consensus.cc:2468] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:29.577936  2505 raft_consensus.cc:2804] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:29.579739  2505 raft_consensus.cc:697] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:29.580322  2049 leader_election.cc:304] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:29.581887  2378 raft_consensus.cc:2804] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:29.582381  2378 raft_consensus.cc:493] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.582870  2378 raft_consensus.cc:3060] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.583530  1859 raft_consensus.cc:2468] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:29.580678  2505 consensus_queue.cc:237] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.594545  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:29.598773  2049 leader_election.cc:343] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.606019  2130 tablet_bootstrap.cc:654] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.615120  2449 consensus_queue.cc:1048] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:29.618811  2133 tablet_bootstrap.cc:654] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.658896  2129 tablet_bootstrap.cc:654] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.688405  2131 tablet_bootstrap.cc:492] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee: No bootstrap required, opened a new log
I20260709 12:28:29.688998  2131 ts_tablet_manager.cc:1403] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee: Time spent bootstrapping tablet: real 0.163s	user 0.013s	sys 0.011s
I20260709 12:28:29.691958  2131 raft_consensus.cc:359] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.692649  2131 raft_consensus.cc:385] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.692907  2131 raft_consensus.cc:740] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.693581  2131 consensus_queue.cc:260] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.696659  2131 ts_tablet_manager.cc:1434] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:28:29.708302  1937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:29.708966  1937 raft_consensus.cc:3060] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.710736  1930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4631b580b30e4f57b7a7a09b2452dc75" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:29.711536  1930 raft_consensus.cc:3060] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.722672  1931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b15f3e29c654279b3811d5ed11c31a2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:29.723477  1931 raft_consensus.cc:2468] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:29.723860  1937 raft_consensus.cc:2468] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:29.658800  2378 raft_consensus.cc:515] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.744234  2133 tablet_bootstrap.cc:492] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:29.744791  2133 ts_tablet_manager.cc:1403] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.171s	user 0.021s	sys 0.017s
I20260709 12:28:29.752590  2315 raft_consensus.cc:493] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.753144  2315 raft_consensus.cc:515] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.755818  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:29.756713  1975 leader_election.cc:304] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:29.752362  2133 raft_consensus.cc:359] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
W20260709 12:28:29.757690  1977 leader_election.cc:343] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.758328  2133 raft_consensus.cc:385] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.758807  2133 raft_consensus.cc:740] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.764680  1975 leader_election.cc:304] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:29.759636  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:29.755911  2315 leader_election.cc:290] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:29.760712  2457 raft_consensus.cc:2804] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:29.759796  1758 catalog_manager.cc:5697] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:29.764953  2133 consensus_queue.cc:260] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.779743  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:29.780618  2011 raft_consensus.cc:3060] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.782461  2521 raft_consensus.cc:2804] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:29.784703  2521 raft_consensus.cc:697] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:29.785661  2521 consensus_queue.cc:237] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.787443  2133 ts_tablet_manager.cc:1434] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 12:28:29.787644  2378 leader_election.cc:290] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:29.787869  2129 tablet_bootstrap.cc:492] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:29.788369  2129 ts_tablet_manager.cc:1403] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.280s	user 0.012s	sys 0.028s
I20260709 12:28:29.778611  2457 raft_consensus.cc:493] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.789386  2133 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:29.794669  2457 raft_consensus.cc:3060] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.797019  2011 raft_consensus.cc:2468] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:29.797650  2133 tablet_bootstrap.cc:654] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.768364  1930 raft_consensus.cc:2468] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:29.799829  2129 raft_consensus.cc:359] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.800215  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:29.800567  2129 raft_consensus.cc:385] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.800860  2129 raft_consensus.cc:740] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.801537  2129 consensus_queue.cc:260] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.803799  2049 leader_election.cc:304] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:29.807421  2505 raft_consensus.cc:2804] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:29.808005  2505 raft_consensus.cc:697] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:29.808969  2505 consensus_queue.cc:237] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
W20260709 12:28:29.806828  2049 leader_election.cc:343] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.815248  2130 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:29.815837  2130 ts_tablet_manager.cc:1403] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.358s	user 0.018s	sys 0.045s
I20260709 12:28:29.818104  2130 raft_consensus.cc:359] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.818624  2130 raft_consensus.cc:385] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.818902  2130 raft_consensus.cc:740] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.819628  2130 consensus_queue.cc:260] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:29.821818  2130 ts_tablet_manager.cc:1434] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:28:29.822883  2130 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:29.812604  1756 catalog_manager.cc:5697] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:29.830653  1756 catalog_manager.cc:5697] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:29.832404  1975 leader_election.cc:343] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.832904  1975 leader_election.cc:304] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:29.833612  2459 raft_consensus.cc:2749] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:29.837186  1975 leader_election.cc:304] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:29.841765  2130 tablet_bootstrap.cc:654] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.842095  2315 raft_consensus.cc:2804] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:29.842985  2315 raft_consensus.cc:697] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:29.843029  2129 ts_tablet_manager.cc:1434] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.001s
I20260709 12:28:29.844038  2315 consensus_queue.cc:237] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:29.844684  2129 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630: Bootstrap starting.
I20260709 12:28:29.862151  2133 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:29.862700  2133 ts_tablet_manager.cc:1403] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.074s	user 0.021s	sys 0.015s
I20260709 12:28:29.878986  2133 raft_consensus.cc:359] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.879920  2133 raft_consensus.cc:385] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.880198  2133 raft_consensus.cc:740] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.880955  2133 consensus_queue.cc:260] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.883237  2133 ts_tablet_manager.cc:1434] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:28:29.884397  2133 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:29.889281  2457 raft_consensus.cc:515] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.892820  2133 tablet_bootstrap.cc:654] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.893340  2505 raft_consensus.cc:493] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.895627  2130 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:29.895857  2129 tablet_bootstrap.cc:654] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:29.902623  2130 ts_tablet_manager.cc:1403] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.080s	user 0.008s	sys 0.018s
I20260709 12:28:29.905411  2130 raft_consensus.cc:359] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.906077  2130 raft_consensus.cc:385] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.906340  2130 raft_consensus.cc:740] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.907477  2130 consensus_queue.cc:260] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.907059  2505 raft_consensus.cc:515] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:29.914698  2505 leader_election.cc:290] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:29.918565  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebf212c059342559ae4781db6ba8e83" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:29.918720  2130 ts_tablet_manager.cc:1434] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:28:29.916169  1933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebf212c059342559ae4781db6ba8e83" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:29.925405  2129 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630: No bootstrap required, opened a new log
I20260709 12:28:29.925961  2129 ts_tablet_manager.cc:1403] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.004s
I20260709 12:28:29.929358  2130 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:29.930527  1758 catalog_manager.cc:5697] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 1 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:29.934209  2049 leader_election.cc:304] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:29.936986  2133 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:29.937428  2133 ts_tablet_manager.cc:1403] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.000s
I20260709 12:28:29.938001  2378 raft_consensus.cc:2749] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:29.942292  2129 raft_consensus.cc:359] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.946105  2133 raft_consensus.cc:359] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.946882  2133 raft_consensus.cc:385] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.945210  2129 raft_consensus.cc:385] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:29.951116  2129 raft_consensus.cc:740] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.947151  2133 raft_consensus.cc:740] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:29.959690  1936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b15f3e29c654279b3811d5ed11c31a2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:29.960491  1936 raft_consensus.cc:3060] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:29.963435  2133 consensus_queue.cc:260] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.957319  2129 consensus_queue.cc:260] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:29.955821  2505 consensus_queue.cc:1048] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:28:29.975271  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b15f3e29c654279b3811d5ed11c31a2" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:29.976332  2085 raft_consensus.cc:3060] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:29.978484  1977 leader_election.cc:343] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:29.989115  2133 ts_tablet_manager.cc:1434] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.002s
I20260709 12:28:29.968286  2457 leader_election.cc:290] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:29.991761  2129 ts_tablet_manager.cc:1434] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.001s
I20260709 12:28:30.001274  2130 tablet_bootstrap.cc:654] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.004804  2133 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:30.009048  2459 raft_consensus.cc:493] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.009377  1936 raft_consensus.cc:2468] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:30.009625  2459 raft_consensus.cc:515] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:30.010545  1975 leader_election.cc:304] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:30.011816  2459 leader_election.cc:290] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:30.012322  2315 raft_consensus.cc:2804] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:30.012910  2315 raft_consensus.cc:697] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:30.013914  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "736e3203de274189acfaef187bfea762" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:30.014650  2082 raft_consensus.cc:2468] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:30.015957  1977 leader_election.cc:304] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:30.016175  1930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "736e3203de274189acfaef187bfea762" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:30.016985  1930 raft_consensus.cc:2468] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:30.017213  2133 tablet_bootstrap.cc:654] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.021746  1756 catalog_manager.cc:5697] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:30.013957  2315 consensus_queue.cc:237] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:30.017491  2459 raft_consensus.cc:2804] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:30.028204  2459 raft_consensus.cc:493] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.028650  2459 raft_consensus.cc:3060] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.041424  2459 raft_consensus.cc:515] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:30.043833  1860 raft_consensus.cc:3060] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.044592  2130 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:30.045110  2130 ts_tablet_manager.cc:1403] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.122s	user 0.012s	sys 0.002s
I20260709 12:28:30.047966  2130 raft_consensus.cc:359] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:30.048622  2130 raft_consensus.cc:385] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.048882  2130 raft_consensus.cc:740] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.051470  1933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "736e3203de274189acfaef187bfea762" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:30.052270  2378 consensus_queue.cc:1048] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:30.052737  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "736e3203de274189acfaef187bfea762" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:30.053318  2082 raft_consensus.cc:3060] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.053694  2378 raft_consensus.cc:493] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.054211  2378 raft_consensus.cc:515] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.052292  1933 raft_consensus.cc:3060] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.062563  2082 raft_consensus.cc:2468] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:30.063964  1977 leader_election.cc:304] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:30.064890  2209 raft_consensus.cc:2804] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:30.065356  2209 raft_consensus.cc:697] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:30.066175  2209 consensus_queue.cc:237] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:30.067272  2133 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:30.068045  2133 ts_tablet_manager.cc:1403] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.064s	user 0.023s	sys 0.012s
I20260709 12:28:30.068099  1760 catalog_manager.cc:5697] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:30.071635  2133 raft_consensus.cc:359] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.072296  2378 leader_election.cc:290] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:30.072598  2133 raft_consensus.cc:385] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.073112  2133 raft_consensus.cc:740] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.075388  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:30.076063  2011 raft_consensus.cc:2468] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
W20260709 12:28:30.076315  1901 consensus_peers.cc:597] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:30.078361  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:30.074003  2133 consensus_queue.cc:260] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.073805  2459 leader_election.cc:290] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:30.085327  2130 consensus_queue.cc:260] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:30.093138  2456 consensus_queue.cc:1048] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:30.088542  2133 ts_tablet_manager.cc:1434] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:28:30.094998  2133 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:30.088510  2049 leader_election.cc:304] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:30.097823  2449 raft_consensus.cc:2804] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:30.098269  2449 raft_consensus.cc:493] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.098629  2449 raft_consensus.cc:3060] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.102895  2130 ts_tablet_manager.cc:1434] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 12:28:30.103986  2130 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:30.107582  2049 leader_election.cc:343] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:30.111886  2133 tablet_bootstrap.cc:654] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.070318  1760 catalog_manager.cc:5697] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:30.119050  2130 tablet_bootstrap.cc:654] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.147526  2378 raft_consensus.cc:493] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.148276  2378 raft_consensus.cc:515] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.158562  2378 leader_election.cc:290] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:30.163892  1933 raft_consensus.cc:2468] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:30.167027  2449 raft_consensus.cc:515] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.176208  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b385e95426fd4870b36475731ab7d0b8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
W20260709 12:28:30.176685  1975 consensus_peers.cc:597] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:30.177165  2011 raft_consensus.cc:2468] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:30.180586  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b385e95426fd4870b36475731ab7d0b8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:30.185495  2544 consensus_queue.cc:1048] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:30.170287  2449 leader_election.cc:290] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:30.187342  2449 consensus_queue.cc:1048] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:30.196849  2130 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:30.197376  2130 ts_tablet_manager.cc:1403] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.094s	user 0.022s	sys 0.022s
I20260709 12:28:30.210739  2130 raft_consensus.cc:359] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:30.218067  2130 raft_consensus.cc:385] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.218402  2130 raft_consensus.cc:740] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.221105  2130 consensus_queue.cc:260] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:30.217916  2560 raft_consensus.cc:493] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:30.229550  2049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 45324.5us
W20260709 12:28:30.230456  2049 leader_election.cc:343] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:30.235503  2560 raft_consensus.cc:515] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.242254  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:30.242892  2011 raft_consensus.cc:3060] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.248451  2130 ts_tablet_manager.cc:1434] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:28:30.252564  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:30.258167  2133 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:30.258751  2133 ts_tablet_manager.cc:1403] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.164s	user 0.032s	sys 0.039s
I20260709 12:28:30.261870  2133 raft_consensus.cc:359] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.262593  2133 raft_consensus.cc:385] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.260869  2445 raft_consensus.cc:493] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.263464  2445 raft_consensus.cc:515] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:30.265525  2445 leader_election.cc:290] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:30.262851  2133 raft_consensus.cc:740] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.266181  2560 leader_election.cc:290] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:30.266812  2133 consensus_queue.cc:260] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.267278  2130 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:30.268533  1860 raft_consensus.cc:3060] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.274279  1933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:30.275166  1933 raft_consensus.cc:2393] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:30.275538  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9e0977a3ed1476f871be3d3a77ae4b2" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:30.275578  2010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:30.277246  2010 raft_consensus.cc:2393] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:30.279291  1826 leader_election.cc:304] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:30.279541  2011 raft_consensus.cc:2468] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:30.280489  2446 raft_consensus.cc:3060] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.292667  2130 tablet_bootstrap.cc:654] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.294260  2521 raft_consensus.cc:493] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.294909  2521 raft_consensus.cc:515] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.299823  2521 leader_election.cc:290] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:30.278689  1756 catalog_manager.cc:5697] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:30.307490  1756 catalog_manager.cc:5697] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:30.308398  2133 ts_tablet_manager.cc:1434] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.000s
I20260709 12:28:30.305636  2560 raft_consensus.cc:493] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.311895  2560 raft_consensus.cc:515] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.314535  2560 leader_election.cc:290] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:30.319264  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf128bd1b4b546dab8e2c0504d06cb87" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:30.319588  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf128bd1b4b546dab8e2c0504d06cb87" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:30.320207  2082 raft_consensus.cc:2468] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 0.
I20260709 12:28:30.322091  1977 leader_election.cc:304] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
W20260709 12:28:30.323899  1975 leader_election.cc:343] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:30.306082  2049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:39583, user_credentials={real_user=slave}} blocked reactor thread for 55477.8us
I20260709 12:28:30.328132  2049 leader_election.cc:304] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:30.324100  2315 raft_consensus.cc:2804] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:30.331171  2376 raft_consensus.cc:2804] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:30.331306  2315 raft_consensus.cc:493] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.331737  2376 raft_consensus.cc:493] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.331879  2315 raft_consensus.cc:3060] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.332163  2376 raft_consensus.cc:3060] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.302896  2209 consensus_queue.cc:1048] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:28:30.337476  1933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:30.339896  2446 raft_consensus.cc:2749] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:30.330019  2049 leader_election.cc:304] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:30.342563  2457 consensus_queue.cc:1048] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:30.344049  2571 raft_consensus.cc:2804] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:30.344619  2133 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:30.344580  2571 raft_consensus.cc:697] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:30.345535  2571 consensus_queue.cc:237] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
W20260709 12:28:30.353286  2049 leader_election.cc:343] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:30.374625  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91363564e93f4297b4e825de5b7e5176" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:30.368259  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146e39d5a724f05b9b3cbc7a8145d98" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:30.388375  1828 leader_election.cc:304] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:30.389240  2446 raft_consensus.cc:3060] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.389694  1828 leader_election.cc:304] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0
I20260709 12:28:30.398793  2378 raft_consensus.cc:493] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.404295  2130 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:30.404819  2130 ts_tablet_manager.cc:1403] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.138s	user 0.026s	sys 0.014s
I20260709 12:28:30.401106  2376 raft_consensus.cc:515] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.405099  2378 raft_consensus.cc:515] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.438999  2376 leader_election.cc:290] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:30.441010  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b385e95426fd4870b36475731ab7d0b8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:30.455864  2133 tablet_bootstrap.cc:654] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.456106  2011 raft_consensus.cc:3060] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.441537  2130 raft_consensus.cc:359] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.460098  2130 raft_consensus.cc:385] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.460459  2130 raft_consensus.cc:740] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.453258  2459 consensus_queue.cc:1048] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:28:30.445729  1932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:30.463044  1932 raft_consensus.cc:2468] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:30.445871  1861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:30.464973  2049 leader_election.cc:304] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:30.440135  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b385e95426fd4870b36475731ab7d0b8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:30.461874  2130 consensus_queue.cc:260] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.467221  2505 raft_consensus.cc:2804] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:30.476136  2505 raft_consensus.cc:493] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.476481  2505 raft_consensus.cc:3060] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.479140  1860 raft_consensus.cc:3060] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.444208  2378 leader_election.cc:290] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:30.478622  2130 ts_tablet_manager.cc:1434] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.001s
I20260709 12:28:30.444309  1757 catalog_manager.cc:5697] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:30.498703  2049 leader_election.cc:343] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:30.464128  2446 raft_consensus.cc:2749] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:30.511577  1860 raft_consensus.cc:2468] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:30.445669  2315 raft_consensus.cc:515] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.521828  2445 raft_consensus.cc:2749] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:30.555369  2544 consensus_queue.cc:1048] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:28:30.563570  2049 leader_election.cc:304] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:30.574136  2571 consensus_queue.cc:1048] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:30.575120  2376 raft_consensus.cc:2804] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:30.575798  2376 raft_consensus.cc:697] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:30.576787  2376 consensus_queue.cc:237] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.577803  2133 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:30.578368  2133 ts_tablet_manager.cc:1403] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.234s	user 0.009s	sys 0.011s
I20260709 12:28:30.582062  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf128bd1b4b546dab8e2c0504d06cb87" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:30.582793  2082 raft_consensus.cc:3060] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.588495  2457 consensus_queue.cc:1048] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:30.595317  2130 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:30.615499  2315 leader_election.cc:290] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:30.619158  2133 raft_consensus.cc:359] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.620039  2133 raft_consensus.cc:385] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.621567  2082 raft_consensus.cc:2468] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:30.623152  1977 leader_election.cc:304] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:30.620338  2133 raft_consensus.cc:740] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.624241  2587 raft_consensus.cc:2804] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:30.639698  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 90290.3us
I20260709 12:28:30.624740  2011 raft_consensus.cc:2468] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:30.646922  2133 consensus_queue.cc:260] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.651449  2133 ts_tablet_manager.cc:1434] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.073s	user 0.011s	sys 0.000s
I20260709 12:28:30.641909  2587 raft_consensus.cc:697] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:30.658046  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf128bd1b4b546dab8e2c0504d06cb87" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:30.669241  2587 consensus_queue.cc:237] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.670070  2133 tablet_bootstrap.cc:492] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0: Bootstrap starting.
I20260709 12:28:30.673089  2130 tablet_bootstrap.cc:654] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.695425  2505 raft_consensus.cc:515] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.697589  2505 leader_election.cc:290] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:30.707228  2133 tablet_bootstrap.cc:654] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:30.724694  1808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:30.729027  1933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:30.729775  1933 raft_consensus.cc:3060] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.730412  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
W20260709 12:28:30.738477  2049 leader_election.cc:343] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:30.742926  1977 consensus_peers.cc:597] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785): Couldn't send request to peer 5edb06eaed8e41868299eb62ade6eef0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:30.747570  1933 raft_consensus.cc:2468] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:30.755738  2049 leader_election.cc:304] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
W20260709 12:28:30.760753  1975 leader_election.cc:343] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:30.774446  2378 raft_consensus.cc:2804] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:30.786584  2378 raft_consensus.cc:697] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:30.786732  1758 catalog_manager.cc:5697] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:30.787570  2378 consensus_queue.cc:237] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.791956  2130 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:30.792502  2130 ts_tablet_manager.cc:1403] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.198s	user 0.015s	sys 0.000s
I20260709 12:28:30.799222  2130 raft_consensus.cc:359] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.800503  2130 raft_consensus.cc:385] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.800851  2130 raft_consensus.cc:740] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.802227  2130 consensus_queue.cc:260] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.804414  2130 ts_tablet_manager.cc:1434] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:28:30.807432  2130 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:30.816732  2544 raft_consensus.cc:493] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.817277  2544 raft_consensus.cc:515] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.819368  2544 leader_election.cc:290] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:30.823052  2133 tablet_bootstrap.cc:492] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0: No bootstrap required, opened a new log
I20260709 12:28:30.823915  2133 ts_tablet_manager.cc:1403] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0: Time spent bootstrapping tablet: real 0.154s	user 0.020s	sys 0.011s
I20260709 12:28:30.831532  2133 raft_consensus.cc:359] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:30.833976  2133 raft_consensus.cc:385] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.834661  2133 raft_consensus.cc:740] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.835503  2133 consensus_queue.cc:260] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:30.837182  2130 tablet_bootstrap.cc:654] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.838878  2505 consensus_queue.cc:1048] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:30.842852  1928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca90853e7a245c090a24e9f2b6b88e1" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:30.843364  1859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca90853e7a245c090a24e9f2b6b88e1" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:30.843602  1928 raft_consensus.cc:2468] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:30.844015  1859 raft_consensus.cc:2468] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 0.
I20260709 12:28:30.846622  2457 consensus_queue.cc:1048] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:30.850541  2133 ts_tablet_manager.cc:1434] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:28:30.856932  2592 consensus_queue.cc:1048] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:30.868108  2049 leader_election.cc:304] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:30.873735  2376 raft_consensus.cc:2804] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:30.874130  2376 raft_consensus.cc:493] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.874468  2376 raft_consensus.cc:3060] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.922544  2130 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:30.923139  2130 ts_tablet_manager.cc:1403] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.116s	user 0.016s	sys 0.000s
I20260709 12:28:30.923247  2376 raft_consensus.cc:515] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.925382  2376 leader_election.cc:290] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:30.925930  2130 raft_consensus.cc:359] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:30.926635  2130 raft_consensus.cc:385] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.926883  2130 raft_consensus.cc:740] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.928126  2130 consensus_queue.cc:260] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:30.943107  1860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca90853e7a245c090a24e9f2b6b88e1" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:30.943825  1860 raft_consensus.cc:3060] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.948747  2130 ts_tablet_manager.cc:1434] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 12:28:30.950085  2130 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:30.951915  1931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ca90853e7a245c090a24e9f2b6b88e1" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:30.952534  1931 raft_consensus.cc:3060] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:30.960383  1931 raft_consensus.cc:2468] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:30.964793  2049 leader_election.cc:304] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:30.965544  2571 raft_consensus.cc:2804] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:30.965689  2049 consensus_peers.cc:597] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:30.966095  2571 raft_consensus.cc:697] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:30.966928  2571 consensus_queue.cc:237] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:30.968112  1860 raft_consensus.cc:2468] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:30.966807  1758 catalog_manager.cc:5697] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:30.969975  2130 tablet_bootstrap.cc:654] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:30.982762  2130 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:30.983290  2130 ts_tablet_manager.cc:1403] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.000s
I20260709 12:28:30.985927  2130 raft_consensus.cc:359] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.986580  2130 raft_consensus.cc:385] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:30.986807  2130 raft_consensus.cc:740] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:30.987541  2130 consensus_queue.cc:260] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:30.989181  2130 ts_tablet_manager.cc:1434] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:30.996297  2130 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:31.001223  2130 tablet_bootstrap.cc:654] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.001603  2571 raft_consensus.cc:493] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.002228  2571 raft_consensus.cc:515] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:31.004360  2571 leader_election.cc:290] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:31.003947  1760 catalog_manager.cc:5697] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:31.005633  1928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b15f3e29c654279b3811d5ed11c31a2" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:31.007676  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b15f3e29c654279b3811d5ed11c31a2" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:31.026386  2049 leader_election.cc:304] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:31.028357  2449 raft_consensus.cc:2749] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:31.029036  2449 consensus_queue.cc:1048] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:31.039155  2130 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:31.039796  2130 ts_tablet_manager.cc:1403] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.020s
I20260709 12:28:31.042618  2130 raft_consensus.cc:359] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:31.043287  2130 raft_consensus.cc:385] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.043555  2130 raft_consensus.cc:740] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.044216  2130 consensus_queue.cc:260] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:31.045909  2130 ts_tablet_manager.cc:1434] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:31.047317  2130 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:31.053112  2130 tablet_bootstrap.cc:654] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.062947  2130 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: No bootstrap required, opened a new log
I20260709 12:28:31.063408  2130 ts_tablet_manager.cc:1403] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:28:31.066223  2130 raft_consensus.cc:359] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:31.066895  2130 raft_consensus.cc:385] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.067217  2130 raft_consensus.cc:740] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.067883  2130 consensus_queue.cc:260] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:31.072760  2130 ts_tablet_manager.cc:1434] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:31.076808  1757 catalog_manager.cc:5697] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 1 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:31.077446  2376 consensus_queue.cc:1048] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:31.106757  2583 raft_consensus.cc:493] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.107299  2583 raft_consensus.cc:515] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:31.109191  2583 leader_election.cc:290] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:31.109676  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf128bd1b4b546dab8e2c0504d06cb87" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:31.110144  2082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf128bd1b4b546dab8e2c0504d06cb87" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:31.110954  2082 raft_consensus.cc:2393] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c59f6f4b90d460e832bcff010c18400 in current term 1: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:31.112092  1828 leader_election.cc:304] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:31.112682  2583 raft_consensus.cc:3060] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.119530  2583 raft_consensus.cc:2749] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:31.210589  1860 raft_consensus.cc:3060] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.217232  2376 consensus_queue.cc:1048] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:31.230165  2376 consensus_queue.cc:1048] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:31.262128  2589 consensus_queue.cc:1048] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:31.263038 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
W20260709 12:28:31.278070  2049 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
I20260709 12:28:31.299515  2085 raft_consensus.cc:3060] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.315201  2560 raft_consensus.cc:493] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.316013  2560 raft_consensus.cc:515] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
W20260709 12:28:31.321346  2049 consensus_peers.cc:597] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.321714  2560 leader_election.cc:290] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:31.327090  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:31.332360  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:31.343488  1828 leader_election.cc:304] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:31.345391  2560 raft_consensus.cc:2749] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:31.348246  1901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 51826.7us
W20260709 12:28:31.352686  1901 consensus_peers.cc:597] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.354378  1975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:35551, user_credentials={real_user=slave}} blocked reactor thread for 50199.3us
W20260709 12:28:31.358299  1975 consensus_peers.cc:597] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.362568  1975 consensus_peers.cc:597] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.363227  1975 consensus_peers.cc:597] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.363823  1975 consensus_peers.cc:597] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.375626  2620 raft_consensus.cc:493] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.376454  2620 raft_consensus.cc:515] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:31.379243  2620 leader_election.cc:290] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:31.384251  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:31.385670  1928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:31.388922  1826 leader_election.cc:304] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
W20260709 12:28:31.390354  1975 consensus_peers.cc:597] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.391386  2583 raft_consensus.cc:2749] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:31.437454  2049 consensus_peers.cc:597] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.440295  1975 consensus_peers.cc:597] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.449122  1901 consensus_peers.cc:597] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.449756  2049 consensus_peers.cc:597] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.458801  2049 consensus_peers.cc:597] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.471230 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:31.473219  1975 consensus_peers.cc:597] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.474099 19603 tablet_replica.cc:333] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.474905 19603 raft_consensus.cc:2243] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.475463 19603 raft_consensus.cc:2272] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.475875  1901 consensus_peers.cc:597] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.478077 19603 tablet_replica.cc:333] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.478788 19603 raft_consensus.cc:2243] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.479163 19603 raft_consensus.cc:2272] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.481087 19603 tablet_replica.cc:333] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.481894 19603 raft_consensus.cc:2243] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.482362 19603 raft_consensus.cc:2272] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.485548 19603 tablet_replica.cc:333] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.486501 19603 raft_consensus.cc:2243] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.487076 19603 raft_consensus.cc:2272] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.489950 19603 tablet_replica.cc:333] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.492929 19603 raft_consensus.cc:2243] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.494006 19603 raft_consensus.cc:2272] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.495332  2376 consensus_queue.cc:1048] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:31.497540 19603 tablet_replica.cc:333] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.498497 19603 raft_consensus.cc:2243] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.498965 19603 raft_consensus.cc:2272] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.501446 19603 tablet_replica.cc:333] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.502302 19603 raft_consensus.cc:2243] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.502668 19603 raft_consensus.cc:2272] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.505048 19603 tablet_replica.cc:333] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
W20260709 12:28:31.505662  1901 consensus_peers.cc:597] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.505987 19603 raft_consensus.cc:2243] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.506608 19603 raft_consensus.cc:2272] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.514195 19603 tablet_replica.cc:333] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.515087 19603 raft_consensus.cc:2243] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.515707 19603 raft_consensus.cc:2272] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.518730 19603 tablet_replica.cc:333] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.519667 19603 raft_consensus.cc:2243] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.520251 19603 raft_consensus.cc:2272] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.520860  1975 consensus_peers.cc:597] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.521631  1975 consensus_peers.cc:597] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.523519 19603 tablet_replica.cc:333] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.524432 19603 raft_consensus.cc:2243] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.529357 19603 raft_consensus.cc:2272] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.532047 19603 tablet_replica.cc:333] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.532837 19603 raft_consensus.cc:2243] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.533317 19603 raft_consensus.cc:2272] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.535955 19603 tablet_replica.cc:333] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.536769 19603 raft_consensus.cc:2243] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.537295 19603 raft_consensus.cc:2272] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.539650 19603 tablet_replica.cc:333] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.540346 19603 raft_consensus.cc:2243] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:31.541404 19603 raft_consensus.cc:2272] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.544286 19603 tablet_replica.cc:333] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.545178 19603 raft_consensus.cc:2243] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.545691 19603 raft_consensus.cc:2272] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.548657 19603 tablet_replica.cc:333] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.549635 19603 raft_consensus.cc:2243] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.550122 19603 raft_consensus.cc:2272] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.552802 19603 tablet_replica.cc:333] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.553345 19603 raft_consensus.cc:2243] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.553860 19603 raft_consensus.cc:2272] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.554052  2049 consensus_peers.cc:597] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.556362 19603 tablet_replica.cc:333] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.557318 19603 raft_consensus.cc:2243] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.557864 19603 raft_consensus.cc:2272] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.559535  1975 consensus_peers.cc:597] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.560707 19603 tablet_replica.cc:333] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.561528 19603 raft_consensus.cc:2243] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.562068 19603 raft_consensus.cc:2272] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.564888  1975 consensus_peers.cc:597] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.565099 19603 tablet_replica.cc:333] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.565950 19603 raft_consensus.cc:2243] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.566474 19603 raft_consensus.cc:2272] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.571552 19603 tablet_replica.cc:333] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.572474 19603 raft_consensus.cc:2243] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:31.572808  1901 consensus_peers.cc:597] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.572956 19603 raft_consensus.cc:2272] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.576013 19603 tablet_replica.cc:333] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
W20260709 12:28:31.576408  1975 consensus_peers.cc:597] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.576570 19603 raft_consensus.cc:2243] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.577131 19603 raft_consensus.cc:2272] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.578611  1975 consensus_peers.cc:597] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.579572 19603 tablet_replica.cc:333] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.580183 19603 raft_consensus.cc:2243] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.580605 19603 raft_consensus.cc:2272] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.582412 19603 tablet_replica.cc:333] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.583068 19603 raft_consensus.cc:2243] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:31.583889 19603 raft_consensus.cc:2272] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.586464 19603 tablet_replica.cc:333] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.587301 19603 raft_consensus.cc:2243] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.587792 19603 raft_consensus.cc:2272] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.590211 19603 tablet_replica.cc:333] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.590974 19603 raft_consensus.cc:2243] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.591548 19603 raft_consensus.cc:2272] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.594396 19603 tablet_replica.cc:333] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.595139 19603 raft_consensus.cc:2243] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:31.596237 19603 raft_consensus.cc:2272] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.599504 19603 tablet_replica.cc:333] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.600265 19603 raft_consensus.cc:2243] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.600811 19603 raft_consensus.cc:2272] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.601514  2049 consensus_peers.cc:597] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.603648 19603 tablet_replica.cc:333] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.604522 19603 raft_consensus.cc:2243] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:31.605533 19603 raft_consensus.cc:2272] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.608244 19603 tablet_replica.cc:333] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.608985 19603 raft_consensus.cc:2243] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:31.609963 19603 raft_consensus.cc:2272] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.612365 19603 tablet_replica.cc:333] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.612920 19603 raft_consensus.cc:2243] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.613372 19603 raft_consensus.cc:2272] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.616122 19603 tablet_replica.cc:333] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
W20260709 12:28:31.616667  1975 consensus_peers.cc:597] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.616808 19603 raft_consensus.cc:2243] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.617487 19603 raft_consensus.cc:2272] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.620157 19603 tablet_replica.cc:333] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.620927 19603 raft_consensus.cc:2243] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.621294 19603 raft_consensus.cc:2272] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.623428 19603 tablet_replica.cc:333] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.624118 19603 raft_consensus.cc:2243] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.624590 19603 raft_consensus.cc:2272] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.626878 19603 tablet_replica.cc:333] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.627611 19603 raft_consensus.cc:2243] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.628104 19603 raft_consensus.cc:2272] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.630326 19603 tablet_replica.cc:333] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.630841 19603 raft_consensus.cc:2243] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.631275 19603 raft_consensus.cc:2272] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.633085 19603 tablet_replica.cc:333] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.633620 19603 raft_consensus.cc:2243] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.634020 19603 raft_consensus.cc:2272] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.635973 19603 tablet_replica.cc:333] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.636514 19603 raft_consensus.cc:2243] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:31.637247 19603 raft_consensus.cc:2272] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.639757 19603 tablet_replica.cc:333] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.640465 19603 raft_consensus.cc:2243] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.640933 19603 raft_consensus.cc:2272] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.642928 19603 tablet_replica.cc:333] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
W20260709 12:28:31.643460  1901 consensus_peers.cc:597] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.643639 19603 raft_consensus.cc:2243] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.644125 19603 raft_consensus.cc:2272] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.644769  2049 consensus_peers.cc:597] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.645581  1975 consensus_peers.cc:597] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.646636 19603 tablet_replica.cc:333] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
W20260709 12:28:31.647380  2049 consensus_peers.cc:597] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.647449 19603 raft_consensus.cc:2243] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.648185 19603 raft_consensus.cc:2272] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.650087 19603 tablet_replica.cc:333] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.650589 19603 raft_consensus.cc:2243] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.651001 19603 raft_consensus.cc:2272] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.652872 19603 tablet_replica.cc:333] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.653371 19603 raft_consensus.cc:2243] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.653831 19603 raft_consensus.cc:2272] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.656203 19603 tablet_replica.cc:333] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.656998 19603 raft_consensus.cc:2243] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.657572 19603 raft_consensus.cc:2272] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:31.659744 19603 tablet_replica.cc:333] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: stopping tablet replica
I20260709 12:28:31.660368 19603 raft_consensus.cc:2243] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:31.660816 19603 raft_consensus.cc:2272] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:31.676946  1975 consensus_peers.cc:597] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.680099  2049 consensus_peers.cc:597] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.688697  2049 consensus_peers.cc:597] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.716995  2049 consensus_peers.cc:597] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.758930 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
W20260709 12:28:31.759895  2049 consensus_peers.cc:597] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.781780  2049 consensus_peers.cc:597] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.816751  1975 consensus_peers.cc:597] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:31.836710  1901 consensus_peers.cc:597] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:31.868585 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:31.877041  2626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:31.877573  2627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:31.879733  2629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:31.880618 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:31.881969 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:31.882331 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:31.882623 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600111882603 us; error 0 us; skew 500 ppm
I20260709 12:28:31.883433 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:31.889318 19603 webserver.cc:533] Webserver started at http://127.19.36.193:37467/ using document root <none> and password file <none>
I20260709 12:28:31.890079 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:31.890363 19603 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:28:31.895671 19603 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:28:31.906636  2634 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:31.907500 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:31.907900 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "8c59f6f4b90d460e832bcff010c18400"
format_stamp: "Formatted at 2026-07-09 12:28:20 on dist-test-slave-hfcn"
I20260709 12:28:31.908390 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:31.923069 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:31.924299 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:31.925740 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:31.934104  2641 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:28:32.216325 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:28:32.216698 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.284s	user 0.009s	sys 0.000s
I20260709 12:28:32.217046 19603 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:28:32.222400  2641 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.237238  2641 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.238423  2641 tablet_bootstrap.cc:492] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.239032  2641 ts_tablet_manager.cc:1403] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:28:32.242074  2641 raft_consensus.cc:359] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.242717  2641 raft_consensus.cc:740] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.243460  2641 consensus_queue.cc:260] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.260179  2641 ts_tablet_manager.cc:1434] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 12:28:32.261003  2641 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.279318  2641 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.280264  2641 tablet_bootstrap.cc:492] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.280881  2641 ts_tablet_manager.cc:1403] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 12:28:32.283674  2641 raft_consensus.cc:359] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.284310  2641 raft_consensus.cc:740] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.285002  2641 consensus_queue.cc:260] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.286859  2641 ts_tablet_manager.cc:1434] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:28:32.287616  2641 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.311643  2641 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.312594  2641 tablet_bootstrap.cc:492] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.313198  2641 ts_tablet_manager.cc:1403] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260709 12:28:32.315409  2641 raft_consensus.cc:359] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.316045  2641 raft_consensus.cc:740] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.317698  2641 consensus_queue.cc:260] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.319567  2641 ts_tablet_manager.cc:1434] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:32.320315  2641 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.338699  2641 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.339811  2641 tablet_bootstrap.cc:492] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.340560  2641 ts_tablet_manager.cc:1403] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.006s
I20260709 12:28:32.343108  2641 raft_consensus.cc:359] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.343798  2641 raft_consensus.cc:740] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.346258  2641 consensus_queue.cc:260] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.348121  2641 ts_tablet_manager.cc:1434] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 12:28:32.348836  2641 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.363070  2641 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.364017  2641 tablet_bootstrap.cc:492] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.364611  2641 ts_tablet_manager.cc:1403] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:28:32.367168  2641 raft_consensus.cc:359] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.367817  2641 raft_consensus.cc:740] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.368469  2641 consensus_queue.cc:260] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.370360  2641 ts_tablet_manager.cc:1434] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:28:32.371105  2641 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.386317  2641 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.387104  2641 tablet_bootstrap.cc:492] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.387723  2641 ts_tablet_manager.cc:1403] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:28:32.389868  2641 raft_consensus.cc:359] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.390501  2641 raft_consensus.cc:740] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.391180  2641 consensus_queue.cc:260] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.392686  2641 ts_tablet_manager.cc:1434] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:32.393298  2641 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.406037  2641 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.406785  2641 tablet_bootstrap.cc:492] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.407433  2641 ts_tablet_manager.cc:1403] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.004s
I20260709 12:28:32.409255  2641 raft_consensus.cc:359] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.409759  2641 raft_consensus.cc:740] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.410494  2641 consensus_queue.cc:260] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.412217  2641 ts_tablet_manager.cc:1434] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:32.412914  2641 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.423426  2641 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.424225  2641 tablet_bootstrap.cc:492] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.424830  2641 ts_tablet_manager.cc:1403] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 12:28:32.426889  2641 raft_consensus.cc:359] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.427472  2641 raft_consensus.cc:740] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.428134  2641 consensus_queue.cc:260] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.429654  2641 ts_tablet_manager.cc:1434] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:28:32.430274  2641 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.441874  2641 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.442571  2641 tablet_bootstrap.cc:492] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.443110  2641 ts_tablet_manager.cc:1403] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:28:32.444895  2641 raft_consensus.cc:359] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.445421  2641 raft_consensus.cc:740] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.445950  2641 consensus_queue.cc:260] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.447726  2641 ts_tablet_manager.cc:1434] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:32.448377  2641 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.454633 19603 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:28:32.454946 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.238s	user 0.226s	sys 0.008s
I20260709 12:28:32.463388  2641 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.464298  2641 tablet_bootstrap.cc:492] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.464918  2641 ts_tablet_manager.cc:1403] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 12:28:32.467490  2641 raft_consensus.cc:359] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.468247  2641 raft_consensus.cc:740] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.469267  2641 consensus_queue.cc:260] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.471688  2641 ts_tablet_manager.cc:1434] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:32.472561  2641 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.500284  2641 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.501251  2641 tablet_bootstrap.cc:492] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.501905  2641 ts_tablet_manager.cc:1403] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.004s
I20260709 12:28:32.504400  2641 raft_consensus.cc:359] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.505018  2641 raft_consensus.cc:740] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.505798  2641 consensus_queue.cc:260] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.515669  2641 ts_tablet_manager.cc:1434] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:28:32.516813  2641 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:32.547770  1901 consensus_peers.cc:597] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:32.596014  2315 raft_consensus.cc:493] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.596698  2315 raft_consensus.cc:515] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.663722  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:32.664902  2315 leader_election.cc:290] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:32.686671  2641 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.689049  2641 tablet_bootstrap.cc:492] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.689698  2641 ts_tablet_manager.cc:1403] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.173s	user 0.048s	sys 0.093s
I20260709 12:28:32.692421  2641 raft_consensus.cc:359] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.693131  2641 raft_consensus.cc:740] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.693790  2641 consensus_queue.cc:260] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.695863  2641 ts_tablet_manager.cc:1434] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:32.696611  2641 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:32.705967  1975 leader_election.cc:336] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111)
I20260709 12:28:32.706688  1975 leader_election.cc:304] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:32.707593  2315 raft_consensus.cc:2749] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:32.710783  2641 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.711719  2641 tablet_bootstrap.cc:492] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.712328  2641 ts_tablet_manager.cc:1403] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:28:32.714843  2641 raft_consensus.cc:359] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.715490  2641 raft_consensus.cc:740] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.716207  2641 consensus_queue.cc:260] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.718345  2641 ts_tablet_manager.cc:1434] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:28:32.719084  2641 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.732653  2641 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.733384  2641 tablet_bootstrap.cc:492] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.733952  2641 ts_tablet_manager.cc:1403] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 12:28:32.736358  2641 raft_consensus.cc:359] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.737006  2641 raft_consensus.cc:740] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.737756  2641 consensus_queue.cc:260] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.740010  2641 ts_tablet_manager.cc:1434] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:32.740669  2641 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.748098 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:35551
I20260709 12:28:32.748236  2716 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:35551 every 8 connection(s)
I20260709 12:28:32.783047  2641 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.784142  2641 tablet_bootstrap.cc:492] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.784803  2641 ts_tablet_manager.cc:1403] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.024s
I20260709 12:28:32.787408  2641 raft_consensus.cc:359] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.788112  2641 raft_consensus.cc:740] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.788713  2641 consensus_queue.cc:260] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.790412  2641 ts_tablet_manager.cc:1434] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:28:32.791270  2641 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.779019  2457 raft_consensus.cc:493] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.794185  2457 raft_consensus.cc:515] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.796453  2457 leader_election.cc:290] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:32.796919  2457 raft_consensus.cc:493] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.797382  2457 raft_consensus.cc:515] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.801106  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:32.813710  2641 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.814814  2641 tablet_bootstrap.cc:492] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.816623  2641 ts_tablet_manager.cc:1403] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 12:28:32.820581  2641 raft_consensus.cc:359] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.821349  2641 raft_consensus.cc:740] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.822029  2641 consensus_queue.cc:260] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.824090  2641 ts_tablet_manager.cc:1434] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:32.826148  2641 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.862035  1928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:32.860090  2457 leader_election.cc:290] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:32.865597  2641 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.869587  2641 tablet_bootstrap.cc:492] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.870491  2641 ts_tablet_manager.cc:1403] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.011s
I20260709 12:28:32.875897  2641 raft_consensus.cc:359] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.876798  2641 raft_consensus.cc:740] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.877776  2641 consensus_queue.cc:260] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.880555  2641 ts_tablet_manager.cc:1434] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:32.881570  2641 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:32.897219  2689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:32.898078  2689 raft_consensus.cc:2468] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:32.899179  2355 raft_consensus.cc:493] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
W20260709 12:28:32.900048  1977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:37785, user_credentials={real_user=slave}} blocked reactor thread for 69260.5us
I20260709 12:28:32.899893  2355 raft_consensus.cc:515] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.903885  2689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:32.906391  2355 leader_election.cc:290] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:32.908038  2689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:32.909075  2689 raft_consensus.cc:2468] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:32.909195  2449 raft_consensus.cc:493] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.909735  2449 raft_consensus.cc:515] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.910305  1901 leader_election.cc:304] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:32.911123  2355 raft_consensus.cc:2804] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:32.911599  2355 raft_consensus.cc:493] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.911923  2449 leader_election.cc:290] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:32.912016  2355 raft_consensus.cc:3060] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.920288  2355 raft_consensus.cc:515] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.921595  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:32.922127  1975 leader_election.cc:304] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee
I20260709 12:28:32.922312  2008 raft_consensus.cc:2468] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:32.923156  2589 raft_consensus.cc:2804] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:32.923691  2589 raft_consensus.cc:493] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.924155  2589 raft_consensus.cc:3060] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.930680  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:32.931344  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:32.931422  2080 raft_consensus.cc:2468] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:32.931772  2049 leader_election.cc:304] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:32.932369  2416 raft_consensus.cc:493] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.932780  2642 raft_consensus.cc:493] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.933205  2689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:32.932855  2416 raft_consensus.cc:515] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.933893  2689 raft_consensus.cc:2468] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:32.934983  2378 raft_consensus.cc:2804] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:32.933660  2642 raft_consensus.cc:515] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.935439  2416 leader_election.cc:290] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:32.935442  2378 raft_consensus.cc:493] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.936421  2378 raft_consensus.cc:3060] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:32.936488  1975 leader_election.cc:343] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:32.936961  1975 leader_election.cc:304] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:32.937244  2642 leader_election.cc:290] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:32.937826  2725 raft_consensus.cc:2749] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:32.939347  2718 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40093
I20260709 12:28:32.939862  2718 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:32.942557  2718 heartbeater.cc:507] Master 127.19.36.254:40093 requested a full tablet report, sending...
I20260709 12:28:32.949532  2355 leader_election.cc:290] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:32.942229  2589 raft_consensus.cc:515] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.947177  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:32.951924  2008 raft_consensus.cc:2468] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 1.
I20260709 12:28:32.952778  1901 leader_election.cc:304] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:32.952915  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:32.953588  2355 raft_consensus.cc:2804] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:32.953624  2686 raft_consensus.cc:3060] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.954046  2355 raft_consensus.cc:493] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.954506  2355 raft_consensus.cc:3060] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.955417  2378 raft_consensus.cc:515] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:32.956723  2641 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:32.957576  2641 tablet_bootstrap.cc:492] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:32.957537  2378 leader_election.cc:290] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:32.958532  2641 ts_tablet_manager.cc:1403] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.077s	user 0.016s	sys 0.009s
I20260709 12:28:32.962141  2416 raft_consensus.cc:493] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:32.964349  1932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:32.964872  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:32.965256  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:32.966269  1934 raft_consensus.cc:2393] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 2: Already voted for candidate 3e95870e8ff04dc7ab920b8917898eee in this term.
I20260709 12:28:32.966930  2688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "891bc4cc61ee4451a2cb82d28286d5f8" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:32.965430  2008 raft_consensus.cc:3060] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.969031  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7707b908f046398ca5ee4defefe55f" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:32.969282  2355 raft_consensus.cc:515] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.969719  2684 raft_consensus.cc:3060] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.971452  2355 leader_election.cc:290] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:32.962718  2416 raft_consensus.cc:515] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:32.973325  2416 leader_election.cc:290] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:32.975663  2008 raft_consensus.cc:2468] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 2.
I20260709 12:28:32.976116  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:32.976823  2685 raft_consensus.cc:3060] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.978962  2684 raft_consensus.cc:2468] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 2.
I20260709 12:28:32.981702  2683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:32.982048  2049 leader_election.cc:304] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:32.981684  2008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:32.984102  2008 raft_consensus.cc:3060] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.984877  2689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:32.984900  2685 raft_consensus.cc:2468] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 2.
W20260709 12:28:32.985770  1901 leader_election.cc:343] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:32.986876  1901 leader_election.cc:343] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:32.987375  1975 leader_election.cc:304] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 3e95870e8ff04dc7ab920b8917898eee
I20260709 12:28:32.987843  2686 raft_consensus.cc:2468] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:32.988211  2725 raft_consensus.cc:2804] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:32.992314  2725 raft_consensus.cc:697] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 2 LEADER]: Becoming Leader. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:32.993366  2725 consensus_queue.cc:237] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:32.990279  1901 leader_election.cc:304] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:32.998282  2008 raft_consensus.cc:2468] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:32.959779  2589 leader_election.cc:290] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:32.998754  2416 raft_consensus.cc:2804] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:32.988550  2728 raft_consensus.cc:2804] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:32.999588  2416 raft_consensus.cc:697] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:32.999835  2049 leader_election.cc:304] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:33.000460  1901 leader_election.cc:304] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:32.989065  2641 raft_consensus.cc:359] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.001796  2641 raft_consensus.cc:740] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.003098  2728 raft_consensus.cc:697] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:33.003144  2355 raft_consensus.cc:2804] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:33.004303  2728 consensus_queue.cc:237] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.004897  2416 consensus_queue.cc:237] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.008152  2642 raft_consensus.cc:3060] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:33.017091  2642 raft_consensus.cc:2749] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:33.024375  2641 consensus_queue.cc:260] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.028765  2378 raft_consensus.cc:493] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:33.029444  2378 raft_consensus.cc:515] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.025969  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf8686b20194e3981d420491c825046" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:33.031438  2378 leader_election.cc:290] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:33.031999  2686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
W20260709 12:28:33.033663  1901 leader_election.cc:343] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:33.033179  1758 catalog_manager.cc:5697] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 reported cstate change: term changed from 1 to 2, leader changed from 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193) to 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "8e43e9e956f14e52897d2efc3f361630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:33.036741  2049 leader_election.cc:343] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:33.044559  2641 ts_tablet_manager.cc:1434] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.086s	user 0.004s	sys 0.004s
I20260709 12:28:33.045931  2641 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.056463  2355 raft_consensus.cc:697] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:33.061839  2355 consensus_queue.cc:237] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.064791  1760 catalog_manager.cc:5697] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 1 to 2, leader changed from 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193) to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 2 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:33.067286  2457 raft_consensus.cc:493] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:33.068717  2457 raft_consensus.cc:515] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.075243  2457 leader_election.cc:290] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:33.058449  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:33.080758  1934 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:33.081920  2049 leader_election.cc:304] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:33.082764  2449 raft_consensus.cc:2804] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:33.083300  2449 raft_consensus.cc:493] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:33.083742  2449 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:33.098352  2449 raft_consensus.cc:515] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.099661  1756 ts_manager.cc:194] Re-registered known tserver with Master: 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:33.101475  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
W20260709 12:28:33.102941  2049 leader_election.cc:343] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:33.104341  2732 raft_consensus.cc:493] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:33.104915  2732 raft_consensus.cc:515] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.108162  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:33.108965  2685 raft_consensus.cc:2393] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 2: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:33.110122  2449 leader_election.cc:290] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 election: Requested vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
W20260709 12:28:33.132241  1903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.196:37785, user_credentials={real_user=slave}} blocked reactor thread for 112495us
I20260709 12:28:33.137079  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:33.138113  2085 raft_consensus.cc:2393] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e95870e8ff04dc7ab920b8917898eee in current term 2: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:33.140643  1903 leader_election.cc:304] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:33.111317  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:33.142154  2416 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:33.140451  2732 leader_election.cc:290] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:33.157178  1757 catalog_manager.cc:5697] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 1 to 2, leader changed from 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193) to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 2 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:33.149155  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:33.165541  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "466fe5ff431c463a84050bd3e265a246" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:33.174556  2080 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:33.176120  2449 raft_consensus.cc:493] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:33.176613  2449 raft_consensus.cc:515] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.177146  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:33.177830  2684 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e43e9e956f14e52897d2efc3f361630 in term 1.
I20260709 12:28:33.180073  2449 leader_election.cc:290] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:33.186376  2416 raft_consensus.cc:2749] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:33.182523  1977 leader_election.cc:304] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:33.197002  2736 raft_consensus.cc:2804] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:33.197816  2736 raft_consensus.cc:493] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:33.198145  2736 raft_consensus.cc:3060] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:33.199409  2684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400" is_pre_election: true
I20260709 12:28:33.200724  2684 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 1.
I20260709 12:28:33.202142  2049 leader_election.cc:304] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:33.202989  2738 raft_consensus.cc:2804] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:33.203423  2738 raft_consensus.cc:493] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:33.203788  2738 raft_consensus.cc:3060] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:33.205142  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:33.212960  1901 leader_election.cc:304] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee; no voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:33.213830  2732 raft_consensus.cc:3060] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:33.215512  2738 raft_consensus.cc:515] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.227633  1934 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 2.
I20260709 12:28:33.231045  2641 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.231298  2049 leader_election.cc:304] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:33.238956  2449 raft_consensus.cc:2804] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:33.239436  2449 raft_consensus.cc:697] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:33.240213  2449 consensus_queue.cc:237] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.231844  2738 leader_election.cc:290] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:33.232241  2732 raft_consensus.cc:2749] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:33.242015  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:33.233840  2736 raft_consensus.cc:515] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.243253  2689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "5edb06eaed8e41868299eb62ade6eef0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:33.243510  2011 raft_consensus.cc:2393] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5edb06eaed8e41868299eb62ade6eef0 in current term 2: Already voted for candidate 8e43e9e956f14e52897d2efc3f361630 in this term.
I20260709 12:28:33.243912  2689 raft_consensus.cc:3060] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:33.247408  2641 tablet_bootstrap.cc:492] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.251035  2641 ts_tablet_manager.cc:1403] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.205s	user 0.016s	sys 0.007s
I20260709 12:28:33.251262  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:33.264945  2085 raft_consensus.cc:2393] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e43e9e956f14e52897d2efc3f361630 in current term 2: Already voted for candidate 5edb06eaed8e41868299eb62ade6eef0 in this term.
I20260709 12:28:33.254398  2736 leader_election.cc:290] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551)
I20260709 12:28:33.256215  2685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11795a035a1e482d88901e8aea8ee185" candidate_uuid: "8e43e9e956f14e52897d2efc3f361630" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
I20260709 12:28:33.266625  2641 raft_consensus.cc:359] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.271050  2641 raft_consensus.cc:740] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.274237  1975 leader_election.cc:304] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e43e9e956f14e52897d2efc3f361630; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:33.273546  2641 consensus_queue.cc:260] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.275063  2589 raft_consensus.cc:2749] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:33.276904  2641 ts_tablet_manager.cc:1434] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:28:33.277659  2689 raft_consensus.cc:2468] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5edb06eaed8e41868299eb62ade6eef0 in term 2.
I20260709 12:28:33.277701  2641 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.278934  2049 leader_election.cc:304] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:33.279754  2738 raft_consensus.cc:2804] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:33.280300  2738 raft_consensus.cc:697] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Becoming Leader. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:33.283864  2738 consensus_queue.cc:237] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.295267  1758 catalog_manager.cc:5697] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 1 to 2, leader changed from 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193) to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 2 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:33.310777  2641 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.311604  2641 tablet_bootstrap.cc:492] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.312206  2641 ts_tablet_manager.cc:1403] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.004s
I20260709 12:28:33.314618  2641 raft_consensus.cc:359] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.315150  2641 raft_consensus.cc:385] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:33.315444  2641 raft_consensus.cc:740] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.316213  2641 consensus_queue.cc:260] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.322983  2641 ts_tablet_manager.cc:1434] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:28:33.323812  2641 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.327080  1758 catalog_manager.cc:5697] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 reported cstate change: term changed from 1 to 2, leader changed from 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193) to 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196). New cstate: current_term: 2 leader_uuid: "5edb06eaed8e41868299eb62ade6eef0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:33.327767  1760 catalog_manager.cc:5697] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 1 to 2, leader changed from 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193) to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 2 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:33.351557  2641 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.352480  2641 tablet_bootstrap.cc:492] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.352824  1756 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60948
I20260709 12:28:33.353103  2641 ts_tablet_manager.cc:1403] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.002s
I20260709 12:28:33.357062  2641 raft_consensus.cc:359] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.357447  2718 heartbeater.cc:499] Master 127.19.36.254:40093 was elected leader, sending a full tablet report...
I20260709 12:28:33.357618  2641 raft_consensus.cc:740] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.358791  2641 consensus_queue.cc:260] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.360976  2641 ts_tablet_manager.cc:1434] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:33.361788  2641 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.376737  2641 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.377575  2641 tablet_bootstrap.cc:492] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.378217  2641 ts_tablet_manager.cc:1403] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.005s
I20260709 12:28:33.379431  2747 raft_consensus.cc:493] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.379971  2747 raft_consensus.cc:515] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.380622  2641 raft_consensus.cc:359] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.381232  2641 raft_consensus.cc:740] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.381881  2641 consensus_queue.cc:260] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.387473  2747 leader_election.cc:290] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:33.391399  2641 ts_tablet_manager.cc:1434] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 12:28:33.395167  2641 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.417233  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5edb06eaed8e41868299eb62ade6eef0" is_pre_election: true
I20260709 12:28:33.420425  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:33.421988  2636 leader_election.cc:304] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 5edb06eaed8e41868299eb62ade6eef0, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:33.423027  2641 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.423166  2747 raft_consensus.cc:2749] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:33.424060  2641 tablet_bootstrap.cc:492] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.424669  2641 ts_tablet_manager.cc:1403] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.000s
I20260709 12:28:33.426666  2641 raft_consensus.cc:359] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.427136  2641 raft_consensus.cc:740] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.427780  2641 consensus_queue.cc:260] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.429174  2641 ts_tablet_manager.cc:1434] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:33.429909  2641 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.441205  2641 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.441865  2641 tablet_bootstrap.cc:492] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.442440  2641 ts_tablet_manager.cc:1403] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:28:33.444788  2641 raft_consensus.cc:359] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.445426  2641 raft_consensus.cc:740] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.446033  2641 consensus_queue.cc:260] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.448060  2641 ts_tablet_manager.cc:1434] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:28:33.448846  2641 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.455274  2684 raft_consensus.cc:1275] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Refusing update from remote peer 5edb06eaed8e41868299eb62ade6eef0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.456662  2738 consensus_queue.cc:1048] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.478761  2011 raft_consensus.cc:1275] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 5edb06eaed8e41868299eb62ade6eef0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.480965  2449 consensus_queue.cc:1048] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.482563  2641 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.483582  2641 tablet_bootstrap.cc:492] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.484233  2641 ts_tablet_manager.cc:1403] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.001s
I20260709 12:28:33.487730  2641 raft_consensus.cc:359] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.488495  2641 raft_consensus.cc:740] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.489492  2641 consensus_queue.cc:260] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.492473  2641 ts_tablet_manager.cc:1434] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:28:33.493299  2641 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.498983  2684 raft_consensus.cc:1275] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Refusing update from remote peer 8e43e9e956f14e52897d2efc3f361630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.500609  2736 consensus_queue.cc:1048] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.508059  1931 raft_consensus.cc:1275] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8e43e9e956f14e52897d2efc3f361630: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.509935  2736 consensus_queue.cc:1048] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.510711  2641 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.511680  2641 tablet_bootstrap.cc:492] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.512315  2641 ts_tablet_manager.cc:1403] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.005s
I20260709 12:28:33.514897  2641 raft_consensus.cc:359] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.521337  2085 raft_consensus.cc:1275] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.515578  2641 raft_consensus.cc:740] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.525120  2732 consensus_queue.cc:1048] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.526278  2641 consensus_queue.cc:260] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.512379  2689 raft_consensus.cc:3060] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:33.532840  2641 ts_tablet_manager.cc:1434] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.003s
I20260709 12:28:33.535549  2641 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.593042  2641 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.594194  2641 tablet_bootstrap.cc:492] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.595315  2641 ts_tablet_manager.cc:1403] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.021s
I20260709 12:28:33.605741  2641 raft_consensus.cc:359] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.606444  2641 raft_consensus.cc:740] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.607170  2641 consensus_queue.cc:260] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.611508  2641 ts_tablet_manager.cc:1434] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 12:28:33.612624  2641 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:33.620298  2049 consensus_peers.cc:597] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:33.621440  1975 consensus_peers.cc:597] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:33.667131  2689 raft_consensus.cc:1275] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.674523  2729 consensus_queue.cc:1048] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:33.676357  1901 consensus_peers.cc:597] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:33.678056  1975 consensus_peers.cc:597] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:33.680540  2007 raft_consensus.cc:1275] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.683272  2729 consensus_queue.cc:1048] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.683532  2641 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.684505  2641 tablet_bootstrap.cc:492] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.685165  2641 ts_tablet_manager.cc:1403] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.013s
I20260709 12:28:33.704303  2641 raft_consensus.cc:359] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.705279  2641 raft_consensus.cc:385] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:33.706450  2641 raft_consensus.cc:740] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.709474  2641 consensus_queue.cc:260] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.711673  2641 ts_tablet_manager.cc:1434] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:28:33.712453  2641 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.737893  2641 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.738787  2641 tablet_bootstrap.cc:492] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.739490  2641 ts_tablet_manager.cc:1403] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.000s
I20260709 12:28:33.741971  2641 raft_consensus.cc:359] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.742627  2641 raft_consensus.cc:740] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.743402  2641 consensus_queue.cc:260] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.745643  2641 ts_tablet_manager.cc:1434] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:28:33.746431  2641 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:33.749015  2049 consensus_peers.cc:597] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:33.759835  2641 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.761138  2641 tablet_bootstrap.cc:492] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.761798  2641 ts_tablet_manager.cc:1403] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.003s
I20260709 12:28:33.761943  2685 raft_consensus.cc:1275] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Refusing update from remote peer 5edb06eaed8e41868299eb62ade6eef0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.763413  2449 consensus_queue.cc:1048] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.765574  2641 raft_consensus.cc:359] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.766551  2641 raft_consensus.cc:740] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.767524  2641 consensus_queue.cc:260] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.769459  2641 ts_tablet_manager.cc:1434] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:33.770630  2641 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.776477  2011 raft_consensus.cc:1275] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 5edb06eaed8e41868299eb62ade6eef0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.779258  2728 consensus_queue.cc:1048] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:33.808657  1975 consensus_peers.cc:597] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:33.815584  2641 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.816592  2641 tablet_bootstrap.cc:492] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.817286  2641 ts_tablet_manager.cc:1403] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.047s	user 0.028s	sys 0.003s
W20260709 12:28:33.817379  2049 consensus_peers.cc:597] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:33.819744  2641 raft_consensus.cc:359] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.820402  2641 raft_consensus.cc:740] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.823097  2641 consensus_queue.cc:260] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.827287  2641 ts_tablet_manager.cc:1434] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:28:33.828073  2641 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.831266  1931 raft_consensus.cc:1275] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 5edb06eaed8e41868299eb62ade6eef0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:33.832862  2728 consensus_queue.cc:1048] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:33.852038  2641 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.852952  2641 tablet_bootstrap.cc:492] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.853596  2641 ts_tablet_manager.cc:1403] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.000s
I20260709 12:28:33.856217  2641 raft_consensus.cc:359] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.856885  2641 raft_consensus.cc:740] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.857561  2641 consensus_queue.cc:260] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.859611  2641 ts_tablet_manager.cc:1434] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:33.860435  2641 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.871223  2747 raft_consensus.cc:493] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.871769  2747 raft_consensus.cc:515] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:33.874023  2747 leader_election.cc:290] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:33.877816  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e43e9e956f14e52897d2efc3f361630" is_pre_election: true
I20260709 12:28:33.884544  2641 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.885416  2641 tablet_bootstrap.cc:492] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.886061  2641 ts_tablet_manager.cc:1403] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.000s
I20260709 12:28:33.888521  2641 raft_consensus.cc:359] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.889164  2641 raft_consensus.cc:740] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.889806  2641 consensus_queue.cc:260] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.891793  2641 ts_tablet_manager.cc:1434] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:28:33.895025  2641 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.898248  1931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac43f6f37d474da58e0a962b25fe9235" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e95870e8ff04dc7ab920b8917898eee" is_pre_election: true
I20260709 12:28:33.900098  2636 leader_election.cc:304] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630
I20260709 12:28:33.901400  2747 raft_consensus.cc:2749] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:33.909222  2641 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.909988  2641 tablet_bootstrap.cc:492] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.910591  2641 ts_tablet_manager.cc:1403] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.005s
I20260709 12:28:33.912640  2641 raft_consensus.cc:359] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.913244  2641 raft_consensus.cc:740] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.914002  2641 consensus_queue.cc:260] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.915925  2641 ts_tablet_manager.cc:1434] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:28:33.917086  2641 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.931532  2641 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.932264  2641 tablet_bootstrap.cc:492] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.932806  2641 ts_tablet_manager.cc:1403] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:28:33.934835  2641 raft_consensus.cc:359] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.935453  2641 raft_consensus.cc:740] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.936100  2641 consensus_queue.cc:260] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.938344  2641 ts_tablet_manager.cc:1434] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:33.939333  2641 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.952888  2641 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.953578  2641 tablet_bootstrap.cc:492] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.954133  2641 ts_tablet_manager.cc:1403] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.003s
I20260709 12:28:33.955924  2641 raft_consensus.cc:359] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.956394  2641 raft_consensus.cc:740] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.957044  2641 consensus_queue.cc:260] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:33.958868  2641 ts_tablet_manager.cc:1434] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:33.960043  2641 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:33.975697  2641 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.976634  2641 tablet_bootstrap.cc:492] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.977289  2641 ts_tablet_manager.cc:1403] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 12:28:33.979439  2641 raft_consensus.cc:359] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.979902  2641 raft_consensus.cc:740] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:33.980491  2641 consensus_queue.cc:260] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:33.982239  2641 ts_tablet_manager.cc:1434] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:33.983327  2641 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:33.992003  1975 consensus_peers.cc:597] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:33.998016  2641 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.998880  2641 tablet_bootstrap.cc:492] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:33.999470  2641 ts_tablet_manager.cc:1403] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 12:28:34.001225  2641 raft_consensus.cc:359] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:34.001806  2641 raft_consensus.cc:740] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.002315  2641 consensus_queue.cc:260] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:34.004179  2641 ts_tablet_manager.cc:1434] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:34.005182  2641 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:34.017117  2641 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:34.018049  2641 tablet_bootstrap.cc:492] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:34.019271  2641 ts_tablet_manager.cc:1403] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 12:28:34.021656  2641 raft_consensus.cc:359] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:34.022290  2641 raft_consensus.cc:740] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.022948  2641 consensus_queue.cc:260] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:34.024922  2641 ts_tablet_manager.cc:1434] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:34.025681  2641 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
W20260709 12:28:34.029017  2049 consensus_peers.cc:597] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:34.040450  2641 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:34.041407  2641 tablet_bootstrap.cc:492] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:34.042028  2641 ts_tablet_manager.cc:1403] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.005s
I20260709 12:28:34.044369  2689 raft_consensus.cc:3060] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:34.044314  2641 raft_consensus.cc:359] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:34.045079  2641 raft_consensus.cc:740] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.045771  2641 consensus_queue.cc:260] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:34.047820  2641 ts_tablet_manager.cc:1434] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 12:28:34.054263  2641 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:34.080670  2641 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:34.081722  2641 tablet_bootstrap.cc:492] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:34.082437  2641 ts_tablet_manager.cc:1403] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.000s
I20260709 12:28:34.084903  2641 raft_consensus.cc:359] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:34.085716  2641 raft_consensus.cc:740] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.086380  2641 consensus_queue.cc:260] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:34.088354  2641 ts_tablet_manager.cc:1434] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:34.093215  2641 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:34.111290  2641 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:34.112156  2641 tablet_bootstrap.cc:492] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:34.112694  2641 ts_tablet_manager.cc:1403] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.007s
I20260709 12:28:34.114332  2641 raft_consensus.cc:359] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:34.114830  2641 raft_consensus.cc:740] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.115432  2641 consensus_queue.cc:260] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:34.117290  2641 ts_tablet_manager.cc:1434] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:34.118053  2641 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:34.132778  2641 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:34.133703  2641 tablet_bootstrap.cc:492] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:34.134367  2641 ts_tablet_manager.cc:1403] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.006s
I20260709 12:28:34.136512  2641 raft_consensus.cc:359] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:34.137145  2641 raft_consensus.cc:740] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.137773  2641 consensus_queue.cc:260] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:34.139168  2641 ts_tablet_manager.cc:1434] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:28:34.139993  2641 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:34.146024  2686 raft_consensus.cc:3060] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:34.177594  2641 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:34.178727  2641 tablet_bootstrap.cc:492] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:34.179484  2641 ts_tablet_manager.cc:1403] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.020s
I20260709 12:28:34.181777  2641 raft_consensus.cc:359] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:34.182538  2641 raft_consensus.cc:740] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.183254  2641 consensus_queue.cc:260] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:34.184826  2641 ts_tablet_manager.cc:1434] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:34.185588  2641 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap starting.
I20260709 12:28:34.198899  2641 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:34.200004  2641 tablet_bootstrap.cc:492] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Bootstrap complete.
I20260709 12:28:34.200663  2641 ts_tablet_manager.cc:1403] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 12:28:34.203161  2641 raft_consensus.cc:359] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:34.203846  2641 raft_consensus.cc:740] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Initialized, Role: FOLLOWER
I20260709 12:28:34.204377  2641 consensus_queue.cc:260] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:34.206943  2641 ts_tablet_manager.cc:1434] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260709 12:28:34.299118  2686 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:42.792472 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:42.844170  2085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125"
dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:28:42.845104  2085 raft_consensus.cc:606] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:42.857343  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:42.857818  2007 raft_consensus.cc:606] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:42.860416  2080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830"
dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
mode: GRACEFUL
new_leader_uuid: "3e95870e8ff04dc7ab920b8917898eee"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:28:42.860973  2080 raft_consensus.cc:606] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Received request to transfer leadership to TS 3e95870e8ff04dc7ab920b8917898eee
I20260709 12:28:42.863507  2080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b"
dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:28:42.863920  2080 raft_consensus.cc:606] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:42.865890  2080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90834683513744559ef8f7efe530f459"
dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:28:42.866324  2080 raft_consensus.cc:606] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:42.868935  2080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92"
dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:28:42.869352  2080 raft_consensus.cc:606] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:42.870213 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:28:42.871549 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:42.872346 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:42.906237  2815 raft_consensus.cc:993] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:42.906999  2728 raft_consensus.cc:1081] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:42.907780  2816 raft_consensus.cc:993] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:42.908304  2816 raft_consensus.cc:1081] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:42.908643  2689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90834683513744559ef8f7efe530f459"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53350
I20260709 12:28:42.909215  2689 raft_consensus.cc:493] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:42.909639  2689 raft_consensus.cc:3060] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:42.909991  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:42.910615  2685 raft_consensus.cc:493] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:42.910948  2685 raft_consensus.cc:3060] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:42.918529  2689 raft_consensus.cc:515] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:42.919085  2685 raft_consensus.cc:515] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:42.921008  2689 leader_election.cc:290] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:42.922221  1931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:42.922969  1931 raft_consensus.cc:3060] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:42.924635  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90834683513744559ef8f7efe530f459" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:42.925199  2080 raft_consensus.cc:3055] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:42.925488  2080 raft_consensus.cc:740] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:42.926271  2080 consensus_queue.cc:260] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:42.927657  2080 raft_consensus.cc:3060] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:42.929347  1931 raft_consensus.cc:2468] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:42.930750  2636 leader_election.cc:304] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:42.932725  2685 leader_election.cc:290] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:42.939402  2080 raft_consensus.cc:2468] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:42.941583  2823 raft_consensus.cc:2804] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:42.951864  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:42.952669  1934 raft_consensus.cc:3060] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:42.967533  1934 raft_consensus.cc:2468] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:42.968137  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ee2ecd6eef424bb1bf355492553d8e" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:42.969055  2007 raft_consensus.cc:3055] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:42.969254  2007 raft_consensus.cc:740] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:42.969745  2636 leader_election.cc:304] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:42.970851  2823 raft_consensus.cc:697] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:42.972049  2825 raft_consensus.cc:2804] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:42.970151  2007 consensus_queue.cc:260] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:42.973109  2825 raft_consensus.cc:697] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:42.972482  2823 consensus_queue.cc:237] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:42.973682  2007 raft_consensus.cc:3060] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:42.976583  2825 consensus_queue.cc:237] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:42.984362  2007 raft_consensus.cc:2468] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:42.988636  2815 raft_consensus.cc:993] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:42.989090  2815 raft_consensus.cc:1081] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:42.991016  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53350
I20260709 12:28:42.991623  2685 raft_consensus.cc:493] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:42.992033  2685 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:42.991882  1756 catalog_manager.cc:5697] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:42.999296  2685 raft_consensus.cc:515] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:43.001408  2685 leader_election.cc:290] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 3 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:43.002554  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:43.003352  1934 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:43.005985  2728 raft_consensus.cc:993] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:43.005889  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d17fb96cd6f94ffa9824eb3868a5fe92" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:43.006654  2728 raft_consensus.cc:1081] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:43.006994  2080 raft_consensus.cc:3055] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:43.007503  2080 raft_consensus.cc:740] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:43.008445  2689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53350
I20260709 12:28:43.009027  2689 raft_consensus.cc:493] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:43.008929  2080 consensus_queue.cc:260] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:43.009502  2689 raft_consensus.cc:3060] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.010383  2080 raft_consensus.cc:3060] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:43.014506  1934 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 3.
I20260709 12:28:43.015923  2636 leader_election.cc:304] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:43.016852  2825 raft_consensus.cc:2804] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:43.017378  2825 raft_consensus.cc:697] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 3 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:43.018848  2080 raft_consensus.cc:2468] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 3.
I20260709 12:28:43.018754  2825 consensus_queue.cc:237] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:43.021472  2689 raft_consensus.cc:515] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:43.026086  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:43.026679  2080 raft_consensus.cc:3055] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:43.026840  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d082f4dbaeb47959b209411cd69957b" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:43.027029  2080 raft_consensus.cc:740] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:43.027549  1934 raft_consensus.cc:3060] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.028075  2080 consensus_queue.cc:260] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:43.029307  2080 raft_consensus.cc:3060] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.034130  1934 raft_consensus.cc:2468] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:43.035456  2636 leader_election.cc:304] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:43.034910  1756 catalog_manager.cc:5697] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:43.036845  2080 raft_consensus.cc:2468] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:43.037142  1756 catalog_manager.cc:5697] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 2 to 3, leader changed from 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 3 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:43.039160  2689 leader_election.cc:290] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:43.040305  2825 raft_consensus.cc:2804] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:43.041220  2825 raft_consensus.cc:697] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:43.042019  2825 consensus_queue.cc:237] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:43.055583  1758 catalog_manager.cc:5697] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:43.291525  2818 raft_consensus.cc:993] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0: : Instructing follower 3e95870e8ff04dc7ab920b8917898eee to start an election
I20260709 12:28:43.292012  2815 raft_consensus.cc:1081] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Signalling peer 3e95870e8ff04dc7ab920b8917898eee to start an election
I20260709 12:28:43.293274  1931 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830"
dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
 from {username='slave'} at 127.0.0.1:53556
I20260709 12:28:43.293730  1931 raft_consensus.cc:493] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:43.294116  1931 raft_consensus.cc:3060] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.298702  1931 raft_consensus.cc:515] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:43.300653  1931 leader_election.cc:290] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:43.301541  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:43.301708  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e46e6bd8574e31824d103146fc9830" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:43.302160  2011 raft_consensus.cc:3060] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.302333  2080 raft_consensus.cc:3055] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:43.302727  2080 raft_consensus.cc:740] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:43.303567  2080 consensus_queue.cc:260] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:43.304744  2080 raft_consensus.cc:3060] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.308946  2011 raft_consensus.cc:2468] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:43.310913  1901 leader_election.cc:304] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:43.311719  2080 raft_consensus.cc:2468] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:43.311833  2813 raft_consensus.cc:2804] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:43.312407  2813 raft_consensus.cc:697] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:43.313271  2813 consensus_queue.cc:237] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:43.318125  2818 raft_consensus.cc:993] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:43.318604  2815 raft_consensus.cc:1081] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:43.320029  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53350
I20260709 12:28:43.320611  2685 raft_consensus.cc:493] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:43.320915  2685 raft_consensus.cc:3060] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.323625  1758 catalog_manager.cc:5697] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 1 to 2, leader changed from 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196) to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 2 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:43.326582  2685 raft_consensus.cc:515] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:43.328264  2685 leader_election.cc:290] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:43.329308  2080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:43.329926  2080 raft_consensus.cc:3055] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:43.329849  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5128bcc1e7cb49009f2b376e8fb5e125" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:43.330386  2080 raft_consensus.cc:740] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5edb06eaed8e41868299eb62ade6eef0, State: Running, Role: LEADER
I20260709 12:28:43.330570  2011 raft_consensus.cc:3060] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.331336  2080 consensus_queue.cc:260] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:43.332564  2080 raft_consensus.cc:3060] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:43.337085  2011 raft_consensus.cc:2468] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:43.338140  2636 leader_election.cc:304] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c59f6f4b90d460e832bcff010c18400, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:43.338944  2825 raft_consensus.cc:2804] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:43.339162  2080 raft_consensus.cc:2468] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:43.339521  2825 raft_consensus.cc:697] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:43.340615  2825 consensus_queue.cc:237] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:43.357587  1758 catalog_manager.cc:5697] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:43.431496  1931 raft_consensus.cc:1275] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.432818  2825 consensus_queue.cc:1048] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.444796  1934 raft_consensus.cc:1275] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.446755  2826 consensus_queue.cc:1048] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:43.457434  2080 raft_consensus.cc:1275] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.471028  2825 consensus_queue.cc:1048] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.477110  2085 raft_consensus.cc:1275] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 3 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:43.479033  2825 consensus_queue.cc:1048] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:43.505311  2084 raft_consensus.cc:1275] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.507508  2826 consensus_queue.cc:1048] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.511858  2011 raft_consensus.cc:1275] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.514348  2840 consensus_queue.cc:1048] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:43.571470  1931 raft_consensus.cc:1275] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.573236  2840 consensus_queue.cc:1048] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.590211  1931 raft_consensus.cc:1275] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 3 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:43.591869  2845 consensus_queue.cc:1048] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:43.865283  2007 raft_consensus.cc:1275] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.866603  2817 consensus_queue.cc:1048] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.869285  2084 raft_consensus.cc:1275] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.870726  2817 consensus_queue.cc:1048] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.893242  2007 raft_consensus.cc:1275] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.895256  2840 consensus_queue.cc:1048] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.896733  2084 raft_consensus.cc:1275] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:43.898070  2823 consensus_queue.cc:1048] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:43.910977 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:43.967080  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a8c550169674d72af0524338f68fbee"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:43.967679  2007 raft_consensus.cc:606] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:43.970031  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d4f181266414f459ea67d333f044205"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:43.970463  2007 raft_consensus.cc:606] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:43.973053  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "3e95870e8ff04dc7ab920b8917898eee"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:43.973546  2007 raft_consensus.cc:606] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 3e95870e8ff04dc7ab920b8917898eee
I20260709 12:28:43.975595  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:43.976027  2007 raft_consensus.cc:606] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:43.978106  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:43.978554  2007 raft_consensus.cc:606] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:43.979632 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:43.981082 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:43.981838 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:44.041036  2816 raft_consensus.cc:993] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.041620  2809 raft_consensus.cc:1081] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.043421  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:44.044052  2685 raft_consensus.cc:493] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.044445  2685 raft_consensus.cc:3060] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.050043  2685 raft_consensus.cc:515] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:44.051649  2685 leader_election.cc:290] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:44.052620  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:44.052436  1932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edae6c4094614367a5bd04271ce7b99c" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:44.053139  2007 raft_consensus.cc:3055] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:44.053251  1932 raft_consensus.cc:3060] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.053573  2007 raft_consensus.cc:740] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:44.054563  2007 consensus_queue.cc:260] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:44.056449  2007 raft_consensus.cc:3060] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.060814  1932 raft_consensus.cc:2468] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.062081  2636 leader_election.cc:304] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:44.063060  2846 raft_consensus.cc:2804] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:44.063517  2007 raft_consensus.cc:2468] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.063591  2846 raft_consensus.cc:697] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:44.065310  2846 consensus_queue.cc:237] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:44.076812  1758 catalog_manager.cc:5697] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.175217  2860 raft_consensus.cc:993] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.175720  2860 raft_consensus.cc:1081] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.177395  2689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d4f181266414f459ea67d333f044205"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:44.177831  2689 raft_consensus.cc:493] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.178092  2689 raft_consensus.cc:3060] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.183395  2689 raft_consensus.cc:515] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:44.185096  2689 leader_election.cc:290] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:44.185907  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:44.186192  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f181266414f459ea67d333f044205" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:44.186584  2007 raft_consensus.cc:3055] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:44.186721  2084 raft_consensus.cc:3060] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.186844  2007 raft_consensus.cc:740] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:44.187601  2007 consensus_queue.cc:260] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:44.188539  2007 raft_consensus.cc:3060] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.193233  2084 raft_consensus.cc:2468] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.194416  2638 leader_election.cc:304] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:44.194965  2007 raft_consensus.cc:2468] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.195102  2846 raft_consensus.cc:2804] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:44.195736  2846 raft_consensus.cc:697] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:44.196645  2846 consensus_queue.cc:237] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:44.206570  1758 catalog_manager.cc:5697] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:44.299014  2816 raft_consensus.cc:993] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.299573  2860 raft_consensus.cc:1081] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.301632  2689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:44.302111  2689 raft_consensus.cc:493] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.302408  2689 raft_consensus.cc:3060] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.309451  2689 raft_consensus.cc:515] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:44.311568  2689 leader_election.cc:290] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:44.312498  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:44.312753  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4414b0202fb4cd58bc7e1b4249efe0a" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:44.313223  2007 raft_consensus.cc:3055] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:44.313383  2084 raft_consensus.cc:3060] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.313550  2007 raft_consensus.cc:740] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:44.314218  2007 consensus_queue.cc:260] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:44.315246  2007 raft_consensus.cc:3060] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.318477  2084 raft_consensus.cc:2468] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.319619  2638 leader_election.cc:304] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:44.320299  2846 raft_consensus.cc:2804] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:44.320374  2007 raft_consensus.cc:2468] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.320741  2846 raft_consensus.cc:697] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:44.321730  2846 consensus_queue.cc:237] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:44.329919  2860 raft_consensus.cc:993] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 3e95870e8ff04dc7ab920b8917898eee to start an election
I20260709 12:28:44.330549  2824 raft_consensus.cc:1081] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 3e95870e8ff04dc7ab920b8917898eee to start an election
I20260709 12:28:44.332252  1932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691"
dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
 from {username='slave'} at 127.0.0.1:53540
I20260709 12:28:44.332724  1932 raft_consensus.cc:493] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.333047  1932 raft_consensus.cc:3060] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.340557  1932 raft_consensus.cc:515] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:44.343015  1932 leader_election.cc:290] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:44.344131  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:44.344005  1758 catalog_manager.cc:5697] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:44.344671  2007 raft_consensus.cc:3055] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:44.345060  2007 raft_consensus.cc:740] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:44.345672  2007 consensus_queue.cc:260] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:44.346665  2007 raft_consensus.cc:3060] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.355602  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "985c695826744eb693cfa90b8a3f3691" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:44.356226  2084 raft_consensus.cc:3060] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.357334  2007 raft_consensus.cc:2468] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:44.360124  1901 leader_election.cc:304] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8e43e9e956f14e52897d2efc3f361630; no voters: 
I20260709 12:28:44.361147  2813 raft_consensus.cc:2804] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:44.361574  2813 raft_consensus.cc:697] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:44.362725  2084 raft_consensus.cc:2468] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:44.362470  2813 consensus_queue.cc:237] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:44.374058  1758 catalog_manager.cc:5697] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 2 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.458626  2824 raft_consensus.cc:993] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.459103  2816 raft_consensus.cc:1081] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:44.460644  2689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a8c550169674d72af0524338f68fbee"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:44.461052  2689 raft_consensus.cc:493] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.461282  2689 raft_consensus.cc:3060] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.466401  2689 raft_consensus.cc:515] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:44.467854  2689 leader_election.cc:290] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:44.468586  1932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:44.468835  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8c550169674d72af0524338f68fbee" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:44.469245  1932 raft_consensus.cc:3060] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.469355  2007 raft_consensus.cc:3055] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:44.469761  2007 raft_consensus.cc:740] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:44.470592  2007 consensus_queue.cc:260] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:44.471869  2007 raft_consensus.cc:3060] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.476275  1932 raft_consensus.cc:2468] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.477448  2636 leader_election.cc:304] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:44.478296  2846 raft_consensus.cc:2804] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:44.478446  2007 raft_consensus.cc:2468] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:44.478832  2846 raft_consensus.cc:697] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:44.479838  2846 consensus_queue.cc:237] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:44.489012  1758 catalog_manager.cc:5697] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.541029  1932 raft_consensus.cc:1275] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.542584  2846 consensus_queue.cc:1048] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.558141  2007 raft_consensus.cc:1275] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.563364  2840 consensus_queue.cc:1048] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.619506  2007 raft_consensus.cc:1275] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.620896  2846 consensus_queue.cc:1048] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.631484  2084 raft_consensus.cc:1275] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.632815  2870 consensus_queue.cc:1048] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.781599  2007 raft_consensus.cc:1275] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.783032  2870 consensus_queue.cc:1048] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.793426  2084 raft_consensus.cc:1275] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.799553  2840 consensus_queue.cc:1048] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.825179  2007 raft_consensus.cc:1275] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.826656  2841 consensus_queue.cc:1048] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.837016  2084 raft_consensus.cc:1275] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.839252  2841 consensus_queue.cc:1048] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.903499  2007 raft_consensus.cc:1275] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.904989  2870 consensus_queue.cc:1048] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:44.921723  1934 raft_consensus.cc:1275] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:44.923871  2840 consensus_queue.cc:1048] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.018748 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:45.063405  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:45.063853  2007 raft_consensus.cc:606] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:45.075503  1932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5"
dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:33990
I20260709 12:28:45.076153  1932 raft_consensus.cc:606] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:45.078784  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:45.079327  2007 raft_consensus.cc:606] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:45.081877  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:45.082387  2007 raft_consensus.cc:606] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:45.085207  2007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f"
dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
mode: GRACEFUL
new_leader_uuid: "3e95870e8ff04dc7ab920b8917898eee"
 from {username='slave'} at 127.0.0.1:40328
I20260709 12:28:45.085700  2007 raft_consensus.cc:606] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Received request to transfer leadership to TS 3e95870e8ff04dc7ab920b8917898eee
I20260709 12:28:45.086831 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:45.088114 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:45.088754 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:45.131752  2824 raft_consensus.cc:993] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 3e95870e8ff04dc7ab920b8917898eee to start an election
I20260709 12:28:45.131970  2817 raft_consensus.cc:993] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.132361  2816 raft_consensus.cc:1081] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 3e95870e8ff04dc7ab920b8917898eee to start an election
I20260709 12:28:45.132634  2817 raft_consensus.cc:1081] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.134156  1934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f"
dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
 from {username='slave'} at 127.0.0.1:53540
I20260709 12:28:45.134265  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53374
I20260709 12:28:45.134672  1934 raft_consensus.cc:493] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.134950  2685 raft_consensus.cc:493] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.135052  1934 raft_consensus.cc:3060] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.135465  2685 raft_consensus.cc:3060] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.141587  2685 raft_consensus.cc:515] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:45.141736  1934 raft_consensus.cc:515] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:45.144030  2685 leader_election.cc:290] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:45.153990  1934 leader_election.cc:290] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:45.144091  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:45.148099  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcdf235113b34d5f8237b06b7662479f" candidate_uuid: "3e95870e8ff04dc7ab920b8917898eee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:45.144897  1932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:45.162468  2007 raft_consensus.cc:3055] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.162806  1932 raft_consensus.cc:3055] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.162539  2084 raft_consensus.cc:3060] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.163400  1932 raft_consensus.cc:740] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:45.164109  2007 raft_consensus.cc:740] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:45.164604  1932 consensus_queue.cc:260] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:45.165030  2007 consensus_queue.cc:260] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:45.144091  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "880a5a05cb024c17b64d7ef2621311c5" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:45.166529  2007 raft_consensus.cc:3060] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.167019  2085 raft_consensus.cc:3060] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.166200  1932 raft_consensus.cc:3060] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.171866  2084 raft_consensus.cc:2468] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:45.176220  2085 raft_consensus.cc:2468] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.176837  2007 raft_consensus.cc:2468] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e95870e8ff04dc7ab920b8917898eee in term 2.
I20260709 12:28:45.177271  1903 leader_election.cc:304] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 5edb06eaed8e41868299eb62ade6eef0; no voters: 
I20260709 12:28:45.178881  2638 leader_election.cc:304] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:45.179827  2870 raft_consensus.cc:2804] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.180307  2870 raft_consensus.cc:697] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:45.181216  2870 consensus_queue.cc:237] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:45.183887  1932 raft_consensus.cc:2468] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.184456  2817 raft_consensus.cc:2804] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.184995  2817 raft_consensus.cc:697] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Becoming Leader. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:45.185875  2817 consensus_queue.cc:237] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } }
I20260709 12:28:45.203751  2885 raft_consensus.cc:993] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.204360  2885 raft_consensus.cc:1081] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.207041  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:45.207623  2685 raft_consensus.cc:493] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.207979  2685 raft_consensus.cc:3060] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.222743  2685 raft_consensus.cc:515] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:45.228422  2685 leader_election.cc:290] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:45.231375  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:45.232132  2085 raft_consensus.cc:3060] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.232414  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8e346d58d44c1dbcf74cd388a21904" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:45.233006  2007 raft_consensus.cc:3055] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.233294  2007 raft_consensus.cc:740] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:45.233986  2007 consensus_queue.cc:260] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:45.235167  2007 raft_consensus.cc:3060] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.234967  1756 catalog_manager.cc:5697] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.238997  2085 raft_consensus.cc:2468] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.240427  2638 leader_election.cc:304] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:45.239084  1758 catalog_manager.cc:5697] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194). New cstate: current_term: 2 leader_uuid: "3e95870e8ff04dc7ab920b8917898eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.241253  2845 raft_consensus.cc:2804] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.241732  2845 raft_consensus.cc:697] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:45.241806  2007 raft_consensus.cc:2468] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.242612  2845 consensus_queue.cc:237] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:45.272810  1758 catalog_manager.cc:5697] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:45.293164  2824 raft_consensus.cc:993] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.293658  2885 raft_consensus.cc:1081] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.294992  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:45.295446  2685 raft_consensus.cc:493] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.295745  2685 raft_consensus.cc:3060] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.300390  2685 raft_consensus.cc:515] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:45.302493  2007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:45.302165  2685 leader_election.cc:290] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583), 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785)
I20260709 12:28:45.302662  2085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b4fd397d24490f8e5ad99e1a217e66" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:45.303157  2007 raft_consensus.cc:3055] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.303408  2085 raft_consensus.cc:3060] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.303552  2007 raft_consensus.cc:740] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:45.304358  2007 consensus_queue.cc:260] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:45.305660  2007 raft_consensus.cc:3060] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.310487  2085 raft_consensus.cc:2468] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.311643  2638 leader_election.cc:304] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:45.312549  2870 raft_consensus.cc:2804] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.313275  2870 raft_consensus.cc:697] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:45.314143  2870 consensus_queue.cc:237] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } }
I20260709 12:28:45.316412  2007 raft_consensus.cc:2468] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.327821  1758 catalog_manager.cc:5697] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:45.374699  2824 raft_consensus.cc:993] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.375243  2885 raft_consensus.cc:1081] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:45.376608  2685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53362
I20260709 12:28:45.377058  2685 raft_consensus.cc:493] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.377331  2685 raft_consensus.cc:3060] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.382921  2685 raft_consensus.cc:515] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:45.384922  2685 leader_election.cc:290] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535), 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583)
I20260709 12:28:45.385847  1934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:45.385998  2011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979e0700017a488aa7012c1d228deeb1" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8e43e9e956f14e52897d2efc3f361630"
I20260709 12:28:45.386560  1934 raft_consensus.cc:3060] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.386788  2011 raft_consensus.cc:3055] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.387131  2011 raft_consensus.cc:740] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e43e9e956f14e52897d2efc3f361630, State: Running, Role: LEADER
I20260709 12:28:45.387926  2011 consensus_queue.cc:260] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:45.389302  2011 raft_consensus.cc:3060] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.392844  1934 raft_consensus.cc:2468] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.394059  2636 leader_election.cc:304] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:45.394820  2870 raft_consensus.cc:2804] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.396526  2870 raft_consensus.cc:697] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:45.396648  2011 raft_consensus.cc:2468] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:45.397478  2870 consensus_queue.cc:237] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } }
I20260709 12:28:45.408145  1758 catalog_manager.cc:5697] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.621997  2084 raft_consensus.cc:1275] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.623348  2870 consensus_queue.cc:1048] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.626900  1932 raft_consensus.cc:1275] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.628796  2845 consensus_queue.cc:1048] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.695509  2011 raft_consensus.cc:1275] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.696798  2870 consensus_queue.cc:1048] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.707724  2082 raft_consensus.cc:1275] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.709038  2870 consensus_queue.cc:1048] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.754999  2082 raft_consensus.cc:1275] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.756256  2845 consensus_queue.cc:1048] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.767728  2007 raft_consensus.cc:1275] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.769237  2845 consensus_queue.cc:1048] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.804080  2007 raft_consensus.cc:1275] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.805127  2817 consensus_queue.cc:1048] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.809075  2082 raft_consensus.cc:1275] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 3e95870e8ff04dc7ab920b8917898eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.810369  2813 consensus_queue.cc:1048] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.867887  2007 raft_consensus.cc:1275] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.869208  2845 consensus_queue.cc:1048] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e43e9e956f14e52897d2efc3f361630" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 39583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.880566  1932 raft_consensus.cc:1275] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.884557  2845 consensus_queue.cc:1048] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:46.123961 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:46.162163  1932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04"
dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
mode: GRACEFUL
new_leader_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:33990
I20260709 12:28:46.162672  1932 raft_consensus.cc:606] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Received request to transfer leadership to TS 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:46.163935 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:28:46.165087 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:46.165675 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:46.190744  2817 raft_consensus.cc:993] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee: : Instructing follower 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:46.191365  2902 raft_consensus.cc:1081] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Signalling peer 8c59f6f4b90d460e832bcff010c18400 to start an election
I20260709 12:28:46.193051  2689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04"
dest_uuid: "8c59f6f4b90d460e832bcff010c18400"
 from {username='slave'} at 127.0.0.1:53374
I20260709 12:28:46.193583  2689 raft_consensus.cc:493] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:46.193938  2689 raft_consensus.cc:3060] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:46.202826  2689 raft_consensus.cc:515] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:46.205022  2084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5edb06eaed8e41868299eb62ade6eef0"
I20260709 12:28:46.205587  2084 raft_consensus.cc:3060] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:46.208515  2689 leader_election.cc:290] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Requested vote from peers 5edb06eaed8e41868299eb62ade6eef0 (127.19.36.196:37785), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
I20260709 12:28:46.209689  1932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4773e370acbd4490afddcc7393f8be04" candidate_uuid: "8c59f6f4b90d460e832bcff010c18400" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e95870e8ff04dc7ab920b8917898eee"
I20260709 12:28:46.210340  1932 raft_consensus.cc:3055] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:46.210618  1932 raft_consensus.cc:740] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e95870e8ff04dc7ab920b8917898eee, State: Running, Role: LEADER
I20260709 12:28:46.211373  1932 consensus_queue.cc:260] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:46.211809  2084 raft_consensus.cc:2468] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:46.212620  1932 raft_consensus.cc:3060] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:46.212847  2638 leader_election.cc:304] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0, 8c59f6f4b90d460e832bcff010c18400; no voters: 
I20260709 12:28:46.213686  2846 raft_consensus.cc:2804] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:46.214229  2846 raft_consensus.cc:697] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Becoming Leader. State: Replica: 8c59f6f4b90d460e832bcff010c18400, State: Running, Role: LEADER
I20260709 12:28:46.215106  2846 consensus_queue.cc:237] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:46.219295  1932 raft_consensus.cc:2468] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c59f6f4b90d460e832bcff010c18400 in term 2.
I20260709 12:28:46.226671  1758 catalog_manager.cc:5697] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 reported cstate change: term changed from 1 to 2, leader changed from 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194) to 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "8c59f6f4b90d460e832bcff010c18400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:46.619117  2084 raft_consensus.cc:1275] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:46.620316  2845 consensus_queue.cc:1048] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:46.636248  1932 raft_consensus.cc:1275] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Refusing update from remote peer 8c59f6f4b90d460e832bcff010c18400: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:46.638250  2846 consensus_queue.cc:1048] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:47.208911 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:47.251860 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:47.253159 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:47.253595 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:47.288760 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:47.319979 19603 tablet_server.cc:179] TabletServer@127.19.36.193:35551 shutting down...
W20260709 12:28:47.374468  1901 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 115 similar messages]
W20260709 12:28:47.379496  1901 consensus_peers.cc:597] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.380082  1901 consensus_peers.cc:597] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.387156  1901 consensus_peers.cc:597] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.403649  1975 consensus_peers.cc:597] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.407389 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:47.409345 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.410348 19603 raft_consensus.cc:2243] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.411509 19603 raft_consensus.cc:2272] T e2645174b413446fbf4c9423465821be P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.413950  2049 consensus_peers.cc:597] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.415980 19603 tablet_replica.cc:333] stopping tablet replica
W20260709 12:28:47.416817  2049 consensus_peers.cc:597] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.416949 19603 raft_consensus.cc:2243] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.417778 19603 raft_consensus.cc:2272] T 70ee2ecd6eef424bb1bf355492553d8e P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.419803 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.420373 19603 raft_consensus.cc:2243] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.420807 19603 raft_consensus.cc:2272] T 91363564e93f4297b4e825de5b7e5176 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.423404 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.424206 19603 raft_consensus.cc:2243] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.424685 19603 raft_consensus.cc:2272] T 4631b580b30e4f57b7a7a09b2452dc75 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.426065  2049 consensus_peers.cc:597] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.427438 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.428124 19603 raft_consensus.cc:2243] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.428521 19603 raft_consensus.cc:2272] T 3a6b8329d4924908b3a364b840d691fe P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.430917 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.431537 19603 raft_consensus.cc:2243] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.432219 19603 raft_consensus.cc:2272] T 90834683513744559ef8f7efe530f459 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.434948 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.435770 19603 raft_consensus.cc:2243] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.436651 19603 raft_consensus.cc:2272] T 2e8e346d58d44c1dbcf74cd388a21904 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.439208 19603 tablet_replica.cc:333] stopping tablet replica
W20260709 12:28:47.439791  1975 consensus_peers.cc:597] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.439998 19603 raft_consensus.cc:2243] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.441030 19603 raft_consensus.cc:2272] T dfd2a42648874aef8ca5935e76f879c5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.443295 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.443899 19603 raft_consensus.cc:2243] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.444633 19603 raft_consensus.cc:2272] T d17fb96cd6f94ffa9824eb3868a5fe92 P 8c59f6f4b90d460e832bcff010c18400 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.446621 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.447250 19603 raft_consensus.cc:2243] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.447786 19603 raft_consensus.cc:2272] T f6241be73e134c25909f8054db9f85dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.450739 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.451660 19603 raft_consensus.cc:2243] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.452132 19603 raft_consensus.cc:2272] T ac43f6f37d474da58e0a962b25fe9235 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.454768 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.455474 19603 raft_consensus.cc:2243] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.455955 19603 raft_consensus.cc:2272] T cf093503636049fbb48b9a304d578c38 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.457130  1901 consensus_peers.cc:597] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.458557 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.459583 19603 raft_consensus.cc:2243] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.460028 19603 raft_consensus.cc:2272] T baf92d63594b4e3081ad94cac31690d1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.462271 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.462814 19603 raft_consensus.cc:2243] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.463234 19603 raft_consensus.cc:2272] T 8a38771542bf449280983d8f709895dc P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.465291 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.465843 19603 raft_consensus.cc:2243] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.466281 19603 raft_consensus.cc:2272] T bf128bd1b4b546dab8e2c0504d06cb87 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.468294 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.468897 19603 raft_consensus.cc:2243] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.469305 19603 raft_consensus.cc:2272] T 179c67e242914e64a56c8d1f829cdef5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.471517 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.472285 19603 raft_consensus.cc:2243] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.473189 19603 raft_consensus.cc:2272] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.476176 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.476913 19603 raft_consensus.cc:2243] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.477746 19603 raft_consensus.cc:2272] T e4414b0202fb4cd58bc7e1b4249efe0a P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.480268 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.481045 19603 raft_consensus.cc:2243] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.481879 19603 raft_consensus.cc:2272] T 8d082f4dbaeb47959b209411cd69957b P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.484782 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.485625 19603 raft_consensus.cc:2243] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.486158 19603 raft_consensus.cc:2272] T b385e95426fd4870b36475731ab7d0b8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.489099  2049 consensus_peers.cc:597] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.489351 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.490187 19603 raft_consensus.cc:2243] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.490594 19603 raft_consensus.cc:2272] T 3d0af4a569f84ef9841ad0c788c5d6ca P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.492795 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.493332 19603 raft_consensus.cc:2243] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.493695 19603 raft_consensus.cc:2272] T 5ca90853e7a245c090a24e9f2b6b88e1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.495867 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.496625 19603 raft_consensus.cc:2243] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.497097 19603 raft_consensus.cc:2272] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.499866 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.500546 19603 raft_consensus.cc:2243] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.501534 19603 raft_consensus.cc:2272] T 1d4f181266414f459ea67d333f044205 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.504542 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.505458 19603 raft_consensus.cc:2243] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:47.505481  1901 consensus_peers.cc:597] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.506096 19603 raft_consensus.cc:2272] T 891bc4cc61ee4451a2cb82d28286d5f8 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.509100 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.509872 19603 raft_consensus.cc:2243] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.510356 19603 raft_consensus.cc:2272] T 29f59bee90544d80a4b32bfcd1116d4a P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.513202 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.513891 19603 raft_consensus.cc:2243] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.514420 19603 raft_consensus.cc:2272] T 11795a035a1e482d88901e8aea8ee185 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.517405 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.518208 19603 raft_consensus.cc:2243] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.518769 19603 raft_consensus.cc:2272] T 466fe5ff431c463a84050bd3e265a246 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.521760  2049 consensus_peers.cc:597] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.521874 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.522848 19603 raft_consensus.cc:2243] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.523723 19603 raft_consensus.cc:2272] T 979e0700017a488aa7012c1d228deeb1 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.526160 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.526671 19603 raft_consensus.cc:2243] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.527055 19603 raft_consensus.cc:2272] T 6146e39d5a724f05b9b3cbc7a8145d98 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.529443 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.530174 19603 raft_consensus.cc:2243] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.530704 19603 raft_consensus.cc:2272] T 32982b9d743f4ceb80f17292b281fd6f P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.533041 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.533546 19603 raft_consensus.cc:2243] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.533959 19603 raft_consensus.cc:2272] T 2c7707b908f046398ca5ee4defefe55f P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.536128 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.536823 19603 raft_consensus.cc:2243] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.537788 19603 raft_consensus.cc:2272] T 4773e370acbd4490afddcc7393f8be04 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.539654  1975 consensus_peers.cc:597] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.539687  2049 consensus_peers.cc:597] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.541194 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.541863 19603 raft_consensus.cc:2243] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.542625 19603 raft_consensus.cc:2272] T b1b4fd397d24490f8e5ad99e1a217e66 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.544701 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.545251 19603 raft_consensus.cc:2243] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.545619 19603 raft_consensus.cc:2272] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.547787 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.548509 19603 raft_consensus.cc:2243] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.548950 19603 raft_consensus.cc:2272] T 2c6aef41b0274acb8e13d886aa250805 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.550678  1975 consensus_peers.cc:597] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.551273 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.551911 19603 raft_consensus.cc:2243] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.552314 19603 raft_consensus.cc:2272] T 172b29319c78428499e095bf5e76b74e P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.554308 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.554854 19603 raft_consensus.cc:2243] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.555632 19603 raft_consensus.cc:2272] T 0a8c550169674d72af0524338f68fbee P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.557602 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.558108 19603 raft_consensus.cc:2243] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.558508 19603 raft_consensus.cc:2272] T ee564988f3774725bff3cd527794c6a1 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.561506 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.562222 19603 raft_consensus.cc:2243] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.563773 19603 raft_consensus.cc:2272] T eb6c6f10db0b41ae8960133a8573f0b5 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.566114 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.566612 19603 raft_consensus.cc:2243] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.566934 19603 raft_consensus.cc:2272] T db6a2096401b4cec9b542b6eb99f705b P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.568897 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.569514 19603 raft_consensus.cc:2243] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.569962 19603 raft_consensus.cc:2272] T bbf8686b20194e3981d420491c825046 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.572711 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.573299 19603 raft_consensus.cc:2243] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.573992 19603 raft_consensus.cc:2272] T 880a5a05cb024c17b64d7ef2621311c5 P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.576135 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.576643 19603 raft_consensus.cc:2243] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.577076 19603 raft_consensus.cc:2272] T e922d7ff58764a529d119fa6491f1768 P 8c59f6f4b90d460e832bcff010c18400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.579586 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:47.580029 19603 raft_consensus.cc:2243] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.580709 19603 raft_consensus.cc:2272] T edae6c4094614367a5bd04271ce7b99c P 8c59f6f4b90d460e832bcff010c18400 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.611631  2049 consensus_peers.cc:597] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.624521  2049 consensus_peers.cc:597] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.649819  1975 consensus_peers.cc:597] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.651571  1901 consensus_peers.cc:597] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.655256  1975 consensus_peers.cc:597] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.657698 19603 tablet_server.cc:196] TabletServer@127.19.36.193:35551 shutdown complete.
W20260709 12:28:47.665939  1901 consensus_peers.cc:597] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.671614  2049 consensus_peers.cc:597] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.674482  1975 consensus_peers.cc:597] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.682577  1975 consensus_peers.cc:597] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.687623  1975 consensus_peers.cc:597] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.702464  2049 consensus_peers.cc:597] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.716246  2049 consensus_peers.cc:597] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.747053 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
W20260709 12:28:47.754479  2049 consensus_peers.cc:597] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.762025  2049 consensus_peers.cc:597] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.762571  1901 consensus_peers.cc:597] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.770845  1975 consensus_peers.cc:597] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.784981  2049 consensus_peers.cc:597] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.799857  1975 consensus_peers.cc:597] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.805289  2049 consensus_peers.cc:597] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.809806 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:47.810561 19603 tablet_replica.cc:333] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.811273 19603 raft_consensus.cc:2243] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.811827 19603 raft_consensus.cc:2272] T 736e3203de274189acfaef187bfea762 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.814324 19603 tablet_replica.cc:333] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.814993 19603 raft_consensus.cc:2243] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.815552 19603 raft_consensus.cc:2272] T d9e0977a3ed1476f871be3d3a77ae4b2 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.818598  1975 consensus_peers.cc:597] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 -> Peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Couldn't send request to peer 8c59f6f4b90d460e832bcff010c18400. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.820938 19603 tablet_replica.cc:333] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.821928 19603 raft_consensus.cc:2243] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.822453 19603 raft_consensus.cc:2272] T 90834683513744559ef8f7efe530f459 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.823820  1975 consensus_peers.cc:597] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.825037 19603 tablet_replica.cc:333] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.825858 19603 raft_consensus.cc:2243] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.826246 19603 raft_consensus.cc:2272] T 4631b580b30e4f57b7a7a09b2452dc75 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.828231 19603 tablet_replica.cc:333] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.828920 19603 raft_consensus.cc:2243] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.829363 19603 raft_consensus.cc:2272] T 5ca90853e7a245c090a24e9f2b6b88e1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.832127 19603 tablet_replica.cc:333] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.833031 19603 raft_consensus.cc:2243] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.833532 19603 raft_consensus.cc:2272] T 3d0af4a569f84ef9841ad0c788c5d6ca P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.835258  1975 consensus_peers.cc:597] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.835916 19603 tablet_replica.cc:333] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.836519 19603 raft_consensus.cc:2243] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.836879 19603 raft_consensus.cc:2272] T 0a8c550169674d72af0524338f68fbee P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.838753 19603 tablet_replica.cc:333] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.839339 19603 raft_consensus.cc:2243] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.839707 19603 raft_consensus.cc:2272] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.841607 19603 tablet_replica.cc:333] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.842134 19603 raft_consensus.cc:2243] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.842895 19603 raft_consensus.cc:2272] T 5ebf212c059342559ae4781db6ba8e83 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.844725 19603 tablet_replica.cc:333] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.845232 19603 raft_consensus.cc:2243] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.845592 19603 raft_consensus.cc:2272] T 3a6b8329d4924908b3a364b840d691fe P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.847388 19603 tablet_replica.cc:333] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.848065 19603 raft_consensus.cc:2243] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.848500 19603 raft_consensus.cc:2272] T c5ff50d36854429790ff6c78da6f2c8b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.850523 19603 tablet_replica.cc:333] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.851013 19603 raft_consensus.cc:2243] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.851380 19603 raft_consensus.cc:2272] T 179c67e242914e64a56c8d1f829cdef5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.853216 19603 tablet_replica.cc:333] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.853698 19603 raft_consensus.cc:2243] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.854048 19603 raft_consensus.cc:2272] T d17fb96cd6f94ffa9824eb3868a5fe92 P 3e95870e8ff04dc7ab920b8917898eee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.855916 19603 tablet_replica.cc:333] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.856681 19603 raft_consensus.cc:2243] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.857194 19603 raft_consensus.cc:2272] T 70ee2ecd6eef424bb1bf355492553d8e P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.859315 19603 tablet_replica.cc:333] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
W20260709 12:28:47.859475  1975 consensus_peers.cc:597] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.860005 19603 raft_consensus.cc:2243] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.860466 19603 raft_consensus.cc:2272] T edae6c4094614367a5bd04271ce7b99c P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.862290 19603 tablet_replica.cc:333] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.862778 19603 raft_consensus.cc:2243] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.863557 19603 raft_consensus.cc:2272] T e922d7ff58764a529d119fa6491f1768 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.865727 19603 tablet_replica.cc:333] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.866489 19603 raft_consensus.cc:2243] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.867270 19603 raft_consensus.cc:2272] T dcdf235113b34d5f8237b06b7662479f P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.867856  1975 consensus_peers.cc:597] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.869251 19603 tablet_replica.cc:333] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.869894 19603 raft_consensus.cc:2243] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.870304 19603 raft_consensus.cc:2272] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.872502 19603 tablet_replica.cc:333] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.873142 19603 raft_consensus.cc:2243] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.873663 19603 raft_consensus.cc:2272] T 8d082f4dbaeb47959b209411cd69957b P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.876183 19603 tablet_replica.cc:333] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.876921 19603 raft_consensus.cc:2243] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.877372 19603 raft_consensus.cc:2272] T 4773e370acbd4490afddcc7393f8be04 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.877525  2049 consensus_peers.cc:597] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.879673 19603 tablet_replica.cc:333] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.880242 19603 raft_consensus.cc:2243] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.881018 19603 raft_consensus.cc:2272] T 78d17658b0174e638c0041193b99bf82 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.883894 19603 tablet_replica.cc:333] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.884651 19603 raft_consensus.cc:2243] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.885233 19603 raft_consensus.cc:2272] T db6a2096401b4cec9b542b6eb99f705b P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.887989 19603 tablet_replica.cc:333] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.888713 19603 raft_consensus.cc:2243] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.889176 19603 raft_consensus.cc:2272] T 880a5a05cb024c17b64d7ef2621311c5 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.891812 19603 tablet_replica.cc:333] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.892537 19603 raft_consensus.cc:2243] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.893546 19603 raft_consensus.cc:2272] T bbf8686b20194e3981d420491c825046 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.896417 19603 tablet_replica.cc:333] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.897200 19603 raft_consensus.cc:2243] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.897735 19603 raft_consensus.cc:2272] T 466fe5ff431c463a84050bd3e265a246 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.900368 19603 tablet_replica.cc:333] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.901118 19603 raft_consensus.cc:2243] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.902133 19603 raft_consensus.cc:2272] T 812870f8eb264d3caf48fefa3d70cfa1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.904874 19603 tablet_replica.cc:333] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.905615 19603 raft_consensus.cc:2243] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.906653 19603 raft_consensus.cc:2272] T 8b2a8fdea8624729ae4b5191c85fb520 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.909137 19603 tablet_replica.cc:333] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.909807 19603 raft_consensus.cc:2243] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.910326 19603 raft_consensus.cc:2272] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.913061 19603 tablet_replica.cc:333] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.913770 19603 raft_consensus.cc:2243] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.914285 19603 raft_consensus.cc:2272] T ed23ad21c29f479f8dc03b1a48fd9061 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.916981 19603 tablet_replica.cc:333] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.917719 19603 raft_consensus.cc:2243] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.918681 19603 raft_consensus.cc:2272] T 985c695826744eb693cfa90b8a3f3691 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.921324 19603 tablet_replica.cc:333] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.922044 19603 raft_consensus.cc:2243] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.923094 19603 raft_consensus.cc:2272] T ee564988f3774725bff3cd527794c6a1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.925858 19603 tablet_replica.cc:333] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.926618 19603 raft_consensus.cc:2243] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.927176 19603 raft_consensus.cc:2272] T eb6c6f10db0b41ae8960133a8573f0b5 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.930298 19603 tablet_replica.cc:333] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.931025 19603 raft_consensus.cc:2243] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.932010 19603 raft_consensus.cc:2272] T baf92d63594b4e3081ad94cac31690d1 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.934636 19603 tablet_replica.cc:333] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.935434 19603 raft_consensus.cc:2243] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.936100 19603 raft_consensus.cc:2272] T e2645174b413446fbf4c9423465821be P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.938771 19603 tablet_replica.cc:333] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.939548 19603 raft_consensus.cc:2243] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.940119 19603 raft_consensus.cc:2272] T 0b15f3e29c654279b3811d5ed11c31a2 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.942804 19603 tablet_replica.cc:333] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.943564 19603 raft_consensus.cc:2243] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.944613 19603 raft_consensus.cc:2272] T 2c6aef41b0274acb8e13d886aa250805 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.947310 19603 tablet_replica.cc:333] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.948045 19603 raft_consensus.cc:2243] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.949087 19603 raft_consensus.cc:2272] T 74e46e6bd8574e31824d103146fc9830 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.951807 19603 tablet_replica.cc:333] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.952524 19603 raft_consensus.cc:2243] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.953543 19603 raft_consensus.cc:2272] T 8a38771542bf449280983d8f709895dc P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.954763  1975 consensus_peers.cc:597] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.957442 19603 tablet_replica.cc:333] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.958259 19603 raft_consensus.cc:2243] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.959290 19603 raft_consensus.cc:2272] T 891bc4cc61ee4451a2cb82d28286d5f8 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.960875  1975 consensus_peers.cc:597] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.961938 19603 tablet_replica.cc:333] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.962548 19603 raft_consensus.cc:2243] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.962987 19603 raft_consensus.cc:2272] T 29f59bee90544d80a4b32bfcd1116d4a P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.965304 19603 tablet_replica.cc:333] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.965830 19603 raft_consensus.cc:2243] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.966573 19603 raft_consensus.cc:2272] T ac43f6f37d474da58e0a962b25fe9235 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.968470 19603 tablet_replica.cc:333] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.968976 19603 raft_consensus.cc:2243] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.969362 19603 raft_consensus.cc:2272] T f2bbe2dde38842ab846f9c2d8a260918 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.971297 19603 tablet_replica.cc:333] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.971760 19603 raft_consensus.cc:2243] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.972157 19603 raft_consensus.cc:2272] T 6146e39d5a724f05b9b3cbc7a8145d98 P 3e95870e8ff04dc7ab920b8917898eee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.974021 19603 tablet_replica.cc:333] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee: stopping tablet replica
I20260709 12:28:47.974459 19603 raft_consensus.cc:2243] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.974857 19603 raft_consensus.cc:2272] T 979e0700017a488aa7012c1d228deeb1 P 3e95870e8ff04dc7ab920b8917898eee [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:48.049224  2049 consensus_peers.cc:597] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.061461 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
W20260709 12:28:48.109918  2049 consensus_peers.cc:597] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.132004  2049 consensus_peers.cc:597] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.134531  1975 consensus_peers.cc:597] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.151686  2049 consensus_peers.cc:597] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.154048 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
W20260709 12:28:48.166642  1975 consensus_peers.cc:597] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.202092  1975 consensus_peers.cc:597] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.202219 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:48.203150 19603 tablet_replica.cc:333] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.203930 19603 raft_consensus.cc:2243] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.204749 19603 raft_consensus.cc:2272] T 736e3203de274189acfaef187bfea762 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.206866 19603 tablet_replica.cc:333] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.207518 19603 raft_consensus.cc:2243] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.208281 19603 raft_consensus.cc:2272] T d9e0977a3ed1476f871be3d3a77ae4b2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.210165 19603 tablet_replica.cc:333] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.210721 19603 raft_consensus.cc:2243] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.211580 19603 raft_consensus.cc:2272] T 4631b580b30e4f57b7a7a09b2452dc75 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.214425 19603 tablet_replica.cc:333] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
W20260709 12:28:48.214664  2049 consensus_peers.cc:597] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.215318 19603 raft_consensus.cc:2243] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.215975 19603 raft_consensus.cc:2272] T 5ebf212c059342559ae4781db6ba8e83 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.218493 19603 tablet_replica.cc:333] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.219164 19603 raft_consensus.cc:2243] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:48.219164  1975 consensus_peers.cc:597] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.219995 19603 raft_consensus.cc:2272] T 91363564e93f4297b4e825de5b7e5176 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.222013 19603 tablet_replica.cc:333] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.222592 19603 raft_consensus.cc:2243] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.223004 19603 raft_consensus.cc:2272] T 0a8c550169674d72af0524338f68fbee P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.224928 19603 tablet_replica.cc:333] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.225481 19603 raft_consensus.cc:2243] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.226239 19603 raft_consensus.cc:2272] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.228502 19603 tablet_replica.cc:333] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.229094 19603 raft_consensus.cc:2243] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.229568 19603 raft_consensus.cc:2272] T edae6c4094614367a5bd04271ce7b99c P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.232120 19603 tablet_replica.cc:333] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
W20260709 12:28:48.232174  2049 consensus_peers.cc:597] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Couldn't send request to peer 3e95870e8ff04dc7ab920b8917898eee. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.232918 19603 raft_consensus.cc:2243] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.233855 19603 raft_consensus.cc:2272] T c5ff50d36854429790ff6c78da6f2c8b P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.235828 19603 tablet_replica.cc:333] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.236328 19603 raft_consensus.cc:2243] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.236989 19603 raft_consensus.cc:2272] T bf128bd1b4b546dab8e2c0504d06cb87 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.238720 19603 tablet_replica.cc:333] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.239230 19603 raft_consensus.cc:2243] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.239619 19603 raft_consensus.cc:2272] T 5128bcc1e7cb49009f2b376e8fb5e125 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.241354 19603 tablet_replica.cc:333] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.241850 19603 raft_consensus.cc:2243] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.242684 19603 raft_consensus.cc:2272] T 179c67e242914e64a56c8d1f829cdef5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.244830 19603 tablet_replica.cc:333] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.245393 19603 raft_consensus.cc:2243] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.245855 19603 raft_consensus.cc:2272] T f6241be73e134c25909f8054db9f85dc P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.247870 19603 tablet_replica.cc:333] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.248427 19603 raft_consensus.cc:2243] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.248896 19603 raft_consensus.cc:2272] T 74e46e6bd8574e31824d103146fc9830 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.251353 19603 tablet_replica.cc:333] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.252108 19603 raft_consensus.cc:2243] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.252660 19603 raft_consensus.cc:2272] T 78d17658b0174e638c0041193b99bf82 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.255330 19603 tablet_replica.cc:333] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.256124 19603 raft_consensus.cc:2243] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.256644 19603 raft_consensus.cc:2272] T 2e8e346d58d44c1dbcf74cd388a21904 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.259093 19603 tablet_replica.cc:333] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
W20260709 12:28:48.259789  2049 consensus_peers.cc:597] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.259878 19603 raft_consensus.cc:2243] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.260675 19603 raft_consensus.cc:2272] T 70ee2ecd6eef424bb1bf355492553d8e P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.263011 19603 tablet_replica.cc:333] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.263820 19603 raft_consensus.cc:2243] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.264295 19603 raft_consensus.cc:2272] T dcdf235113b34d5f8237b06b7662479f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.266821 19603 tablet_replica.cc:333] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.267431 19603 raft_consensus.cc:2243] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.268203 19603 raft_consensus.cc:2272] T 6d5ff55130ab4cbf8e84dc429552a7e8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.270833 19603 tablet_replica.cc:333] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.271589 19603 raft_consensus.cc:2243] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.272133 19603 raft_consensus.cc:2272] T bbf8686b20194e3981d420491c825046 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.274658 19603 tablet_replica.cc:333] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.275249 19603 raft_consensus.cc:2243] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.275738 19603 raft_consensus.cc:2272] T b1b4fd397d24490f8e5ad99e1a217e66 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.277721 19603 tablet_replica.cc:333] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.278586 19603 raft_consensus.cc:2243] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.279075 19603 raft_consensus.cc:2272] T 1d4f181266414f459ea67d333f044205 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.281415 19603 tablet_replica.cc:333] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.281958 19603 raft_consensus.cc:2243] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.282754 19603 raft_consensus.cc:2272] T 29f59bee90544d80a4b32bfcd1116d4a P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.284598 19603 tablet_replica.cc:333] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.285096 19603 raft_consensus.cc:2243] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.285790 19603 raft_consensus.cc:2272] T 466fe5ff431c463a84050bd3e265a246 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.287856 19603 tablet_replica.cc:333] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.288372 19603 raft_consensus.cc:2243] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.288776 19603 raft_consensus.cc:2272] T 11795a035a1e482d88901e8aea8ee185 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.290640 19603 tablet_replica.cc:333] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.291158 19603 raft_consensus.cc:2243] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.291610 19603 raft_consensus.cc:2272] T 32982b9d743f4ceb80f17292b281fd6f P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.294118 19603 tablet_replica.cc:333] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.294797 19603 raft_consensus.cc:2243] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.295370 19603 raft_consensus.cc:2272] T 812870f8eb264d3caf48fefa3d70cfa1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.297866 19603 tablet_replica.cc:333] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.298583 19603 raft_consensus.cc:2243] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.299131 19603 raft_consensus.cc:2272] T 8b2a8fdea8624729ae4b5191c85fb520 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.301563 19603 tablet_replica.cc:333] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.302232 19603 raft_consensus.cc:2243] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.303134 19603 raft_consensus.cc:2272] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.305723 19603 tablet_replica.cc:333] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.306360 19603 raft_consensus.cc:2243] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.307377 19603 raft_consensus.cc:2272] T dfd2a42648874aef8ca5935e76f879c5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.309707 19603 tablet_replica.cc:333] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.310273 19603 raft_consensus.cc:2243] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.310739 19603 raft_consensus.cc:2272] T cf093503636049fbb48b9a304d578c38 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.313150 19603 tablet_replica.cc:333] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.313807 19603 raft_consensus.cc:2243] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:48.313894  2049 consensus_peers.cc:597] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.314245 19603 raft_consensus.cc:2272] T baf92d63594b4e3081ad94cac31690d1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.316229 19603 tablet_replica.cc:333] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.316769 19603 raft_consensus.cc:2243] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.317174 19603 raft_consensus.cc:2272] T 2c7707b908f046398ca5ee4defefe55f P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.318979 19603 tablet_replica.cc:333] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.319545 19603 raft_consensus.cc:2243] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.319933 19603 raft_consensus.cc:2272] T 985c695826744eb693cfa90b8a3f3691 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.321703 19603 tablet_replica.cc:333] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.322216 19603 raft_consensus.cc:2243] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.322624 19603 raft_consensus.cc:2272] T ed23ad21c29f479f8dc03b1a48fd9061 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.324594 19603 tablet_replica.cc:333] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.325107 19603 raft_consensus.cc:2243] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.325493 19603 raft_consensus.cc:2272] T ac43f6f37d474da58e0a962b25fe9235 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.327450 19603 tablet_replica.cc:333] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.327943 19603 raft_consensus.cc:2243] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.328337 19603 raft_consensus.cc:2272] T f2bbe2dde38842ab846f9c2d8a260918 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.330214 19603 tablet_replica.cc:333] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.330714 19603 raft_consensus.cc:2243] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.331552 19603 raft_consensus.cc:2272] T 0b15f3e29c654279b3811d5ed11c31a2 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.333670 19603 tablet_replica.cc:333] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.334334 19603 raft_consensus.cc:2243] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.335141 19603 raft_consensus.cc:2272] T 172b29319c78428499e095bf5e76b74e P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.337574 19603 tablet_replica.cc:333] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.338043 19603 raft_consensus.cc:2243] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.338389 19603 raft_consensus.cc:2272] T 979e0700017a488aa7012c1d228deeb1 P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.340186 19603 tablet_replica.cc:333] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.340612 19603 raft_consensus.cc:2243] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.341424 19603 raft_consensus.cc:2272] T eb6c6f10db0b41ae8960133a8573f0b5 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.343279 19603 tablet_replica.cc:333] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.343719 19603 raft_consensus.cc:2243] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.344085 19603 raft_consensus.cc:2272] T ee564988f3774725bff3cd527794c6a1 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.345819 19603 tablet_replica.cc:333] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.346216 19603 raft_consensus.cc:2243] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.346558 19603 raft_consensus.cc:2272] T b385e95426fd4870b36475731ab7d0b8 P 8e43e9e956f14e52897d2efc3f361630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.348455 19603 tablet_replica.cc:333] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630: stopping tablet replica
I20260709 12:28:48.348886 19603 raft_consensus.cc:2243] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.349418 19603 raft_consensus.cc:2272] T e4414b0202fb4cd58bc7e1b4249efe0a P 8e43e9e956f14e52897d2efc3f361630 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:48.431921  2049 consensus_peers.cc:597] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.195:39583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.440330 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
W20260709 12:28:48.501806  2049 consensus_peers.cc:597] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.195:39583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.508037  2049 consensus_peers.cc:597] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.195:39583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.514714  2049 consensus_peers.cc:597] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.195:39583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.523950 19603 tablet_server.cc:179] TabletServer@127.19.36.196:0 shutting down...
W20260709 12:28:48.524187  2049 consensus_peers.cc:597] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.195:39583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.527773  2933 raft_consensus.cc:493] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8c59f6f4b90d460e832bcff010c18400)
I20260709 12:28:48.528430  2933 raft_consensus.cc:515] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5edb06eaed8e41868299eb62ade6eef0" member_type: VOTER last_known_addr { host: "127.19.36.196" port: 37785 } } peers { permanent_uuid: "8c59f6f4b90d460e832bcff010c18400" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 35551 } } peers { permanent_uuid: "3e95870e8ff04dc7ab920b8917898eee" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 39535 } }
I20260709 12:28:48.530521  2933 leader_election.cc:290] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551), 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535)
W20260709 12:28:48.538313  2049 leader_election.cc:336] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8c59f6f4b90d460e832bcff010c18400 (127.19.36.193:35551): Network error: Client connection negotiation failed: client connection to 127.19.36.193:35551: connect: Connection refused (error 111)
W20260709 12:28:48.540722  2049 leader_election.cc:336] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3e95870e8ff04dc7ab920b8917898eee (127.19.36.194:39535): Network error: Client connection negotiation failed: client connection to 127.19.36.194:39535: connect: Connection refused (error 111)
I20260709 12:28:48.541217  2049 leader_election.cc:304] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5edb06eaed8e41868299eb62ade6eef0; no voters: 3e95870e8ff04dc7ab920b8917898eee, 8c59f6f4b90d460e832bcff010c18400
I20260709 12:28:48.542194  2815 raft_consensus.cc:2749] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:48.565949 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:48.566702 19603 tablet_replica.cc:333] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.567438 19603 raft_consensus.cc:2243] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.567960 19603 raft_consensus.cc:2272] T 90834683513744559ef8f7efe530f459 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.570178 19603 tablet_replica.cc:333] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.571079 19603 raft_consensus.cc:2243] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.571743 19603 raft_consensus.cc:2272] T bf128bd1b4b546dab8e2c0504d06cb87 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.574361 19603 tablet_replica.cc:333] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.574896 19603 raft_consensus.cc:2243] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.575321 19603 raft_consensus.cc:2272] T 5128bcc1e7cb49009f2b376e8fb5e125 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.577169 19603 tablet_replica.cc:333] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.577705 19603 raft_consensus.cc:2243] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.578099 19603 raft_consensus.cc:2272] T 736e3203de274189acfaef187bfea762 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.580029 19603 tablet_replica.cc:333] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.580559 19603 raft_consensus.cc:2243] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.581259 19603 raft_consensus.cc:2272] T 91363564e93f4297b4e825de5b7e5176 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.583128 19603 tablet_replica.cc:333] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.583752 19603 raft_consensus.cc:2243] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.584501 19603 raft_consensus.cc:2272] T 5ca90853e7a245c090a24e9f2b6b88e1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.586510 19603 tablet_replica.cc:333] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.587078 19603 raft_consensus.cc:2243] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.587950 19603 raft_consensus.cc:2272] T 3d0af4a569f84ef9841ad0c788c5d6ca P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.589807 19603 tablet_replica.cc:333] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.590307 19603 raft_consensus.cc:2243] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.591097 19603 raft_consensus.cc:2272] T 6146e39d5a724f05b9b3cbc7a8145d98 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.593096 19603 tablet_replica.cc:333] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.593647 19603 raft_consensus.cc:2243] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.594012 19603 raft_consensus.cc:2272] T ff2bca7c619a4c36bb0cb4bdf08c8e78 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.595983 19603 tablet_replica.cc:333] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.596496 19603 raft_consensus.cc:2243] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.597473 19603 raft_consensus.cc:2272] T cf093503636049fbb48b9a304d578c38 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.599584 19603 tablet_replica.cc:333] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.600124 19603 raft_consensus.cc:2243] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.600492 19603 raft_consensus.cc:2272] T c5ff50d36854429790ff6c78da6f2c8b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.602268 19603 tablet_replica.cc:333] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.602767 19603 raft_consensus.cc:2243] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.603171 19603 raft_consensus.cc:2272] T 5ebf212c059342559ae4781db6ba8e83 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.605113 19603 tablet_replica.cc:333] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.605674 19603 raft_consensus.cc:2243] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.606402 19603 raft_consensus.cc:2272] T 3a6b8329d4924908b3a364b840d691fe P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.608346 19603 tablet_replica.cc:333] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.608906 19603 raft_consensus.cc:2243] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.609328 19603 raft_consensus.cc:2272] T dcdf235113b34d5f8237b06b7662479f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.611239 19603 tablet_replica.cc:333] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.611774 19603 raft_consensus.cc:2243] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.612151 19603 raft_consensus.cc:2272] T d17fb96cd6f94ffa9824eb3868a5fe92 P 5edb06eaed8e41868299eb62ade6eef0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.613998 19603 tablet_replica.cc:333] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.614542 19603 raft_consensus.cc:2243] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.615320 19603 raft_consensus.cc:2272] T f2bbe2dde38842ab846f9c2d8a260918 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.617147 19603 tablet_replica.cc:333] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.617722 19603 raft_consensus.cc:2243] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.618162 19603 raft_consensus.cc:2272] T 0b15f3e29c654279b3811d5ed11c31a2 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.620136 19603 tablet_replica.cc:333] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.620867 19603 raft_consensus.cc:2243] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.621407 19603 raft_consensus.cc:2272] T 2c6aef41b0274acb8e13d886aa250805 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.624300 19603 tablet_replica.cc:333] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.624972 19603 raft_consensus.cc:2243] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.625612 19603 raft_consensus.cc:2272] T 172b29319c78428499e095bf5e76b74e P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:48.626060  2049 consensus_peers.cc:597] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 -> Peer 8e43e9e956f14e52897d2efc3f361630 (127.19.36.195:39583): Couldn't send request to peer 8e43e9e956f14e52897d2efc3f361630. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.195:39583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.628333 19603 tablet_replica.cc:333] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.629253 19603 raft_consensus.cc:2243] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.629766 19603 raft_consensus.cc:2272] T e922d7ff58764a529d119fa6491f1768 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.631958 19603 tablet_replica.cc:333] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.632505 19603 raft_consensus.cc:2243] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.632941 19603 raft_consensus.cc:2272] T 2e8e346d58d44c1dbcf74cd388a21904 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.634764 19603 tablet_replica.cc:333] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.635345 19603 raft_consensus.cc:2243] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.635759 19603 raft_consensus.cc:2272] T 78d17658b0174e638c0041193b99bf82 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.637585 19603 tablet_replica.cc:333] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.638165 19603 raft_consensus.cc:2243] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.638971 19603 raft_consensus.cc:2272] T db6a2096401b4cec9b542b6eb99f705b P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.640872 19603 tablet_replica.cc:333] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.641394 19603 raft_consensus.cc:2243] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.641803 19603 raft_consensus.cc:2272] T 880a5a05cb024c17b64d7ef2621311c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.643608 19603 tablet_replica.cc:333] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.644124 19603 raft_consensus.cc:2243] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.644862 19603 raft_consensus.cc:2272] T b385e95426fd4870b36475731ab7d0b8 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.646699 19603 tablet_replica.cc:333] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.647267 19603 raft_consensus.cc:2243] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.647686 19603 raft_consensus.cc:2272] T e4414b0202fb4cd58bc7e1b4249efe0a P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.649451 19603 tablet_replica.cc:333] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.649986 19603 raft_consensus.cc:2243] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.650359 19603 raft_consensus.cc:2272] T 8d082f4dbaeb47959b209411cd69957b P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.652146 19603 tablet_replica.cc:333] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.652707 19603 raft_consensus.cc:2243] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.653107 19603 raft_consensus.cc:2272] T b1b4fd397d24490f8e5ad99e1a217e66 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.654922 19603 tablet_replica.cc:333] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.655493 19603 raft_consensus.cc:2243] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.655977 19603 raft_consensus.cc:2272] T 4773e370acbd4490afddcc7393f8be04 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.658339 19603 tablet_replica.cc:333] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.658859 19603 raft_consensus.cc:2243] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.659256 19603 raft_consensus.cc:2272] T 8b2a8fdea8624729ae4b5191c85fb520 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.660991 19603 tablet_replica.cc:333] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.661442 19603 raft_consensus.cc:2243] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.661805 19603 raft_consensus.cc:2272] T 8da8ed38ed6a4a8cb9ad7301aae1035d P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.663630 19603 tablet_replica.cc:333] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.664098 19603 raft_consensus.cc:2243] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.664844 19603 raft_consensus.cc:2272] T f6241be73e134c25909f8054db9f85dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.666796 19603 tablet_replica.cc:333] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.667358 19603 raft_consensus.cc:2243] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.668064 19603 raft_consensus.cc:2272] T 11795a035a1e482d88901e8aea8ee185 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.669970 19603 tablet_replica.cc:333] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.670496 19603 raft_consensus.cc:2243] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.670917 19603 raft_consensus.cc:2272] T 985c695826744eb693cfa90b8a3f3691 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.673103 19603 tablet_replica.cc:333] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.673662 19603 raft_consensus.cc:2243] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.674513 19603 raft_consensus.cc:2272] T ed23ad21c29f479f8dc03b1a48fd9061 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.676474 19603 tablet_replica.cc:333] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.676992 19603 raft_consensus.cc:2243] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.677397 19603 raft_consensus.cc:2272] T 1d4f181266414f459ea67d333f044205 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.679453 19603 tablet_replica.cc:333] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.679998 19603 raft_consensus.cc:2243] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.680399 19603 raft_consensus.cc:2272] T 891bc4cc61ee4451a2cb82d28286d5f8 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.682269 19603 tablet_replica.cc:333] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.682818 19603 raft_consensus.cc:2243] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.683673 19603 raft_consensus.cc:2272] T 2c7707b908f046398ca5ee4defefe55f P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.685566 19603 tablet_replica.cc:333] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.686132 19603 raft_consensus.cc:2243] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.686861 19603 raft_consensus.cc:2272] T e2645174b413446fbf4c9423465821be P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.688659 19603 tablet_replica.cc:333] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.689141 19603 raft_consensus.cc:2243] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.689874 19603 raft_consensus.cc:2272] T 32982b9d743f4ceb80f17292b281fd6f P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.691738 19603 tablet_replica.cc:333] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.692202 19603 raft_consensus.cc:2243] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.692611 19603 raft_consensus.cc:2272] T 812870f8eb264d3caf48fefa3d70cfa1 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.694276 19603 tablet_replica.cc:333] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.694736 19603 raft_consensus.cc:2243] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.695129 19603 raft_consensus.cc:2272] T dfd2a42648874aef8ca5935e76f879c5 P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.696841 19603 tablet_replica.cc:333] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.697283 19603 raft_consensus.cc:2243] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.697723 19603 raft_consensus.cc:2272] T 74e46e6bd8574e31824d103146fc9830 P 5edb06eaed8e41868299eb62ade6eef0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.699448 19603 tablet_replica.cc:333] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0: stopping tablet replica
I20260709 12:28:48.699923 19603 raft_consensus.cc:2243] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:48.700291 19603 raft_consensus.cc:2272] T 8a38771542bf449280983d8f709895dc P 5edb06eaed8e41868299eb62ade6eef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.766388 19603 tablet_server.cc:196] TabletServer@127.19.36.196:0 shutdown complete.
I20260709 12:28:48.829121 19603 master.cc:562] Master@127.19.36.254:40093 shutting down...
I20260709 12:28:48.855294 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:48.855863 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:48.856208 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P 07aa32703cc844cda506fcb5d3109a62: stopping tablet replica
I20260709 12:28:48.876899 19603 master.cc:584] Master@127.19.36.254:40093 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (28821 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:28:48.964272 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:40081
I20260709 12:28:48.965417 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:48.971794  2938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:48.972038  2940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:48.971813  2937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:48.974256 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:48.975288 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:48.975463 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:48.975580 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600128975570 us; error 0 us; skew 500 ppm
I20260709 12:28:48.976033 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:48.978446 19603 webserver.cc:533] Webserver started at http://127.19.36.254:33255/ using document root <none> and password file <none>
I20260709 12:28:48.978876 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:48.979022 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:48.979676 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:48.980724 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "e41ae3d6200440e98528a524567da36d"
format_stamp: "Formatted at 2026-07-09 12:28:48 on dist-test-slave-hfcn"
I20260709 12:28:48.984997 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:28:48.988260  2945 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:48.989084 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:48.989368 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "e41ae3d6200440e98528a524567da36d"
format_stamp: "Formatted at 2026-07-09 12:28:48 on dist-test-slave-hfcn"
I20260709 12:28:48.989658 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:49.003135 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:49.004411 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:49.042862 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:40081
I20260709 12:28:49.042992  2996 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:40081 every 8 connection(s)
I20260709 12:28:49.046629  2997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.057263  2997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d: Bootstrap starting.
I20260709 12:28:49.062028  2997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.066123  2997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d: No bootstrap required, opened a new log
I20260709 12:28:49.068249  2997 raft_consensus.cc:359] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41ae3d6200440e98528a524567da36d" member_type: VOTER }
I20260709 12:28:49.068608  2997 raft_consensus.cc:385] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.068783  2997 raft_consensus.cc:740] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41ae3d6200440e98528a524567da36d, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.069314  2997 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41ae3d6200440e98528a524567da36d" member_type: VOTER }
I20260709 12:28:49.069747  2997 raft_consensus.cc:399] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:49.069932  2997 raft_consensus.cc:493] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:49.070133  2997 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.075042  2997 raft_consensus.cc:515] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41ae3d6200440e98528a524567da36d" member_type: VOTER }
I20260709 12:28:49.075583  2997 leader_election.cc:304] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41ae3d6200440e98528a524567da36d; no voters: 
I20260709 12:28:49.076623  2997 leader_election.cc:290] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:49.076941  3000 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.078724  3000 raft_consensus.cc:697] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 1 LEADER]: Becoming Leader. State: Replica: e41ae3d6200440e98528a524567da36d, State: Running, Role: LEADER
I20260709 12:28:49.079509  3000 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41ae3d6200440e98528a524567da36d" member_type: VOTER }
I20260709 12:28:49.079965  2997 sys_catalog.cc:565] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:49.084362  3001 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e41ae3d6200440e98528a524567da36d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41ae3d6200440e98528a524567da36d" member_type: VOTER } }
I20260709 12:28:49.084424  3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41ae3d6200440e98528a524567da36d. Latest consensus state: current_term: 1 leader_uuid: "e41ae3d6200440e98528a524567da36d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41ae3d6200440e98528a524567da36d" member_type: VOTER } }
I20260709 12:28:49.085408  3001 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:49.085467  3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:49.090211  3007 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:49.095407  3007 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:49.098562 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:49.104908  3007 catalog_manager.cc:1383] Generated new cluster ID: b49f577b170740e99cb52f0b327ac43a
I20260709 12:28:49.105234  3007 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:49.119942  3007 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:49.121235  3007 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:49.141490  3007 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d: Generated new TSK 0
I20260709 12:28:49.142092  3007 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:49.192378 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:49.199954  3019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:49.200573  3020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:49.202054  3022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:49.203398 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:49.204561 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:49.204790 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:49.205018 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600129204994 us; error 0 us; skew 500 ppm
I20260709 12:28:49.205610 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:49.208392 19603 webserver.cc:533] Webserver started at http://127.19.36.193:39585/ using document root <none> and password file <none>
I20260709 12:28:49.209000 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:49.209215 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:49.209565 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:49.211225 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "148b0777ed6f426ab1b748decd6e7561"
format_stamp: "Formatted at 2026-07-09 12:28:49 on dist-test-slave-hfcn"
I20260709 12:28:49.216337 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:49.220065  3027 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.220840 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:28:49.221151 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "148b0777ed6f426ab1b748decd6e7561"
format_stamp: "Formatted at 2026-07-09 12:28:49 on dist-test-slave-hfcn"
I20260709 12:28:49.221468 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:49.250140 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:49.251515 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:49.253068 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:49.255527 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:49.255746 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.255983 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:49.256168 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.300393 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:38865
I20260709 12:28:49.300501  3089 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:38865 every 8 connection(s)
I20260709 12:28:49.306207 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:49.313694  3094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:49.315404  3095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:49.317667 19603 server_base.cc:1061] running on GCE node
W20260709 12:28:49.318065  3097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:49.318928 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:49.319160 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:49.319437 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600129319381 us; error 0 us; skew 500 ppm
I20260709 12:28:49.319998 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:49.322515 19603 webserver.cc:533] Webserver started at http://127.19.36.194:36745/ using document root <none> and password file <none>
I20260709 12:28:49.323153 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:49.323410 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:49.323827 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:49.325325 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "7c70962ac2244068ab924dc9f68c98aa"
format_stamp: "Formatted at 2026-07-09 12:28:49 on dist-test-slave-hfcn"
I20260709 12:28:49.325570  3090 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40081
I20260709 12:28:49.326025  3090 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:49.326800  3090 heartbeater.cc:507] Master 127.19.36.254:40081 requested a full tablet report, sending...
I20260709 12:28:49.329155  2962 ts_manager.cc:194] Registered new tserver with Master: 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865)
I20260709 12:28:49.330541 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:49.331712  2962 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58972
I20260709 12:28:49.334558  3102 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.335335 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:49.335618 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "7c70962ac2244068ab924dc9f68c98aa"
format_stamp: "Formatted at 2026-07-09 12:28:49 on dist-test-slave-hfcn"
I20260709 12:28:49.335913 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:49.361558 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:49.362762 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:49.364233 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:49.366454 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:49.366684 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.366916 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:49.367087 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.408928 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:37223
I20260709 12:28:49.409051  3164 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:37223 every 8 connection(s)
I20260709 12:28:49.414194 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:49.422053  3168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:49.422895  3169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:49.426465 19603 server_base.cc:1061] running on GCE node
W20260709 12:28:49.427479  3171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:49.428184  3165 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40081
I20260709 12:28:49.428619  3165 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:49.428689 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:49.429013 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:49.429221 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600129429202 us; error 0 us; skew 500 ppm
I20260709 12:28:49.429641  3165 heartbeater.cc:507] Master 127.19.36.254:40081 requested a full tablet report, sending...
I20260709 12:28:49.430083 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:49.432133  2962 ts_manager.cc:194] Registered new tserver with Master: 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:49.433248 19603 webserver.cc:533] Webserver started at http://127.19.36.195:42327/ using document root <none> and password file <none>
I20260709 12:28:49.433928 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:49.434167 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:49.434118  2962 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58982
I20260709 12:28:49.434810 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:49.436654 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "d4bb183960864493901feedeb761ae8e"
format_stamp: "Formatted at 2026-07-09 12:28:49 on dist-test-slave-hfcn"
I20260709 12:28:49.441409 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:28:49.444768  3176 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.445533 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:28:49.445842 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "d4bb183960864493901feedeb761ae8e"
format_stamp: "Formatted at 2026-07-09 12:28:49 on dist-test-slave-hfcn"
I20260709 12:28:49.446146 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:49.459645 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:49.460801 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:49.462168 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:49.464359 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:49.464583 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.464815 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:49.464982 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:49.506958 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:40447
I20260709 12:28:49.507066  3238 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:40447 every 8 connection(s)
I20260709 12:28:49.521080  3239 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40081
I20260709 12:28:49.521544  3239 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:49.522258  3239 heartbeater.cc:507] Master 127.19.36.254:40081 requested a full tablet report, sending...
I20260709 12:28:49.524237  2962 ts_manager.cc:194] Registered new tserver with Master: d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:49.525091 19603 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014664189s
I20260709 12:28:49.525614  2962 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58984
I20260709 12:28:49.550302  2962 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58990:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:49.553308  2962 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:49.630170  3204 tablet_service.cc:1511] Processing CreateTablet for tablet 861bf7cca3d9446bb46dffc150e42bd8 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:49.632849  3203 tablet_service.cc:1511] Processing CreateTablet for tablet 119b1c9bc89746d497bcb03fd8a6d5dc (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:49.634974  3055 tablet_service.cc:1511] Processing CreateTablet for tablet 861bf7cca3d9446bb46dffc150e42bd8 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:49.636970  3204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861bf7cca3d9446bb46dffc150e42bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.639516  3054 tablet_service.cc:1511] Processing CreateTablet for tablet 119b1c9bc89746d497bcb03fd8a6d5dc (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:49.640342  3201 tablet_service.cc:1511] Processing CreateTablet for tablet b7bbb0668ef9405b8e4dcaa137f53339 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:49.640906  3055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861bf7cca3d9446bb46dffc150e42bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.641631  3054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 119b1c9bc89746d497bcb03fd8a6d5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.638460  3052 tablet_service.cc:1511] Processing CreateTablet for tablet b7bbb0668ef9405b8e4dcaa137f53339 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:49.638772  3053 tablet_service.cc:1511] Processing CreateTablet for tablet c4e16762ec3840b2952fe56ae6039c0b (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:49.642783  3051 tablet_service.cc:1511] Processing CreateTablet for tablet 490f22e467d94f3f91541007124a59ad (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:49.645272  3202 tablet_service.cc:1511] Processing CreateTablet for tablet c4e16762ec3840b2952fe56ae6039c0b (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:49.647167  3203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 119b1c9bc89746d497bcb03fd8a6d5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.648000  3201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7bbb0668ef9405b8e4dcaa137f53339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.648789  3202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e16762ec3840b2952fe56ae6039c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.642809  3052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7bbb0668ef9405b8e4dcaa137f53339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.652078  3051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490f22e467d94f3f91541007124a59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.656599  3200 tablet_service.cc:1511] Processing CreateTablet for tablet 490f22e467d94f3f91541007124a59ad (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:49.658095  3200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490f22e467d94f3f91541007124a59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.659005  3199 tablet_service.cc:1511] Processing CreateTablet for tablet a877ac06651d482f8390c1c14aa3b5e4 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:49.660429  3050 tablet_service.cc:1511] Processing CreateTablet for tablet a877ac06651d482f8390c1c14aa3b5e4 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:49.660631  3049 tablet_service.cc:1511] Processing CreateTablet for tablet 6fb93f2a811249d18eb651d2489d9141 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:49.661923  3050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a877ac06651d482f8390c1c14aa3b5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.660547  3199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a877ac06651d482f8390c1c14aa3b5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.667569  3053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e16762ec3840b2952fe56ae6039c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.679176  3048 tablet_service.cc:1511] Processing CreateTablet for tablet d46c1af825d54b138e97c6396918abfc (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:49.680882  3048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46c1af825d54b138e97c6396918abfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.681751  3049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fb93f2a811249d18eb651d2489d9141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.683761  3047 tablet_service.cc:1511] Processing CreateTablet for tablet 635ee0c36e9e4e49a7d2cbb4c669235a (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:49.684511  3198 tablet_service.cc:1511] Processing CreateTablet for tablet d46c1af825d54b138e97c6396918abfc (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:49.685326  3047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635ee0c36e9e4e49a7d2cbb4c669235a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.685935  3198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46c1af825d54b138e97c6396918abfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.689442  3046 tablet_service.cc:1511] Processing CreateTablet for tablet 38cc74f58e7d4221ba39cf9286077cca (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:49.690874  3046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38cc74f58e7d4221ba39cf9286077cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.690791  3197 tablet_service.cc:1511] Processing CreateTablet for tablet 635ee0c36e9e4e49a7d2cbb4c669235a (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:49.691397  3196 tablet_service.cc:1511] Processing CreateTablet for tablet 38cc74f58e7d4221ba39cf9286077cca (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:49.692359  3197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635ee0c36e9e4e49a7d2cbb4c669235a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.693054  3196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38cc74f58e7d4221ba39cf9286077cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.694016  3204 tablet_service.cc:1511] Processing CreateTablet for tablet 6fb93f2a811249d18eb651d2489d9141 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:49.695446  3204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fb93f2a811249d18eb651d2489d9141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.784178  3254 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:49.791227  3254 tablet_bootstrap.cc:654] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.828166  3130 tablet_service.cc:1511] Processing CreateTablet for tablet 861bf7cca3d9446bb46dffc150e42bd8 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:49.829639  3130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861bf7cca3d9446bb46dffc150e42bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.830192  3255 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:49.830022  3125 tablet_service.cc:1511] Processing CreateTablet for tablet a877ac06651d482f8390c1c14aa3b5e4 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:49.831326  3124 tablet_service.cc:1511] Processing CreateTablet for tablet 6fb93f2a811249d18eb651d2489d9141 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:49.832688  3124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fb93f2a811249d18eb651d2489d9141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.832413  3126 tablet_service.cc:1511] Processing CreateTablet for tablet 490f22e467d94f3f91541007124a59ad (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:49.833866  3126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490f22e467d94f3f91541007124a59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.834803  3123 tablet_service.cc:1511] Processing CreateTablet for tablet d46c1af825d54b138e97c6396918abfc (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:49.836082  3123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46c1af825d54b138e97c6396918abfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.836601  3128 tablet_service.cc:1511] Processing CreateTablet for tablet c4e16762ec3840b2952fe56ae6039c0b (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:49.837662  3255 tablet_bootstrap.cc:654] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.837944  3128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e16762ec3840b2952fe56ae6039c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.838742  3122 tablet_service.cc:1511] Processing CreateTablet for tablet 635ee0c36e9e4e49a7d2cbb4c669235a (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:49.840130  3122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635ee0c36e9e4e49a7d2cbb4c669235a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.840269  3129 tablet_service.cc:1511] Processing CreateTablet for tablet 119b1c9bc89746d497bcb03fd8a6d5dc (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:49.842013  3129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 119b1c9bc89746d497bcb03fd8a6d5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.844408  3121 tablet_service.cc:1511] Processing CreateTablet for tablet 38cc74f58e7d4221ba39cf9286077cca (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:49.844862  3127 tablet_service.cc:1511] Processing CreateTablet for tablet b7bbb0668ef9405b8e4dcaa137f53339 (DEFAULT_TABLE table=test-workload [id=f3fabe430e0a430db87c7531197ff7f3]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:49.851518  3121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38cc74f58e7d4221ba39cf9286077cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.858204  3125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a877ac06651d482f8390c1c14aa3b5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.863638  3127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7bbb0668ef9405b8e4dcaa137f53339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:49.885610  3254 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:49.886263  3254 ts_tablet_manager.cc:1403] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.103s	user 0.022s	sys 0.020s
I20260709 12:28:49.889611  3254 raft_consensus.cc:359] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:49.890525  3254 raft_consensus.cc:385] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.890837  3254 raft_consensus.cc:740] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.896983  3254 consensus_queue.cc:260] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:49.928248  3239 heartbeater.cc:499] Master 127.19.36.254:40081 was elected leader, sending a full tablet report...
I20260709 12:28:49.928943  3254 ts_tablet_manager.cc:1434] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.017s
I20260709 12:28:49.930094  3254 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:49.945580  3254 tablet_bootstrap.cc:654] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.957460  3255 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:49.957952  3258 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:49.958069  3255 ts_tablet_manager.cc:1403] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.128s	user 0.031s	sys 0.042s
I20260709 12:28:49.961405  3255 raft_consensus.cc:359] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:49.961979  3255 raft_consensus.cc:385] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.962224  3255 raft_consensus.cc:740] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.962940  3254 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:49.962879  3255 consensus_queue.cc:260] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:49.963557  3254 ts_tablet_manager.cc:1403] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.007s
I20260709 12:28:49.965032  3090 heartbeater.cc:499] Master 127.19.36.254:40081 was elected leader, sending a full tablet report...
I20260709 12:28:49.965569  3255 ts_tablet_manager.cc:1434] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:49.965713  3258 tablet_bootstrap.cc:654] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.966578  3255 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:49.966784  3254 raft_consensus.cc:359] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:49.967579  3254 raft_consensus.cc:385] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.967872  3254 raft_consensus.cc:740] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.968688  3254 consensus_queue.cc:260] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:49.972031  3254 ts_tablet_manager.cc:1434] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:28:49.973048  3255 tablet_bootstrap.cc:654] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.973289  3254 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:49.983103  3254 tablet_bootstrap.cc:654] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.024938  3254 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.025714  3254 ts_tablet_manager.cc:1403] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.053s	user 0.031s	sys 0.015s
I20260709 12:28:50.026403  3255 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.027309  3255 ts_tablet_manager.cc:1403] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.061s	user 0.027s	sys 0.031s
I20260709 12:28:50.028899  3258 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.029696  3258 ts_tablet_manager.cc:1403] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.072s	user 0.036s	sys 0.031s
I20260709 12:28:50.030701  3254 raft_consensus.cc:359] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.031531  3254 raft_consensus.cc:385] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.031351  3255 raft_consensus.cc:359] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.031915  3254 raft_consensus.cc:740] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.032099  3255 raft_consensus.cc:385] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.032451  3255 raft_consensus.cc:740] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.032819  3254 consensus_queue.cc:260] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.033094  3258 raft_consensus.cc:359] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.033179  3255 consensus_queue.cc:260] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.033890  3258 raft_consensus.cc:385] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.034233  3258 raft_consensus.cc:740] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.035334  3254 ts_tablet_manager.cc:1434] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.007s
I20260709 12:28:50.035436  3255 ts_tablet_manager.cc:1434] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:50.035126  3258 consensus_queue.cc:260] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.036525  3255 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.043968  3254 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:50.051077  3254 tablet_bootstrap.cc:654] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.051564  3255 tablet_bootstrap.cc:654] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.052249  3165 heartbeater.cc:499] Master 127.19.36.254:40081 was elected leader, sending a full tablet report...
I20260709 12:28:50.052810  3258 ts_tablet_manager.cc:1434] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.023s	user 0.019s	sys 0.002s
I20260709 12:28:50.053992  3258 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.056977  3255 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.057626  3255 ts_tablet_manager.cc:1403] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.002s
I20260709 12:28:50.060719  3258 tablet_bootstrap.cc:654] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.061739  3255 raft_consensus.cc:359] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.062840  3255 raft_consensus.cc:385] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.063311  3255 raft_consensus.cc:740] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.064431  3255 consensus_queue.cc:260] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.071382  3255 ts_tablet_manager.cc:1434] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:28:50.072748  3258 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.072914  3255 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.073345  3258 ts_tablet_manager.cc:1403] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.008s
I20260709 12:28:50.076668  3258 raft_consensus.cc:359] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.077437  3258 raft_consensus.cc:385] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.077749  3258 raft_consensus.cc:740] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.078634  3258 consensus_queue.cc:260] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.081143  3258 ts_tablet_manager.cc:1434] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:50.082072  3258 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.082674  3255 tablet_bootstrap.cc:654] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.087042  3254 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.087598  3254 ts_tablet_manager.cc:1403] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.017s
I20260709 12:28:50.088727  3258 tablet_bootstrap.cc:654] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.090655  3254 raft_consensus.cc:359] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.091490  3254 raft_consensus.cc:385] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.092077  3254 raft_consensus.cc:740] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
W20260709 12:28:50.092797  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:50.092922  3254 consensus_queue.cc:260] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.095325  3254 ts_tablet_manager.cc:1434] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:28:50.096333  3254 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:50.096822  3258 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.097347  3258 ts_tablet_manager.cc:1403] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.004s
I20260709 12:28:50.100562  3258 raft_consensus.cc:359] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.101298  3258 raft_consensus.cc:385] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.101352  3255 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.101652  3258 raft_consensus.cc:740] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.101977  3255 ts_tablet_manager.cc:1403] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.007s
I20260709 12:28:50.102813  3254 tablet_bootstrap.cc:654] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.102612  3258 consensus_queue.cc:260] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.105095  3255 raft_consensus.cc:359] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.106307  3255 raft_consensus.cc:385] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.106585  3255 raft_consensus.cc:740] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.107407  3255 consensus_queue.cc:260] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.109687  3255 ts_tablet_manager.cc:1434] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:50.110154  3258 ts_tablet_manager.cc:1434] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:28:50.110683  3255 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.111573  3258 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.117295  3255 tablet_bootstrap.cc:654] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.121989  3254 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.122645  3254 ts_tablet_manager.cc:1403] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.016s
I20260709 12:28:50.126375  3258 tablet_bootstrap.cc:654] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.128551  3254 raft_consensus.cc:359] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.129623  3254 raft_consensus.cc:385] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.129830  3254 raft_consensus.cc:740] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.130686  3254 consensus_queue.cc:260] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.132647  3258 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.133167  3258 ts_tablet_manager.cc:1403] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.015s
I20260709 12:28:50.133637  3255 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.134209  3255 ts_tablet_manager.cc:1403] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.008s
I20260709 12:28:50.135985  3254 ts_tablet_manager.cc:1434] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.001s
I20260709 12:28:50.135973  3258 raft_consensus.cc:359] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.136988  3258 raft_consensus.cc:385] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.137264  3254 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:50.137290  3258 raft_consensus.cc:740] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.137383  3255 raft_consensus.cc:359] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.138592  3255 raft_consensus.cc:385] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.138523  3258 consensus_queue.cc:260] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.139026  3255 raft_consensus.cc:740] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.139989  3255 consensus_queue.cc:260] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.142432  3255 ts_tablet_manager.cc:1434] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:50.143611  3255 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.143723  3258 ts_tablet_manager.cc:1434] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 12:28:50.144518  3254 tablet_bootstrap.cc:654] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.144841  3258 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.153151  3255 tablet_bootstrap.cc:654] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.153245  3258 tablet_bootstrap.cc:654] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.165899  3255 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.166438  3255 ts_tablet_manager.cc:1403] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 12:28:50.166877  3254 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.167466  3254 ts_tablet_manager.cc:1403] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.011s
I20260709 12:28:50.169437  3255 raft_consensus.cc:359] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.170230  3255 raft_consensus.cc:385] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.170579  3255 raft_consensus.cc:740] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.170344  3254 raft_consensus.cc:359] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.171168  3254 raft_consensus.cc:385] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.171528  3254 raft_consensus.cc:740] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.171387  3255 consensus_queue.cc:260] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.172353  3254 consensus_queue.cc:260] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.173913  3255 ts_tablet_manager.cc:1434] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 12:28:50.174960  3255 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.178860  3258 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.179559  3258 ts_tablet_manager.cc:1403] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.015s
I20260709 12:28:50.181514  3255 tablet_bootstrap.cc:654] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.183168  3258 raft_consensus.cc:359] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.184109  3258 raft_consensus.cc:385] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.184480  3258 raft_consensus.cc:740] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.185446  3258 consensus_queue.cc:260] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.186385  3254 ts_tablet_manager.cc:1434] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.004s
I20260709 12:28:50.188210  3258 ts_tablet_manager.cc:1434] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:50.189324  3258 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.189579  3254 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:50.196134  3255 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.196704  3255 ts_tablet_manager.cc:1403] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.005s
I20260709 12:28:50.198984  3254 tablet_bootstrap.cc:654] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.200116  3258 tablet_bootstrap.cc:654] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.199865  3255 raft_consensus.cc:359] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.200690  3255 raft_consensus.cc:385] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.201017  3255 raft_consensus.cc:740] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.202059  3255 consensus_queue.cc:260] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.208644  3255 ts_tablet_manager.cc:1434] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 12:28:50.209995  3255 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.212807  3258 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.213385  3258 ts_tablet_manager.cc:1403] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.012s
I20260709 12:28:50.216789  3258 raft_consensus.cc:359] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.217600  3258 raft_consensus.cc:385] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.217922  3258 raft_consensus.cc:740] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.218870  3258 consensus_queue.cc:260] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.219013  3255 tablet_bootstrap.cc:654] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.221423  3258 ts_tablet_manager.cc:1434] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:50.221771  3265 raft_consensus.cc:493] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.222595  3258 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.222381  3265 raft_consensus.cc:515] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.225023  3254 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.225723  3254 ts_tablet_manager.cc:1403] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.017s
I20260709 12:28:50.229784  3254 raft_consensus.cc:359] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.230552  3254 raft_consensus.cc:385] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.230868  3254 raft_consensus.cc:740] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.231910  3254 consensus_queue.cc:260] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.232704  3255 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.233201  3255 ts_tablet_manager.cc:1403] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.007s
I20260709 12:28:50.235368  3258 tablet_bootstrap.cc:654] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.233693  3265 leader_election.cc:290] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.250998  3261 raft_consensus.cc:493] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.251948  3261 raft_consensus.cc:515] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.253249  3254 ts_tablet_manager.cc:1434] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 12:28:50.254529  3254 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:50.255563  3255 raft_consensus.cc:359] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.256363  3255 raft_consensus.cc:385] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.256664  3255 raft_consensus.cc:740] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.257500  3255 consensus_queue.cc:260] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.264137  3254 tablet_bootstrap.cc:654] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.267620  3261 leader_election.cc:290] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.291903  3259 raft_consensus.cc:493] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.292610  3259 raft_consensus.cc:515] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.300051  3259 leader_election.cc:290] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.305197  3255 ts_tablet_manager.cc:1434] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.028s
I20260709 12:28:50.306428  3255 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.310779  3258 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.311393  3258 ts_tablet_manager.cc:1403] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.006s
I20260709 12:28:50.314513  3258 raft_consensus.cc:359] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.315332  3258 raft_consensus.cc:385] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.315447  3255 tablet_bootstrap.cc:654] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.315707  3258 raft_consensus.cc:740] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.316658  3258 consensus_queue.cc:260] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.324952  3259 raft_consensus.cc:493] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.326251  3259 raft_consensus.cc:515] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.328284  3258 ts_tablet_manager.cc:1434] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:28:50.330787  3258 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.328599  3259 leader_election.cc:290] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.350067  3265 raft_consensus.cc:493] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.350759  3265 raft_consensus.cc:515] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.351471  3254 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.352010  3254 ts_tablet_manager.cc:1403] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.098s	user 0.009s	sys 0.032s
I20260709 12:28:50.352051  3255 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.352705  3255 ts_tablet_manager.cc:1403] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.008s
I20260709 12:28:50.353210  3265 leader_election.cc:290] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.354069  3065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.354944  3065 raft_consensus.cc:2468] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 0.
I20260709 12:28:50.355355  3254 raft_consensus.cc:359] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.356510  3254 raft_consensus.cc:385] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.356752  3254 raft_consensus.cc:740] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.356065  3255 raft_consensus.cc:359] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.358513  3255 raft_consensus.cc:385] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.358850  3255 raft_consensus.cc:740] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.359004  3106 leader_election.cc:304] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:50.359812  3255 consensus_queue.cc:260] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.360260  3065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.360944  3065 raft_consensus.cc:2468] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 0.
I20260709 12:28:50.362205  3255 ts_tablet_manager.cc:1434] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:50.362104  3214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.362937  3214 raft_consensus.cc:2468] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 0.
I20260709 12:28:50.363687  3255 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:50.364291  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.365000  3213 raft_consensus.cc:2468] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 0.
I20260709 12:28:50.366371  3106 leader_election.cc:304] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:50.368083  3258 tablet_bootstrap.cc:654] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.368410  3265 raft_consensus.cc:2804] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.370543  3254 consensus_queue.cc:260] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.371013  3265 raft_consensus.cc:493] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.372241  3287 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.372841  3254 ts_tablet_manager.cc:1434] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:28:50.372905  3287 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.374059  3254 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:50.375602  3255 tablet_bootstrap.cc:654] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.388346  3287 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.388988  3265 raft_consensus.cc:3060] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.388805  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.391449  3255 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: No bootstrap required, opened a new log
I20260709 12:28:50.392028  3255 ts_tablet_manager.cc:1403] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.004s
I20260709 12:28:50.391743  3214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.392727  3214 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 0.
W20260709 12:28:50.394315  3029 leader_election.cc:343] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4bb183960864493901feedeb761ae8e (127.19.36.195:40447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.395794  3029 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.396975  3287 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.397370  3287 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.397740  3287 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.397934  3255 raft_consensus.cc:359] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.398747  3255 raft_consensus.cc:385] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.399062  3255 raft_consensus.cc:740] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.400228  3255 consensus_queue.cc:260] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.402540  3255 ts_tablet_manager.cc:1434] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:28:50.409819  3254 tablet_bootstrap.cc:654] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.412307  3261 raft_consensus.cc:493] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.413012  3261 raft_consensus.cc:515] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.418737  3299 raft_consensus.cc:2804] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.419059  3296 raft_consensus.cc:493] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.419288  3299 raft_consensus.cc:493] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.419703  3299 raft_consensus.cc:3060] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.419571  3296 raft_consensus.cc:515] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.421838  3065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.422746  3065 raft_consensus.cc:2468] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
I20260709 12:28:50.423588  3259 raft_consensus.cc:493] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.424114  3259 raft_consensus.cc:515] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.424341  3064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.425278  3064 raft_consensus.cc:2468] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
I20260709 12:28:50.426252  3259 leader_election.cc:290] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.423027  3214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46c1af825d54b138e97c6396918abfc" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.427261  3214 raft_consensus.cc:2468] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 0.
I20260709 12:28:50.427996  3299 raft_consensus.cc:515] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.428491  3180 leader_election.cc:304] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.429944  3180 leader_election.cc:304] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.428726  3104 leader_election.cc:304] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.431411  3287 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.432480  3064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.432585  3265 raft_consensus.cc:515] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.432942  3301 raft_consensus.cc:2804] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.433333  3064 raft_consensus.cc:2468] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
I20260709 12:28:50.434527  3287 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Requested vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.434649  3214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.435369  3214 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.435762  3301 raft_consensus.cc:493] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.434087  3296 leader_election.cc:290] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.434427  3259 raft_consensus.cc:2804] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.433985  3300 raft_consensus.cc:493] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.436255  3261 leader_election.cc:290] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.430215  3299 leader_election.cc:290] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.436993  3259 raft_consensus.cc:493] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.443975  3259 raft_consensus.cc:3060] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.436827  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.444403  3302 raft_consensus.cc:2804] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.437743  3180 leader_election.cc:304] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.439568  3138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.440119  3136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.440723  3212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.440816  3137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.446265  3300 raft_consensus.cc:515] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.446848  3212 raft_consensus.cc:2468] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 0.
I20260709 12:28:50.448202  3140 raft_consensus.cc:2468] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
I20260709 12:28:50.449817  3300 leader_election.cc:290] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.453133  3029 leader_election.cc:304] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.454316  3287 raft_consensus.cc:493] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.445709  3302 raft_consensus.cc:493] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.454938  3287 raft_consensus.cc:515] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.455696  3302 raft_consensus.cc:3060] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.458438  3287 leader_election.cc:290] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.438679  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.441462  3135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.460773  3212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.442884  3134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.442195  3211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.438300  3265 leader_election.cc:290] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.438485  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.445824  3138 raft_consensus.cc:2468] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
W20260709 12:28:50.443289  3106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:38865, user_credentials={real_user=slave}} blocked reactor thread for 77023.6us
I20260709 12:28:50.445561  3301 raft_consensus.cc:3060] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.462692  3210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.464288  3139 raft_consensus.cc:2393] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4bb183960864493901feedeb761ae8e in current term 1: Already voted for candidate 7c70962ac2244068ab924dc9f68c98aa in this term.
I20260709 12:28:50.464468  3211 raft_consensus.cc:3060] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:50.464974  3030 leader_election.cc:343] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.460309  3287 raft_consensus.cc:2804] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.467787  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.468078  3287 raft_consensus.cc:493] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.467419  3065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.468516  3063 raft_consensus.cc:3060] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.468688  3287 raft_consensus.cc:3060] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.468863  3065 raft_consensus.cc:3060] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.468019  3030 leader_election.cc:304] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561; no voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e
W20260709 12:28:50.471364  3030 leader_election.cc:343] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.471711  3062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.472411  3062 raft_consensus.cc:2468] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 0.
I20260709 12:28:50.473219  3261 raft_consensus.cc:2749] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.466079  3064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46c1af825d54b138e97c6396918abfc" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.473858  3064 raft_consensus.cc:2468] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 0.
W20260709 12:28:50.474500  3029 leader_election.cc:343] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4bb183960864493901feedeb761ae8e (127.19.36.195:40447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.476307  3063 raft_consensus.cc:2468] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 1.
I20260709 12:28:50.476395  3287 raft_consensus.cc:515] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.477718  3259 raft_consensus.cc:515] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.478730  3287 leader_election.cc:290] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Requested vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.478904  3212 raft_consensus.cc:2393] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c70962ac2244068ab924dc9f68c98aa in current term 1: Already voted for candidate d4bb183960864493901feedeb761ae8e in this term.
I20260709 12:28:50.480012  3210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.480625  3210 raft_consensus.cc:3060] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.481432  3065 raft_consensus.cc:2468] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 1.
I20260709 12:28:50.481988  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.482471  3106 leader_election.cc:304] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:50.482880  3063 raft_consensus.cc:3060] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.483421  3265 raft_consensus.cc:2804] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.483804  3265 raft_consensus.cc:493] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.483981  3106 leader_election.cc:304] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:50.484838  3300 raft_consensus.cc:2804] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.484171  3265 raft_consensus.cc:3060] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.485474  3300 raft_consensus.cc:697] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:50.485579  3211 raft_consensus.cc:2468] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 1.
I20260709 12:28:50.486536  3300 consensus_queue.cc:237] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.484934  3106 leader_election.cc:304] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:50.492467  3299 raft_consensus.cc:2804] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.492945  3259 leader_election.cc:290] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.493383  3258 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.493012  3299 raft_consensus.cc:697] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:50.490934  3063 raft_consensus.cc:2468] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:50.493944  3258 ts_tablet_manager.cc:1403] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.163s	user 0.032s	sys 0.052s
I20260709 12:28:50.494343  3299 consensus_queue.cc:237] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.495709  3180 leader_election.cc:304] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.497280  3258 raft_consensus.cc:359] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.498013  3258 raft_consensus.cc:385] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.498267  3258 raft_consensus.cc:740] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.498896  3258 consensus_queue.cc:260] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.499524  3265 raft_consensus.cc:515] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.501413  3258 ts_tablet_manager.cc:1434] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:50.502475  3258 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.503079  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.503860  3063 raft_consensus.cc:2393] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c70962ac2244068ab924dc9f68c98aa in current term 1: Already voted for candidate d4bb183960864493901feedeb761ae8e in this term.
I20260709 12:28:50.508080  3265 leader_election.cc:290] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.512459  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.512720  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.513177  3139 raft_consensus.cc:2468] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 0.
I20260709 12:28:50.513311  3140 raft_consensus.cc:3060] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:50.513878  3030 leader_election.cc:343] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.515301  3303 raft_consensus.cc:2804] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.525010  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.526165  3139 raft_consensus.cc:2393] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4bb183960864493901feedeb761ae8e in current term 1: Already voted for candidate 7c70962ac2244068ab924dc9f68c98aa in this term.
I20260709 12:28:50.525766  3030 leader_election.cc:304] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: d4bb183960864493901feedeb761ae8e
I20260709 12:28:50.527091  3287 raft_consensus.cc:2804] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.527474  3287 raft_consensus.cc:493] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.527793  3287 raft_consensus.cc:3060] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.523701  3302 raft_consensus.cc:515] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.528479  3210 raft_consensus.cc:2468] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 1.
I20260709 12:28:50.529747  3211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.535564  3213 raft_consensus.cc:2393] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c70962ac2244068ab924dc9f68c98aa in current term 1: Already voted for candidate d4bb183960864493901feedeb761ae8e in this term.
I20260709 12:28:50.539654  3287 raft_consensus.cc:515] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.540372  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.542506  3287 leader_election.cc:290] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Requested vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.542658  3214 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 1.
I20260709 12:28:50.537812  3259 raft_consensus.cc:2764] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:50.538367  3302 leader_election.cc:290] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.537838  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e16762ec3840b2952fe56ae6039c0b" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.545528  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.548645  3258 tablet_bootstrap.cc:654] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.551380  3303 raft_consensus.cc:697] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:50.553771  3063 raft_consensus.cc:2393] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4bb183960864493901feedeb761ae8e in current term 1: Already voted for candidate 7c70962ac2244068ab924dc9f68c98aa in this term.
I20260709 12:28:50.558459  3301 raft_consensus.cc:515] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.558967  3180 leader_election.cc:304] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:50.558559  2962 catalog_manager.cc:5697] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 1 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.559351  3254 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.555747  3029 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:50.560037  3254 ts_tablet_manager.cc:1403] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.186s	user 0.020s	sys 0.023s
I20260709 12:28:50.560567  3287 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:50.560997  3029 leader_election.cc:343] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4bb183960864493901feedeb761ae8e (127.19.36.195:40447): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.561443  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46c1af825d54b138e97c6396918abfc" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.561741  3029 leader_election.cc:304] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.562047  3261 raft_consensus.cc:493] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.556381  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.562618  3261 raft_consensus.cc:515] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.563133  3139 raft_consensus.cc:3060] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.563906  3210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46c1af825d54b138e97c6396918abfc" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.564538  3210 raft_consensus.cc:3060] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.564786  3261 leader_election.cc:290] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.565305  3261 raft_consensus.cc:2804] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.565745  3261 raft_consensus.cc:697] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Becoming Leader. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Running, Role: LEADER
I20260709 12:28:50.567070  3301 leader_election.cc:290] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.568392  3138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbb0668ef9405b8e4dcaa137f53339" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.569049  3138 raft_consensus.cc:2468] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 0.
I20260709 12:28:50.570338  3030 leader_election.cc:304] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:50.561041  3287 raft_consensus.cc:697] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Becoming Leader. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Running, Role: LEADER
I20260709 12:28:50.572108  3287 consensus_queue.cc:237] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.572685  3210 raft_consensus.cc:2468] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 1.
I20260709 12:28:50.574687  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbb0668ef9405b8e4dcaa137f53339" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.574326  3309 raft_consensus.cc:2804] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.575748  3309 raft_consensus.cc:493] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.575663  3213 raft_consensus.cc:2468] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 0.
I20260709 12:28:50.576689  3309 raft_consensus.cc:3060] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.577091  3104 leader_election.cc:304] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.562080  3063 raft_consensus.cc:3060] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.578073  3301 raft_consensus.cc:2804] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.578812  3301 raft_consensus.cc:697] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:50.579698  3301 consensus_queue.cc:237] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.581705  3140 raft_consensus.cc:2468] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 1.
I20260709 12:28:50.581859  3307 raft_consensus.cc:2749] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.566556  3261 consensus_queue.cc:237] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.585639  3139 raft_consensus.cc:2468] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 1.
I20260709 12:28:50.585412  3309 raft_consensus.cc:515] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.592988  3309 leader_election.cc:290] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Requested vote from peers 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.582733  3303 consensus_queue.cc:237] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:50.595774  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbb0668ef9405b8e4dcaa137f53339" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:50.596442  3213 raft_consensus.cc:3060] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.600142  3258 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.600836  3258 ts_tablet_manager.cc:1403] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.001s
I20260709 12:28:50.605253  3213 raft_consensus.cc:2468] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 1.
I20260709 12:28:50.605161  3254 raft_consensus.cc:359] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.606634  3254 raft_consensus.cc:385] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.606920  3029 leader_election.cc:304] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.608799  3309 raft_consensus.cc:2804] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.609282  3309 raft_consensus.cc:697] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Becoming Leader. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Running, Role: LEADER
I20260709 12:28:50.610193  3309 consensus_queue.cc:237] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.611506  3063 raft_consensus.cc:2468] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 1.
I20260709 12:28:50.615708  3104 leader_election.cc:304] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa; no voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e
I20260709 12:28:50.607034  3254 raft_consensus.cc:740] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.627559  3254 consensus_queue.cc:260] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.623558  3300 raft_consensus.cc:2749] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.628134  3303 raft_consensus.cc:493] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:50.632460  3030 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.194:37223, user_credentials={real_user=slave}} blocked reactor thread for 62187.6us
I20260709 12:28:50.635658  3303 raft_consensus.cc:515] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.638922  3258 raft_consensus.cc:359] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.639742  3258 raft_consensus.cc:385] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.640002  3258 raft_consensus.cc:740] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.640901  3258 consensus_queue.cc:260] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.638247  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbb0668ef9405b8e4dcaa137f53339" candidate_uuid: "148b0777ed6f426ab1b748decd6e7561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.638279  3030 leader_election.cc:304] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: d4bb183960864493901feedeb761ae8e
I20260709 12:28:50.644570  3309 raft_consensus.cc:2804] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.645239  3309 raft_consensus.cc:697] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Becoming Leader. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Running, Role: LEADER
I20260709 12:28:50.645809  3140 raft_consensus.cc:3060] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.645882  3303 leader_election.cc:290] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.646071  3309 consensus_queue.cc:237] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.646996  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.647787  2960 catalog_manager.cc:5697] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 0 to 1, leader changed from <none> to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 1 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:50.651010  3179 leader_election.cc:343] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.651943  3258 ts_tablet_manager.cc:1434] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.051s	user 0.010s	sys 0.000s
I20260709 12:28:50.651981  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.652973  3063 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
I20260709 12:28:50.653505  3258 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa: Bootstrap starting.
I20260709 12:28:50.652814  2961 catalog_manager.cc:5697] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "148b0777ed6f426ab1b748decd6e7561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.654165  3180 leader_election.cc:304] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:50.655407  3303 raft_consensus.cc:2804] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.655767  3303 raft_consensus.cc:493] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.656080  3303 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.661000  3258 tablet_bootstrap.cc:654] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.663431  3254 ts_tablet_manager.cc:1434] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.011s
I20260709 12:28:50.668848  3254 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e: Bootstrap starting.
I20260709 12:28:50.674659  3254 tablet_bootstrap.cc:654] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.664979  3303 raft_consensus.cc:515] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.678725  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.679358  3063 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.683908  3300 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.684569  3300 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.686827  3300 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:50.689060  3065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.690450  3211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:50.691323  3211 raft_consensus.cc:2393] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c70962ac2244068ab924dc9f68c98aa in current term 1: Already voted for candidate 148b0777ed6f426ab1b748decd6e7561 in this term.
I20260709 12:28:50.692343  3303 leader_election.cc:290] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.693445  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.695894  3106 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa; no voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e
W20260709 12:28:50.697132  3179 leader_election.cc:343] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.664144  3140 raft_consensus.cc:2468] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 148b0777ed6f426ab1b748decd6e7561 in term 1.
I20260709 12:28:50.699004  3300 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.700301  3258 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa: No bootstrap required, opened a new log
I20260709 12:28:50.706583  3258 ts_tablet_manager.cc:1403] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.020s
I20260709 12:28:50.687348  3063 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:50.708616  3180 leader_election.cc:304] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:50.710249  3303 raft_consensus.cc:2804] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.710702  3258 raft_consensus.cc:359] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.710892  3303 raft_consensus.cc:697] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:50.712595  3303 consensus_queue.cc:237] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.713898  3300 raft_consensus.cc:2749] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.715904  2962 catalog_manager.cc:5697] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 1 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.717512  3254 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e: No bootstrap required, opened a new log
I20260709 12:28:50.718006  3254 ts_tablet_manager.cc:1403] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.001s
I20260709 12:28:50.717897  2962 catalog_manager.cc:5697] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 1 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.719477  3258 raft_consensus.cc:385] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.719787  3258 raft_consensus.cc:740] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.720522  3254 raft_consensus.cc:359] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.720593  3258 consensus_queue.cc:260] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.721405  3254 raft_consensus.cc:385] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.721714  3254 raft_consensus.cc:740] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.723399  3258 ts_tablet_manager.cc:1434] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:28:50.726119  3254 consensus_queue.cc:260] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.726269  2961 catalog_manager.cc:5697] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 0 to 1, leader changed from <none> to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 1 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:50.728319  3254 ts_tablet_manager.cc:1434] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:28:50.737839  2960 catalog_manager.cc:5697] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "148b0777ed6f426ab1b748decd6e7561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.739681  2960 catalog_manager.cc:5697] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "148b0777ed6f426ab1b748decd6e7561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.741253  2960 catalog_manager.cc:5697] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "148b0777ed6f426ab1b748decd6e7561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.763557  3303 raft_consensus.cc:493] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.764047  3303 raft_consensus.cc:515] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.765678  3303 leader_election.cc:290] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.766608  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.766739  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.767484  3140 raft_consensus.cc:2393] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4bb183960864493901feedeb761ae8e in current term 1: Already voted for candidate 148b0777ed6f426ab1b748decd6e7561 in this term.
I20260709 12:28:50.768390  3179 leader_election.cc:304] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:50.769007  3303 raft_consensus.cc:3060] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.773949  3303 raft_consensus.cc:2749] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.902714  3303 raft_consensus.cc:493] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.903180  3303 raft_consensus.cc:515] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.905447  3303 leader_election.cc:290] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.906164  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561" is_pre_election: true
I20260709 12:28:50.906551  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:50.906816  3063 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
I20260709 12:28:50.906387  3300 consensus_queue.cc:1048] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.907354  3140 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 0.
I20260709 12:28:50.908023  3180 leader_election.cc:304] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.908890  3303 raft_consensus.cc:2804] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.909272  3303 raft_consensus.cc:493] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.909552  3303 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.916484  3303 raft_consensus.cc:515] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.919008  3303 leader_election.cc:290] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:50.919880  3063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:50.920506  3063 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.920410  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:50.921245  3140 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.921470  3300 consensus_queue.cc:1048] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:50.928663  3063 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:50.928705  3140 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:50.929870  3180 leader_election.cc:304] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:50.930831  3307 raft_consensus.cc:2804] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.931298  3307 raft_consensus.cc:697] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:50.932144  3307 consensus_queue.cc:237] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:50.947551  2960 catalog_manager.cc:5697] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 0 to 1, leader changed from <none> to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 1 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.005592 19603 ts_manager.cc:295] Set tserver state for 148b0777ed6f426ab1b748decd6e7561 to MAINTENANCE_MODE
I20260709 12:28:51.006376 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
I20260709 12:28:51.022360  3300 raft_consensus.cc:493] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.023838  3300 raft_consensus.cc:515] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:51.026235  3300 leader_election.cc:290] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:51.027940 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:51.028625 19603 tablet_replica.cc:333] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.029538 19603 raft_consensus.cc:2243] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:51.030131 19603 raft_consensus.cc:2272] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.032333  3211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:51.033604  3309 consensus_queue.cc:1048] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:51.035079  3106 consensus_peers.cc:597] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:51.035480  3180 consensus_peers.cc:597] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:51.035848  3106 leader_election.cc:336] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:51.036334  3106 leader_election.cc:304] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa; no voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e
I20260709 12:28:51.037083  3300 raft_consensus.cc:2749] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.039594 19603 tablet_replica.cc:333] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.040508 19603 raft_consensus.cc:2243] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:51.041428 19603 raft_consensus.cc:2272] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.043943 19603 tablet_replica.cc:333] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.044489 19603 raft_consensus.cc:2243] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:51.045194 19603 pending_rounds.cc:70] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Trying to abort 1 pending ops.
I20260709 12:28:51.045395 19603 pending_rounds.cc:77] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626135137181696 op_type: NO_OP noop_request { }
I20260709 12:28:51.045684 19603 raft_consensus.cc:2889] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:51.045961 19603 raft_consensus.cc:2272] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.048141 19603 tablet_replica.cc:333] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.048687 19603 raft_consensus.cc:2243] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:51.049115 19603 raft_consensus.cc:2272] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.049021  3287 consensus_queue.cc:1048] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.051545 19603 tablet_replica.cc:333] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.052321 19603 raft_consensus.cc:2243] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:51.052775 19603 raft_consensus.cc:2272] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.056211 19603 tablet_replica.cc:333] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.056355  3287 consensus_queue.cc:1048] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.056931 19603 raft_consensus.cc:2243] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:51.057503 19603 raft_consensus.cc:2272] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.059636 19603 tablet_replica.cc:333] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.060194 19603 raft_consensus.cc:2243] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:51.060940 19603 raft_consensus.cc:2272] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.063397 19603 tablet_replica.cc:333] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.064116 19603 raft_consensus.cc:2243] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:51.064951 19603 pending_rounds.cc:70] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Trying to abort 1 pending ops.
I20260709 12:28:51.065214 19603 pending_rounds.cc:77] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626134992367616 op_type: NO_OP noop_request { }
I20260709 12:28:51.065544 19603 raft_consensus.cc:2889] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:51.065884 19603 raft_consensus.cc:2272] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.070725  3309 consensus_queue.cc:1048] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.073580 19603 tablet_replica.cc:333] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.074483 19603 raft_consensus.cc:2243] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:51.075037 19603 raft_consensus.cc:2272] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:51.076296  3300 consensus_queue.cc:1048] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.077654 19603 tablet_replica.cc:333] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: stopping tablet replica
I20260709 12:28:51.078459 19603 raft_consensus.cc:2243] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:51.079654 19603 raft_consensus.cc:2272] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:51.093808  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:51.104601  3140 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.114049  3307 consensus_queue.cc:1048] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.114441 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
W20260709 12:28:51.116536  3106 consensus_peers.cc:597] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:51.137828  3180 consensus_peers.cc:597] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:51.146871  3302 consensus_queue.cc:1048] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.172678  3301 consensus_queue.cc:1048] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.381394  3307 consensus_queue.cc:1048] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:51.395907  3180 consensus_peers.cc:597] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:51.431655  3106 consensus_peers.cc:597] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:52.094659  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:52.387475  3180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111) [suppressed 116 similar messages]
I20260709 12:28:52.471657  3345 raft_consensus.cc:493] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.472168  3345 raft_consensus.cc:515] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.473810  3345 leader_election.cc:290] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:52.474990  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
W20260709 12:28:52.477506  3106 leader_election.cc:336] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.477880  3106 leader_election.cc:304] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa; no voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e
I20260709 12:28:52.478480  3345 raft_consensus.cc:2749] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:52.496163  3302 raft_consensus.cc:493] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.496687  3302 raft_consensus.cc:515] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.498353  3302 leader_election.cc:290] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.499504  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:52.500204  3140 raft_consensus.cc:2468] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:52.501287  3179 leader_election.cc:304] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:52.501930  3302 raft_consensus.cc:2804] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:28:52.502157  3180 leader_election.cc:336] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.502308  3302 raft_consensus.cc:493] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.502740  3302 raft_consensus.cc:3060] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.507848  3302 raft_consensus.cc:515] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.509369  3302 leader_election.cc:290] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.510473  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "119b1c9bc89746d497bcb03fd8a6d5dc" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:52.511013  3140 raft_consensus.cc:3060] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:52.512856  3180 leader_election.cc:336] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.515769  3140 raft_consensus.cc:2468] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 2.
I20260709 12:28:52.516786  3179 leader_election.cc:304] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561
I20260709 12:28:52.517597  3302 raft_consensus.cc:2804] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.518596  3302 raft_consensus.cc:697] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:52.519480  3302 consensus_queue.cc:237] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.526994  2961 catalog_manager.cc:5697] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 1 to 2, leader changed from 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193) to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 2 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.545135  3346 raft_consensus.cc:493] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
I20260709 12:28:52.545711  3346 raft_consensus.cc:515] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.547504  3346 leader_election.cc:290] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.548694  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:52.549400  3140 raft_consensus.cc:2468] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:52.550518  3179 leader_election.cc:304] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:52.551324  3346 raft_consensus.cc:2804] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:28:52.551357  3180 leader_election.cc:336] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.551803  3346 raft_consensus.cc:493] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
I20260709 12:28:52.552377  3346 raft_consensus.cc:3060] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.552378  3302 raft_consensus.cc:493] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
I20260709 12:28:52.553016  3302 raft_consensus.cc:515] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.554555  3302 leader_election.cc:290] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.555568  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbb0668ef9405b8e4dcaa137f53339" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:52.556146  3140 raft_consensus.cc:2468] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:52.557233  3179 leader_election.cc:304] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:52.557984  3302 raft_consensus.cc:2804] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:52.558341  3302 raft_consensus.cc:493] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
W20260709 12:28:52.558492  3180 leader_election.cc:336] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.558707  3302 raft_consensus.cc:3060] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.559286  3346 raft_consensus.cc:515] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.561136  3346 leader_election.cc:290] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.562201  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb93f2a811249d18eb651d2489d9141" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:52.562817  3140 raft_consensus.cc:3060] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:52.564577  3180 leader_election.cc:336] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.564833  3302 raft_consensus.cc:515] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.567026  3302 leader_election.cc:290] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.568077  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7bbb0668ef9405b8e4dcaa137f53339" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:52.568660  3139 raft_consensus.cc:3060] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.570250  3140 raft_consensus.cc:2468] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 2.
W20260709 12:28:52.570477  3180 leader_election.cc:336] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.571377  3179 leader_election.cc:304] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561
I20260709 12:28:52.572041  3302 raft_consensus.cc:2804] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.572438  3302 raft_consensus.cc:697] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:52.572788  3345 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
I20260709 12:28:52.573172  3302 consensus_queue.cc:237] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.573287  3345 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.574601  3139 raft_consensus.cc:2468] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 2.
I20260709 12:28:52.574954  3346 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
I20260709 12:28:52.575608  3345 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:52.575469  3346 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.575812  3179 leader_election.cc:304] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561
I20260709 12:28:52.577651  3346 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.578056  3346 raft_consensus.cc:2804] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.578326  3346 raft_consensus.cc:697] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:52.581607  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa" is_pre_election: true
I20260709 12:28:52.578924  3346 consensus_queue.cc:237] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.582295  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:52.583154  3213 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 1.
W20260709 12:28:52.583338  3106 leader_election.cc:336] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.582370  3139 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 1.
I20260709 12:28:52.584501  3104 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561
I20260709 12:28:52.585328  3179 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:52.585718  3345 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:52.586094  3302 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:52.586131  3345 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
I20260709 12:28:52.586602  3302 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 148b0777ed6f426ab1b748decd6e7561)
I20260709 12:28:52.586804  3345 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.587074  3302 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:52.586143  3180 leader_election.cc:336] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.591958  2961 catalog_manager.cc:5697] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 1 to 2, leader changed from 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193) to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 2 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.593958  3302 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.594959  3345 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:52.595964  3302 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:52.597061  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:52.597898  3139 raft_consensus.cc:2393] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d4bb183960864493901feedeb761ae8e in current term 2: Already voted for candidate 7c70962ac2244068ab924dc9f68c98aa in this term.
W20260709 12:28:52.600875  3106 leader_election.cc:336] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.601888  3345 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
W20260709 12:28:52.602239  3180 leader_election.cc:336] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:52.602734  3180 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:52.602955  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:52.603790  3213 raft_consensus.cc:2393] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c70962ac2244068ab924dc9f68c98aa in current term 2: Already voted for candidate d4bb183960864493901feedeb761ae8e in this term.
I20260709 12:28:52.604118  3302 raft_consensus.cc:2749] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:52.605234  3104 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa; no voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e
I20260709 12:28:52.605878  3345 raft_consensus.cc:2749] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:52.608915  2961 catalog_manager.cc:5697] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 1 to 2, leader changed from 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193) to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 2 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.973160  3139 raft_consensus.cc:1275] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 12:28:52.974524  3180 consensus_peers.cc:597] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:52.974632  3302 consensus_queue.cc:1048] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.026046  3139 raft_consensus.cc:1275] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.027329  3302 consensus_queue.cc:1048] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:53.040611  3180 consensus_peers.cc:597] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:53.095656  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:53.159567  3139 raft_consensus.cc:1275] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.161266  3302 consensus_queue.cc:1048] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:53.164121  3180 consensus_peers.cc:597] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:53.442406  3106 consensus_peers.cc:597] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:53.509354  3180 consensus_peers.cc:597] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:53.804051  3106 consensus_peers.cc:597] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:53.883019  3180 consensus_peers.cc:597] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:53.922745  3106 consensus_peers.cc:597] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:53.996497  3180 consensus_peers.cc:597] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:54.096822  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:54.254323  3360 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.254917  3360 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:54.256673  3360 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:54.257843  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e" is_pre_election: true
I20260709 12:28:54.258481  3213 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 2.
I20260709 12:28:54.259538  3104 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:54.260434  3360 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:28:54.260504  3106 leader_election.cc:336] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:54.260983  3360 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.261336  3360 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:54.266449  3360 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:54.268155  3360 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 3 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:54.269301  3213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:54.269923  3213 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:28:54.271662  3106 leader_election.cc:336] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111)
I20260709 12:28:54.275489  3213 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 3.
I20260709 12:28:54.276645  3104 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 148b0777ed6f426ab1b748decd6e7561
I20260709 12:28:54.277514  3360 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:54.278632  3360 raft_consensus.cc:697] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:54.279946  3360 consensus_queue.cc:237] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:54.287250  2961 catalog_manager.cc:5697] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 1 to 3, leader changed from 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193) to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 3 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:54.684823  3106 consensus_peers.cc:597] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:54.778335  3213 raft_consensus.cc:1275] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:54.779670  3361 consensus_queue.cc:1048] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:55.097869  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:55.482395  3180 consensus_peers.cc:597] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:55.526894  3180 consensus_peers.cc:597] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:55.786597  3180 consensus_peers.cc:597] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:56.002692  3180 consensus_peers.cc:597] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:56.098855  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:56.139110  3106 consensus_peers.cc:597] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:56.140081 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:56.145874  3372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:56.146518  3373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:56.147838  3375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:56.148757 19603 server_base.cc:1061] running on GCE node
I20260709 12:28:56.149664 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:56.149905 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:56.150147 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600136150114 us; error 0 us; skew 500 ppm
I20260709 12:28:56.150637 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:56.152953 19603 webserver.cc:533] Webserver started at http://127.19.36.193:39585/ using document root <none> and password file <none>
I20260709 12:28:56.153375 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:56.153528 19603 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:28:56.156975 19603 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:28:56.160588  3380 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:56.161353 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:28:56.161613 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "148b0777ed6f426ab1b748decd6e7561"
format_stamp: "Formatted at 2026-07-09 12:28:49 on dist-test-slave-hfcn"
I20260709 12:28:56.161856 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:28:56.170615  3106 consensus_peers.cc:597] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:56.175174 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:56.176334 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:56.177742 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:56.181254  3387 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:28:56.208132  3180 consensus_peers.cc:597] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:56.230904 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:28:56.231600 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.051s	user 0.001s	sys 0.002s
I20260709 12:28:56.231997 19603 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:28:56.237533  3387 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.248917  3387 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.249814  3387 tablet_bootstrap.cc:492] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.250514  3387 ts_tablet_manager.cc:1403] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:28:56.253087  3387 raft_consensus.cc:359] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.253635  3387 raft_consensus.cc:740] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.254364  3387 consensus_queue.cc:260] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.256662  3387 ts_tablet_manager.cc:1434] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.004s
I20260709 12:28:56.257373  3387 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
W20260709 12:28:56.264492  3106 consensus_peers.cc:597] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:56.269295  3387 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.270038  3387 tablet_bootstrap.cc:492] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.270648  3387 ts_tablet_manager.cc:1403] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.013s	user 0.003s	sys 0.007s
I20260709 12:28:56.272775  3387 raft_consensus.cc:359] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.273305  3387 raft_consensus.cc:740] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.273880  3387 consensus_queue.cc:260] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.275416  3387 ts_tablet_manager.cc:1434] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:28:56.276088  3387 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.277083 19603 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:28:56.277426 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.045s	user 0.042s	sys 0.002s
I20260709 12:28:56.289167  3387 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:28:56.290040  3387 tablet_bootstrap.cc:492] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.290699  3387 ts_tablet_manager.cc:1403] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.004s
I20260709 12:28:56.293068  3387 raft_consensus.cc:359] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.293830  3387 raft_consensus.cc:740] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.294667  3387 consensus_queue.cc:260] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.296628  3387 ts_tablet_manager.cc:1434] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:56.297467  3387 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.310643  3387 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:28:56.311602  3387 tablet_bootstrap.cc:492] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.312150  3387 ts_tablet_manager.cc:1403] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 12:28:56.313900  3387 raft_consensus.cc:359] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.314599  3387 raft_consensus.cc:740] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.315331  3387 consensus_queue.cc:260] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.316959  3387 ts_tablet_manager.cc:1434] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:28:56.317597  3387 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.327373  3387 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.328004  3387 tablet_bootstrap.cc:492] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.328542  3387 ts_tablet_manager.cc:1403] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:28:56.330680  3387 raft_consensus.cc:359] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.331213  3387 raft_consensus.cc:740] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.331863  3387 consensus_queue.cc:260] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.334086  3387 ts_tablet_manager.cc:1434] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:28:56.334903  3387 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.345240 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:38865
I20260709 12:28:56.345353  3451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:38865 every 8 connection(s)
I20260709 12:28:56.346850  3387 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.347821  3387 tablet_bootstrap.cc:492] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.348361  3387 ts_tablet_manager.cc:1403] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:28:56.350168  3387 raft_consensus.cc:359] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.350852  3387 raft_consensus.cc:740] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.351661  3387 consensus_queue.cc:260] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
W20260709 12:28:56.352229  3453 tablet.cc:2406] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:56.353560  3387 ts_tablet_manager.cc:1434] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:56.355119  3387 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.352968 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:56.375816  3387 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.377200  3387 tablet_bootstrap.cc:492] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.378113  3387 ts_tablet_manager.cc:1403] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.000s
I20260709 12:28:56.381143  3387 raft_consensus.cc:359] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.381740  3387 raft_consensus.cc:740] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.382427  3387 consensus_queue.cc:260] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.384032  3387 ts_tablet_manager.cc:1434] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:28:56.384845  3387 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.389170  3452 heartbeater.cc:344] Connected to a master server at 127.19.36.254:40081
I20260709 12:28:56.389652  3452 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:56.390588  3452 heartbeater.cc:507] Master 127.19.36.254:40081 requested a full tablet report, sending...
I20260709 12:28:56.392294  3211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
mode: GRACEFUL
new_leader_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:59956
I20260709 12:28:56.393018  3211 raft_consensus.cc:606] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Received request to transfer leadership to TS 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:56.395885  3211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
mode: GRACEFUL
new_leader_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:59956
I20260709 12:28:56.396463  3211 raft_consensus.cc:606] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Received request to transfer leadership to TS 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:56.397498  2961 ts_manager.cc:194] Re-registered known tserver with Master: 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865)
I20260709 12:28:56.397639 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:56.397841  3387 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.398407 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:56.398705  3387 tablet_bootstrap.cc:492] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.399238 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:56.399484  3387 ts_tablet_manager.cc:1403] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 12:28:56.401933  3387 raft_consensus.cc:359] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:56.402611  3387 raft_consensus.cc:740] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.403270  3387 consensus_queue.cc:260] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:28:56.405236  3387 ts_tablet_manager.cc:1434] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:28:56.405979  3387 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.406019  2961 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46520
I20260709 12:28:56.409161  3452 heartbeater.cc:499] Master 127.19.36.254:40081 was elected leader, sending a full tablet report...
I20260709 12:28:56.417805  3387 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.418550  3387 tablet_bootstrap.cc:492] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.419142  3387 ts_tablet_manager.cc:1403] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 12:28:56.421079  3387 raft_consensus.cc:359] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.421612  3387 raft_consensus.cc:740] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.422271  3387 consensus_queue.cc:260] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.424337  3387 ts_tablet_manager.cc:1434] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:56.425209  3387 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap starting.
I20260709 12:28:56.437525  3463 raft_consensus.cc:993] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: : Instructing follower 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:56.438144  3371 raft_consensus.cc:1081] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Signalling peer 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:56.440641  3387 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:56.440716  3140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:45494
I20260709 12:28:56.441450  3140 raft_consensus.cc:493] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.441635  3387 tablet_bootstrap.cc:492] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Bootstrap complete.
I20260709 12:28:56.441816  3140 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.442227  3387 ts_tablet_manager.cc:1403] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Time spent bootstrapping tablet: real 0.017s	user 0.006s	sys 0.008s
I20260709 12:28:56.442299  3425 raft_consensus.cc:3060] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.444084  3387 raft_consensus.cc:359] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.444595  3387 raft_consensus.cc:740] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148b0777ed6f426ab1b748decd6e7561, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.445187  3387 consensus_queue.cc:260] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.447052  3387 ts_tablet_manager.cc:1434] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:56.449903  3425 pending_rounds.cc:85] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Aborting all ops after (but not including) 0
I20260709 12:28:56.450259  3425 pending_rounds.cc:107] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:28:56.450644  3425 raft_consensus.cc:2889] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:56.450659  3140 raft_consensus.cc:515] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.453955  3140 leader_election.cc:290] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:56.454125  3211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:56.454698  3211 raft_consensus.cc:3055] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.455088  3211 raft_consensus.cc:740] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:56.455884  3211 consensus_queue.cc:260] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.457047  3211 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.463438  3211 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 2.
I20260709 12:28:56.464557  3104 leader_election.cc:304] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:56.465296  3388 raft_consensus.cc:2804] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.465741  3388 raft_consensus.cc:697] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:56.466462  3388 consensus_queue.cc:237] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.468132  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:56.468828  3424 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.476899  2960 catalog_manager.cc:5697] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 1 to 2, leader changed from d4bb183960864493901feedeb761ae8e (127.19.36.195) to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 2 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.481148  3424 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 2.
I20260709 12:28:56.496752  3424 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:28:56.548264  3424 raft_consensus.cc:3060] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.800251  3424 raft_consensus.cc:3060] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.925855  3463 raft_consensus.cc:993] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: : Instructing follower 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:56.926296  3371 raft_consensus.cc:1081] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Signalling peer 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:56.927760  3140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:45494
I20260709 12:28:56.928256  3140 raft_consensus.cc:493] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.928566  3140 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.933717  3140 raft_consensus.cc:515] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.935369  3140 leader_election.cc:290] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:56.936210  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:56.936410  3211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:56.936883  3424 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.936942  3211 raft_consensus.cc:3055] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.937366  3211 raft_consensus.cc:740] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:56.938251  3211 consensus_queue.cc:260] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.939591  3211 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.944243  3424 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 2.
I20260709 12:28:56.945442  3106 leader_election.cc:304] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:56.946178  3388 raft_consensus.cc:2804] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.946223  3211 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 2.
I20260709 12:28:56.946632  3388 raft_consensus.cc:697] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:56.947661  3388 consensus_queue.cc:237] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:56.955760  2961 catalog_manager.cc:5697] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 1 to 2, leader changed from d4bb183960864493901feedeb761ae8e (127.19.36.195) to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 2 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:57.010842  3424 raft_consensus.cc:1275] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.012291  3388 consensus_queue.cc:1048] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:57.037067  3211 raft_consensus.cc:1275] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.039043  3388 consensus_queue.cc:1048] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:57.100100  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:57.381208  3211 raft_consensus.cc:1275] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.382632  3388 consensus_queue.cc:1048] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.394030  3424 raft_consensus.cc:1275] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.395399  3361 consensus_queue.cc:1048] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.409691 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:57.425534  3140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "490f22e467d94f3f91541007124a59ad"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
mode: GRACEFUL
new_leader_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:54270
I20260709 12:28:57.426088  3140 raft_consensus.cc:606] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 LEADER]: Received request to transfer leadership to TS d4bb183960864493901feedeb761ae8e
I20260709 12:28:57.428539  3140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
mode: GRACEFUL
new_leader_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:54270
I20260709 12:28:57.428994  3140 raft_consensus.cc:606] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Received request to transfer leadership to TS d4bb183960864493901feedeb761ae8e
I20260709 12:28:57.430018 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:57.430493 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:57.431072 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:58.101256  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:58.147007  3491 raft_consensus.cc:993] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa: : Instructing follower d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:58.147773  3361 raft_consensus.cc:1081] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 LEADER]: Signalling peer d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:58.148296  3491 raft_consensus.cc:993] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa: : Instructing follower d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:58.148734  3484 raft_consensus.cc:1081] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Signalling peer d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:58.149369  3211 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "490f22e467d94f3f91541007124a59ad"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:43364
I20260709 12:28:58.149957  3211 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:58.150297  3211 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:58.150375  3210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:43364
I20260709 12:28:58.150995  3210 raft_consensus.cc:493] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:58.151408  3210 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.158519  3211 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:58.159847  3210 raft_consensus.cc:515] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:58.160948  3211 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 4 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:58.161656  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:58.162037  3210 leader_election.cc:290] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 3 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:58.162487  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:58.163096  3422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:58.163930  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:58.164223  3139 raft_consensus.cc:3055] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:58.164294  3422 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.164644  3140 raft_consensus.cc:3055] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:58.164724  3139 raft_consensus.cc:740] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:58.165150  3140 raft_consensus.cc:740] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:58.165969  3140 consensus_queue.cc:260] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:58.165915  3139 consensus_queue.cc:260] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:58.167016  3140 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.167356  3139 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:58.171712  3422 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 3.
I20260709 12:28:58.164659  3424 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:58.172571  3140 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 3.
I20260709 12:28:58.173722  3179 leader_election.cc:304] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:58.174460  3492 raft_consensus.cc:2804] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:58.174853  3139 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 4.
I20260709 12:28:58.175223  3492 raft_consensus.cc:697] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:58.176371  3179 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:58.176244  3492 consensus_queue.cc:237] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:58.177263  3482 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:58.177771  3482 raft_consensus.cc:697] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:58.178577  3482 consensus_queue.cc:237] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:58.179337  3424 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 4.
I20260709 12:28:58.185505  2960 catalog_manager.cc:5697] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 2 to 3, leader changed from 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194) to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 3 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:58.200106  2960 catalog_manager.cc:5697] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 3 to 4, leader changed from 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194) to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 4 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:58.573112  3139 raft_consensus.cc:1275] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:28:58.574429  3482 consensus_queue.cc:1048] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:58.586709  3424 raft_consensus.cc:1275] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:28:58.588109  3482 consensus_queue.cc:1048] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:58.594132  3422 raft_consensus.cc:1275] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 3 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:58.595880  3482 consensus_queue.cc:1048] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:58.609223  3139 raft_consensus.cc:1275] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:58.610896  3492 consensus_queue.cc:1048] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:58.658648 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:58.666088  3210 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "490f22e467d94f3f91541007124a59ad"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
mode: GRACEFUL
new_leader_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:59956
I20260709 12:28:58.666630  3210 raft_consensus.cc:606] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 LEADER]: Received request to transfer leadership to TS 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:58.668951  3210 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
mode: GRACEFUL
new_leader_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:59956
I20260709 12:28:58.669430  3210 raft_consensus.cc:606] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Received request to transfer leadership to TS 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:28:58.670405 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:58.670826 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:58.671411 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:58.820773  3161 debug-util.cc:398] Leaking SignalData structure 0x7b08002656a0 after lost signal to thread 19606
W20260709 12:28:58.821946  3161 debug-util.cc:398] Leaking SignalData structure 0x7b0800200580 after lost signal to thread 2996
W20260709 12:28:58.823092  3161 debug-util.cc:398] Leaking SignalData structure 0x7b08001e9440 after lost signal to thread 3164
W20260709 12:28:58.824164  3161 debug-util.cc:398] Leaking SignalData structure 0x7b08002c9020 after lost signal to thread 3238
W20260709 12:28:58.825248  3161 debug-util.cc:398] Leaking SignalData structure 0x7b0800372c60 after lost signal to thread 3451
I20260709 12:28:59.026068  3481 raft_consensus.cc:993] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e: : Instructing follower 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:59.026643  3492 raft_consensus.cc:1081] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 LEADER]: Signalling peer 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:59.028353  3139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "490f22e467d94f3f91541007124a59ad"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:45494
I20260709 12:28:59.029007  3139 raft_consensus.cc:493] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:59.029410  3139 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:28:59.037925  3139 raft_consensus.cc:515] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.040336  3139 leader_election.cc:290] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 5 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:59.041374  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:59.041364  3210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490f22e467d94f3f91541007124a59ad" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:59.042862  3210 raft_consensus.cc:3055] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:28:59.042809  3424 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:28:59.043313  3210 raft_consensus.cc:740] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:59.044276  3210 consensus_queue.cc:260] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.045562  3210 raft_consensus.cc:3060] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:28:59.050871  3424 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 5.
I20260709 12:28:59.052129  3106 leader_election.cc:304] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:28:59.052913  3210 raft_consensus.cc:2468] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 5.
I20260709 12:28:59.052923  3484 raft_consensus.cc:2804] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:28:59.054430  3484 raft_consensus.cc:697] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 5 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:59.055478  3484 consensus_queue.cc:237] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.063437  2961 catalog_manager.cc:5697] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 4 to 5, leader changed from d4bb183960864493901feedeb761ae8e (127.19.36.195) to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 5 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:59.102269  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:59.137401  3482 raft_consensus.cc:993] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: : Instructing follower 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:59.137931  3492 raft_consensus.cc:1081] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Signalling peer 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:28:59.139400  3139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:45494
I20260709 12:28:59.139920  3139 raft_consensus.cc:493] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:59.140234  3139 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:59.145416  3139 raft_consensus.cc:515] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.147631  3139 leader_election.cc:290] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 4 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:28:59.147925  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:59.148141  3210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635ee0c36e9e4e49a7d2cbb4c669235a" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:28:59.148682  3424 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:59.148793  3210 raft_consensus.cc:3055] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:59.149215  3210 raft_consensus.cc:740] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:59.149864  3210 consensus_queue.cc:260] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.150734  3210 raft_consensus.cc:3060] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:59.155663  3210 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 4.
I20260709 12:28:59.156570  3424 raft_consensus.cc:2468] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 4.
I20260709 12:28:59.156798  3104 leader_election.cc:304] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:59.157698  3484 raft_consensus.cc:2804] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:59.158187  3484 raft_consensus.cc:697] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 4 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:59.159039  3484 consensus_queue.cc:237] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.166663  2961 catalog_manager.cc:5697] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 3 to 4, leader changed from d4bb183960864493901feedeb761ae8e (127.19.36.195) to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 4 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.527567  3210 raft_consensus.cc:1275] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 5 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:28:59.529547  3506 consensus_queue.cc:1048] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:28:59.552431  3424 raft_consensus.cc:1275] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 5 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:28:59.555639  3506 consensus_queue.cc:1048] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:59.611974  3210 raft_consensus.cc:1275] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:59.613576  3506 consensus_queue.cc:1048] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:59.636238  3424 raft_consensus.cc:1275] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:59.638012  3511 consensus_queue.cc:1048] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:59.679093 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:59.694890  3139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
mode: GRACEFUL
new_leader_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:54270
I20260709 12:28:59.695705  3139 raft_consensus.cc:606] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Received request to transfer leadership to TS d4bb183960864493901feedeb761ae8e
I20260709 12:28:59.698825  3139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
mode: GRACEFUL
new_leader_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:54270
I20260709 12:28:59.699345  3139 raft_consensus.cc:606] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Received request to transfer leadership to TS d4bb183960864493901feedeb761ae8e
I20260709 12:28:59.700515 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:59.701010 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:59.701820 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:59.752739  3511 raft_consensus.cc:993] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa: : Instructing follower d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:59.753448  3484 raft_consensus.cc:1081] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Signalling peer d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:59.755398  3210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:43364
I20260709 12:28:59.756258  3210 raft_consensus.cc:493] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:59.756719  3210 raft_consensus.cc:3060] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.762573  3511 raft_consensus.cc:993] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa: : Instructing follower d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:59.763248  3484 raft_consensus.cc:1081] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Signalling peer d4bb183960864493901feedeb761ae8e to start an election
I20260709 12:28:59.765347  3211 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
 from {username='slave'} at 127.0.0.1:43364
I20260709 12:28:59.765990  3211 raft_consensus.cc:493] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:59.766403  3211 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.767771  3210 raft_consensus.cc:515] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.770152  3210 leader_election.cc:290] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:59.771111  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:59.772117  3424 raft_consensus.cc:3060] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.772464  3139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38cc74f58e7d4221ba39cf9286077cca" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:59.773203  3139 raft_consensus.cc:3055] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:59.773550  3139 raft_consensus.cc:740] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:59.774807  3139 consensus_queue.cc:260] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.776530  3139 raft_consensus.cc:3060] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.776314  3211 raft_consensus.cc:515] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.779469  3211 leader_election.cc:290] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 3 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223)
I20260709 12:28:59.780128  3422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:28:59.780993  3422 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.785542  3140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "d4bb183960864493901feedeb761ae8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
I20260709 12:28:59.786307  3140 raft_consensus.cc:3055] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:59.786640  3140 raft_consensus.cc:740] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:28:59.787853  3140 consensus_queue.cc:260] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.788475  3139 raft_consensus.cc:2468] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 2.
I20260709 12:28:59.789264  3140 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.792234  3422 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 3.
I20260709 12:28:59.793992  3180 leader_election.cc:304] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:59.794703  3424 raft_consensus.cc:2468] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 2.
I20260709 12:28:59.795429  3482 raft_consensus.cc:2804] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.796633  3482 raft_consensus.cc:697] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:59.795384  3179 leader_election.cc:304] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c70962ac2244068ab924dc9f68c98aa, d4bb183960864493901feedeb761ae8e; no voters: 
I20260709 12:28:59.797793  3482 consensus_queue.cc:237] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.802263  3140 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb183960864493901feedeb761ae8e in term 3.
I20260709 12:28:59.805192  3518 raft_consensus.cc:2804] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:59.810822  3518 raft_consensus.cc:697] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Becoming Leader. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:28:59.811998  3518 consensus_queue.cc:237] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:28:59.821586  2960 catalog_manager.cc:5697] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 2 to 3, leader changed from 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194) to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 3 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.823000  2960 catalog_manager.cc:5697] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e reported cstate change: term changed from 1 to 2, leader changed from 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194) to d4bb183960864493901feedeb761ae8e (127.19.36.195). New cstate: current_term: 2 leader_uuid: "d4bb183960864493901feedeb761ae8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:00.103758  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:00.182080  3424 raft_consensus.cc:1275] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 3 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:00.183784  3520 consensus_queue.cc:1048] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:00.197237  3140 raft_consensus.cc:1275] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:00.198909  3482 consensus_queue.cc:1048] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:00.387018  3140 raft_consensus.cc:1275] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:00.388285  3518 consensus_queue.cc:1048] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:00.402014  3424 raft_consensus.cc:1275] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Refusing update from remote peer d4bb183960864493901feedeb761ae8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:00.405591  3482 consensus_queue.cc:1048] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:00.708195 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:00.715700  3210 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
mode: GRACEFUL
new_leader_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:59956
I20260709 12:29:00.716260  3210 raft_consensus.cc:606] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Received request to transfer leadership to TS 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:29:00.719359  3210 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4"
dest_uuid: "d4bb183960864493901feedeb761ae8e"
mode: GRACEFUL
new_leader_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:59956
I20260709 12:29:00.719983  3210 raft_consensus.cc:606] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Received request to transfer leadership to TS 7c70962ac2244068ab924dc9f68c98aa
I20260709 12:29:00.721230 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:00.721832 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:00.722436 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:00.811923  3482 raft_consensus.cc:993] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: : Instructing follower 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:29:00.812585  3482 raft_consensus.cc:1081] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Signalling peer 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:29:00.814492  3140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:45494
I20260709 12:29:00.814981  3140 raft_consensus.cc:493] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:00.815364  3140 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:00.820617  3140 raft_consensus.cc:515] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:29:00.822646  3140 leader_election.cc:290] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 4 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:29:00.823221  3210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:29:00.823973  3210 raft_consensus.cc:3055] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:00.823089  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a877ac06651d482f8390c1c14aa3b5e4" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:29:00.824393  3210 raft_consensus.cc:740] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:29:00.824699  3424 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:00.825475  3210 consensus_queue.cc:260] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:29:00.826885  3210 raft_consensus.cc:3060] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:00.829594  3424 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 4.
I20260709 12:29:00.830813  3106 leader_election.cc:304] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:29:00.831558  3506 raft_consensus.cc:2804] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:00.832656  3506 raft_consensus.cc:697] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 4 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:29:00.833470  3506 consensus_queue.cc:237] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } }
I20260709 12:29:00.838474  3210 raft_consensus.cc:2468] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 4.
I20260709 12:29:00.841394  2961 catalog_manager.cc:5697] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 3 to 4, leader changed from d4bb183960864493901feedeb761ae8e (127.19.36.195) to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 4 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:00.972352  3520 raft_consensus.cc:993] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e: : Instructing follower 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:29:00.972870  3482 raft_consensus.cc:1081] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Signalling peer 7c70962ac2244068ab924dc9f68c98aa to start an election
I20260709 12:29:00.974570  3140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8"
dest_uuid: "7c70962ac2244068ab924dc9f68c98aa"
 from {username='slave'} at 127.0.0.1:45494
I20260709 12:29:00.975057  3140 raft_consensus.cc:493] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:00.975440  3140 raft_consensus.cc:3060] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.980756  3140 raft_consensus.cc:515] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:29:00.982321  3140 leader_election.cc:290] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Requested vote from peers 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865), d4bb183960864493901feedeb761ae8e (127.19.36.195:40447)
I20260709 12:29:00.983099  3424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "148b0777ed6f426ab1b748decd6e7561"
I20260709 12:29:00.983644  3210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861bf7cca3d9446bb46dffc150e42bd8" candidate_uuid: "7c70962ac2244068ab924dc9f68c98aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4bb183960864493901feedeb761ae8e"
I20260709 12:29:00.983830  3424 raft_consensus.cc:3060] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.984295  3210 raft_consensus.cc:3055] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:00.984638  3210 raft_consensus.cc:740] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4bb183960864493901feedeb761ae8e, State: Running, Role: LEADER
I20260709 12:29:00.985532  3210 consensus_queue.cc:260] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:29:00.986613  3210 raft_consensus.cc:3060] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.991227  3424 raft_consensus.cc:2468] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 2.
I20260709 12:29:00.992246  3210 raft_consensus.cc:2468] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c70962ac2244068ab924dc9f68c98aa in term 2.
I20260709 12:29:00.992435  3106 leader_election.cc:304] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 148b0777ed6f426ab1b748decd6e7561, 7c70962ac2244068ab924dc9f68c98aa; no voters: 
I20260709 12:29:00.993502  3506 raft_consensus.cc:2804] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:00.994025  3506 raft_consensus.cc:697] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Becoming Leader. State: Replica: 7c70962ac2244068ab924dc9f68c98aa, State: Running, Role: LEADER
I20260709 12:29:00.995224  3506 consensus_queue.cc:237] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } }
I20260709 12:29:01.004099  2961 catalog_manager.cc:5697] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa reported cstate change: term changed from 1 to 2, leader changed from d4bb183960864493901feedeb761ae8e (127.19.36.195) to 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194). New cstate: current_term: 2 leader_uuid: "7c70962ac2244068ab924dc9f68c98aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c70962ac2244068ab924dc9f68c98aa" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 37223 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:01.105010  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:01.244401  3210 raft_consensus.cc:1275] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:01.245767  3506 consensus_queue.cc:1048] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:01.262264  3424 raft_consensus.cc:1275] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:01.263696  3533 consensus_queue.cc:1048] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:01.531230  3424 raft_consensus.cc:1275] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:01.532621  3533 consensus_queue.cc:1048] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "148b0777ed6f426ab1b748decd6e7561" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:01.546311  3210 raft_consensus.cc:1275] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Refusing update from remote peer 7c70962ac2244068ab924dc9f68c98aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:01.549067  3533 consensus_queue.cc:1048] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb183960864493901feedeb761ae8e" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 40447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:01.729396 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:01.773612 19603 tablet_server.cc:179] TabletServer@127.19.36.193:38865 shutting down...
W20260709 12:29:01.792521  3180 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 86 similar messages]
W20260709 12:29:01.796381  3180 consensus_peers.cc:597] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:01.813935 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:01.814901 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.815876 19603 raft_consensus.cc:2243] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.816421 19603 raft_consensus.cc:2272] T 861bf7cca3d9446bb46dffc150e42bd8 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.820449 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.821197 19603 raft_consensus.cc:2243] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.821683 19603 raft_consensus.cc:2272] T 490f22e467d94f3f91541007124a59ad P 148b0777ed6f426ab1b748decd6e7561 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.824498 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.825160 19603 raft_consensus.cc:2243] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.825717 19603 raft_consensus.cc:2272] T c4e16762ec3840b2952fe56ae6039c0b P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.828372 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.828926 19603 raft_consensus.cc:2243] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.829450 19603 raft_consensus.cc:2272] T 635ee0c36e9e4e49a7d2cbb4c669235a P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.832132 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.832674 19603 raft_consensus.cc:2243] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.833192 19603 raft_consensus.cc:2272] T 6fb93f2a811249d18eb651d2489d9141 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.835855 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.836493 19603 raft_consensus.cc:2243] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.837009 19603 raft_consensus.cc:2272] T 119b1c9bc89746d497bcb03fd8a6d5dc P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.839787 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.840376 19603 raft_consensus.cc:2243] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.840986 19603 raft_consensus.cc:2272] T b7bbb0668ef9405b8e4dcaa137f53339 P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.843449 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.843995 19603 raft_consensus.cc:2243] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.844480 19603 raft_consensus.cc:2272] T d46c1af825d54b138e97c6396918abfc P 148b0777ed6f426ab1b748decd6e7561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.847054 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.847563 19603 raft_consensus.cc:2243] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.848011 19603 raft_consensus.cc:2272] T 38cc74f58e7d4221ba39cf9286077cca P 148b0777ed6f426ab1b748decd6e7561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.850569 19603 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:01.851070 19603 raft_consensus.cc:2243] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.851560 19603 raft_consensus.cc:2272] T a877ac06651d482f8390c1c14aa3b5e4 P 148b0777ed6f426ab1b748decd6e7561 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:01.864696  3106 consensus_peers.cc:597] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:01.882601 19603 tablet_server.cc:196] TabletServer@127.19.36.193:38865 shutdown complete.
I20260709 12:29:01.918220 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
W20260709 12:29:01.924171  3179 consensus_peers.cc:597] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e -> Peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Couldn't send request to peer 7c70962ac2244068ab924dc9f68c98aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:01.947950 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:01.948912 19603 tablet_replica.cc:333] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.949739 19603 raft_consensus.cc:2243] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.950600 19603 raft_consensus.cc:2272] T 635ee0c36e9e4e49a7d2cbb4c669235a P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.953289 19603 tablet_replica.cc:333] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.953940 19603 raft_consensus.cc:2243] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.954751 19603 raft_consensus.cc:2272] T 490f22e467d94f3f91541007124a59ad P 7c70962ac2244068ab924dc9f68c98aa [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.957077 19603 tablet_replica.cc:333] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.957674 19603 raft_consensus.cc:2243] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.958442 19603 raft_consensus.cc:2272] T 861bf7cca3d9446bb46dffc150e42bd8 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.961216 19603 tablet_replica.cc:333] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.961827 19603 raft_consensus.cc:2243] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.962296 19603 raft_consensus.cc:2272] T b7bbb0668ef9405b8e4dcaa137f53339 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.964645 19603 tablet_replica.cc:333] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.965257 19603 raft_consensus.cc:2243] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.966079 19603 raft_consensus.cc:2272] T a877ac06651d482f8390c1c14aa3b5e4 P 7c70962ac2244068ab924dc9f68c98aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.969599 19603 tablet_replica.cc:333] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.970271 19603 raft_consensus.cc:2243] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.970777 19603 raft_consensus.cc:2272] T 38cc74f58e7d4221ba39cf9286077cca P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:01.971448  3179 consensus_peers.cc:597] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e -> Peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Couldn't send request to peer 7c70962ac2244068ab924dc9f68c98aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:01.973285 19603 tablet_replica.cc:333] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.973883 19603 raft_consensus.cc:2243] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.974701 19603 raft_consensus.cc:2272] T d46c1af825d54b138e97c6396918abfc P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.976980 19603 tablet_replica.cc:333] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.977504 19603 raft_consensus.cc:2243] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.977958 19603 raft_consensus.cc:2272] T 6fb93f2a811249d18eb651d2489d9141 P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.980176 19603 tablet_replica.cc:333] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.980641 19603 raft_consensus.cc:2243] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.981009 19603 raft_consensus.cc:2272] T 119b1c9bc89746d497bcb03fd8a6d5dc P 7c70962ac2244068ab924dc9f68c98aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.983137 19603 tablet_replica.cc:333] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa: stopping tablet replica
I20260709 12:29:01.983695 19603 raft_consensus.cc:2243] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.984524 19603 raft_consensus.cc:2272] T c4e16762ec3840b2952fe56ae6039c0b P 7c70962ac2244068ab924dc9f68c98aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.035465 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
W20260709 12:29:02.042055  3179 consensus_peers.cc:597] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e -> Peer 7c70962ac2244068ab924dc9f68c98aa (127.19.36.194:37223): Couldn't send request to peer 7c70962ac2244068ab924dc9f68c98aa. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:37223: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:02.062168 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
I20260709 12:29:02.095036 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:29:02.095531  3180 consensus_peers.cc:597] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:02.096151 19603 tablet_replica.cc:333] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.097091 19603 raft_consensus.cc:2243] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:02.097702 19603 raft_consensus.cc:2272] T 635ee0c36e9e4e49a7d2cbb4c669235a P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.100998 19603 tablet_replica.cc:333] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e: stopping tablet replica
W20260709 12:29:02.101645  3180 consensus_peers.cc:597] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e -> Peer 148b0777ed6f426ab1b748decd6e7561 (127.19.36.193:38865): Couldn't send request to peer 148b0777ed6f426ab1b748decd6e7561. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:38865: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:02.101890 19603 raft_consensus.cc:2243] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:02.102788 19603 raft_consensus.cc:2272] T b7bbb0668ef9405b8e4dcaa137f53339 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.105280 19603 tablet_replica.cc:333] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.105933 19603 raft_consensus.cc:2243] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:02.106025  3010 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:02.106406 19603 raft_consensus.cc:2272] T a877ac06651d482f8390c1c14aa3b5e4 P d4bb183960864493901feedeb761ae8e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.108990 19603 tablet_replica.cc:333] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.109622 19603 raft_consensus.cc:2243] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:02.110383 19603 raft_consensus.cc:2272] T 38cc74f58e7d4221ba39cf9286077cca P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.112471 19603 tablet_replica.cc:333] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.113018 19603 raft_consensus.cc:2243] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:02.113476 19603 raft_consensus.cc:2272] T d46c1af825d54b138e97c6396918abfc P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.115646 19603 tablet_replica.cc:333] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.116165 19603 raft_consensus.cc:2243] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:02.116966 19603 raft_consensus.cc:2272] T 119b1c9bc89746d497bcb03fd8a6d5dc P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.119261 19603 tablet_replica.cc:333] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.119791 19603 raft_consensus.cc:2243] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:02.120607 19603 raft_consensus.cc:2272] T 6fb93f2a811249d18eb651d2489d9141 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.122867 19603 tablet_replica.cc:333] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.123373 19603 raft_consensus.cc:2243] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:02.123746 19603 raft_consensus.cc:2272] T 490f22e467d94f3f91541007124a59ad P d4bb183960864493901feedeb761ae8e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.125734 19603 tablet_replica.cc:333] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.126185 19603 raft_consensus.cc:2243] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:02.126518 19603 raft_consensus.cc:2272] T 861bf7cca3d9446bb46dffc150e42bd8 P d4bb183960864493901feedeb761ae8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.128938 19603 tablet_replica.cc:333] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e: stopping tablet replica
I20260709 12:29:02.129372 19603 raft_consensus.cc:2243] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:02.129767 19603 raft_consensus.cc:2272] T c4e16762ec3840b2952fe56ae6039c0b P d4bb183960864493901feedeb761ae8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.161398 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
I20260709 12:29:02.187039 19603 master.cc:562] Master@127.19.36.254:40081 shutting down...
I20260709 12:29:02.208132 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:02.208704 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:02.209051 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P e41ae3d6200440e98528a524567da36d: stopping tablet replica
I20260709 12:29:02.229012 19603 master.cc:584] Master@127.19.36.254:40081 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13308 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:29:02.272555 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:37269
I20260709 12:29:02.273679 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:02.279083  3553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:02.279382  3554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:02.281373 19603 server_base.cc:1061] running on GCE node
W20260709 12:29:02.281428  3556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:02.282749 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:02.282932 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:02.283043 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600142283032 us; error 0 us; skew 500 ppm
I20260709 12:29:02.283844 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:02.286191 19603 webserver.cc:533] Webserver started at http://127.19.36.254:35455/ using document root <none> and password file <none>
I20260709 12:29:02.286651 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:02.286800 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:02.287011 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:02.288167 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "e497cefaa48345c8a9871544ea5e73e3"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.292392 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:29:02.295709  3561 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.296438 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:02.296764 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "e497cefaa48345c8a9871544ea5e73e3"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.297010 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:02.321053 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:02.322460 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:02.360323 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:37269
I20260709 12:29:02.360431  3612 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:37269 every 8 connection(s)
I20260709 12:29:02.364307  3613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:02.375782  3613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3: Bootstrap starting.
I20260709 12:29:02.380833  3613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.385246  3613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3: No bootstrap required, opened a new log
I20260709 12:29:02.387617  3613 raft_consensus.cc:359] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e497cefaa48345c8a9871544ea5e73e3" member_type: VOTER }
I20260709 12:29:02.388046  3613 raft_consensus.cc:385] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.388265  3613 raft_consensus.cc:740] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e497cefaa48345c8a9871544ea5e73e3, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.388885  3613 consensus_queue.cc:260] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e497cefaa48345c8a9871544ea5e73e3" member_type: VOTER }
I20260709 12:29:02.389343  3613 raft_consensus.cc:399] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:02.389586  3613 raft_consensus.cc:493] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:02.389859  3613 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.394840  3613 raft_consensus.cc:515] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e497cefaa48345c8a9871544ea5e73e3" member_type: VOTER }
I20260709 12:29:02.395385  3613 leader_election.cc:304] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e497cefaa48345c8a9871544ea5e73e3; no voters: 
I20260709 12:29:02.396521  3613 leader_election.cc:290] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:02.396936  3616 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:02.398427  3616 raft_consensus.cc:697] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 1 LEADER]: Becoming Leader. State: Replica: e497cefaa48345c8a9871544ea5e73e3, State: Running, Role: LEADER
I20260709 12:29:02.399111  3616 consensus_queue.cc:237] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e497cefaa48345c8a9871544ea5e73e3" member_type: VOTER }
I20260709 12:29:02.399713  3613 sys_catalog.cc:565] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:02.404126  3618 sys_catalog.cc:455] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e497cefaa48345c8a9871544ea5e73e3. Latest consensus state: current_term: 1 leader_uuid: "e497cefaa48345c8a9871544ea5e73e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e497cefaa48345c8a9871544ea5e73e3" member_type: VOTER } }
I20260709 12:29:02.404879  3618 sys_catalog.cc:458] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:02.408282  3617 sys_catalog.cc:455] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e497cefaa48345c8a9871544ea5e73e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e497cefaa48345c8a9871544ea5e73e3" member_type: VOTER } }
I20260709 12:29:02.409901  3617 sys_catalog.cc:458] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:02.413408  3623 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:02.418681  3623 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:02.422501 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:02.427384  3623 catalog_manager.cc:1383] Generated new cluster ID: e5012cd5564649b3a9e0a8a025e0d44d
I20260709 12:29:02.427628  3623 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:02.441339  3623 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:02.442569  3623 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:02.459338  3623 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3: Generated new TSK 0
I20260709 12:29:02.459945  3623 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:02.489014 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:02.495144  3634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:02.496279  3635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:02.499315  3637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:02.500128 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:02.501252 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:02.501425 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:02.501618 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600142501593 us; error 0 us; skew 500 ppm
I20260709 12:29:02.502115 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:02.504336 19603 webserver.cc:533] Webserver started at http://127.19.36.193:40845/ using document root <none> and password file <none>
I20260709 12:29:02.504841 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:02.505005 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:02.505204 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:02.506222 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "1c35ac24fc5a479bbaad775f441d90a2"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.510843 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:29:02.514245  3642 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.515017 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:02.515357 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "1c35ac24fc5a479bbaad775f441d90a2"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.515654 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:02.542920 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:02.544198 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:02.545682 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:02.547924 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:02.548148 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.548386 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:02.548573 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.589924 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:36693
I20260709 12:29:02.590025  3704 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:36693 every 8 connection(s)
I20260709 12:29:02.596076 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:02.602546  3709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:02.606456 19603 server_base.cc:1061] running on GCE node
W20260709 12:29:02.606739  3710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:02.606904  3712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:02.607558  3705 heartbeater.cc:344] Connected to a master server at 127.19.36.254:37269
I20260709 12:29:02.608057  3705 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:02.608376 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:02.608639 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:02.608805 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600142608788 us; error 0 us; skew 500 ppm
I20260709 12:29:02.608906  3705 heartbeater.cc:507] Master 127.19.36.254:37269 requested a full tablet report, sending...
I20260709 12:29:02.609421 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:02.611325  3578 ts_manager.cc:194] Registered new tserver with Master: 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693)
I20260709 12:29:02.612077 19603 webserver.cc:533] Webserver started at http://127.19.36.194:35501/ using document root <none> and password file <none>
I20260709 12:29:02.612598 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:02.612795 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:02.613049 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:02.613328  3578 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58124
I20260709 12:29:02.614202 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "d3c2f4693caa481e840e0dd8179661f3"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.619102 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:29:02.622232  3717 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.622991 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:02.623311 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "d3c2f4693caa481e840e0dd8179661f3"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.623550 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:02.634436 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:02.635572 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:02.636912 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:02.639330 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:02.639551 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.639827 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:02.639992 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.680076 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:34323
I20260709 12:29:02.680192  3779 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:34323 every 8 connection(s)
I20260709 12:29:02.685959 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:02.692992  3783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:02.694950  3784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:02.696434  3786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:02.697410 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:02.698302  3780 heartbeater.cc:344] Connected to a master server at 127.19.36.254:37269
I20260709 12:29:02.698504 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:29:02.698753  3780 heartbeater.cc:461] Registering TS with master...
W20260709 12:29:02.698792 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:02.699118 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600142699099 us; error 0 us; skew 500 ppm
I20260709 12:29:02.699733  3780 heartbeater.cc:507] Master 127.19.36.254:37269 requested a full tablet report, sending...
I20260709 12:29:02.699988 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:02.702132  3578 ts_manager.cc:194] Registered new tserver with Master: d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323)
I20260709 12:29:02.702764 19603 webserver.cc:533] Webserver started at http://127.19.36.195:35219/ using document root <none> and password file <none>
I20260709 12:29:02.703380 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:02.703629 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:02.703797  3578 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58132
I20260709 12:29:02.703900 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:02.705507 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "49d4a5345225457ab5a683399ccec712"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.710279 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:29:02.713591  3791 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.714329 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:29:02.714670 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "49d4a5345225457ab5a683399ccec712"
format_stamp: "Formatted at 2026-07-09 12:29:02 on dist-test-slave-hfcn"
I20260709 12:29:02.714974 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:02.728727 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:02.729877 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:02.731292 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:02.733392 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:02.733614 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.733850 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:02.734020 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:02.773448 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:46283
I20260709 12:29:02.773612  3853 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:46283 every 8 connection(s)
I20260709 12:29:02.787464  3854 heartbeater.cc:344] Connected to a master server at 127.19.36.254:37269
I20260709 12:29:02.787911  3854 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:02.788645  3854 heartbeater.cc:507] Master 127.19.36.254:37269 requested a full tablet report, sending...
I20260709 12:29:02.790429  3578 ts_manager.cc:194] Registered new tserver with Master: 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:02.791375 19603 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01462639s
I20260709 12:29:02.791805  3578 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58134
I20260709 12:29:02.814970  3578 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58146:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:02.817408  3578 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:02.876488  3745 tablet_service.cc:1511] Processing CreateTablet for tablet dee0e8ad86ab4f558107c29471d36f3f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1d70dec2d23b43abb6263f1170ebd6da]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:02.877933  3745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dee0e8ad86ab4f558107c29471d36f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:02.880402  3744 tablet_service.cc:1511] Processing CreateTablet for tablet fa6f763b5315411f91c3bb68370f4fa9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1d70dec2d23b43abb6263f1170ebd6da]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:02.881794  3744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa6f763b5315411f91c3bb68370f4fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:02.881963  3819 tablet_service.cc:1511] Processing CreateTablet for tablet dee0e8ad86ab4f558107c29471d36f3f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1d70dec2d23b43abb6263f1170ebd6da]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:02.882364  3818 tablet_service.cc:1511] Processing CreateTablet for tablet fa6f763b5315411f91c3bb68370f4fa9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1d70dec2d23b43abb6263f1170ebd6da]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:02.882827  3670 tablet_service.cc:1511] Processing CreateTablet for tablet dee0e8ad86ab4f558107c29471d36f3f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1d70dec2d23b43abb6263f1170ebd6da]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:02.883406  3819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dee0e8ad86ab4f558107c29471d36f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:02.884254  3670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dee0e8ad86ab4f558107c29471d36f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:02.884595  3818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa6f763b5315411f91c3bb68370f4fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:02.897066  3669 tablet_service.cc:1511] Processing CreateTablet for tablet fa6f763b5315411f91c3bb68370f4fa9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1d70dec2d23b43abb6263f1170ebd6da]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:02.898499  3669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa6f763b5315411f91c3bb68370f4fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:02.915271  3869 tablet_bootstrap.cc:492] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712: Bootstrap starting.
I20260709 12:29:02.918385  3871 tablet_bootstrap.cc:492] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2: Bootstrap starting.
I20260709 12:29:02.918783  3870 tablet_bootstrap.cc:492] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3: Bootstrap starting.
I20260709 12:29:02.923753  3869 tablet_bootstrap.cc:654] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.926019  3871 tablet_bootstrap.cc:654] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.929963  3869 tablet_bootstrap.cc:492] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712: No bootstrap required, opened a new log
I20260709 12:29:02.930023  3870 tablet_bootstrap.cc:654] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.930576  3869 ts_tablet_manager.cc:1403] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:29:02.932482  3871 tablet_bootstrap.cc:492] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2: No bootstrap required, opened a new log
I20260709 12:29:02.933049  3871 ts_tablet_manager.cc:1403] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 12:29:02.933611  3869 raft_consensus.cc:359] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.934628  3869 raft_consensus.cc:385] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.934971  3869 raft_consensus.cc:740] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d4a5345225457ab5a683399ccec712, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.935854  3869 consensus_queue.cc:260] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.936559  3871 raft_consensus.cc:359] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.937393  3870 tablet_bootstrap.cc:492] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3: No bootstrap required, opened a new log
I20260709 12:29:02.938074  3870 ts_tablet_manager.cc:1403] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.002s
I20260709 12:29:02.937371  3871 raft_consensus.cc:385] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.937992  3854 heartbeater.cc:499] Master 127.19.36.254:37269 was elected leader, sending a full tablet report...
I20260709 12:29:02.938360  3869 ts_tablet_manager.cc:1434] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:02.938618  3871 raft_consensus.cc:740] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.939695  3869 tablet_bootstrap.cc:492] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712: Bootstrap starting.
I20260709 12:29:02.939666  3871 consensus_queue.cc:260] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.941154  3870 raft_consensus.cc:359] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.942168  3870 raft_consensus.cc:385] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.942590  3870 raft_consensus.cc:740] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.943806  3870 consensus_queue.cc:260] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.946949  3869 tablet_bootstrap.cc:654] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.950958  3705 heartbeater.cc:499] Master 127.19.36.254:37269 was elected leader, sending a full tablet report...
I20260709 12:29:02.951246  3871 ts_tablet_manager.cc:1434] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.007s
I20260709 12:29:02.954702  3870 ts_tablet_manager.cc:1434] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.006s
I20260709 12:29:02.955497  3780 heartbeater.cc:499] Master 127.19.36.254:37269 was elected leader, sending a full tablet report...
I20260709 12:29:02.956082  3870 tablet_bootstrap.cc:492] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3: Bootstrap starting.
I20260709 12:29:02.957334  3869 tablet_bootstrap.cc:492] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712: No bootstrap required, opened a new log
I20260709 12:29:02.957849  3869 ts_tablet_manager.cc:1403] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.005s
I20260709 12:29:02.960814  3869 raft_consensus.cc:359] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.961563  3869 raft_consensus.cc:385] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.961863  3869 raft_consensus.cc:740] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d4a5345225457ab5a683399ccec712, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.962594  3869 consensus_queue.cc:260] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.964931  3869 ts_tablet_manager.cc:1434] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 12:29:02.967619  3870 tablet_bootstrap.cc:654] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.969157  3871 tablet_bootstrap.cc:492] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2: Bootstrap starting.
I20260709 12:29:02.974352  3877 raft_consensus.cc:493] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.974884  3877 raft_consensus.cc:515] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.978247  3871 tablet_bootstrap.cc:654] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.980383  3877 leader_election.cc:290] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:02.993019  3680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2" is_pre_election: true
I20260709 12:29:02.994776  3871 tablet_bootstrap.cc:492] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2: No bootstrap required, opened a new log
I20260709 12:29:02.995414  3871 ts_tablet_manager.cc:1403] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.016s
I20260709 12:29:02.996474  3870 tablet_bootstrap.cc:492] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3: No bootstrap required, opened a new log
W20260709 12:29:02.994856  3720 leader_election.cc:343] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.997035  3870 ts_tablet_manager.cc:1403] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.008s
I20260709 12:29:02.997619  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712" is_pre_election: true
I20260709 12:29:02.998231  3829 raft_consensus.cc:2468] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 0.
I20260709 12:29:02.998389  3871 raft_consensus.cc:359] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:02.999289  3871 raft_consensus.cc:385] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.999668  3871 raft_consensus.cc:740] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.999745  3718 leader_election.cc:304] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49d4a5345225457ab5a683399ccec712, d3c2f4693caa481e840e0dd8179661f3; no voters: 1c35ac24fc5a479bbaad775f441d90a2
I20260709 12:29:03.000485  3871 consensus_queue.cc:260] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.000901  3877 raft_consensus.cc:2804] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:03.001432  3877 raft_consensus.cc:493] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.001767  3877 raft_consensus.cc:3060] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.002805  3871 ts_tablet_manager.cc:1434] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:03.000890  3870 raft_consensus.cc:359] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.007993  3870 raft_consensus.cc:385] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:03.008224  3877 raft_consensus.cc:515] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.008257  3870 raft_consensus.cc:740] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.009527  3870 consensus_queue.cc:260] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.010207  3877 leader_election.cc:290] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 election: Requested vote from peers 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.010998  3680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
I20260709 12:29:03.011626  3680 raft_consensus.cc:3060] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.011889  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712"
I20260709 12:29:03.012460  3829 raft_consensus.cc:3060] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.017554  3870 ts_tablet_manager.cc:1434] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3: Time spent starting tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 12:29:03.019634  3680 raft_consensus.cc:2468] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 1.
I20260709 12:29:03.020851  3720 leader_election.cc:304] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, d3c2f4693caa481e840e0dd8179661f3; no voters: 
I20260709 12:29:03.020991  3829 raft_consensus.cc:2468] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 1.
I20260709 12:29:03.021864  3882 raft_consensus.cc:2804] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:03.022421  3882 raft_consensus.cc:697] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Becoming Leader. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Running, Role: LEADER
I20260709 12:29:03.023249  3882 consensus_queue.cc:237] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.031430  3576 catalog_manager.cc:5697] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "d3c2f4693caa481e840e0dd8179661f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.171150  3876 raft_consensus.cc:493] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.171723  3876 raft_consensus.cc:515] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.173799  3876 leader_election.cc:290] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.188845  3755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c2f4693caa481e840e0dd8179661f3" is_pre_election: true
I20260709 12:29:03.189612  3755 raft_consensus.cc:2468] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 0.
I20260709 12:29:03.190097  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712" is_pre_election: true
I20260709 12:29:03.190714  3645 leader_election.cc:304] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, d3c2f4693caa481e840e0dd8179661f3; no voters: 
I20260709 12:29:03.190853  3829 raft_consensus.cc:2468] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 0.
I20260709 12:29:03.191496  3876 raft_consensus.cc:2804] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:03.191890  3876 raft_consensus.cc:493] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.192243  3876 raft_consensus.cc:3060] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.197714  3876 raft_consensus.cc:515] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.199273  3876 leader_election.cc:290] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 election: Requested vote from peers d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.199986  3755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
I20260709 12:29:03.200301  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712"
I20260709 12:29:03.200641  3755 raft_consensus.cc:3060] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.200871  3829 raft_consensus.cc:3060] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.208071  3829 raft_consensus.cc:2468] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 1.
I20260709 12:29:03.208146  3755 raft_consensus.cc:2468] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 1.
I20260709 12:29:03.209331  3643 leader_election.cc:304] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, 49d4a5345225457ab5a683399ccec712; no voters: 
I20260709 12:29:03.210021  3876 raft_consensus.cc:2804] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:03.211072  3876 raft_consensus.cc:697] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 LEADER]: Becoming Leader. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Running, Role: LEADER
I20260709 12:29:03.211802  3876 consensus_queue.cc:237] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.218459  3576 catalog_manager.cc:5697] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "1c35ac24fc5a479bbaad775f441d90a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.296348  3576 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58150:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:03.298205  3576 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:03.320415  3669 tablet_service.cc:1511] Processing CreateTablet for tablet a904c9f91e364331a8158e0f479cb197 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0477f4fedd8b47cc9dac1377b77fb96f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:03.321748  3669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a904c9f91e364331a8158e0f479cb197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:03.321933  3744 tablet_service.cc:1511] Processing CreateTablet for tablet a904c9f91e364331a8158e0f479cb197 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0477f4fedd8b47cc9dac1377b77fb96f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:03.323297  3744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a904c9f91e364331a8158e0f479cb197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:03.323300  3819 tablet_service.cc:1511] Processing CreateTablet for tablet a904c9f91e364331a8158e0f479cb197 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0477f4fedd8b47cc9dac1377b77fb96f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:03.324780  3819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a904c9f91e364331a8158e0f479cb197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:03.326916  3670 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca710cd296849819afb2d650a107769 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0477f4fedd8b47cc9dac1377b77fb96f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:03.328212  3670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca710cd296849819afb2d650a107769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:03.330893  3745 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca710cd296849819afb2d650a107769 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0477f4fedd8b47cc9dac1377b77fb96f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:03.332180  3745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca710cd296849819afb2d650a107769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:03.336715  3818 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca710cd296849819afb2d650a107769 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0477f4fedd8b47cc9dac1377b77fb96f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:03.338004  3818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca710cd296849819afb2d650a107769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:03.342772  3870 tablet_bootstrap.cc:492] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3: Bootstrap starting.
I20260709 12:29:03.343672  3871 tablet_bootstrap.cc:492] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2: Bootstrap starting.
I20260709 12:29:03.349347  3870 tablet_bootstrap.cc:654] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:03.349478  3869 tablet_bootstrap.cc:492] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712: Bootstrap starting.
I20260709 12:29:03.351830  3871 tablet_bootstrap.cc:654] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:03.355985  3869 tablet_bootstrap.cc:654] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:03.373476  3871 tablet_bootstrap.cc:492] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2: No bootstrap required, opened a new log
I20260709 12:29:03.373920  3869 tablet_bootstrap.cc:492] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712: No bootstrap required, opened a new log
I20260709 12:29:03.374094  3871 ts_tablet_manager.cc:1403] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2: Time spent bootstrapping tablet: real 0.031s	user 0.003s	sys 0.009s
I20260709 12:29:03.374569  3869 ts_tablet_manager.cc:1403] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.013s
I20260709 12:29:03.377408  3870 tablet_bootstrap.cc:492] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3: No bootstrap required, opened a new log
I20260709 12:29:03.377976  3870 ts_tablet_manager.cc:1403] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.012s
I20260709 12:29:03.377852  3869 raft_consensus.cc:359] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.377980  3871 raft_consensus.cc:359] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.378770  3869 raft_consensus.cc:385] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:03.379029  3871 raft_consensus.cc:385] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:03.379166  3869 raft_consensus.cc:740] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d4a5345225457ab5a683399ccec712, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.379619  3871 raft_consensus.cc:740] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.380333  3869 consensus_queue.cc:260] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.380610  3871 consensus_queue.cc:260] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.381318  3870 raft_consensus.cc:359] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.382133  3870 raft_consensus.cc:385] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:03.382457  3870 raft_consensus.cc:740] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.382946  3869 ts_tablet_manager.cc:1434] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:29:03.384204  3869 tablet_bootstrap.cc:492] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712: Bootstrap starting.
I20260709 12:29:03.386206  3871 ts_tablet_manager.cc:1434] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:29:03.387277  3871 tablet_bootstrap.cc:492] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2: Bootstrap starting.
I20260709 12:29:03.383227  3870 consensus_queue.cc:260] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.390141  3870 ts_tablet_manager.cc:1434] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:29:03.391251  3869 tablet_bootstrap.cc:654] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:03.391251  3870 tablet_bootstrap.cc:492] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: Bootstrap starting.
I20260709 12:29:03.399078  3871 tablet_bootstrap.cc:654] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:03.400853  3870 tablet_bootstrap.cc:654] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:03.417972  3626 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:03.421576  3870 tablet_bootstrap.cc:492] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: No bootstrap required, opened a new log
I20260709 12:29:03.421653  3871 tablet_bootstrap.cc:492] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2: No bootstrap required, opened a new log
I20260709 12:29:03.422188  3869 tablet_bootstrap.cc:492] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712: No bootstrap required, opened a new log
I20260709 12:29:03.422413  3870 ts_tablet_manager.cc:1403] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.011s
I20260709 12:29:03.422448  3871 ts_tablet_manager.cc:1403] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.018s
I20260709 12:29:03.422698  3869 ts_tablet_manager.cc:1403] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.016s
I20260709 12:29:03.425424  3869 raft_consensus.cc:359] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.426043  3869 raft_consensus.cc:385] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:03.426614  3869 raft_consensus.cc:740] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d4a5345225457ab5a683399ccec712, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.425966  3870 raft_consensus.cc:359] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.426219  3871 raft_consensus.cc:359] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.427053  3870 raft_consensus.cc:385] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:03.427096  3871 raft_consensus.cc:385] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:03.427644  3871 raft_consensus.cc:740] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.427539  3870 raft_consensus.cc:740] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.427459  3869 consensus_queue.cc:260] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.428634  3871 consensus_queue.cc:260] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.429190  3870 consensus_queue.cc:260] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.429953  3869 ts_tablet_manager.cc:1434] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:29:03.432458  3870 ts_tablet_manager.cc:1434] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:03.435799  3871 ts_tablet_manager.cc:1434] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:29:03.444936  3882 consensus_queue.cc:1048] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:03.459372  3882 consensus_queue.cc:1048] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:03.489840  3876 raft_consensus.cc:493] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.490325  3876 raft_consensus.cc:515] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.492065  3876 leader_election.cc:290] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.492908  3755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a904c9f91e364331a8158e0f479cb197" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c2f4693caa481e840e0dd8179661f3" is_pre_election: true
I20260709 12:29:03.493304  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a904c9f91e364331a8158e0f479cb197" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712" is_pre_election: true
I20260709 12:29:03.493566  3755 raft_consensus.cc:2468] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 0.
I20260709 12:29:03.494107  3829 raft_consensus.cc:2468] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 0.
I20260709 12:29:03.494632  3645 leader_election.cc:304] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, d3c2f4693caa481e840e0dd8179661f3; no voters: 
I20260709 12:29:03.495556  3876 raft_consensus.cc:2804] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:03.495886  3876 raft_consensus.cc:493] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.496109  3876 raft_consensus.cc:3060] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.501329  3876 raft_consensus.cc:515] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.502959  3876 leader_election.cc:290] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 election: Requested vote from peers d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.503721  3755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a904c9f91e364331a8158e0f479cb197" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
I20260709 12:29:03.503976  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a904c9f91e364331a8158e0f479cb197" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712"
I20260709 12:29:03.504352  3755 raft_consensus.cc:3060] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.504537  3829 raft_consensus.cc:3060] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.511386  3829 raft_consensus.cc:2468] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 1.
I20260709 12:29:03.511387  3755 raft_consensus.cc:2468] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 1.
I20260709 12:29:03.512533  3643 leader_election.cc:304] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, 49d4a5345225457ab5a683399ccec712; no voters: 
I20260709 12:29:03.513163  3876 raft_consensus.cc:2804] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:03.513672  3876 raft_consensus.cc:697] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 LEADER]: Becoming Leader. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Running, Role: LEADER
I20260709 12:29:03.514289  3876 consensus_queue.cc:237] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.521747  3577 catalog_manager.cc:5697] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "1c35ac24fc5a479bbaad775f441d90a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.617781  3882 raft_consensus.cc:493] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.618386  3882 raft_consensus.cc:515] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.620433  3882 leader_election.cc:290] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.621171  3680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2" is_pre_election: true
I20260709 12:29:03.621457  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712" is_pre_election: true
I20260709 12:29:03.621793  3680 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 0.
I20260709 12:29:03.622152  3829 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 0.
I20260709 12:29:03.622816  3720 leader_election.cc:304] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, d3c2f4693caa481e840e0dd8179661f3; no voters: 
I20260709 12:29:03.623677  3882 raft_consensus.cc:2804] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:03.624027  3882 raft_consensus.cc:493] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.624320  3882 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.629878  3882 raft_consensus.cc:515] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.631563  3882 leader_election.cc:290] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 election: Requested vote from peers 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.632316  3680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
I20260709 12:29:03.632572  3829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d4a5345225457ab5a683399ccec712"
I20260709 12:29:03.632980  3680 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.633129  3829 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:03.640111  3829 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 1.
I20260709 12:29:03.640111  3680 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 1.
I20260709 12:29:03.641403  3718 leader_election.cc:304] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49d4a5345225457ab5a683399ccec712, d3c2f4693caa481e840e0dd8179661f3; no voters: 
I20260709 12:29:03.642071  3882 raft_consensus.cc:2804] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:03.642503  3882 raft_consensus.cc:697] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Becoming Leader. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Running, Role: LEADER
I20260709 12:29:03.643312  3882 consensus_queue.cc:237] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.649783  3577 catalog_manager.cc:5697] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "d3c2f4693caa481e840e0dd8179661f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.676638  3755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f"
dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
mode: GRACEFUL
new_leader_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
 from {username='slave'} at 127.0.0.1:40144
I20260709 12:29:03.677325  3755 raft_consensus.cc:606] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Received request to transfer leadership to TS 1c35ac24fc5a479bbaad775f441d90a2
I20260709 12:29:03.692977  3680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9"
dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
mode: GRACEFUL
new_leader_uuid: "49d4a5345225457ab5a683399ccec712"
 from {username='slave'} at 127.0.0.1:44284
I20260709 12:29:03.693542  3680 raft_consensus.cc:606] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 LEADER]: Received request to transfer leadership to TS 49d4a5345225457ab5a683399ccec712
I20260709 12:29:03.702108  3755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ca710cd296849819afb2d650a107769"
dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
mode: GRACEFUL
new_leader_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
 from {username='slave'} at 127.0.0.1:40144
I20260709 12:29:03.702678  3755 raft_consensus.cc:606] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Received request to transfer leadership to TS 1c35ac24fc5a479bbaad775f441d90a2
I20260709 12:29:03.722298  3876 consensus_queue.cc:1048] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:03.729209  3886 raft_consensus.cc:993] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2: : Instructing follower 49d4a5345225457ab5a683399ccec712 to start an election
I20260709 12:29:03.730000  3876 raft_consensus.cc:1081] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 LEADER]: Signalling peer 49d4a5345225457ab5a683399ccec712 to start an election
I20260709 12:29:03.732372  3828 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9"
dest_uuid: "49d4a5345225457ab5a683399ccec712"
 from {username='slave'} at 127.0.0.1:44124
I20260709 12:29:03.733157  3828 raft_consensus.cc:493] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:03.733542  3828 raft_consensus.cc:3060] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.740061  3886 consensus_queue.cc:1048] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:03.745390  3828 raft_consensus.cc:515] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.749202  3828 leader_election.cc:290] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [CANDIDATE]: Term 2 election: Requested vote from peers 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693), d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323)
I20260709 12:29:03.765659  3680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9" candidate_uuid: "49d4a5345225457ab5a683399ccec712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
I20260709 12:29:03.766301  3680 raft_consensus.cc:3055] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.766754  3680 raft_consensus.cc:740] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Running, Role: LEADER
I20260709 12:29:03.767606  3680 consensus_queue.cc:260] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.768762  3680 raft_consensus.cc:3060] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.770898  3755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa6f763b5315411f91c3bb68370f4fa9" candidate_uuid: "49d4a5345225457ab5a683399ccec712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
I20260709 12:29:03.771631  3755 raft_consensus.cc:3060] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.776155  3680 raft_consensus.cc:2468] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d4a5345225457ab5a683399ccec712 in term 2.
I20260709 12:29:03.777544  3794 leader_election.cc:304] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, 49d4a5345225457ab5a683399ccec712; no voters: 
I20260709 12:29:03.778512  3875 raft_consensus.cc:2804] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.778721  3755 raft_consensus.cc:2468] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d4a5345225457ab5a683399ccec712 in term 2.
I20260709 12:29:03.780089  3875 raft_consensus.cc:697] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 2 LEADER]: Becoming Leader. State: Replica: 49d4a5345225457ab5a683399ccec712, State: Running, Role: LEADER
I20260709 12:29:03.780890  3875 consensus_queue.cc:237] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.787472  3577 catalog_manager.cc:5697] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 reported cstate change: term changed from 1 to 2, leader changed from 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193) to 49d4a5345225457ab5a683399ccec712 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "49d4a5345225457ab5a683399ccec712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:03.921639  3877 raft_consensus.cc:993] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3: : Instructing follower 1c35ac24fc5a479bbaad775f441d90a2 to start an election
I20260709 12:29:03.922209  3914 raft_consensus.cc:1081] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Signalling peer 1c35ac24fc5a479bbaad775f441d90a2 to start an election
I20260709 12:29:03.923734  3680 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f"
dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
 from {username='slave'} at 127.0.0.1:44270
I20260709 12:29:03.924211  3680 raft_consensus.cc:493] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:03.924608  3680 raft_consensus.cc:3060] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.929765  3680 raft_consensus.cc:515] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.931656  3680 leader_election.cc:290] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 2 election: Requested vote from peers d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:03.932376  3755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
I20260709 12:29:03.932595  3828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dee0e8ad86ab4f558107c29471d36f3f" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49d4a5345225457ab5a683399ccec712"
I20260709 12:29:03.932971  3755 raft_consensus.cc:3055] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.933223  3828 raft_consensus.cc:3060] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.933342  3755 raft_consensus.cc:740] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Running, Role: LEADER
I20260709 12:29:03.934291  3755 consensus_queue.cc:260] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.935324  3755 raft_consensus.cc:3060] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.939023  3828 raft_consensus.cc:2468] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 2.
I20260709 12:29:03.940454  3755 raft_consensus.cc:2468] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 2.
I20260709 12:29:03.940582  3643 leader_election.cc:304] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, 49d4a5345225457ab5a683399ccec712; no voters: 
I20260709 12:29:03.941524  3876 raft_consensus.cc:2804] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.942022  3876 raft_consensus.cc:697] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 LEADER]: Becoming Leader. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Running, Role: LEADER
I20260709 12:29:03.942595  3909 consensus_queue.cc:1048] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:03.942893  3876 consensus_queue.cc:237] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:03.951371  3578 catalog_manager.cc:5697] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 reported cstate change: term changed from 1 to 2, leader changed from d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194) to 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "1c35ac24fc5a479bbaad775f441d90a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.953589  3909 consensus_queue.cc:1048] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:04.047676  3914 consensus_queue.cc:1048] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:04.054206  3877 raft_consensus.cc:993] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: : Instructing follower 1c35ac24fc5a479bbaad775f441d90a2 to start an election
I20260709 12:29:04.054746  3882 raft_consensus.cc:1081] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Signalling peer 1c35ac24fc5a479bbaad775f441d90a2 to start an election
I20260709 12:29:04.056509  3679 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ca710cd296849819afb2d650a107769"
dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
 from {username='slave'} at 127.0.0.1:44270
I20260709 12:29:04.057103  3679 raft_consensus.cc:493] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:04.057487  3679 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.060185  3914 consensus_queue.cc:1048] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:04.065259  3679 raft_consensus.cc:515] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:04.068205  3679 leader_election.cc:290] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 2 election: Requested vote from peers d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194:34323), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:04.069451  3755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
I20260709 12:29:04.070155  3755 raft_consensus.cc:3055] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:04.070428  3828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "1c35ac24fc5a479bbaad775f441d90a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49d4a5345225457ab5a683399ccec712"
I20260709 12:29:04.070549  3755 raft_consensus.cc:740] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Running, Role: LEADER
I20260709 12:29:04.071292  3828 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.072001  3755 consensus_queue.cc:260] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:04.073192  3755 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.076642  3828 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 2.
I20260709 12:29:04.077867  3643 leader_election.cc:304] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c35ac24fc5a479bbaad775f441d90a2, 49d4a5345225457ab5a683399ccec712; no voters: 
I20260709 12:29:04.078560  3909 raft_consensus.cc:2804] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:04.078590  3755 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c35ac24fc5a479bbaad775f441d90a2 in term 2.
I20260709 12:29:04.079062  3909 raft_consensus.cc:697] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 LEADER]: Becoming Leader. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Running, Role: LEADER
I20260709 12:29:04.080354  3909 consensus_queue.cc:237] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:04.086323  3577 catalog_manager.cc:5697] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 reported cstate change: term changed from 1 to 2, leader changed from d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194) to 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "1c35ac24fc5a479bbaad775f441d90a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.156488 19603 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:29:04.157954 19603 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:29:04.158285 19603 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:29:04.169866  3679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ca710cd296849819afb2d650a107769"
dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
mode: GRACEFUL
new_leader_uuid: "d3c2f4693caa481e840e0dd8179661f3"
 from {username='slave'} at 127.0.0.1:44294
I20260709 12:29:04.170428  3679 raft_consensus.cc:606] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 LEADER]: Received request to transfer leadership to TS d3c2f4693caa481e840e0dd8179661f3
I20260709 12:29:04.171517 19603 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:29:04.279948  3679 raft_consensus.cc:1275] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Refusing update from remote peer 49d4a5345225457ab5a683399ccec712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.281276  3875 consensus_queue.cc:1048] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.290494  3755 raft_consensus.cc:1275] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Refusing update from remote peer 49d4a5345225457ab5a683399ccec712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.291905  3921 consensus_queue.cc:1048] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.354435  3828 raft_consensus.cc:1275] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Refusing update from remote peer 1c35ac24fc5a479bbaad775f441d90a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.355684  3909 consensus_queue.cc:1048] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.363090  3755 raft_consensus.cc:1275] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Refusing update from remote peer 1c35ac24fc5a479bbaad775f441d90a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.364584  3886 consensus_queue.cc:1048] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.542903  3828 raft_consensus.cc:1275] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Refusing update from remote peer 1c35ac24fc5a479bbaad775f441d90a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.544201  3886 consensus_queue.cc:1048] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.553601  3755 raft_consensus.cc:1275] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Refusing update from remote peer 1c35ac24fc5a479bbaad775f441d90a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.555128  3909 consensus_queue.cc:1048] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.561232  3909 raft_consensus.cc:993] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2: : Instructing follower d3c2f4693caa481e840e0dd8179661f3 to start an election
I20260709 12:29:04.561652  3922 raft_consensus.cc:1081] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 LEADER]: Signalling peer d3c2f4693caa481e840e0dd8179661f3 to start an election
I20260709 12:29:04.563046  3755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ca710cd296849819afb2d650a107769"
dest_uuid: "d3c2f4693caa481e840e0dd8179661f3"
 from {username='slave'} at 127.0.0.1:40128
I20260709 12:29:04.563553  3755 raft_consensus.cc:493] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:04.563856  3755 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:04.569084  3755 raft_consensus.cc:515] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:04.570704  3755 leader_election.cc:290] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 3 election: Requested vote from peers 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193:36693), 49d4a5345225457ab5a683399ccec712 (127.19.36.195:46283)
I20260709 12:29:04.571625  3679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1c35ac24fc5a479bbaad775f441d90a2"
I20260709 12:29:04.571889  3828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca710cd296849819afb2d650a107769" candidate_uuid: "d3c2f4693caa481e840e0dd8179661f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49d4a5345225457ab5a683399ccec712"
I20260709 12:29:04.572297  3679 raft_consensus.cc:3055] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:04.572517  3828 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:04.572719  3679 raft_consensus.cc:740] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1c35ac24fc5a479bbaad775f441d90a2, State: Running, Role: LEADER
I20260709 12:29:04.573632  3679 consensus_queue.cc:260] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:04.574587  3679 raft_consensus.cc:3060] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:04.578471  3828 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 3.
I20260709 12:29:04.579521  3718 leader_election.cc:304] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49d4a5345225457ab5a683399ccec712, d3c2f4693caa481e840e0dd8179661f3; no voters: 
I20260709 12:29:04.579763  3679 raft_consensus.cc:2468] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2f4693caa481e840e0dd8179661f3 in term 3.
I20260709 12:29:04.580229  3914 raft_consensus.cc:2804] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:04.581390  3914 raft_consensus.cc:697] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 3 LEADER]: Becoming Leader. State: Replica: d3c2f4693caa481e840e0dd8179661f3, State: Running, Role: LEADER
I20260709 12:29:04.582180  3914 consensus_queue.cc:237] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } }
I20260709 12:29:04.587687  3577 catalog_manager.cc:5697] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 reported cstate change: term changed from 2 to 3, leader changed from 1c35ac24fc5a479bbaad775f441d90a2 (127.19.36.193) to d3c2f4693caa481e840e0dd8179661f3 (127.19.36.194). New cstate: current_term: 3 leader_uuid: "d3c2f4693caa481e840e0dd8179661f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c35ac24fc5a479bbaad775f441d90a2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3c2f4693caa481e840e0dd8179661f3" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 34323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49d4a5345225457ab5a683399ccec712" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46283 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.791801 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
I20260709 12:29:04.813354 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:04.814024 19603 tablet_replica.cc:333] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2: stopping tablet replica
I20260709 12:29:04.814548 19603 raft_consensus.cc:2243] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.814963 19603 raft_consensus.cc:2272] T 6ca710cd296849819afb2d650a107769 P 1c35ac24fc5a479bbaad775f441d90a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.817468 19603 tablet_replica.cc:333] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2: stopping tablet replica
I20260709 12:29:04.817968 19603 raft_consensus.cc:2243] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:04.818689 19603 raft_consensus.cc:2272] T a904c9f91e364331a8158e0f479cb197 P 1c35ac24fc5a479bbaad775f441d90a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.820875 19603 tablet_replica.cc:333] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2: stopping tablet replica
I20260709 12:29:04.821563 19603 raft_consensus.cc:2243] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:04.822443 19603 raft_consensus.cc:2272] T dee0e8ad86ab4f558107c29471d36f3f P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.824940 19603 tablet_replica.cc:333] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2: stopping tablet replica
I20260709 12:29:04.825500 19603 raft_consensus.cc:2243] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.825969 19603 raft_consensus.cc:2272] T fa6f763b5315411f91c3bb68370f4fa9 P 1c35ac24fc5a479bbaad775f441d90a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.849718 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
I20260709 12:29:04.861770 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
I20260709 12:29:04.878682 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:04.879359 19603 tablet_replica.cc:333] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3: stopping tablet replica
I20260709 12:29:04.879931 19603 raft_consensus.cc:2243] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.880375 19603 raft_consensus.cc:2272] T fa6f763b5315411f91c3bb68370f4fa9 P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.882714 19603 tablet_replica.cc:333] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: stopping tablet replica
I20260709 12:29:04.883297 19603 raft_consensus.cc:2243] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:04.883919 19603 pending_rounds.cc:70] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: Trying to abort 1 pending ops.
I20260709 12:29:04.884083 19603 pending_rounds.cc:77] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626192215846912 op_type: NO_OP noop_request { }
I20260709 12:29:04.884348 19603 raft_consensus.cc:2889] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:04.884593 19603 raft_consensus.cc:2272] T 6ca710cd296849819afb2d650a107769 P d3c2f4693caa481e840e0dd8179661f3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.887250 19603 tablet_replica.cc:333] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3: stopping tablet replica
I20260709 12:29:04.887774 19603 raft_consensus.cc:2243] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.888159 19603 raft_consensus.cc:2272] T a904c9f91e364331a8158e0f479cb197 P d3c2f4693caa481e840e0dd8179661f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.890724 19603 tablet_replica.cc:333] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3: stopping tablet replica
I20260709 12:29:04.891249 19603 raft_consensus.cc:2243] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.891651 19603 raft_consensus.cc:2272] T dee0e8ad86ab4f558107c29471d36f3f P d3c2f4693caa481e840e0dd8179661f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.915863 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
I20260709 12:29:04.928337 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
I20260709 12:29:04.947156 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:04.947885 19603 tablet_replica.cc:333] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712: stopping tablet replica
I20260709 12:29:04.948503 19603 raft_consensus.cc:2243] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:04.949366 19603 raft_consensus.cc:2272] T fa6f763b5315411f91c3bb68370f4fa9 P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.951819 19603 tablet_replica.cc:333] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712: stopping tablet replica
I20260709 12:29:04.952349 19603 raft_consensus.cc:2243] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.952800 19603 raft_consensus.cc:2272] T 6ca710cd296849819afb2d650a107769 P 49d4a5345225457ab5a683399ccec712 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.955511 19603 tablet_replica.cc:333] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712: stopping tablet replica
I20260709 12:29:04.956035 19603 raft_consensus.cc:2243] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.956458 19603 raft_consensus.cc:2272] T a904c9f91e364331a8158e0f479cb197 P 49d4a5345225457ab5a683399ccec712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.958321 19603 tablet_replica.cc:333] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712: stopping tablet replica
I20260709 12:29:04.958815 19603 raft_consensus.cc:2243] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:04.959300 19603 raft_consensus.cc:2272] T dee0e8ad86ab4f558107c29471d36f3f P 49d4a5345225457ab5a683399ccec712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:04.971873 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
I20260709 12:29:04.984076 19603 master.cc:562] Master@127.19.36.254:37269 shutting down...
I20260709 12:29:05.000576 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:05.001081 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:05.001415 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P e497cefaa48345c8a9871544ea5e73e3: stopping tablet replica
I20260709 12:29:05.012858 19603 master.cc:584] Master@127.19.36.254:37269 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2767 ms)
[----------] 6 tests from LeaderRebalancerTest (545133 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:29:05.040654 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:43127
I20260709 12:29:05.041694 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:05.046725  3926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.046919  3927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.048524  3929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:05.049396 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:05.050377 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:05.050668 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:05.050855 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600145050836 us; error 0 us; skew 500 ppm
I20260709 12:29:05.051527 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:05.053707 19603 webserver.cc:533] Webserver started at http://127.19.36.254:44567/ using document root <none> and password file <none>
I20260709 12:29:05.054195 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:05.054382 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:05.054661 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:05.055810 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "8aff479924f94418980445175024eed4"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.060207 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:29:05.063675  3934 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.064433 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:05.064740 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "8aff479924f94418980445175024eed4"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.065038 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:05.092541 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:05.093585 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:05.129467 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:43127
I20260709 12:29:05.129596  3985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:43127 every 8 connection(s)
I20260709 12:29:05.133248  3986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.143838  3986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4: Bootstrap starting.
I20260709 12:29:05.148358  3986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.152315  3986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4: No bootstrap required, opened a new log
I20260709 12:29:05.154668  3986 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aff479924f94418980445175024eed4" member_type: VOTER }
I20260709 12:29:05.155112  3986 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.155431  3986 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aff479924f94418980445175024eed4, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.156221  3986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aff479924f94418980445175024eed4" member_type: VOTER }
I20260709 12:29:05.156911  3986 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:05.157222  3986 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:05.157497  3986 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:05.162276  3986 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aff479924f94418980445175024eed4" member_type: VOTER }
I20260709 12:29:05.162802  3986 leader_election.cc:304] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aff479924f94418980445175024eed4; no voters: 
I20260709 12:29:05.163964  3986 leader_election.cc:290] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:05.164297  3989 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:05.165676  3989 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 1 LEADER]: Becoming Leader. State: Replica: 8aff479924f94418980445175024eed4, State: Running, Role: LEADER
I20260709 12:29:05.166383  3989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aff479924f94418980445175024eed4" member_type: VOTER }
I20260709 12:29:05.167016  3986 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:05.169173  3990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8aff479924f94418980445175024eed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aff479924f94418980445175024eed4" member_type: VOTER } }
I20260709 12:29:05.169931  3990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:05.169204  3991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aff479924f94418980445175024eed4. Latest consensus state: current_term: 1 leader_uuid: "8aff479924f94418980445175024eed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aff479924f94418980445175024eed4" member_type: VOTER } }
I20260709 12:29:05.170970  3991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:05.177191  3994 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:05.185487  3994 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:05.187036 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:05.194916  3994 catalog_manager.cc:1383] Generated new cluster ID: c392dcdeacbb464ca38508bb334299ce
I20260709 12:29:05.195232  3994 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:05.208549  3994 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:05.209812  3994 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:05.226745  3994 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4: Generated new TSK 0
I20260709 12:29:05.227386  3994 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:05.254283 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:05.260282  4007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.261499  4008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.262893  4010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:05.265888 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:05.266820 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:05.266999 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:05.267247 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600145267225 us; error 0 us; skew 500 ppm
I20260709 12:29:05.267813 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:05.270401 19603 webserver.cc:533] Webserver started at http://127.19.36.193:45207/ using document root <none> and password file <none>
I20260709 12:29:05.270854 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:05.271016 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:05.271296 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:05.272367 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.276764 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:29:05.280563  4015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.281337 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:29:05.281615 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.281875 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:05.296166 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:05.297212 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:05.298599 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:05.300954 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:05.301128 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.301419 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:05.301563 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.338981 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:39303
I20260709 12:29:05.339148  4077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:39303 every 8 connection(s)
I20260709 12:29:05.343922 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:05.351208  4082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.352304  4083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.355762  4085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:05.356333  4078 heartbeater.cc:344] Connected to a master server at 127.19.36.254:43127
I20260709 12:29:05.356887  4078 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:05.357779  4078 heartbeater.cc:507] Master 127.19.36.254:43127 requested a full tablet report, sending...
I20260709 12:29:05.358471 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:05.359576 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:05.359836 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:05.360040 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600145360015 us; error 0 us; skew 500 ppm
I20260709 12:29:05.360204  3951 ts_manager.cc:194] Registered new tserver with Master: b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303)
I20260709 12:29:05.360659 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:05.362320  3951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44524
I20260709 12:29:05.363511 19603 webserver.cc:533] Webserver started at http://127.19.36.194:35663/ using document root <none> and password file <none>
I20260709 12:29:05.364121 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:05.364346 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:05.364640 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:05.365818 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "6d5500e215154de78c642bdb2cfa8417"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.371304 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:29:05.374850  4090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.375768 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:29:05.376111 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "6d5500e215154de78c642bdb2cfa8417"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.376441 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:05.404307 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:05.405529 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:05.407089 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:05.409534 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:05.409780 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.410058 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:05.410251 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.449234 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:46091
I20260709 12:29:05.449338  4153 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:46091 every 8 connection(s)
I20260709 12:29:05.454231 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:05.461678  4157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.462579  4158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:05.465292  4160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:05.466501  4154 heartbeater.cc:344] Connected to a master server at 127.19.36.254:43127
I20260709 12:29:05.467002  4154 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:05.467417 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:05.468052  4154 heartbeater.cc:507] Master 127.19.36.254:43127 requested a full tablet report, sending...
I20260709 12:29:05.468556 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:05.468868 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:05.469090 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600145469070 us; error 0 us; skew 500 ppm
I20260709 12:29:05.469740 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:05.470476  3951 ts_manager.cc:194] Registered new tserver with Master: 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:05.472183  3951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44536
I20260709 12:29:05.472647 19603 webserver.cc:533] Webserver started at http://127.19.36.195:46425/ using document root <none> and password file <none>
I20260709 12:29:05.473278 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:05.473539 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:05.473891 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:05.475482 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "825e407a29994c42b0b1196a872be0bf"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.480343 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:05.484227  4165 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.485087 19603 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:29:05.485368 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "825e407a29994c42b0b1196a872be0bf"
format_stamp: "Formatted at 2026-07-09 12:29:05 on dist-test-slave-hfcn"
I20260709 12:29:05.485646 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:05.502905 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:05.504069 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:05.505523 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:05.507901 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:05.508152 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.508416 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:05.508618 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:05.547610 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:38885
I20260709 12:29:05.547713  4227 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:38885 every 8 connection(s)
I20260709 12:29:05.561067  4228 heartbeater.cc:344] Connected to a master server at 127.19.36.254:43127
I20260709 12:29:05.561474  4228 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:05.562115  4228 heartbeater.cc:507] Master 127.19.36.254:43127 requested a full tablet report, sending...
I20260709 12:29:05.564536  3951 ts_manager.cc:194] Registered new tserver with Master: 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:05.565688 19603 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014817975s
I20260709 12:29:05.566571  3951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50320
I20260709 12:29:05.591257  3951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50326:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:05.593703  3951 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:05.662061  4193 tablet_service.cc:1511] Processing CreateTablet for tablet 7c06ad9cf1ce4740900f95114a2eadea (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:05.663558  4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c06ad9cf1ce4740900f95114a2eadea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.675060  4119 tablet_service.cc:1511] Processing CreateTablet for tablet 7c06ad9cf1ce4740900f95114a2eadea (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:05.675124  4118 tablet_service.cc:1511] Processing CreateTablet for tablet b5f60b5ab23e43f5a3acb4ad0bc76e8d (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:05.676705  4119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c06ad9cf1ce4740900f95114a2eadea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.677628  4118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5f60b5ab23e43f5a3acb4ad0bc76e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.682121  4192 tablet_service.cc:1511] Processing CreateTablet for tablet b5f60b5ab23e43f5a3acb4ad0bc76e8d (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:05.683655  4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5f60b5ab23e43f5a3acb4ad0bc76e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.689524  4117 tablet_service.cc:1511] Processing CreateTablet for tablet db06f827b69242e0a6d1d38f10292858 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:05.691110  4117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db06f827b69242e0a6d1d38f10292858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.692397  4191 tablet_service.cc:1511] Processing CreateTablet for tablet db06f827b69242e0a6d1d38f10292858 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:05.693773  4191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db06f827b69242e0a6d1d38f10292858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.696642  4116 tablet_service.cc:1511] Processing CreateTablet for tablet ecf13ba63b734e3d9e946d9954deaf3a (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:05.698180  4116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf13ba63b734e3d9e946d9954deaf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.699159  4190 tablet_service.cc:1511] Processing CreateTablet for tablet ecf13ba63b734e3d9e946d9954deaf3a (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:05.700569  4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf13ba63b734e3d9e946d9954deaf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.706630  4115 tablet_service.cc:1511] Processing CreateTablet for tablet e05b4adf9f3845fdb787602c9c168ca7 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:05.708066  4115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e05b4adf9f3845fdb787602c9c168ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.709022  4189 tablet_service.cc:1511] Processing CreateTablet for tablet e05b4adf9f3845fdb787602c9c168ca7 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:05.710372  4189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e05b4adf9f3845fdb787602c9c168ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.712714  4114 tablet_service.cc:1511] Processing CreateTablet for tablet 2dd5b469c5674294af772a0de19886ce (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:05.714119  4114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dd5b469c5674294af772a0de19886ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.718744  4192 tablet_service.cc:1511] Processing CreateTablet for tablet 2dd5b469c5674294af772a0de19886ce (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:05.718748  4113 tablet_service.cc:1511] Processing CreateTablet for tablet 6794ce7d4a5142c0b33d6387baee030c (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:05.723652  4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dd5b469c5674294af772a0de19886ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.724355  4113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6794ce7d4a5142c0b33d6387baee030c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.729027  4188 tablet_service.cc:1511] Processing CreateTablet for tablet 6794ce7d4a5142c0b33d6387baee030c (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:05.730422  4188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6794ce7d4a5142c0b33d6387baee030c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.731631  4112 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec374ac1d804bec951b2b7e65d5ebc8 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:05.732986  4112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec374ac1d804bec951b2b7e65d5ebc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.734238  4187 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec374ac1d804bec951b2b7e65d5ebc8 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:05.735615  4187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec374ac1d804bec951b2b7e65d5ebc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.736744  4111 tablet_service.cc:1511] Processing CreateTablet for tablet 37fe0f56980f422fa17effc8cf63940b (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:05.737960  4111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fe0f56980f422fa17effc8cf63940b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.758392  4244 tablet_bootstrap.cc:492] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:05.759941  4192 tablet_service.cc:1511] Processing CreateTablet for tablet 37fe0f56980f422fa17effc8cf63940b (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:05.762154  4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fe0f56980f422fa17effc8cf63940b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.763293  4038 tablet_service.cc:1511] Processing CreateTablet for tablet 2dd5b469c5674294af772a0de19886ce (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:05.764053  4043 tablet_service.cc:1511] Processing CreateTablet for tablet 7c06ad9cf1ce4740900f95114a2eadea (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:05.764770  4038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dd5b469c5674294af772a0de19886ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.766096  4042 tablet_service.cc:1511] Processing CreateTablet for tablet b5f60b5ab23e43f5a3acb4ad0bc76e8d (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:05.767475  4042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5f60b5ab23e43f5a3acb4ad0bc76e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.771416  4041 tablet_service.cc:1511] Processing CreateTablet for tablet db06f827b69242e0a6d1d38f10292858 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:05.772718  4041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db06f827b69242e0a6d1d38f10292858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.776243  4040 tablet_service.cc:1511] Processing CreateTablet for tablet ecf13ba63b734e3d9e946d9954deaf3a (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:05.777577  4040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf13ba63b734e3d9e946d9954deaf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.777267  4037 tablet_service.cc:1511] Processing CreateTablet for tablet 6794ce7d4a5142c0b33d6387baee030c (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:05.778581  4037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6794ce7d4a5142c0b33d6387baee030c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.780892  4039 tablet_service.cc:1511] Processing CreateTablet for tablet e05b4adf9f3845fdb787602c9c168ca7 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:05.776244  4036 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec374ac1d804bec951b2b7e65d5ebc8 (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:05.779686  4035 tablet_service.cc:1511] Processing CreateTablet for tablet 37fe0f56980f422fa17effc8cf63940b (DEFAULT_TABLE table=test-workload [id=582e95601c524f96816e91d9cc934fb2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:05.784090  4043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c06ad9cf1ce4740900f95114a2eadea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.784745  4039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e05b4adf9f3845fdb787602c9c168ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.787482  4036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec374ac1d804bec951b2b7e65d5ebc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.795643  4035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fe0f56980f422fa17effc8cf63940b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.844743  4244 tablet_bootstrap.cc:654] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.868712  4243 tablet_bootstrap.cc:492] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:05.875990  4245 tablet_bootstrap.cc:492] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:05.877135  4244 tablet_bootstrap.cc:492] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:05.877866  4244 ts_tablet_manager.cc:1403] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.004s
I20260709 12:29:05.880688  4244 raft_consensus.cc:359] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:05.881453  4244 raft_consensus.cc:385] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.881803  4244 raft_consensus.cc:740] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.882732  4245 tablet_bootstrap.cc:654] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.882694  4244 consensus_queue.cc:260] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:05.886672  4243 tablet_bootstrap.cc:654] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.887087  4244 ts_tablet_manager.cc:1434] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:29:05.888085  4154 heartbeater.cc:499] Master 127.19.36.254:43127 was elected leader, sending a full tablet report...
I20260709 12:29:05.888546  4244 tablet_bootstrap.cc:492] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:05.897643  4244 tablet_bootstrap.cc:654] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.907306  4243 tablet_bootstrap.cc:492] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:05.907310  4245 tablet_bootstrap.cc:492] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:05.907492  4244 tablet_bootstrap.cc:492] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:05.908053  4243 ts_tablet_manager.cc:1403] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.005s
I20260709 12:29:05.908150  4245 ts_tablet_manager.cc:1403] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.020s
I20260709 12:29:05.908265  4244 ts_tablet_manager.cc:1403] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 12:29:05.910677  4244 raft_consensus.cc:359] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.911266  4244 raft_consensus.cc:385] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.911525  4244 raft_consensus.cc:740] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.911443  4243 raft_consensus.cc:359] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:05.911640  4245 raft_consensus.cc:359] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.912671  4245 raft_consensus.cc:385] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.912660  4244 consensus_queue.cc:260] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.913277  4245 raft_consensus.cc:740] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.914721  4244 ts_tablet_manager.cc:1434] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:29:05.914336  4245 consensus_queue.cc:260] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.915697  4244 tablet_bootstrap.cc:492] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:05.917106  4078 heartbeater.cc:499] Master 127.19.36.254:43127 was elected leader, sending a full tablet report...
I20260709 12:29:05.917728  4245 ts_tablet_manager.cc:1434] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:29:05.912564  4243 raft_consensus.cc:385] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.918407  4243 raft_consensus.cc:740] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.918852  4245 tablet_bootstrap.cc:492] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:05.919469  4243 consensus_queue.cc:260] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:05.922250  4244 tablet_bootstrap.cc:654] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.926781  4245 tablet_bootstrap.cc:654] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.936317  4243 ts_tablet_manager.cc:1434] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.028s	user 0.001s	sys 0.014s
I20260709 12:29:05.937284  4228 heartbeater.cc:499] Master 127.19.36.254:43127 was elected leader, sending a full tablet report...
I20260709 12:29:05.939386  4245 tablet_bootstrap.cc:492] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:05.940052  4245 ts_tablet_manager.cc:1403] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 12:29:05.942250  4243 tablet_bootstrap.cc:492] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:05.949733  4243 tablet_bootstrap.cc:654] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.948266  4245 raft_consensus.cc:359] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.955420  4245 raft_consensus.cc:385] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.955888  4245 raft_consensus.cc:740] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.956998  4245 consensus_queue.cc:260] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.959848  4245 ts_tablet_manager.cc:1434] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.002s
I20260709 12:29:05.959925  4244 tablet_bootstrap.cc:492] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:05.960595  4244 ts_tablet_manager.cc:1403] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.045s	user 0.023s	sys 0.020s
I20260709 12:29:05.960980  4245 tablet_bootstrap.cc:492] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:05.961370  4243 tablet_bootstrap.cc:492] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:05.962019  4243 ts_tablet_manager.cc:1403] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 12:29:05.963706  4244 raft_consensus.cc:359] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.964491  4244 raft_consensus.cc:385] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.964797  4244 raft_consensus.cc:740] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.965600  4243 raft_consensus.cc:359] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.966624  4243 raft_consensus.cc:385] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.966988  4243 raft_consensus.cc:740] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.965731  4244 consensus_queue.cc:260] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.967861  4243 consensus_queue.cc:260] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.968652  4245 tablet_bootstrap.cc:654] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.969471  4244 ts_tablet_manager.cc:1434] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:29:05.970227  4243 ts_tablet_manager.cc:1434] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:29:05.970556  4244 tablet_bootstrap.cc:492] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:05.971344  4243 tablet_bootstrap.cc:492] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:05.977185  4244 tablet_bootstrap.cc:654] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.977876  4243 tablet_bootstrap.cc:654] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.987470  4244 tablet_bootstrap.cc:492] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:05.988193  4244 ts_tablet_manager.cc:1403] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:29:05.992026  4244 raft_consensus.cc:359] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.993000  4244 raft_consensus.cc:385] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.993383  4244 raft_consensus.cc:740] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.995544  4244 consensus_queue.cc:260] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:05.997997  4244 ts_tablet_manager.cc:1434] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:05.999100  4244 tablet_bootstrap.cc:492] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:05.999498  4243 tablet_bootstrap.cc:492] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:06.000069  4243 ts_tablet_manager.cc:1403] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.012s
I20260709 12:29:06.003093  4243 raft_consensus.cc:359] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.003916  4243 raft_consensus.cc:385] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.004225  4243 raft_consensus.cc:740] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.005026  4243 consensus_queue.cc:260] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.005887  4244 tablet_bootstrap.cc:654] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.007473  4243 ts_tablet_manager.cc:1434] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:06.013152  4247 raft_consensus.cc:493] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.013875  4247 raft_consensus.cc:515] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.024734  4247 leader_election.cc:290] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.028095  4247 raft_consensus.cc:493] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.028620  4245 tablet_bootstrap.cc:492] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:06.029430  4245 ts_tablet_manager.cc:1403] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.069s	user 0.024s	sys 0.036s
I20260709 12:29:06.028740  4247 raft_consensus.cc:515] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:06.030148  4243 tablet_bootstrap.cc:492] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:06.031776  4247 leader_election.cc:290] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885), b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303)
I20260709 12:29:06.032603  4245 raft_consensus.cc:359] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:06.037252  4245 raft_consensus.cc:385] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.037568  4245 raft_consensus.cc:740] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.037869  4243 tablet_bootstrap.cc:654] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.038525  4245 consensus_queue.cc:260] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:06.039697  4251 raft_consensus.cc:493] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.040200  4251 raft_consensus.cc:515] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.041121  4244 tablet_bootstrap.cc:492] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:06.041720  4244 ts_tablet_manager.cc:1403] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.007s
I20260709 12:29:06.044888  4244 raft_consensus.cc:359] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.045707  4244 raft_consensus.cc:385] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.045994  4244 raft_consensus.cc:740] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.046790  4245 ts_tablet_manager.cc:1434] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:29:06.046736  4244 consensus_queue.cc:260] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.047935  4245 tablet_bootstrap.cc:492] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:06.049222  4244 ts_tablet_manager.cc:1434] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:06.049589  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.050300  4244 tablet_bootstrap.cc:492] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:06.050256  4053 raft_consensus.cc:2468] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:06.051625  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.052352  4052 raft_consensus.cc:2468] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:06.055142  4245 tablet_bootstrap.cc:654] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.056896  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.057314  4244 tablet_bootstrap.cc:654] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.058514  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.059273  4202 raft_consensus.cc:2468] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:06.061010  4093 leader_election.cc:304] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.062419  4093 leader_election.cc:304] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.063412  4247 raft_consensus.cc:2804] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.063946  4247 raft_consensus.cc:493] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.064383  4247 raft_consensus.cc:3060] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.066721  4245 tablet_bootstrap.cc:492] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:06.067444  4245 ts_tablet_manager.cc:1403] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.001s
W20260709 12:29:06.068202  4091 leader_election.cc:343] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.069825  4251 leader_election.cc:290] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.070708  4245 raft_consensus.cc:359] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.071738  4245 raft_consensus.cc:385] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.072093  4245 raft_consensus.cc:740] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.073104  4245 consensus_queue.cc:260] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.075591  4245 ts_tablet_manager.cc:1434] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:06.076947  4245 tablet_bootstrap.cc:492] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:06.082463  4252 raft_consensus.cc:493] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.083333  4252 raft_consensus.cc:515] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.095165  4252 leader_election.cc:290] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.105888  4247 raft_consensus.cc:515] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.109822  4267 raft_consensus.cc:2804] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.110466  4267 raft_consensus.cc:493] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.110900  4267 raft_consensus.cc:3060] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.111691  4245 tablet_bootstrap.cc:654] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:06.112704  4093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:39303, user_credentials={real_user=slave}} blocked reactor thread for 50405.7us
I20260709 12:29:06.128021  4245 tablet_bootstrap.cc:492] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:06.128607  4247 leader_election.cc:290] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.128753  4245 ts_tablet_manager.cc:1403] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.052s	user 0.006s	sys 0.009s
I20260709 12:29:06.130252  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:06.132001  4243 tablet_bootstrap.cc:492] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:06.132617  4243 ts_tablet_manager.cc:1403] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.103s	user 0.019s	sys 0.000s
I20260709 12:29:06.132977  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:06.132941  4245 raft_consensus.cc:359] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.133908  4245 raft_consensus.cc:385] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.133976  4052 raft_consensus.cc:3060] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.134258  4245 raft_consensus.cc:740] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.135166  4245 consensus_queue.cc:260] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.136180  4243 raft_consensus.cc:359] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.138728  4243 raft_consensus.cc:385] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.142164  4243 raft_consensus.cc:740] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.144826  4245 ts_tablet_manager.cc:1434] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 12:29:06.145017  4243 consensus_queue.cc:260] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:06.139912  4091 leader_election.cc:343] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.147295  4245 tablet_bootstrap.cc:492] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:06.153280  4052 raft_consensus.cc:2468] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:06.146018  4244 tablet_bootstrap.cc:492] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:06.154484  4244 ts_tablet_manager.cc:1403] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.104s	user 0.039s	sys 0.053s
I20260709 12:29:06.155227  4245 tablet_bootstrap.cc:654] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.155412  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.155776  4093 leader_election.cc:304] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 825e407a29994c42b0b1196a872be0bf
W20260709 12:29:06.157051  4016 leader_election.cc:343] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.157667  4244 raft_consensus.cc:359] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.158416  4244 raft_consensus.cc:385] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.158720  4244 raft_consensus.cc:740] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.159512  4244 consensus_queue.cc:260] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.161803  4247 raft_consensus.cc:2804] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.155416  4243 ts_tablet_manager.cc:1434] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:29:06.173018  4243 tablet_bootstrap.cc:492] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:06.175673  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.174058  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.172647  4247 raft_consensus.cc:697] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:06.180253  4247 consensus_queue.cc:237] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:06.181305  3999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:06.182121  4245 tablet_bootstrap.cc:492] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:06.177213  4129 raft_consensus.cc:2468] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:06.182718  4245 ts_tablet_manager.cc:1403] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.007s
I20260709 12:29:06.185884  4245 raft_consensus.cc:359] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.186609  4245 raft_consensus.cc:385] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.186889  4245 raft_consensus.cc:740] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.187652  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.187718  4245 consensus_queue.cc:260] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:06.189958  4168 leader_election.cc:343] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.190654  4245 ts_tablet_manager.cc:1434] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:29:06.191143  4243 tablet_bootstrap.cc:654] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.192958  4167 leader_election.cc:304] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.194213  4252 raft_consensus.cc:2804] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.193989  4267 raft_consensus.cc:515] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:06.194577  4252 raft_consensus.cc:493] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.194967  4252 raft_consensus.cc:3060] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.197502  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:06.198127  4052 raft_consensus.cc:3060] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.204041  4267 leader_election.cc:290] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Requested vote from peers 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885), b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303)
I20260709 12:29:06.191777  4245 tablet_bootstrap.cc:492] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:06.205817  4244 ts_tablet_manager.cc:1434] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.051s	user 0.010s	sys 0.015s
I20260709 12:29:06.207821  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:06.202881  4252 raft_consensus.cc:515] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.211414  4202 raft_consensus.cc:3060] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.211994  4244 tablet_bootstrap.cc:492] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:06.213723  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:06.214088  3948 catalog_manager.cc:5697] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.215363  4251 raft_consensus.cc:493] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.215868  4251 raft_consensus.cc:515] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.210228  4052 raft_consensus.cc:2468] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:06.218469  4251 leader_election.cc:290] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
W20260709 12:29:06.218911  4168 leader_election.cc:343] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.220104  4244 tablet_bootstrap.cc:654] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.220860  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.221057  4017 leader_election.cc:304] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:06.224225  4093 leader_election.cc:304] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.224607  4202 raft_consensus.cc:2468] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:06.225867  4245 tablet_bootstrap.cc:654] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:06.228089  4016 leader_election.cc:343] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.228288  4251 raft_consensus.cc:2749] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.228288  4243 tablet_bootstrap.cc:492] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:06.230902  4243 ts_tablet_manager.cc:1403] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.000s
I20260709 12:29:06.231346  4267 raft_consensus.cc:2804] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.236495  4267 raft_consensus.cc:697] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:06.236588  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.237533  4267 consensus_queue.cc:237] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
W20260709 12:29:06.238628  4017 leader_election.cc:343] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.239804  4017 leader_election.cc:304] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:06.240525  4251 raft_consensus.cc:2749] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.247398  4243 raft_consensus.cc:359] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.248289  4243 raft_consensus.cc:385] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.248298  4244 tablet_bootstrap.cc:492] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:06.248379  4245 tablet_bootstrap.cc:492] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:06.248770  4243 raft_consensus.cc:740] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.249167  4244 ts_tablet_manager.cc:1403] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.002s
I20260709 12:29:06.249233  4245 ts_tablet_manager.cc:1403] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.014s
I20260709 12:29:06.249507  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:06.250366  4128 raft_consensus.cc:3060] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.249946  4243 consensus_queue.cc:260] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.251490  4252 leader_election.cc:290] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.253428  4244 raft_consensus.cc:359] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.254482  4244 raft_consensus.cc:385] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.254864  4244 raft_consensus.cc:740] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.255954  4244 consensus_queue.cc:260] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.258988  4244 ts_tablet_manager.cc:1434] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:29:06.260406  4244 tablet_bootstrap.cc:492] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:06.261358  4128 raft_consensus.cc:2468] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:06.262904  4167 leader_election.cc:304] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.264067  4252 raft_consensus.cc:2804] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.263814  4245 raft_consensus.cc:359] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.264588  4245 raft_consensus.cc:385] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.264573  4252 raft_consensus.cc:697] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:06.265061  4245 raft_consensus.cc:740] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.265664  4252 consensus_queue.cc:237] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.265941  4245 consensus_queue.cc:260] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.268957  4245 ts_tablet_manager.cc:1434] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:29:06.269230  4244 tablet_bootstrap.cc:654] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.270087  4245 tablet_bootstrap.cc:492] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:06.274358  4243 ts_tablet_manager.cc:1434] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.042s	user 0.017s	sys 0.010s
I20260709 12:29:06.276059  4243 tablet_bootstrap.cc:492] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:06.278867  4245 tablet_bootstrap.cc:654] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.285187  4243 tablet_bootstrap.cc:654] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.290460  4245 tablet_bootstrap.cc:492] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:06.292546  4245 ts_tablet_manager.cc:1403] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.011s
I20260709 12:29:06.295452  4245 raft_consensus.cc:359] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.296147  4245 raft_consensus.cc:385] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.296419  4245 raft_consensus.cc:740] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.297184  4245 consensus_queue.cc:260] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.298976  4243 tablet_bootstrap.cc:492] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:06.299678  4245 ts_tablet_manager.cc:1434] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:29:06.299855  4243 ts_tablet_manager.cc:1403] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.004s
I20260709 12:29:06.300683  4245 tablet_bootstrap.cc:492] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:06.304261  4243 raft_consensus.cc:359] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.305312  4243 raft_consensus.cc:385] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.305763  4243 raft_consensus.cc:740] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.306912  4243 consensus_queue.cc:260] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.309348  4245 tablet_bootstrap.cc:654] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.309872  4243 ts_tablet_manager.cc:1434] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:29:06.311415  4243 tablet_bootstrap.cc:492] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:06.315413  4251 raft_consensus.cc:493] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.316114  4251 raft_consensus.cc:515] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.318789  3948 catalog_manager.cc:5697] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.321110  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf13ba63b734e3d9e946d9954deaf3a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.321779  4202 raft_consensus.cc:2468] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 0.
I20260709 12:29:06.323042  4016 leader_election.cc:304] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.323879  4262 raft_consensus.cc:2804] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.324283  4262 raft_consensus.cc:493] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.324437  4251 leader_election.cc:290] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.324647  4262 raft_consensus.cc:3060] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.324591  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf13ba63b734e3d9e946d9954deaf3a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
W20260709 12:29:06.326400  4017 leader_election.cc:343] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.329434  4243 tablet_bootstrap.cc:654] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.333179  4262 raft_consensus.cc:515] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.333400  3951 catalog_manager.cc:5697] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 1 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.323601  4244 tablet_bootstrap.cc:492] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:06.335819  4244 ts_tablet_manager.cc:1403] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.076s	user 0.021s	sys 0.038s
I20260709 12:29:06.339574  4244 raft_consensus.cc:359] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.340885  4267 raft_consensus.cc:493] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.341598  4267 raft_consensus.cc:515] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.345765  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.346663  4052 raft_consensus.cc:2468] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:06.353058  4093 leader_election.cc:304] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.354373  4274 raft_consensus.cc:2804] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.354877  4274 raft_consensus.cc:493] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.355331  4274 raft_consensus.cc:3060] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.361419  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.362032  4267 leader_election.cc:290] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.362236  4202 raft_consensus.cc:2468] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:06.362857  4245 tablet_bootstrap.cc:492] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:06.363377  4245 ts_tablet_manager.cc:1403] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.028s
I20260709 12:29:06.364039  4262 leader_election.cc:290] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.340471  4244 raft_consensus.cc:385] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.364642  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf13ba63b734e3d9e946d9954deaf3a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:06.364995  4244 raft_consensus.cc:740] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.365859  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf13ba63b734e3d9e946d9954deaf3a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:06.365808  4244 consensus_queue.cc:260] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:06.366192  4017 leader_election.cc:343] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.366508  4202 raft_consensus.cc:3060] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.368259  4244 ts_tablet_manager.cc:1434] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:29:06.369247  4247 raft_consensus.cc:493] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.369827  4247 raft_consensus.cc:515] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.370385  4245 raft_consensus.cc:359] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.371102  4245 raft_consensus.cc:385] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.371388  4245 raft_consensus.cc:740] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.372077  4245 consensus_queue.cc:260] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.374428  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.375093  4202 raft_consensus.cc:2468] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 1.
W20260709 12:29:06.376236  4091 leader_election.cc:343] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.376883  4245 ts_tablet_manager.cc:1434] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:29:06.369247  4244 tablet_bootstrap.cc:492] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:06.377717  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.381104  4052 raft_consensus.cc:2468] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:06.381327  4016 leader_election.cc:304] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:06.371970  4247 leader_election.cc:290] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.382431  4093 leader_election.cc:304] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:06.383039  4243 tablet_bootstrap.cc:492] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:06.379715  4252 raft_consensus.cc:493] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.382551  4262 raft_consensus.cc:2804] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.383644  4243 ts_tablet_manager.cc:1403] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.027s
I20260709 12:29:06.384083  4247 raft_consensus.cc:2804] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.384503  4247 raft_consensus.cc:493] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.383877  4262 raft_consensus.cc:697] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:06.383723  4252 raft_consensus.cc:515] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.385775  4262 consensus_queue.cc:237] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.386389  4244 tablet_bootstrap.cc:654] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.384842  4247 raft_consensus.cc:3060] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.387387  4252 leader_election.cc:290] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.387038  4243 raft_consensus.cc:359] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.387949  4243 raft_consensus.cc:385] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.388284  4243 raft_consensus.cc:740] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.378876  4274 raft_consensus.cc:515] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.389072  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.389451  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.389827  4052 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:06.390358  4128 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:06.391034  4168 leader_election.cc:304] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.392618  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:06.393487  4052 raft_consensus.cc:3060] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.392532  4243 consensus_queue.cc:260] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.394403  4283 raft_consensus.cc:2804] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.394898  4283 raft_consensus.cc:493] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.395417  4283 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.404312  4243 ts_tablet_manager.cc:1434] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:29:06.410490  4052 raft_consensus.cc:2468] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:06.411520  4247 raft_consensus.cc:515] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.411962  4093 leader_election.cc:304] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.413653  4267 raft_consensus.cc:2804] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.414165  4267 raft_consensus.cc:697] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:06.415769  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:06.415433  4267 consensus_queue.cc:237] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.416416  4202 raft_consensus.cc:3060] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.420310  3948 catalog_manager.cc:5697] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.423539  4274 leader_election.cc:290] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.424288  4243 tablet_bootstrap.cc:492] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:06.429095  4202 raft_consensus.cc:2468] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:06.414146  4283 raft_consensus.cc:515] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.437404  4251 raft_consensus.cc:493] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.438148  4251 raft_consensus.cc:515] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.441071  4251 leader_election.cc:290] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.441354  3951 catalog_manager.cc:5697] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.442371  4247 leader_election.cc:290] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.443948  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.444224  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:06.444649  4128 raft_consensus.cc:2468] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 0.
I20260709 12:29:06.444859  4202 raft_consensus.cc:3060] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.445886  4244 tablet_bootstrap.cc:492] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:06.445737  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.446466  4244 ts_tablet_manager.cc:1403] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.077s	user 0.024s	sys 0.039s
I20260709 12:29:06.447232  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:06.447597  4017 leader_election.cc:304] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.447853  4053 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:06.448619  4016 leader_election.cc:343] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.449213  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:06.449576  4251 raft_consensus.cc:2804] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.449796  4052 raft_consensus.cc:3060] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.450063  4251 raft_consensus.cc:493] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.449714  4244 raft_consensus.cc:359] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.450579  4251 raft_consensus.cc:3060] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.450780  4244 raft_consensus.cc:385] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.451241  4244 raft_consensus.cc:740] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.452052  4244 consensus_queue.cc:260] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.454417  4244 ts_tablet_manager.cc:1434] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:29:06.455545  4053 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:06.455646  4202 raft_consensus.cc:2468] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:06.456962  4283 leader_election.cc:290] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.457132  4091 leader_election.cc:304] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: 
I20260709 12:29:06.457679  4168 leader_election.cc:304] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.458541  4247 raft_consensus.cc:2804] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.458843  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:06.459072  4247 raft_consensus.cc:697] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:06.459501  4128 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.460399  4243 tablet_bootstrap.cc:654] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.460142  4247 consensus_queue.cc:237] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.461542  4052 raft_consensus.cc:2468] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:06.468804  4283 raft_consensus.cc:2804] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.473536  4283 raft_consensus.cc:697] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:06.474540  4283 consensus_queue.cc:237] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.475818  4128 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:06.477473  4251 raft_consensus.cc:515] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.481120  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:06.480432  3951 catalog_manager.cc:5697] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.481973  4128 raft_consensus.cc:3060] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.484508  4251 leader_election.cc:290] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.485639  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
W20260709 12:29:06.487139  4016 leader_election.cc:343] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.491640  4128 raft_consensus.cc:2468] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 1.
I20260709 12:29:06.492915  4017 leader_election.cc:304] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:06.494040  4251 raft_consensus.cc:2804] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.495316  4251 raft_consensus.cc:697] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:06.496234  4251 consensus_queue.cc:237] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.496733  3948 catalog_manager.cc:5697] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 1 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.501418  4243 tablet_bootstrap.cc:492] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:06.501920  4243 ts_tablet_manager.cc:1403] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.078s	user 0.028s	sys 0.018s
I20260709 12:29:06.504460  4243 raft_consensus.cc:359] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.505133  4243 raft_consensus.cc:385] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.505393  4243 raft_consensus.cc:740] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.506150  4243 consensus_queue.cc:260] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.510274  4243 ts_tablet_manager.cc:1434] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:29:06.511063  4243 tablet_bootstrap.cc:492] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:06.512041  3948 catalog_manager.cc:5697] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.516502  4243 tablet_bootstrap.cc:654] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.521654  4243 tablet_bootstrap.cc:492] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:06.522082  4243 ts_tablet_manager.cc:1403] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 12:29:06.524039  4243 raft_consensus.cc:359] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.524544  4243 raft_consensus.cc:385] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.524770  4243 raft_consensus.cc:740] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.525375  4243 consensus_queue.cc:260] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.526747  4243 ts_tablet_manager.cc:1434] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:29:06.546536  4283 raft_consensus.cc:493] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.547039  4283 raft_consensus.cc:515] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.549762  4283 leader_election.cc:290] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.549913  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.550405  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6794ce7d4a5142c0b33d6387baee030c" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.550873  4052 raft_consensus.cc:2393] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 825e407a29994c42b0b1196a872be0bf in current term 1: Already voted for candidate 6d5500e215154de78c642bdb2cfa8417 in this term.
I20260709 12:29:06.551934  4168 leader_election.cc:304] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf; no voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.552587  4283 raft_consensus.cc:3060] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.557754  4283 raft_consensus.cc:2749] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.573189  4247 consensus_queue.cc:1048] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.582046  4283 raft_consensus.cc:493] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.582647  4283 raft_consensus.cc:515] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.584376  4267 consensus_queue.cc:1048] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.585860  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.586474  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.587291  4128 raft_consensus.cc:2393] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 825e407a29994c42b0b1196a872be0bf in current term 1: Already voted for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in this term.
I20260709 12:29:06.588876  4167 leader_election.cc:304] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf; no voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.589933  4252 raft_consensus.cc:3060] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.589972  4283 leader_election.cc:290] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.598892  4252 raft_consensus.cc:2749] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.648169  4252 raft_consensus.cc:493] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.648684  4252 raft_consensus.cc:515] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.650558  4252 leader_election.cc:290] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.651554  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.651928  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:06.652439  4052 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:06.652738  4128 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:06.653568  4168 leader_election.cc:304] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.654403  4252 raft_consensus.cc:2804] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.654807  4252 raft_consensus.cc:493] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.655093  4252 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.660179  4252 raft_consensus.cc:515] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.661798  4252 leader_election.cc:290] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:06.662803  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:06.662997  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:06.663530  4052 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.663684  4128 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.669003  4052 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:06.669003  4128 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:06.670488  4168 leader_election.cc:304] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.671294  4252 raft_consensus.cc:2804] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.671816  4252 raft_consensus.cc:697] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:06.672632  4252 consensus_queue.cc:237] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.674888  4247 consensus_queue.cc:1048] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.681102  3948 catalog_manager.cc:5697] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 1 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.687687  4275 consensus_queue.cc:1048] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.688715  4252 consensus_queue.cc:1048] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.703462  4052 raft_consensus.cc:3060] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.714829  4283 consensus_queue.cc:1048] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.783783  4129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
mode: GRACEFUL
new_leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:42744
I20260709 12:29:06.784430  4129 raft_consensus.cc:606] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Received request to transfer leadership to TS b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.797945  4202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
mode: GRACEFUL
new_leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:42284
I20260709 12:29:06.798482  4202 raft_consensus.cc:606] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Received request to transfer leadership to TS b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.803601  4129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
mode: GRACEFUL
new_leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:42744
I20260709 12:29:06.804121  4129 raft_consensus.cc:606] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Received request to transfer leadership to TS b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.812356  4202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
mode: GRACEFUL
new_leader_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42284
I20260709 12:29:06.812894  4202 raft_consensus.cc:606] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Received request to transfer leadership to TS 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:06.817802  4202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
mode: GRACEFUL
new_leader_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42284
I20260709 12:29:06.818331  4202 raft_consensus.cc:606] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Received request to transfer leadership to TS 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:06.834146  4052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8"
dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
mode: GRACEFUL
new_leader_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:41828
I20260709 12:29:06.834687  4052 raft_consensus.cc:606] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Received request to transfer leadership to TS 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:06.839782  4129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
mode: GRACEFUL
new_leader_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42744
I20260709 12:29:06.840292  4129 raft_consensus.cc:606] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Received request to transfer leadership to TS 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:06.867473  4275 consensus_queue.cc:1048] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.877669  4275 consensus_queue.cc:1048] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.885401  4275 consensus_queue.cc:1048] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.886626  4274 raft_consensus.cc:993] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417: : Instructing follower b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:06.887241  4247 raft_consensus.cc:1081] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Signalling peer b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:06.889081  4053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858"
dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:41808
I20260709 12:29:06.889657  4053 raft_consensus.cc:493] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:06.890082  4053 raft_consensus.cc:3060] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.895584  4247 raft_consensus.cc:493] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.895666  4283 consensus_queue.cc:1048] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.896214  4247 raft_consensus.cc:515] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.900914  4050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf13ba63b734e3d9e946d9954deaf3a" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:06.902823  4267 consensus_queue.cc:1048] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.903134  4251 consensus_queue.cc:1048] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.903541  4247 leader_election.cc:290] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.903215  4053 raft_consensus.cc:515] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.906708  4053 leader_election.cc:290] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.907860  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:06.908440  4129 raft_consensus.cc:3055] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:06.908710  4129 raft_consensus.cc:740] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:06.913106  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf13ba63b734e3d9e946d9954deaf3a" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:06.913939  4202 raft_consensus.cc:2393] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6d5500e215154de78c642bdb2cfa8417 in current term 1: Already voted for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in this term.
I20260709 12:29:06.913792  4251 consensus_queue.cc:1048] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.915041  4283 raft_consensus.cc:993] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf: : Instructing follower b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:06.915546  4283 raft_consensus.cc:1081] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Signalling peer b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:06.915333  4129 consensus_queue.cc:260] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.916158  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db06f827b69242e0a6d1d38f10292858" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:06.916596  4129 raft_consensus.cc:3060] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.916703  4202 raft_consensus.cc:3060] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.917987  4091 leader_election.cc:304] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417; no voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:06.919430  4053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d"
dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:41824
I20260709 12:29:06.924857  4053 raft_consensus.cc:493] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:06.926793  4267 raft_consensus.cc:3060] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.925496  4053 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.930939  4252 consensus_queue.cc:1048] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.935796  4202 raft_consensus.cc:2468] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:06.937536  4016 leader_election.cc:304] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:06.938172  4267 raft_consensus.cc:2749] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.938483  4251 raft_consensus.cc:2804] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:06.939234  4251 raft_consensus.cc:697] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:06.939949  4251 consensus_queue.cc:237] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.943691  4129 raft_consensus.cc:2468] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:06.957607  4053 raft_consensus.cc:515] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.961403  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:06.962018  4203 raft_consensus.cc:3055] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:06.962354  4203 raft_consensus.cc:740] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:06.964757  4251 raft_consensus.cc:993] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2: : Instructing follower 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:06.965330  4052 raft_consensus.cc:1240] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Rejecting Update request from peer 825e407a29994c42b0b1196a872be0bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:06.965494  4053 leader_election.cc:290] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:06.965265  4251 raft_consensus.cc:1081] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Signalling peer 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:06.973117  4127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42728
I20260709 12:29:06.975327  4128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:06.976222  4128 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.975399  3951 catalog_manager.cc:5697] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 1 to 2, leader changed from 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194) to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.966171  4203 consensus_queue.cc:260] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:06.987922  4308 consensus_queue.cc:1059] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.985697  4127 raft_consensus.cc:493] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:06.988863  4127 raft_consensus.cc:3060] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.989471  4251 consensus_queue.cc:1048] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.991930  4203 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.992792  4267 raft_consensus.cc:993] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417: : Instructing follower 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:06.993290  4267 raft_consensus.cc:1081] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Signalling peer 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:06.995388  4201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42274
I20260709 12:29:06.995893  4201 raft_consensus.cc:493] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:06.996277  4201 raft_consensus.cc:3060] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.004963  4127 raft_consensus.cc:515] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.007622  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:07.007936  4202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec374ac1d804bec951b2b7e65d5ebc8" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:07.008270  4052 raft_consensus.cc:3055] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.008651  4052 raft_consensus.cc:740] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:07.008646  4202 raft_consensus.cc:3060] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.009577  4052 consensus_queue.cc:260] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.010540  4052 raft_consensus.cc:3060] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.016321  4202 raft_consensus.cc:2468] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:07.017670  4091 leader_election.cc:304] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: 
I20260709 12:29:07.018852  4247 raft_consensus.cc:2804] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.019374  4247 raft_consensus.cc:697] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:07.020275  4247 consensus_queue.cc:237] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.022130  4052 raft_consensus.cc:2468] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:07.024852  4262 consensus_queue.cc:1048] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:07.025102  4128 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:07.026942  4201 raft_consensus.cc:515] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.028684  4203 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:07.029078  4017 leader_election.cc:304] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:07.029733  4127 leader_election.cc:290] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:07.029996  4262 raft_consensus.cc:2804] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.031884  4201 leader_election.cc:290] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:07.032415  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:07.030560  4262 raft_consensus.cc:697] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:07.033208  4052 raft_consensus.cc:3060] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.033706  4262 consensus_queue.cc:237] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.033862  3949 catalog_manager.cc:5697] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 1 to 2, leader changed from b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193) to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:07.041787  4052 raft_consensus.cc:2468] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 2.
I20260709 12:29:07.043046  4168 leader_election.cc:304] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:07.043891  4308 raft_consensus.cc:2804] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.044185  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fe0f56980f422fa17effc8cf63940b" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:07.044433  4308 raft_consensus.cc:697] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 2 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:07.044822  4127 raft_consensus.cc:3055] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.042212  3948 catalog_manager.cc:5697] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 1 to 2, leader changed from 825e407a29994c42b0b1196a872be0bf (127.19.36.195) to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:07.045228  4127 raft_consensus.cc:740] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:07.045512  4308 consensus_queue.cc:237] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.046952  4127 consensus_queue.cc:260] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.048542  4127 raft_consensus.cc:3060] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.056855  4127 raft_consensus.cc:2468] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 2.
I20260709 12:29:07.062302  3951 catalog_manager.cc:5697] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 1 to 2, leader changed from 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194) to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 2 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.140678  4308 consensus_queue.cc:1048] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:07.148480  4283 raft_consensus.cc:993] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf: : Instructing follower 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:07.149022  4283 raft_consensus.cc:1081] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Signalling peer 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:07.150786  4129 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42742
I20260709 12:29:07.151441  4129 raft_consensus.cc:493] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.151851  4129 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.153486  4252 consensus_queue.cc:1048] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:07.159909  4129 raft_consensus.cc:515] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.161057  4127 raft_consensus.cc:1240] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Rejecting Update request from peer 825e407a29994c42b0b1196a872be0bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:07.162669  4129 leader_election.cc:290] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:07.164533  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:07.165187  4201 raft_consensus.cc:3055] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.165561  4201 raft_consensus.cc:740] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:07.165675  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:07.166354  4052 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.166545  4201 consensus_queue.cc:260] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.167400  4252 consensus_queue.cc:1059] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:07.168614  4201 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.171990  4052 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:07.173317  4093 leader_election.cc:304] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:07.174273  4274 raft_consensus.cc:2804] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.174836  4274 raft_consensus.cc:697] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:07.174892  4201 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:07.175921  4274 consensus_queue.cc:237] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.189394  3949 catalog_manager.cc:5697] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 1 to 2, leader changed from 825e407a29994c42b0b1196a872be0bf (127.19.36.195) to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:07.198815  4283 raft_consensus.cc:993] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf: : Instructing follower 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:07.199332  4252 raft_consensus.cc:1081] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Signalling peer 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:07.200764  4127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42742
I20260709 12:29:07.201269  4127 raft_consensus.cc:493] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.201586  4127 raft_consensus.cc:3060] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.206493  4127 raft_consensus.cc:515] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.208287  4127 leader_election.cc:290] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:07.209136  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:07.209431  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e05b4adf9f3845fdb787602c9c168ca7" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:07.209873  4052 raft_consensus.cc:3060] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.210023  4201 raft_consensus.cc:3055] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.210439  4201 raft_consensus.cc:740] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:07.211151  4201 consensus_queue.cc:260] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.212152  4201 raft_consensus.cc:3060] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.215556  4052 raft_consensus.cc:2468] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:07.216740  4093 leader_election.cc:304] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:07.217154  4201 raft_consensus.cc:2468] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:07.217504  4274 raft_consensus.cc:2804] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.218034  4274 raft_consensus.cc:697] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:07.218865  4274 consensus_queue.cc:237] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:07.225724  3949 catalog_manager.cc:5697] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 1 to 2, leader changed from 825e407a29994c42b0b1196a872be0bf (127.19.36.195) to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:07.232529  4247 raft_consensus.cc:993] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417: : Instructing follower b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:07.233081  4274 raft_consensus.cc:1081] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Signalling peer b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:07.234812  4052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea"
dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:41808
I20260709 12:29:07.235487  4052 raft_consensus.cc:493] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.235819  4052 raft_consensus.cc:3060] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.242092  4052 raft_consensus.cc:515] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:07.243947  4052 leader_election.cc:290] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Requested vote from peers 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:07.244778  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:07.245126  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c06ad9cf1ce4740900f95114a2eadea" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:07.245487  4201 raft_consensus.cc:3060] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.245784  4127 raft_consensus.cc:3055] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.246110  4127 raft_consensus.cc:740] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:07.246822  4127 consensus_queue.cc:260] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:07.247854  4127 raft_consensus.cc:3060] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.251156  4201 raft_consensus.cc:2468] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:07.252351  4016 leader_election.cc:304] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:07.252977  4127 raft_consensus.cc:2468] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:07.253165  4262 raft_consensus.cc:2804] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.253674  4262 raft_consensus.cc:697] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:07.254457  4262 consensus_queue.cc:237] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } }
I20260709 12:29:07.260975  3949 catalog_manager.cc:5697] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 1 to 2, leader changed from 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194) to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.432631  4127 raft_consensus.cc:1275] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.433843  4262 consensus_queue.cc:1048] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.437384  4201 raft_consensus.cc:1275] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.438906  4262 consensus_queue.cc:1048] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.477984  4127 raft_consensus.cc:1275] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.478240  4052 raft_consensus.cc:1275] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.480108  4274 consensus_queue.cc:1048] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.480744  4322 consensus_queue.cc:1048] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.488914  4201 raft_consensus.cc:1275] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.490549  4267 consensus_queue.cc:1048] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:07.493312  4203 raft_consensus.cc:1275] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.497598  4251 consensus_queue.cc:1048] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.570263  4052 raft_consensus.cc:1275] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.571728  4247 consensus_queue.cc:1048] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.582002  4203 raft_consensus.cc:1275] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.583425  4247 consensus_queue.cc:1048] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.611716  4052 raft_consensus.cc:1275] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.613045  4252 consensus_queue.cc:1048] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.620330  4129 raft_consensus.cc:1275] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.622080  4283 consensus_queue.cc:1048] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.664294  4203 raft_consensus.cc:1275] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.665575  4262 consensus_queue.cc:1048] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.673431  4129 raft_consensus.cc:1275] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.675621  4251 consensus_queue.cc:1048] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.746706  4203 raft_consensus.cc:1275] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.748463  4274 consensus_queue.cc:1048] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.757650  4052 raft_consensus.cc:1275] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.759505  4328 consensus_queue.cc:1048] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:09.873531  3951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50332:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:09.876358  3951 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:09.934391  4039 tablet_service.cc:1511] Processing CreateTablet for tablet 81f2a0380d0842d7aca2005755834e79 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:09.936553  4039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f2a0380d0842d7aca2005755834e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.936684  4116 tablet_service.cc:1511] Processing CreateTablet for tablet 81f2a0380d0842d7aca2005755834e79 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:09.939455  4116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f2a0380d0842d7aca2005755834e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.939254  4193 tablet_service.cc:1511] Processing CreateTablet for tablet 81f2a0380d0842d7aca2005755834e79 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:09.941726  4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f2a0380d0842d7aca2005755834e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.941938  4114 tablet_service.cc:1511] Processing CreateTablet for tablet 79651f9639b24c159a6d7dc913a6357a (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:09.942770  4187 tablet_service.cc:1511] Processing CreateTablet for tablet 79651f9639b24c159a6d7dc913a6357a (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:09.943499  4114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79651f9639b24c159a6d7dc913a6357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.944221  4187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79651f9639b24c159a6d7dc913a6357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.947211  4035 tablet_service.cc:1511] Processing CreateTablet for tablet 9af04ac7f8b4478dbe38c821153a6d1b (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:09.947430  4112 tablet_service.cc:1511] Processing CreateTablet for tablet 9af04ac7f8b4478dbe38c821153a6d1b (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:09.947944  4188 tablet_service.cc:1511] Processing CreateTablet for tablet 9af04ac7f8b4478dbe38c821153a6d1b (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:09.948799  4112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9af04ac7f8b4478dbe38c821153a6d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.949460  4188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9af04ac7f8b4478dbe38c821153a6d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.950197  4041 tablet_service.cc:1511] Processing CreateTablet for tablet ebbf50548b1a4bd48d9af0b5768f973c (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:09.939250  4036 tablet_service.cc:1511] Processing CreateTablet for tablet 79651f9639b24c159a6d7dc913a6357a (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:09.953749  4192 tablet_service.cc:1511] Processing CreateTablet for tablet ebbf50548b1a4bd48d9af0b5768f973c (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:09.955215  4037 tablet_service.cc:1511] Processing CreateTablet for tablet bd3f9c90732a480fab5f6e1171ab74c8 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:09.956508  4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebbf50548b1a4bd48d9af0b5768f973c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.957723  4190 tablet_service.cc:1511] Processing CreateTablet for tablet bd3f9c90732a480fab5f6e1171ab74c8 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:09.958109  4040 tablet_service.cc:1511] Processing CreateTablet for tablet 527b5e51667649bdb98ef7b017072212 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:09.953795  4115 tablet_service.cc:1511] Processing CreateTablet for tablet ebbf50548b1a4bd48d9af0b5768f973c (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:09.960747  4115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebbf50548b1a4bd48d9af0b5768f973c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.961758  4189 tablet_service.cc:1511] Processing CreateTablet for tablet 527b5e51667649bdb98ef7b017072212 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:09.963202  4042 tablet_service.cc:1511] Processing CreateTablet for tablet b3182bf9f5024432beb478c5d81b1e13 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:09.965333  4117 tablet_service.cc:1511] Processing CreateTablet for tablet bd3f9c90732a480fab5f6e1171ab74c8 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:09.966688  4117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3f9c90732a480fab5f6e1171ab74c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.968111  4043 tablet_service.cc:1511] Processing CreateTablet for tablet 75f5f8fdb07f484facd47ff95233db65 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:09.968439  4111 tablet_service.cc:1511] Processing CreateTablet for tablet 527b5e51667649bdb98ef7b017072212 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:09.969815  4111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 527b5e51667649bdb98ef7b017072212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.972563  4191 tablet_service.cc:1511] Processing CreateTablet for tablet b3182bf9f5024432beb478c5d81b1e13 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:09.975940  4118 tablet_service.cc:1511] Processing CreateTablet for tablet b3182bf9f5024432beb478c5d81b1e13 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:09.976176  4038 tablet_service.cc:1511] Processing CreateTablet for tablet 839c8a7e470649a6a1f87137a4b76bba (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:09.977365  4118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3182bf9f5024432beb478c5d81b1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.981001  4113 tablet_service.cc:1511] Processing CreateTablet for tablet 75f5f8fdb07f484facd47ff95233db65 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:09.982493  4113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f5f8fdb07f484facd47ff95233db65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.959152  4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3f9c90732a480fab5f6e1171ab74c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.983505  4042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3182bf9f5024432beb478c5d81b1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.986840  4191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3182bf9f5024432beb478c5d81b1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.987569  4041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebbf50548b1a4bd48d9af0b5768f973c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.991269  4040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 527b5e51667649bdb98ef7b017072212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.995551  4037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3f9c90732a480fab5f6e1171ab74c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.995688  4189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 527b5e51667649bdb98ef7b017072212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.998497  4036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79651f9639b24c159a6d7dc913a6357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:09.999130  4038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839c8a7e470649a6a1f87137a4b76bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:10.003724  4035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9af04ac7f8b4478dbe38c821153a6d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:10.005371  4186 tablet_service.cc:1511] Processing CreateTablet for tablet 75f5f8fdb07f484facd47ff95233db65 (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:10.006857  4186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f5f8fdb07f484facd47ff95233db65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:10.008493  4185 tablet_service.cc:1511] Processing CreateTablet for tablet 839c8a7e470649a6a1f87137a4b76bba (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:10.009277  4119 tablet_service.cc:1511] Processing CreateTablet for tablet 839c8a7e470649a6a1f87137a4b76bba (DEFAULT_TABLE table=soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:10.010084  4043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f5f8fdb07f484facd47ff95233db65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:10.010748  4119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839c8a7e470649a6a1f87137a4b76bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:10.023613  4185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839c8a7e470649a6a1f87137a4b76bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:10.089385  4341 tablet_bootstrap.cc:492] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.092778  4343 tablet_bootstrap.cc:492] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.106359  4343 tablet_bootstrap.cc:654] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.135804  4343 tablet_bootstrap.cc:492] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.136355  4343 ts_tablet_manager.cc:1403] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.006s
I20260709 12:29:10.139004  4343 raft_consensus.cc:359] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.139835  4343 raft_consensus.cc:385] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.140154  4343 raft_consensus.cc:740] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.140985  4343 consensus_queue.cc:260] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.143446  4343 ts_tablet_manager.cc:1434] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.008s
I20260709 12:29:10.144645  4343 tablet_bootstrap.cc:492] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.145675  4341 tablet_bootstrap.cc:654] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.148892  4342 tablet_bootstrap.cc:492] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.155421  4343 tablet_bootstrap.cc:654] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.158116  4342 tablet_bootstrap.cc:654] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.160979  4343 tablet_bootstrap.cc:492] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.161468  4343 ts_tablet_manager.cc:1403] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.002s
I20260709 12:29:10.162859  4341 tablet_bootstrap.cc:492] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.163587  4341 ts_tablet_manager.cc:1403] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.002s
I20260709 12:29:10.164491  4343 raft_consensus.cc:359] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.165233  4343 raft_consensus.cc:385] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.165513  4343 raft_consensus.cc:740] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.166370  4343 consensus_queue.cc:260] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.167742  4341 raft_consensus.cc:359] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.168851  4341 raft_consensus.cc:385] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.169081  4343 ts_tablet_manager.cc:1434] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:29:10.169265  4341 raft_consensus.cc:740] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.170243  4343 tablet_bootstrap.cc:492] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.170567  4341 consensus_queue.cc:260] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.173666  4341 ts_tablet_manager.cc:1434] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:29:10.174681  4341 tablet_bootstrap.cc:492] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.182049  4341 tablet_bootstrap.cc:654] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.182307  4343 tablet_bootstrap.cc:654] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.189677  4342 tablet_bootstrap.cc:492] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.189669  4341 tablet_bootstrap.cc:492] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.190379  4342 ts_tablet_manager.cc:1403] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.004s
I20260709 12:29:10.190382  4341 ts_tablet_manager.cc:1403] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.002s
I20260709 12:29:10.193912  4342 raft_consensus.cc:359] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.194118  4341 raft_consensus.cc:359] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.194414  4343 tablet_bootstrap.cc:492] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.194720  4342 raft_consensus.cc:385] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.194834  4341 raft_consensus.cc:385] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.195252  4342 raft_consensus.cc:740] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.195334  4343 ts_tablet_manager.cc:1403] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.004s
I20260709 12:29:10.195302  4341 raft_consensus.cc:740] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.196173  4342 consensus_queue.cc:260] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.198901  4342 ts_tablet_manager.cc:1434] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:29:10.199421  4343 raft_consensus.cc:359] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.199997  4342 tablet_bootstrap.cc:492] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.200241  4343 raft_consensus.cc:385] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.200596  4343 raft_consensus.cc:740] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.196591  4341 consensus_queue.cc:260] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.201337  4343 consensus_queue.cc:260] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.203697  4343 ts_tablet_manager.cc:1434] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:10.206023  4341 ts_tablet_manager.cc:1434] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 12:29:10.207127  4341 tablet_bootstrap.cc:492] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.207676  4342 tablet_bootstrap.cc:654] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.208065  4343 tablet_bootstrap.cc:492] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.215107  4341 tablet_bootstrap.cc:654] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.215477  4343 tablet_bootstrap.cc:654] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.217753  4283 raft_consensus.cc:493] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.218484  4283 raft_consensus.cc:515] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.222311  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:10.223795  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
W20260709 12:29:10.224123  4168 leader_election.cc:343] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.229050  4342 tablet_bootstrap.cc:492] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
W20260709 12:29:10.229605  3999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:10.231027  4343 tablet_bootstrap.cc:492] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.231603  4343 ts_tablet_manager.cc:1403] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
W20260709 12:29:10.232396  4167 leader_election.cc:343] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.233031  4167 leader_election.cc:304] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf; no voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.229647  4342 ts_tablet_manager.cc:1403] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.016s
I20260709 12:29:10.236760  4283 leader_election.cc:290] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.237553  4283 raft_consensus.cc:2749] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:10.238519  4342 raft_consensus.cc:359] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.239337  4342 raft_consensus.cc:385] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.239642  4342 raft_consensus.cc:740] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.240334  4342 consensus_queue.cc:260] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.242518  4342 ts_tablet_manager.cc:1434] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:10.234754  4343 raft_consensus.cc:359] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.243599  4343 raft_consensus.cc:385] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.243911  4343 raft_consensus.cc:740] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.244704  4343 consensus_queue.cc:260] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.247164  4343 ts_tablet_manager.cc:1434] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:29:10.247771  4342 tablet_bootstrap.cc:492] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.251227  4343 tablet_bootstrap.cc:492] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.257529  4341 tablet_bootstrap.cc:492] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.258117  4341 ts_tablet_manager.cc:1403] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.021s
I20260709 12:29:10.259259  4342 tablet_bootstrap.cc:654] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.260303  4343 tablet_bootstrap.cc:654] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.261412  4341 raft_consensus.cc:359] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.262224  4341 raft_consensus.cc:385] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.262579  4341 raft_consensus.cc:740] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.263587  4341 consensus_queue.cc:260] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.266887  4341 ts_tablet_manager.cc:1434] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.005s
I20260709 12:29:10.268101  4341 tablet_bootstrap.cc:492] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.275045  4342 tablet_bootstrap.cc:492] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.275375  4341 tablet_bootstrap.cc:654] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.275750  4342 ts_tablet_manager.cc:1403] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.008s
I20260709 12:29:10.279464  4342 raft_consensus.cc:359] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.279888  4343 tablet_bootstrap.cc:492] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.280236  4342 raft_consensus.cc:385] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.280592  4342 raft_consensus.cc:740] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.280632  4343 ts_tablet_manager.cc:1403] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.013s
I20260709 12:29:10.281533  4342 consensus_queue.cc:260] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.285005  4343 raft_consensus.cc:359] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.285847  4343 raft_consensus.cc:385] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.286149  4343 raft_consensus.cc:740] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.286805  4342 ts_tablet_manager.cc:1434] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:29:10.286965  4343 consensus_queue.cc:260] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.287912  4341 tablet_bootstrap.cc:492] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.288787  4341 ts_tablet_manager.cc:1403] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.013s
I20260709 12:29:10.289546  4343 ts_tablet_manager.cc:1434] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:10.290683  4343 tablet_bootstrap.cc:492] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.291806  4341 raft_consensus.cc:359] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.292564  4341 raft_consensus.cc:385] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.288035  4342 tablet_bootstrap.cc:492] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.298136  4343 tablet_bootstrap.cc:654] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.299989  4342 tablet_bootstrap.cc:654] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.292862  4341 raft_consensus.cc:740] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.302481  4341 consensus_queue.cc:260] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.304643  4343 tablet_bootstrap.cc:492] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.305207  4343 ts_tablet_manager.cc:1403] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:29:10.306628  4341 ts_tablet_manager.cc:1434] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 12:29:10.308130  4341 tablet_bootstrap.cc:492] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.309100  4343 raft_consensus.cc:359] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.309880  4343 raft_consensus.cc:385] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.310220  4343 raft_consensus.cc:740] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.311051  4343 consensus_queue.cc:260] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.313427  4343 ts_tablet_manager.cc:1434] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:10.314507  4343 tablet_bootstrap.cc:492] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.316157  4354 raft_consensus.cc:493] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.316679  4354 raft_consensus.cc:515] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.317266  4341 tablet_bootstrap.cc:654] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.319507  4354 leader_election.cc:290] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.319840  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
W20260709 12:29:10.321347  4168 leader_election.cc:343] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.321702  4343 tablet_bootstrap.cc:654] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.322542  4342 tablet_bootstrap.cc:492] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.322675  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.323503  4129 raft_consensus.cc:2468] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:10.323483  4342 ts_tablet_manager.cc:1403] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.014s
I20260709 12:29:10.325053  4167 leader_election.cc:304] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.326261  4354 raft_consensus.cc:2804] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.326746  4354 raft_consensus.cc:493] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.327154  4354 raft_consensus.cc:3060] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.330482  4322 raft_consensus.cc:493] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.330852  4322 raft_consensus.cc:515] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.337590  4341 tablet_bootstrap.cc:492] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.338292  4341 ts_tablet_manager.cc:1403] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.011s
I20260709 12:29:10.338918  4322 leader_election.cc:290] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.340353  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.341336  4341 raft_consensus.cc:359] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.342056  4341 raft_consensus.cc:385] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.342363  4341 raft_consensus.cc:740] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.342350  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
W20260709 12:29:10.343532  4017 leader_election.cc:343] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.344434  4354 raft_consensus.cc:515] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.345292  4341 consensus_queue.cc:260] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:10.345777  4016 leader_election.cc:343] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.347898  4341 ts_tablet_manager.cc:1434] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:10.348338  4016 leader_election.cc:304] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:10.348739  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:10.349359  4129 raft_consensus.cc:3060] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.349601  4341 tablet_bootstrap.cc:492] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.349857  4322 raft_consensus.cc:2749] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:10.351107  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
W20260709 12:29:10.352315  4168 leader_election.cc:343] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.356391  4341 tablet_bootstrap.cc:654] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.356906  4343 tablet_bootstrap.cc:492] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.357450  4343 ts_tablet_manager.cc:1403] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.004s
I20260709 12:29:10.359284  4354 leader_election.cc:290] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.359005  4129 raft_consensus.cc:2468] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:10.360829  4343 raft_consensus.cc:359] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.360955  4167 leader_election.cc:304] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.361496  4343 raft_consensus.cc:385] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.361809  4343 raft_consensus.cc:740] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.337532  4342 raft_consensus.cc:359] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.361922  4354 raft_consensus.cc:2804] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.362314  4342 raft_consensus.cc:385] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.362742  4354 raft_consensus.cc:697] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:10.363415  4342 raft_consensus.cc:740] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.362757  4343 consensus_queue.cc:260] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.363749  4354 consensus_queue.cc:237] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.365042  4343 ts_tablet_manager.cc:1434] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:29:10.364650  4342 consensus_queue.cc:260] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.366294  4343 tablet_bootstrap.cc:492] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.375411  4343 tablet_bootstrap.cc:654] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.388010  4283 raft_consensus.cc:493] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.388828  4283 raft_consensus.cc:515] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.393649  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
W20260709 12:29:10.395787  4168 leader_election.cc:343] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.397326  4283 leader_election.cc:290] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.398947  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.400079  4129 raft_consensus.cc:2468] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:10.401985  4343 tablet_bootstrap.cc:492] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.402065  4167 leader_election.cc:304] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.402947  4343 ts_tablet_manager.cc:1403] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.013s
I20260709 12:29:10.404000  4283 raft_consensus.cc:2804] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.404606  4283 raft_consensus.cc:493] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.405047  4283 raft_consensus.cc:3060] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.407351  4343 raft_consensus.cc:359] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.408424  4343 raft_consensus.cc:385] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.408830  4343 raft_consensus.cc:740] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.409917  4343 consensus_queue.cc:260] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.402100  4342 ts_tablet_manager.cc:1434] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.003s
I20260709 12:29:10.415694  4342 tablet_bootstrap.cc:492] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.420308  4348 raft_consensus.cc:493] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.420892  4348 raft_consensus.cc:515] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.421934  4343 ts_tablet_manager.cc:1434] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.007s
I20260709 12:29:10.423147  4348 leader_election.cc:290] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.423360  4343 tablet_bootstrap.cc:492] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf: Bootstrap starting.
I20260709 12:29:10.429709  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:10.430687  4201 raft_consensus.cc:2468] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:10.433143  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:10.434047  4052 raft_consensus.cc:2468] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:10.436275  4093 leader_election.cc:304] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:10.436743  4343 tablet_bootstrap.cc:654] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.437525  4348 raft_consensus.cc:2804] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.438064  4348 raft_consensus.cc:493] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.438489  4348 raft_consensus.cc:3060] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.447914  4342 tablet_bootstrap.cc:654] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.450655  3950 catalog_manager.cc:5697] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 1 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:10.457902  4283 raft_consensus.cc:515] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.467835  4283 leader_election.cc:290] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.467772  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:10.469309  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
W20260709 12:29:10.469574  4168 leader_election.cc:343] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.470191  4129 raft_consensus.cc:3060] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.476328  4343 tablet_bootstrap.cc:492] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf: No bootstrap required, opened a new log
I20260709 12:29:10.477077  4343 ts_tablet_manager.cc:1403] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.008s
I20260709 12:29:10.478785  4283 raft_consensus.cc:493] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.479774  4283 raft_consensus.cc:515] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.482120  4129 raft_consensus.cc:2468] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:10.485028  4343 raft_consensus.cc:359] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.485602  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:10.485817  4343 raft_consensus.cc:385] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.486322  4343 raft_consensus.cc:740] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.487078  4343 consensus_queue.cc:260] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:10.487382  4168 leader_election.cc:343] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.488821  4283 leader_election.cc:290] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.488947  4341 tablet_bootstrap.cc:492] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.489426  4343 ts_tablet_manager.cc:1434] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 12:29:10.489599  4341 ts_tablet_manager.cc:1403] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.140s	user 0.033s	sys 0.084s
I20260709 12:29:10.490183  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.485523  4348 raft_consensus.cc:515] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.491508  4322 raft_consensus.cc:493] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.492033  4322 raft_consensus.cc:515] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.485852  4342 tablet_bootstrap.cc:492] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.493175  4342 ts_tablet_manager.cc:1403] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.078s	user 0.008s	sys 0.012s
I20260709 12:29:10.492456  4348 leader_election.cc:290] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.494207  4322 leader_election.cc:290] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.495303  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:10.495371  4167 leader_election.cc:304] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.496037  4052 raft_consensus.cc:3060] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.495956  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbf50548b1a4bd48d9af0b5768f973c" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
W20260709 12:29:10.496804  4167 leader_election.cc:343] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.496804  4203 raft_consensus.cc:2468] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 0.
I20260709 12:29:10.497452  4167 leader_election.cc:304] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf; no voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.498353  4016 leader_election.cc:304] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:10.498605  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbf50548b1a4bd48d9af0b5768f973c" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.500478  4283 raft_consensus.cc:2804] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.499725  4322 raft_consensus.cc:2804] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.501068  4322 raft_consensus.cc:493] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.501451  4322 raft_consensus.cc:3060] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:10.501590  4017 leader_election.cc:343] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.497249  4342 raft_consensus.cc:359] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.500051  4354 raft_consensus.cc:2749] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:10.504302  4342 raft_consensus.cc:385] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.504686  4342 raft_consensus.cc:740] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.497829  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:10.506595  4342 consensus_queue.cc:260] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.507169  4201 raft_consensus.cc:3060] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.503786  4283 raft_consensus.cc:697] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:10.509552  4283 consensus_queue.cc:237] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.498029  4341 raft_consensus.cc:359] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.511780  4341 raft_consensus.cc:385] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.512221  4341 raft_consensus.cc:740] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.527716  4341 consensus_queue.cc:260] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.530589  4341 ts_tablet_manager.cc:1434] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.002s
I20260709 12:29:10.543126  4052 raft_consensus.cc:2468] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:10.545603  4341 tablet_bootstrap.cc:492] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.548950  4201 raft_consensus.cc:2468] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:10.551173  4091 leader_election.cc:304] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: 
I20260709 12:29:10.552198  4373 raft_consensus.cc:493] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.552436  4348 raft_consensus.cc:2804] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.553555  4348 raft_consensus.cc:697] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:10.552944  4373 raft_consensus.cc:515] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.554731  4348 consensus_queue.cc:237] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.558398  4373 leader_election.cc:290] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.559273  4129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f2a0380d0842d7aca2005755834e79" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.562134  4283 raft_consensus.cc:493] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.562709  4283 raft_consensus.cc:515] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:10.563560  4017 leader_election.cc:343] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.563651  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f2a0380d0842d7aca2005755834e79" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:10.564494  4203 raft_consensus.cc:2468] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 0.
I20260709 12:29:10.564702  4342 ts_tablet_manager.cc:1434] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.071s	user 0.036s	sys 0.017s
I20260709 12:29:10.566004  4342 tablet_bootstrap.cc:492] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.566167  4283 leader_election.cc:290] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.567304  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f2a0380d0842d7aca2005755834e79" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.570345  4322 raft_consensus.cc:515] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
W20260709 12:29:10.570405  4167 leader_election.cc:343] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.577811  4267 raft_consensus.cc:493] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.579864  4267 raft_consensus.cc:515] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.580518  4322 leader_election.cc:290] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.583318  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527b5e51667649bdb98ef7b017072212" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:10.581110  3949 catalog_manager.cc:5697] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 1 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:10.584086  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbf50548b1a4bd48d9af0b5768f973c" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:10.585750  4341 tablet_bootstrap.cc:654] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:10.586810  4017 leader_election.cc:343] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:10.585636  4093 leader_election.cc:343] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.588531  4267 leader_election.cc:290] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.589016  4267 raft_consensus.cc:493] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.589461  4267 raft_consensus.cc:515] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.592738  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527b5e51667649bdb98ef7b017072212" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:10.593410  4201 raft_consensus.cc:2468] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:10.593478  4267 leader_election.cc:290] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.593837  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:10.586376  4342 tablet_bootstrap.cc:654] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.594430  4052 raft_consensus.cc:2468] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:10.595726  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:10.595777  4093 leader_election.cc:304] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:10.596562  4201 raft_consensus.cc:2468] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 0.
I20260709 12:29:10.596892  4267 raft_consensus.cc:2804] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.597393  4267 raft_consensus.cc:493] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.597788  4267 raft_consensus.cc:3060] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.597862  4091 leader_election.cc:304] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.599215  4016 leader_election.cc:304] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:10.600546  4322 raft_consensus.cc:2804] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.600960  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f2a0380d0842d7aca2005755834e79" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:10.600921  4322 raft_consensus.cc:493] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.601747  4322 raft_consensus.cc:3060] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.606943  4267 raft_consensus.cc:515] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.608269  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbf50548b1a4bd48d9af0b5768f973c" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:10.608893  4203 raft_consensus.cc:3060] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.610105  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:10.610764  4053 raft_consensus.cc:3060] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.614410  4348 raft_consensus.cc:2804] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.614944  4348 raft_consensus.cc:493] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.620770  4203 raft_consensus.cc:2468] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 1.
I20260709 12:29:10.622014  4016 leader_election.cc:304] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:10.622651  4267 leader_election.cc:290] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.623482  4372 raft_consensus.cc:2804] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.624082  4372 raft_consensus.cc:697] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:10.624871  4372 consensus_queue.cc:237] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.628194  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:10.617404  4348 raft_consensus.cc:3060] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.628816  4201 raft_consensus.cc:3060] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.635792  4201 raft_consensus.cc:2468] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:10.637019  4091 leader_election.cc:304] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: 
I20260709 12:29:10.638108  4267 raft_consensus.cc:2804] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.638559  4267 raft_consensus.cc:697] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:10.639535  4267 consensus_queue.cc:237] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.642977  3950 catalog_manager.cc:5697] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.652318  4341 tablet_bootstrap.cc:492] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.653170  4341 ts_tablet_manager.cc:1403] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.108s	user 0.010s	sys 0.004s
I20260709 12:29:10.657372  4341 raft_consensus.cc:359] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.658205  4354 raft_consensus.cc:493] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.665151  4341 raft_consensus.cc:385] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.665606  4341 raft_consensus.cc:740] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.665292  4354 raft_consensus.cc:515] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.667562  4348 raft_consensus.cc:515] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.667603  4354 leader_election.cc:290] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.623523  4053 raft_consensus.cc:2468] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:10.669255  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.669840  4348 leader_election.cc:290] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.669975  4127 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:10.671416  4167 leader_election.cc:304] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: 
I20260709 12:29:10.672102  4342 tablet_bootstrap.cc:492] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.672364  4050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:10.650637  4322 raft_consensus.cc:515] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.672595  4051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527b5e51667649bdb98ef7b017072212" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:10.672835  4354 raft_consensus.cc:2804] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.673085  4050 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 0.
I20260709 12:29:10.673286  4354 raft_consensus.cc:493] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.673458  4052 raft_consensus.cc:2393] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 825e407a29994c42b0b1196a872be0bf in current term 1: Already voted for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in this term.
I20260709 12:29:10.673704  4354 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:10.674160  4093 leader_election.cc:343] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:10.672657  4342 ts_tablet_manager.cc:1403] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.107s	user 0.036s	sys 0.025s
I20260709 12:29:10.661806  3949 catalog_manager.cc:5697] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.675925  4341 consensus_queue.cc:260] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.677064  4168 leader_election.cc:304] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf; no voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.678290  4283 raft_consensus.cc:3060] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.678087  4342 raft_consensus.cc:359] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.678876  4342 raft_consensus.cc:385] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.679209  4342 raft_consensus.cc:740] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.679579  4341 ts_tablet_manager.cc:1434] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.002s
I20260709 12:29:10.680023  4342 consensus_queue.cc:260] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.680907  4341 tablet_bootstrap.cc:492] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.682405  4342 ts_tablet_manager.cc:1434] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:10.683457  4342 tablet_bootstrap.cc:492] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.672499  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527b5e51667649bdb98ef7b017072212" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:10.683943  4203 raft_consensus.cc:3060] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.690146  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f2a0380d0842d7aca2005755834e79" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:10.690827  4127 raft_consensus.cc:3060] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.691249  4342 tablet_bootstrap.cc:654] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.691249  4341 tablet_bootstrap.cc:654] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.695302  4322 leader_election.cc:290] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.698201  4127 raft_consensus.cc:2468] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 1.
I20260709 12:29:10.699299  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f2a0380d0842d7aca2005755834e79" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:10.703986  4283 raft_consensus.cc:2749] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:10.705714  4017 leader_election.cc:304] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:10.706575  4322 raft_consensus.cc:2804] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.707338  4322 raft_consensus.cc:697] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:10.708256  4322 consensus_queue.cc:237] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.711005  4203 raft_consensus.cc:2468] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 1.
I20260709 12:29:10.701805  4341 tablet_bootstrap.cc:492] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.702952  4342 tablet_bootstrap.cc:492] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.712163  4342 ts_tablet_manager.cc:1403] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.008s
I20260709 12:29:10.712992  4091 leader_election.cc:304] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:10.712661  3949 catalog_manager.cc:5697] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.715229  4267 raft_consensus.cc:2804] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.715117  4342 raft_consensus.cc:359] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.715947  4342 raft_consensus.cc:385] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.716005  4267 raft_consensus.cc:697] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:10.716320  4342 raft_consensus.cc:740] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.717132  4342 consensus_queue.cc:260] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.717260  4267 consensus_queue.cc:237] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.718267  4341 ts_tablet_manager.cc:1403] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.006s
I20260709 12:29:10.721298  4341 raft_consensus.cc:359] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.721941  4341 raft_consensus.cc:385] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.722180  4341 raft_consensus.cc:740] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.722380  4201 raft_consensus.cc:2468] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 1.
I20260709 12:29:10.722791  4341 consensus_queue.cc:260] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.702596  4354 raft_consensus.cc:515] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.726643  4342 ts_tablet_manager.cc:1434] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:29:10.727943  4052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:10.728564  4052 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.728888  4341 ts_tablet_manager.cc:1434] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:29:10.728742  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:10.729355  4127 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.729856  4341 tablet_bootstrap.cc:492] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417: Bootstrap starting.
I20260709 12:29:10.731900  4354 leader_election.cc:290] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:10.727962  4342 tablet_bootstrap.cc:492] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.734825  3950 catalog_manager.cc:5697] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.737553  4052 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:10.738854  4168 leader_election.cc:304] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:10.739583  4354 raft_consensus.cc:2804] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.740125  4354 raft_consensus.cc:697] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:10.740275  4341 tablet_bootstrap.cc:654] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.741055  4354 consensus_queue.cc:237] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.742159  4342 tablet_bootstrap.cc:654] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.743705  4127 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 1.
I20260709 12:29:10.770119  3949 catalog_manager.cc:5697] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.801990  4341 tablet_bootstrap.cc:492] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417: No bootstrap required, opened a new log
I20260709 12:29:10.802632  4341 ts_tablet_manager.cc:1403] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.025s
I20260709 12:29:10.805817  4341 raft_consensus.cc:359] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.806200  4342 tablet_bootstrap.cc:492] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.806591  4341 raft_consensus.cc:385] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.806886  4342 ts_tablet_manager.cc:1403] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.048s
I20260709 12:29:10.806928  4341 raft_consensus.cc:740] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.806923  3950 catalog_manager.cc:5697] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 1 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:10.808023  4341 consensus_queue.cc:260] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.809507  4342 raft_consensus.cc:359] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.810218  4342 raft_consensus.cc:385] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.810225  4341 ts_tablet_manager.cc:1434] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:10.810554  4342 raft_consensus.cc:740] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.811398  4342 consensus_queue.cc:260] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.813180  4342 ts_tablet_manager.cc:1434] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:10.814073  4342 tablet_bootstrap.cc:492] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2: Bootstrap starting.
I20260709 12:29:10.821193  4342 tablet_bootstrap.cc:654] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:10.825487  4342 tablet_bootstrap.cc:492] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2: No bootstrap required, opened a new log
I20260709 12:29:10.825986  4342 ts_tablet_manager.cc:1403] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:29:10.828091  4342 raft_consensus.cc:359] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.828543  4342 raft_consensus.cc:385] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:10.828719  4342 raft_consensus.cc:740] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Initialized, Role: FOLLOWER
I20260709 12:29:10.829167  4342 consensus_queue.cc:260] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.830550  4342 ts_tablet_manager.cc:1434] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.002s
I20260709 12:29:10.848424  4052 raft_consensus.cc:3060] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.854205  4354 consensus_queue.cc:1048] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:10.866001  4354 consensus_queue.cc:1048] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:10.928704  4322 raft_consensus.cc:493] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.929370  4322 raft_consensus.cc:515] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.931627  4322 leader_election.cc:290] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.932602  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.932780  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:10.933355  4127 raft_consensus.cc:2468] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 0.
I20260709 12:29:10.933370  4201 raft_consensus.cc:2468] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 0.
I20260709 12:29:10.934660  4017 leader_election.cc:304] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:10.935352  4322 raft_consensus.cc:2804] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:10.935686  4322 raft_consensus.cc:493] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:10.935963  4322 raft_consensus.cc:3060] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.941999  4322 raft_consensus.cc:515] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.943847  4322 leader_election.cc:290] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.944645  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:10.944914  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:10.945405  4127 raft_consensus.cc:3060] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.945576  4201 raft_consensus.cc:3060] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.950572  4127 raft_consensus.cc:2468] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 1.
I20260709 12:29:10.951045  4201 raft_consensus.cc:2468] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 1.
I20260709 12:29:10.951668  4017 leader_election.cc:304] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:10.952515  4322 raft_consensus.cc:2804] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:10.952948  4322 raft_consensus.cc:697] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:10.953691  4322 consensus_queue.cc:237] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.958873  4354 consensus_queue.cc:1048] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:10.961820  3949 catalog_manager.cc:5697] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 1 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.966858  4372 raft_consensus.cc:493] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:10.967427  4372 raft_consensus.cc:515] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:10.969592  4372 leader_election.cc:290] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:10.972595  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527b5e51667649bdb98ef7b017072212" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5500e215154de78c642bdb2cfa8417" is_pre_election: true
I20260709 12:29:10.975304  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527b5e51667649bdb98ef7b017072212" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:10.976035  4201 raft_consensus.cc:2393] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b86b8cfb4a184d21bd69a5b1a039bea2 in current term 1: Already voted for candidate 6d5500e215154de78c642bdb2cfa8417 in this term.
I20260709 12:29:10.977937  4052 raft_consensus.cc:3060] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.979799  4016 leader_election.cc:304] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:10.980728  4322 raft_consensus.cc:3060] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:10.986182  4365 consensus_queue.cc:1048] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:10.988144  4322 raft_consensus.cc:2749] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:10.994635  4348 consensus_queue.cc:1048] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.007678  4348 consensus_queue.cc:1048] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.014791  4348 raft_consensus.cc:493] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:11.015293  4348 raft_consensus.cc:515] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.017005  4348 leader_election.cc:290] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:11.017886  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbf50548b1a4bd48d9af0b5768f973c" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" is_pre_election: true
I20260709 12:29:11.019637  4201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbf50548b1a4bd48d9af0b5768f973c" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825e407a29994c42b0b1196a872be0bf" is_pre_election: true
I20260709 12:29:11.020462  4201 raft_consensus.cc:2393] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6d5500e215154de78c642bdb2cfa8417 in current term 1: Already voted for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in this term.
I20260709 12:29:11.021612  4091 leader_election.cc:304] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417; no voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:11.022223  4348 raft_consensus.cc:3060] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:11.028568  4348 raft_consensus.cc:2749] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:11.030802  4322 consensus_queue.cc:1048] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.041782  4372 consensus_queue.cc:1048] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.064044  4348 consensus_queue.cc:1048] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.074445  4267 consensus_queue.cc:1048] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.148051  4372 consensus_queue.cc:1048] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.158742  4267 consensus_queue.cc:1048] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:11.160175  4373 consensus_queue.cc:1048] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:11.189275  4376 consensus_queue.cc:1048] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.194527  4283 consensus_queue.cc:1048] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.197230  4127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
mode: GRACEFUL
new_leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:42744
I20260709 12:29:11.198091  4127 raft_consensus.cc:606] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Received request to transfer leadership to TS b86b8cfb4a184d21bd69a5b1a039bea2
I20260709 12:29:11.214188  4201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
mode: GRACEFUL
new_leader_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42284
I20260709 12:29:11.214826  4201 raft_consensus.cc:606] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Received request to transfer leadership to TS 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:11.215255  4283 consensus_queue.cc:1048] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:11.228334  4201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
mode: GRACEFUL
new_leader_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42284
I20260709 12:29:11.229034  4201 raft_consensus.cc:606] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Received request to transfer leadership to TS 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:11.237203  4203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
mode: GRACEFUL
new_leader_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42284
I20260709 12:29:11.237881  4203 raft_consensus.cc:606] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Received request to transfer leadership to TS 6d5500e215154de78c642bdb2cfa8417
I20260709 12:29:11.243928  4129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
mode: GRACEFUL
new_leader_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42744
I20260709 12:29:11.244622  4129 raft_consensus.cc:606] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Received request to transfer leadership to TS 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:11.324817  4365 raft_consensus.cc:993] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: : Instructing follower 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:11.325403  4354 raft_consensus.cc:1081] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Signalling peer 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:11.326965  4129 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42742
I20260709 12:29:11.327598  4129 raft_consensus.cc:493] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.327972  4129 raft_consensus.cc:3060] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.333427  4129 raft_consensus.cc:515] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.335436  4129 leader_election.cc:290] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:11.336098  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:11.336486  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f5f8fdb07f484facd47ff95233db65" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:11.336755  4053 raft_consensus.cc:3060] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.337144  4203 raft_consensus.cc:3055] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:11.337479  4203 raft_consensus.cc:740] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:11.338286  4203 consensus_queue.cc:260] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.339604  4203 raft_consensus.cc:3060] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.343925  4053 raft_consensus.cc:2468] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:11.345325  4093 leader_election.cc:304] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:11.346102  4348 raft_consensus.cc:2804] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:11.346503  4348 raft_consensus.cc:697] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:11.347043  4203 raft_consensus.cc:2468] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:11.347332  4348 consensus_queue.cc:237] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.354941  3948 catalog_manager.cc:5697] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 1 to 2, leader changed from 825e407a29994c42b0b1196a872be0bf (127.19.36.195) to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:11.412495  4372 consensus_queue.cc:1048] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.422186  4372 consensus_queue.cc:1048] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:11.455047  4365 raft_consensus.cc:993] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: : Instructing follower 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:11.455574  4354 raft_consensus.cc:1081] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Signalling peer 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:11.457069  4127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42742
I20260709 12:29:11.457593  4127 raft_consensus.cc:493] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.457911  4127 raft_consensus.cc:3060] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.462754  4127 raft_consensus.cc:515] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.464440  4127 leader_election.cc:290] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:11.465153  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:11.465510  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3182bf9f5024432beb478c5d81b1e13" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:11.465775  4053 raft_consensus.cc:3060] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.466140  4203 raft_consensus.cc:3055] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:11.466459  4203 raft_consensus.cc:740] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:11.467242  4203 consensus_queue.cc:260] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.468261  4203 raft_consensus.cc:3060] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.471266  4053 raft_consensus.cc:2468] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:11.472366  4093 leader_election.cc:304] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:11.473173  4348 raft_consensus.cc:2804] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:11.473623  4348 raft_consensus.cc:697] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:11.473958  4203 raft_consensus.cc:2468] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:11.474435  4348 consensus_queue.cc:237] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.482079  3950 catalog_manager.cc:5697] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 1 to 2, leader changed from 825e407a29994c42b0b1196a872be0bf (127.19.36.195) to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:11.542340  4267 raft_consensus.cc:993] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: : Instructing follower 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:11.542886  4348 raft_consensus.cc:1081] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Signalling peer 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:11.544461  4203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42274
I20260709 12:29:11.544952  4203 raft_consensus.cc:493] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.545260  4203 raft_consensus.cc:3060] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.550549  4203 raft_consensus.cc:515] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.552995  4203 leader_election.cc:290] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:11.553061  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:11.553839  4053 raft_consensus.cc:3060] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.553769  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839c8a7e470649a6a1f87137a4b76bba" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:11.554643  4127 raft_consensus.cc:3055] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:11.555052  4127 raft_consensus.cc:740] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:11.555949  4127 consensus_queue.cc:260] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.557250  4127 raft_consensus.cc:3060] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.561482  4053 raft_consensus.cc:2468] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 2.
I20260709 12:29:11.562821  4168 leader_election.cc:304] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:11.563683  4354 raft_consensus.cc:2804] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:11.564174  4354 raft_consensus.cc:697] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 2 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:11.564476  4127 raft_consensus.cc:2468] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 2.
I20260709 12:29:11.565024  4354 consensus_queue.cc:237] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.570256  4348 raft_consensus.cc:993] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: : Instructing follower b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:11.570832  4267 raft_consensus.cc:1081] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Signalling peer b86b8cfb4a184d21bd69a5b1a039bea2 to start an election
I20260709 12:29:11.572772  4053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b"
dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
 from {username='slave'} at 127.0.0.1:41808
I20260709 12:29:11.573392  4053 raft_consensus.cc:493] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.573774  4053 raft_consensus.cc:3060] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.573632  3948 catalog_manager.cc:5697] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 1 to 2, leader changed from 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194) to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 2 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.581696  4053 raft_consensus.cc:515] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.583911  4053 leader_election.cc:290] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Requested vote from peers 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:11.584975  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:11.585654  4203 raft_consensus.cc:3060] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.588418  4127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9af04ac7f8b4478dbe38c821153a6d1b" candidate_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:11.589623  4127 raft_consensus.cc:3055] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:11.589972  4127 raft_consensus.cc:740] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:11.590863  4127 consensus_queue.cc:260] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.592216  4127 raft_consensus.cc:3060] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.593575  4203 raft_consensus.cc:2468] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:11.594919  4016 leader_election.cc:304] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:11.595791  4322 raft_consensus.cc:2804] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:11.596271  4322 raft_consensus.cc:697] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Becoming Leader. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:11.597054  4322 consensus_queue.cc:237] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.601856  4127 raft_consensus.cc:2468] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86b8cfb4a184d21bd69a5b1a039bea2 in term 2.
I20260709 12:29:11.606117  3948 catalog_manager.cc:5697] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 reported cstate change: term changed from 1 to 2, leader changed from 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194) to b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193). New cstate: current_term: 2 leader_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:11.706779  4283 raft_consensus.cc:993] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: : Instructing follower 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:11.707258  4354 raft_consensus.cc:1081] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Signalling peer 6d5500e215154de78c642bdb2cfa8417 to start an election
I20260709 12:29:11.708601  4127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
 from {username='slave'} at 127.0.0.1:42742
I20260709 12:29:11.708987  4127 raft_consensus.cc:493] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.709208  4127 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.713567  4127 raft_consensus.cc:515] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.714941  4127 leader_election.cc:290] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885)
I20260709 12:29:11.715797  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:11.715912  4203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "6d5500e215154de78c642bdb2cfa8417" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "825e407a29994c42b0b1196a872be0bf"
I20260709 12:29:11.716673  4053 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.716778  4203 raft_consensus.cc:3055] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:11.717234  4203 raft_consensus.cc:740] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:11.717938  4203 consensus_queue.cc:260] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.718951  4203 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.721837  4053 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:11.722894  4093 leader_election.cc:304] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:11.723448  4203 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5500e215154de78c642bdb2cfa8417 in term 2.
I20260709 12:29:11.723616  4267 raft_consensus.cc:2804] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:11.724128  4267 raft_consensus.cc:697] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Leader. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:11.724929  4267 consensus_queue.cc:237] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:11.731340  3948 catalog_manager.cc:5697] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 reported cstate change: term changed from 1 to 2, leader changed from 825e407a29994c42b0b1196a872be0bf (127.19.36.195) to 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d5500e215154de78c642bdb2cfa8417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:11.786486  4203 raft_consensus.cc:1275] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.787874  4267 consensus_queue.cc:1048] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:11.795706  4053 raft_consensus.cc:1275] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.797519  4267 consensus_queue.cc:1048] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:11.873095  4203 raft_consensus.cc:1275] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.874382  4267 consensus_queue.cc:1048] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:11.882339  4053 raft_consensus.cc:1275] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.884074  4411 consensus_queue.cc:1048] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:11.989136  4127 raft_consensus.cc:1275] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.990336  4373 consensus_queue.cc:1048] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:11.997601  4203 raft_consensus.cc:1275] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer b86b8cfb4a184d21bd69a5b1a039bea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.999300  4373 consensus_queue.cc:1048] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:12.061082  4053 raft_consensus.cc:1275] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.062364  4354 consensus_queue.cc:1048] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:12.073134  4127 raft_consensus.cc:1275] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.074990  4283 consensus_queue.cc:1048] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:12.134490  4053 raft_consensus.cc:1275] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.135819  4411 consensus_queue.cc:1048] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:12.143887  4201 raft_consensus.cc:1275] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Refusing update from remote peer 6d5500e215154de78c642bdb2cfa8417: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.147086  4411 consensus_queue.cc:1048] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:14.248517  3948 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50332:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:29:14.270223  3948 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50332:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:29:14.339027 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:14.432744  4448 tablet.cc:1724] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:14.467583  4229 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:b3182bf9f5024432beb478c5d81b1e13)
W20260709 12:29:14.534924  4229 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:b3182bf9f5024432beb478c5d81b1e13)
W20260709 12:29:14.539336  4229 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:b3182bf9f5024432beb478c5d81b1e13)
I20260709 12:29:14.539202  4457 tablet.cc:1724] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:14.544847  4167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:46091, user_credentials={real_user=slave}} blocked reactor thread for 44850.2us
W20260709 12:29:14.636801  4444 log.cc:927] Time spent T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 12:29:14.630079  4168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:39303, user_credentials={real_user=slave}} blocked reactor thread for 89473.7us
W20260709 12:29:14.643292  4093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:39303, user_credentials={real_user=slave}} blocked reactor thread for 49274.7us
W20260709 12:29:14.691454  4456 log.cc:927] Time spent T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:29:14.692432  4050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a"
dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
mode: GRACEFUL
new_leader_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:41838
I20260709 12:29:14.693231  4050 raft_consensus.cc:606] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Received request to transfer leadership to TS 825e407a29994c42b0b1196a872be0bf
W20260709 12:29:14.694581  4458 log.cc:927] Time spent T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:29:14.657783  4469 tablet.cc:1724] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:14.697293  4455 log.cc:927] Time spent T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:29:14.658783  4470 tablet.cc:1724] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:14.740661  4016 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:38885, user_credentials={real_user=slave}} blocked reactor thread for 104359us
W20260709 12:29:14.719331  4461 log.cc:927] Time spent T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 12:29:14.769851  4488 tablet.cc:1724] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.773195  4480 tablet.cc:1724] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.743608  4457 tablet.cc:1724] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.761575  4484 tablet.cc:1724] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.784287  4475 tablet.cc:1724] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.803851  4448 tablet.cc:1724] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.829105  4490 tablet.cc:1724] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.762961  4483 tablet.cc:1724] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.855820  4489 tablet.cc:1724] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.858340  4418 raft_consensus.cc:993] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: : Instructing follower 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:14.867576  4418 raft_consensus.cc:1081] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Signalling peer 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:14.863600  4469 tablet.cc:1724] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.878499  4201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42276
I20260709 12:29:14.879339  4201 raft_consensus.cc:493] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:14.879817  4201 raft_consensus.cc:3060] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.884922  4470 tablet.cc:1724] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.891095  4126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
mode: GRACEFUL
new_leader_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42758
I20260709 12:29:14.891845  4126 raft_consensus.cc:606] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Received request to transfer leadership to TS 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:14.893550 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:29:14.894183 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:14.905230  4126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce"
dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
mode: GRACEFUL
new_leader_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42758
I20260709 12:29:14.905926  4126 raft_consensus.cc:606] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Received request to transfer leadership to TS 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:14.907605  4489 tablet.cc:1724] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.907853  4480 tablet.cc:1724] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.911131  4488 tablet.cc:1724] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.912897  4457 tablet.cc:1724] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.896883  4201 raft_consensus.cc:515] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:14.925177  4053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d"
dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
mode: GRACEFUL
new_leader_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:41838
I20260709 12:29:14.925812  4053 raft_consensus.cc:606] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Received request to transfer leadership to TS 825e407a29994c42b0b1196a872be0bf
I20260709 12:29:14.924288  4493 tablet.cc:1724] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.929536  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:14.930171  4053 raft_consensus.cc:3055] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:14.930516  4053 raft_consensus.cc:740] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:14.931219  4201 leader_election.cc:290] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 2 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:14.934537  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79651f9639b24c159a6d7dc913a6357a" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:14.935158  4126 raft_consensus.cc:3060] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.940121  4053 consensus_queue.cc:260] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:14.942266  4053 raft_consensus.cc:3060] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.943413  4469 tablet.cc:1724] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.946812  4475 tablet.cc:1724] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.951438 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:14.955343  4053 raft_consensus.cc:2468] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 2.
I20260709 12:29:14.956499  4168 leader_election.cc:304] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:14.957533  4417 raft_consensus.cc:2804] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:14.957998  4417 raft_consensus.cc:697] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 2 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:14.958863  4126 raft_consensus.cc:2468] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 2.
I20260709 12:29:14.958809  4417 consensus_queue.cc:237] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:14.956514  4488 tablet.cc:1724] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.963899  4470 tablet.cc:1724] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.967319  4448 tablet.cc:1724] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.978683 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:14.979547  4490 tablet.cc:1724] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:14.921393  4484 tablet.cc:1724] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:15.000005  3949 catalog_manager.cc:5697] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 1 to 2, leader changed from b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193) to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 2 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:15.001523  3949 catalog_manager.cc:5789] TS 825e407a29994c42b0b1196a872be0bf (127.19.36.195:38885) does not have the latest schema for tablet 79651f9639b24c159a6d7dc913a6357a (table soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c]). Expected version 1 got 0
I20260709 12:29:15.012102  3948 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4: soft_deleted_table [id=5c5ee57d7a664bef87c1ade7626ca12c] alter complete (version 1)
I20260709 12:29:15.069252  4491 raft_consensus.cc:993] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2: : Instructing follower 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:15.069892  4465 raft_consensus.cc:1081] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Signalling peer 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:15.071380  4201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42276
I20260709 12:29:15.071839  4201 raft_consensus.cc:493] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:15.072170  4201 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.076870  4201 raft_consensus.cc:515] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.078598  4201 leader_election.cc:290] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 3 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:15.079272  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:15.079668  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5f60b5ab23e43f5a3acb4ad0bc76e8d" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:15.079931  4053 raft_consensus.cc:3055] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:15.080245  4053 raft_consensus.cc:740] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b86b8cfb4a184d21bd69a5b1a039bea2, State: Running, Role: LEADER
I20260709 12:29:15.080293  4126 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.081084  4053 consensus_queue.cc:260] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.082297  4053 raft_consensus.cc:3060] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.087496  4126 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 3.
I20260709 12:29:15.088850  4167 leader_election.cc:304] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5500e215154de78c642bdb2cfa8417, 825e407a29994c42b0b1196a872be0bf; no voters: 
I20260709 12:29:15.088987  4053 raft_consensus.cc:2468] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 3.
I20260709 12:29:15.089644  4417 raft_consensus.cc:2804] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:15.090138  4417 raft_consensus.cc:697] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 3 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:15.090912  4417 consensus_queue.cc:237] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.098037  4466 raft_consensus.cc:993] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417: : Instructing follower 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:15.098487  4411 raft_consensus.cc:1081] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Signalling peer 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:15.099346  3950 catalog_manager.cc:5697] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 2 to 3, leader changed from b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193) to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 3 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:15.099985  4201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42274
I20260709 12:29:15.100663  4201 raft_consensus.cc:493] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:15.101003  4201 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.106287  4201 raft_consensus.cc:515] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.108180  4201 leader_election.cc:290] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 3 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:15.109103  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:15.109249  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dd5b469c5674294af772a0de19886ce" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:15.109760  4053 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.109890  4126 raft_consensus.cc:3055] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:15.110338  4126 raft_consensus.cc:740] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:15.111130  4126 consensus_queue.cc:260] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.112375  4126 raft_consensus.cc:3060] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.115155  4053 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 3.
I20260709 12:29:15.116330  4168 leader_election.cc:304] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:15.117167  4417 raft_consensus.cc:2804] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:15.117563  4417 raft_consensus.cc:697] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 3 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:15.117898  4126 raft_consensus.cc:2468] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 3.
I20260709 12:29:15.118314  4417 consensus_queue.cc:237] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.125032  3950 catalog_manager.cc:5697] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 2 to 3, leader changed from 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194) to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 3 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:15.155611  4466 raft_consensus.cc:993] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: : Instructing follower 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:15.156153  4411 raft_consensus.cc:1081] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Signalling peer 825e407a29994c42b0b1196a872be0bf to start an election
I20260709 12:29:15.157580  4201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8"
dest_uuid: "825e407a29994c42b0b1196a872be0bf"
 from {username='slave'} at 127.0.0.1:42274
I20260709 12:29:15.158047  4201 raft_consensus.cc:493] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:15.158334  4201 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.162914  4201 raft_consensus.cc:515] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.164435  4201 leader_election.cc:290] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 3 election: Requested vote from peers b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303), 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091)
I20260709 12:29:15.165207  4053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2"
I20260709 12:29:15.165486  4126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd3f9c90732a480fab5f6e1171ab74c8" candidate_uuid: "825e407a29994c42b0b1196a872be0bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "6d5500e215154de78c642bdb2cfa8417"
I20260709 12:29:15.165846  4053 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.166078  4126 raft_consensus.cc:3055] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:15.166483  4126 raft_consensus.cc:740] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d5500e215154de78c642bdb2cfa8417, State: Running, Role: LEADER
I20260709 12:29:15.167343  4126 consensus_queue.cc:260] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.168599  4126 raft_consensus.cc:3060] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.172611  4053 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 3.
I20260709 12:29:15.173810  4168 leader_election.cc:304] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 825e407a29994c42b0b1196a872be0bf, b86b8cfb4a184d21bd69a5b1a039bea2; no voters: 
I20260709 12:29:15.174572  4417 raft_consensus.cc:2804] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:15.174965  4417 raft_consensus.cc:697] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 3 LEADER]: Becoming Leader. State: Replica: 825e407a29994c42b0b1196a872be0bf, State: Running, Role: LEADER
I20260709 12:29:15.175205  4126 raft_consensus.cc:2468] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 825e407a29994c42b0b1196a872be0bf in term 3.
I20260709 12:29:15.175735  4417 consensus_queue.cc:237] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } }
I20260709 12:29:15.184317  3950 catalog_manager.cc:5697] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf reported cstate change: term changed from 2 to 3, leader changed from 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194) to 825e407a29994c42b0b1196a872be0bf (127.19.36.195). New cstate: current_term: 3 leader_uuid: "825e407a29994c42b0b1196a872be0bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825e407a29994c42b0b1196a872be0bf" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 38885 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:15.381551  4126 raft_consensus.cc:1275] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:29:15.382830  4417 consensus_queue.cc:1048] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:15.392350  4053 raft_consensus.cc:1275] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:29:15.393621  4417 consensus_queue.cc:1048] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:15.485765  4053 raft_consensus.cc:1275] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:15.486963  4417 consensus_queue.cc:1048] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:15.499725  4126 raft_consensus.cc:1275] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:15.503925  4417 consensus_queue.cc:1048] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:15.676872  4053 raft_consensus.cc:1275] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:15.678287  4417 consensus_queue.cc:1048] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:15.687779  4126 raft_consensus.cc:1275] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:15.689872  4417 consensus_queue.cc:1048] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:15.753319  4128 raft_consensus.cc:1275] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:15.754565  4507 consensus_queue.cc:1048] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5500e215154de78c642bdb2cfa8417" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 46091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:15.762157  4053 raft_consensus.cc:1275] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Refusing update from remote peer 825e407a29994c42b0b1196a872be0bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:15.764547  4481 consensus_queue.cc:1048] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86b8cfb4a184d21bd69a5b1a039bea2" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:15.980192 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:15.985337 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:15.985790 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:15.990806 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:15.991288 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:16.992395 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:16.998232 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:16.998708 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:17.003979 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:17.004401 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:18.005532 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:18.010501 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:18.010984 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:18.016701 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:18.017076 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:19.018187 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:19.023876 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:19.024299 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:19.029031 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:19.029460 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:20.030786 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:20.037271 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:20.037818 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:20.042711 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:20.043112 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:21.044339 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:21.049695 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:21.050161 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:21.054787 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:21.055199 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:29:22.001001  4093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:39303, user_credentials={real_user=slave}} blocked reactor thread for 210928us
I20260709 12:29:22.293315 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:22.301410 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:22.301980 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:22.310089 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:22.310643 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:29:23.092783  4150 debug-util.cc:398] Leaking SignalData structure 0x7b08001a99e0 after lost signal to thread 19606
W20260709 12:29:23.093775  4150 debug-util.cc:398] Leaking SignalData structure 0x7b08002b7340 after lost signal to thread 3985
W20260709 12:29:23.094857  4150 debug-util.cc:398] Leaking SignalData structure 0x7b080025ca60 after lost signal to thread 4077
W20260709 12:29:23.096158  4150 debug-util.cc:398] Leaking SignalData structure 0x7b080028dda0 after lost signal to thread 4153
W20260709 12:29:23.097219  4150 debug-util.cc:398] Leaking SignalData structure 0x7b0800384a40 after lost signal to thread 4227
I20260709 12:29:23.311728 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:23.318109 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:23.318503 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:23.323171 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:23.323617 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:24.324518 19603 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:24.329488 19603 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:24.329994 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:24.334915 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:24.335342 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:25.407040 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
W20260709 12:29:25.412292  4168 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 12:29:25.419420  4168 consensus_peers.cc:597] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.440938  4093 consensus_peers.cc:597] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.462062 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:25.462965 19603 tablet_replica.cc:333] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.463975 19603 raft_consensus.cc:2243] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.464848 19603 raft_consensus.cc:2272] T 79651f9639b24c159a6d7dc913a6357a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.467890 19603 tablet_replica.cc:333] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.468492 19603 raft_consensus.cc:2243] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.469259 19603 raft_consensus.cc:2272] T 75f5f8fdb07f484facd47ff95233db65 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.472064 19603 tablet_replica.cc:333] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.472867 19603 raft_consensus.cc:2243] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.473920 19603 raft_consensus.cc:2272] T 9af04ac7f8b4478dbe38c821153a6d1b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.476819 19603 tablet_replica.cc:333] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.477546 19603 raft_consensus.cc:2243] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.478466 19603 raft_consensus.cc:2272] T db06f827b69242e0a6d1d38f10292858 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.481235 19603 tablet_replica.cc:333] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.481901 19603 raft_consensus.cc:2243] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.482887 19603 raft_consensus.cc:2272] T ebbf50548b1a4bd48d9af0b5768f973c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.486565 19603 tablet_replica.cc:333] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.487505 19603 raft_consensus.cc:2243] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.488116 19603 raft_consensus.cc:2272] T bd3f9c90732a480fab5f6e1171ab74c8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.491695 19603 tablet_replica.cc:333] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.492760 19603 raft_consensus.cc:2243] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.493449 19603 raft_consensus.cc:2272] T 2ec374ac1d804bec951b2b7e65d5ebc8 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.497089 19603 tablet_replica.cc:333] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.497771 19603 raft_consensus.cc:2243] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.498416 19603 raft_consensus.cc:2272] T b3182bf9f5024432beb478c5d81b1e13 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.502040 19603 tablet_replica.cc:333] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
W20260709 12:29:25.502820  4168 consensus_peers.cc:597] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.502861 19603 raft_consensus.cc:2243] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.504256 19603 raft_consensus.cc:2272] T 81f2a0380d0842d7aca2005755834e79 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.506999 19603 tablet_replica.cc:333] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.507637 19603 raft_consensus.cc:2243] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.508189 19603 raft_consensus.cc:2272] T 37fe0f56980f422fa17effc8cf63940b P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.511009 19603 tablet_replica.cc:333] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.511606 19603 raft_consensus.cc:2243] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.512101 19603 raft_consensus.cc:2272] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.514966 19603 tablet_replica.cc:333] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.515554 19603 raft_consensus.cc:2243] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.516636 19603 raft_consensus.cc:2272] T 7c06ad9cf1ce4740900f95114a2eadea P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.519484 19603 tablet_replica.cc:333] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.520033 19603 raft_consensus.cc:2243] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.520690 19603 raft_consensus.cc:2272] T 527b5e51667649bdb98ef7b017072212 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.523368 19603 tablet_replica.cc:333] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.523923 19603 raft_consensus.cc:2243] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.524483 19603 raft_consensus.cc:2272] T 839c8a7e470649a6a1f87137a4b76bba P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.527235 19603 tablet_replica.cc:333] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.527776 19603 raft_consensus.cc:2243] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.528791 19603 raft_consensus.cc:2272] T ecf13ba63b734e3d9e946d9954deaf3a P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.531960 19603 tablet_replica.cc:333] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.532526 19603 raft_consensus.cc:2243] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.533269 19603 raft_consensus.cc:2272] T 2dd5b469c5674294af772a0de19886ce P b86b8cfb4a184d21bd69a5b1a039bea2 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.533552  4093 consensus_peers.cc:597] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.536763 19603 tablet_replica.cc:333] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.537237 19603 raft_consensus.cc:2243] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.537789 19603 raft_consensus.cc:2272] T e05b4adf9f3845fdb787602c9c168ca7 P b86b8cfb4a184d21bd69a5b1a039bea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.540647 19603 tablet_replica.cc:333] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2: stopping tablet replica
I20260709 12:29:25.541224 19603 raft_consensus.cc:2243] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.541749 19603 raft_consensus.cc:2272] T 6794ce7d4a5142c0b33d6387baee030c P b86b8cfb4a184d21bd69a5b1a039bea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.589228 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
W20260709 12:29:25.641748  4093 consensus_peers.cc:597] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:39303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.653270 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
W20260709 12:29:25.675508  4168 consensus_peers.cc:597] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:39303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.693253 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:25.696520 19603 tablet_replica.cc:333] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.697734 19603 raft_consensus.cc:2243] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.698495 19603 raft_consensus.cc:2272] T 79651f9639b24c159a6d7dc913a6357a P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.700139  4167 consensus_peers.cc:597] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf -> Peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Couldn't send request to peer 6d5500e215154de78c642bdb2cfa8417. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.702332 19603 tablet_replica.cc:333] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.703424 19603 raft_consensus.cc:2243] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.704137 19603 raft_consensus.cc:2272] T ebbf50548b1a4bd48d9af0b5768f973c P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.705291  4093 consensus_peers.cc:597] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:39303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.707504 19603 tablet_replica.cc:333] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.708552 19603 raft_consensus.cc:2243] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.709367 19603 raft_consensus.cc:2272] T 9af04ac7f8b4478dbe38c821153a6d1b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.712934 19603 tablet_replica.cc:333] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
W20260709 12:29:25.712857  4093 consensus_peers.cc:597] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:39303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.713949 19603 raft_consensus.cc:2243] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.714584 19603 raft_consensus.cc:2272] T 81f2a0380d0842d7aca2005755834e79 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.717247 19603 tablet_replica.cc:333] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.717886 19603 raft_consensus.cc:2243] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.718799 19603 raft_consensus.cc:2272] T b3182bf9f5024432beb478c5d81b1e13 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.721272 19603 tablet_replica.cc:333] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.721817 19603 raft_consensus.cc:2243] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.722723 19603 raft_consensus.cc:2272] T 75f5f8fdb07f484facd47ff95233db65 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.725477 19603 tablet_replica.cc:333] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.726104 19603 raft_consensus.cc:2243] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.726624 19603 raft_consensus.cc:2272] T 2dd5b469c5674294af772a0de19886ce P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.729132 19603 tablet_replica.cc:333] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.729784 19603 raft_consensus.cc:2243] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.730254 19603 raft_consensus.cc:2272] T bd3f9c90732a480fab5f6e1171ab74c8 P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.732823 19603 tablet_replica.cc:333] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.733378 19603 raft_consensus.cc:2243] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.734336 19603 raft_consensus.cc:2272] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.736783 19603 tablet_replica.cc:333] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.737327 19603 raft_consensus.cc:2243] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.737843 19603 raft_consensus.cc:2272] T db06f827b69242e0a6d1d38f10292858 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.740363 19603 tablet_replica.cc:333] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.740924 19603 raft_consensus.cc:2243] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.741484 19603 raft_consensus.cc:2272] T 839c8a7e470649a6a1f87137a4b76bba P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.743835 19603 tablet_replica.cc:333] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.744376 19603 raft_consensus.cc:2243] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.744839 19603 raft_consensus.cc:2272] T ecf13ba63b734e3d9e946d9954deaf3a P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.747478 19603 tablet_replica.cc:333] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.748090 19603 raft_consensus.cc:2243] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.748911 19603 raft_consensus.cc:2272] T 527b5e51667649bdb98ef7b017072212 P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.751415 19603 tablet_replica.cc:333] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.751956 19603 raft_consensus.cc:2243] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.752477 19603 raft_consensus.cc:2272] T 37fe0f56980f422fa17effc8cf63940b P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.754853 19603 tablet_replica.cc:333] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.755472 19603 raft_consensus.cc:2243] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.756078 19603 raft_consensus.cc:2272] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 6d5500e215154de78c642bdb2cfa8417 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.758481 19603 tablet_replica.cc:333] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.758981 19603 raft_consensus.cc:2243] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.759982 19603 raft_consensus.cc:2272] T e05b4adf9f3845fdb787602c9c168ca7 P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.762277 19603 tablet_replica.cc:333] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.762830 19603 raft_consensus.cc:2243] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:25.763836 19603 raft_consensus.cc:2272] T 6794ce7d4a5142c0b33d6387baee030c P 6d5500e215154de78c642bdb2cfa8417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:25.766264 19603 tablet_replica.cc:333] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417: stopping tablet replica
I20260709 12:29:25.766803 19603 raft_consensus.cc:2243] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:25.767378 19603 raft_consensus.cc:2272] T 7c06ad9cf1ce4740900f95114a2eadea P 6d5500e215154de78c642bdb2cfa8417 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:25.780840  4167 consensus_peers.cc:597] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf -> Peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Couldn't send request to peer 6d5500e215154de78c642bdb2cfa8417. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.796098  4168 consensus_peers.cc:597] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:39303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.862758  4168 consensus_peers.cc:597] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:39303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.865762  4167 consensus_peers.cc:597] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf -> Peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Couldn't send request to peer 6d5500e215154de78c642bdb2cfa8417. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:46091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.882989  4167 consensus_peers.cc:597] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf -> Peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Couldn't send request to peer 6d5500e215154de78c642bdb2cfa8417. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:46091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.905965  4168 consensus_peers.cc:597] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf -> Peer b86b8cfb4a184d21bd69a5b1a039bea2 (127.19.36.193:39303): Couldn't send request to peer b86b8cfb4a184d21bd69a5b1a039bea2. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:39303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:25.937307  4167 consensus_peers.cc:597] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf -> Peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Couldn't send request to peer 6d5500e215154de78c642bdb2cfa8417. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:46091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:26.114981 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
I20260709 12:29:26.155043 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
W20260709 12:29:26.183287  4167 consensus_peers.cc:597] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf -> Peer 6d5500e215154de78c642bdb2cfa8417 (127.19.36.194:46091): Couldn't send request to peer 6d5500e215154de78c642bdb2cfa8417. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.194:46091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:26.184437 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:26.185295 19603 tablet_replica.cc:333] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.186414 19603 raft_consensus.cc:2243] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.187263 19603 raft_consensus.cc:2272] T 9af04ac7f8b4478dbe38c821153a6d1b P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.189605 19603 tablet_replica.cc:333] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.190243 19603 raft_consensus.cc:2243] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:26.191076 19603 raft_consensus.cc:2272] T 79651f9639b24c159a6d7dc913a6357a P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.193269 19603 tablet_replica.cc:333] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.193856 19603 raft_consensus.cc:2243] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.194345 19603 raft_consensus.cc:2272] T ebbf50548b1a4bd48d9af0b5768f973c P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.196476 19603 tablet_replica.cc:333] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.196990 19603 raft_consensus.cc:2243] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.197526 19603 raft_consensus.cc:2272] T 75f5f8fdb07f484facd47ff95233db65 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.199692 19603 tablet_replica.cc:333] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.200271 19603 raft_consensus.cc:2243] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.200753 19603 raft_consensus.cc:2272] T 81f2a0380d0842d7aca2005755834e79 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.202806 19603 tablet_replica.cc:333] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.203392 19603 raft_consensus.cc:2243] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.203881 19603 raft_consensus.cc:2272] T b3182bf9f5024432beb478c5d81b1e13 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.206008 19603 tablet_replica.cc:333] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.206573 19603 raft_consensus.cc:2243] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:26.207525 19603 raft_consensus.cc:2272] T bd3f9c90732a480fab5f6e1171ab74c8 P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.209502 19603 tablet_replica.cc:333] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.210039 19603 raft_consensus.cc:2243] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.210486 19603 raft_consensus.cc:2272] T 2ec374ac1d804bec951b2b7e65d5ebc8 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.212534 19603 tablet_replica.cc:333] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.213042 19603 raft_consensus.cc:2243] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:26.213865 19603 raft_consensus.cc:2272] T 2dd5b469c5674294af772a0de19886ce P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.216001 19603 tablet_replica.cc:333] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.216535 19603 raft_consensus.cc:2243] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.216984 19603 raft_consensus.cc:2272] T db06f827b69242e0a6d1d38f10292858 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.219116 19603 tablet_replica.cc:333] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.219727 19603 raft_consensus.cc:2243] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.220199 19603 raft_consensus.cc:2272] T 6794ce7d4a5142c0b33d6387baee030c P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.222350 19603 tablet_replica.cc:333] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.222859 19603 raft_consensus.cc:2243] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.223327 19603 raft_consensus.cc:2272] T e05b4adf9f3845fdb787602c9c168ca7 P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.225376 19603 tablet_replica.cc:333] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.225873 19603 raft_consensus.cc:2243] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:26.226681 19603 raft_consensus.cc:2272] T 37fe0f56980f422fa17effc8cf63940b P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.229348 19603 tablet_replica.cc:333] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.230127 19603 raft_consensus.cc:2243] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:26.230968 19603 raft_consensus.cc:2272] T b5f60b5ab23e43f5a3acb4ad0bc76e8d P 825e407a29994c42b0b1196a872be0bf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.233117 19603 tablet_replica.cc:333] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.233659 19603 raft_consensus.cc:2243] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.234123 19603 raft_consensus.cc:2272] T 527b5e51667649bdb98ef7b017072212 P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.236183 19603 tablet_replica.cc:333] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.236718 19603 raft_consensus.cc:2243] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:26.237469 19603 raft_consensus.cc:2272] T 839c8a7e470649a6a1f87137a4b76bba P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.239601 19603 tablet_replica.cc:333] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.240111 19603 raft_consensus.cc:2243] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.240551 19603 raft_consensus.cc:2272] T ecf13ba63b734e3d9e946d9954deaf3a P 825e407a29994c42b0b1196a872be0bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.242728 19603 tablet_replica.cc:333] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf: stopping tablet replica
I20260709 12:29:26.243261 19603 raft_consensus.cc:2243] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:26.243711 19603 raft_consensus.cc:2272] T 7c06ad9cf1ce4740900f95114a2eadea P 825e407a29994c42b0b1196a872be0bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.286991 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
I20260709 12:29:26.324944 19603 master.cc:562] Master@127.19.36.254:43127 shutting down...
I20260709 12:29:26.350589 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:26.351284 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:26.351583 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8aff479924f94418980445175024eed4: stopping tablet replica
I20260709 12:29:26.373087 19603 master.cc:584] Master@127.19.36.254:43127 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21388 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:29:26.429221 19603 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.36.254:32959
I20260709 12:29:26.430351 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:26.436323  4544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:26.436314  4543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:26.437386  4546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:26.437757 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:26.438699 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:26.438918 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:26.439040 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600166439029 us; error 0 us; skew 500 ppm
I20260709 12:29:26.439555 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:26.441902 19603 webserver.cc:533] Webserver started at http://127.19.36.254:37557/ using document root <none> and password file <none>
I20260709 12:29:26.442342 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:26.442493 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:26.442710 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:26.443876 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/master-0-root/instance:
uuid: "874cc058dc204191b51fd0b57de88a88"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.448217 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:29:26.451416  4551 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.452174 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:29:26.452411 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/master-0-root
uuid: "874cc058dc204191b51fd0b57de88a88"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.452823 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:26.495098 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:26.496309 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:26.535389 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.254:32959
I20260709 12:29:26.535508  4602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.254:32959 every 8 connection(s)
I20260709 12:29:26.539273  4603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:26.551301  4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88: Bootstrap starting.
I20260709 12:29:26.556470  4603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:26.560948  4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88: No bootstrap required, opened a new log
I20260709 12:29:26.563376  4603 raft_consensus.cc:359] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874cc058dc204191b51fd0b57de88a88" member_type: VOTER }
I20260709 12:29:26.563840  4603 raft_consensus.cc:385] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:26.564088  4603 raft_consensus.cc:740] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874cc058dc204191b51fd0b57de88a88, State: Initialized, Role: FOLLOWER
I20260709 12:29:26.564754  4603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874cc058dc204191b51fd0b57de88a88" member_type: VOTER }
I20260709 12:29:26.565251  4603 raft_consensus.cc:399] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:26.565454  4603 raft_consensus.cc:493] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:26.565773  4603 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:26.570647  4603 raft_consensus.cc:515] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874cc058dc204191b51fd0b57de88a88" member_type: VOTER }
I20260709 12:29:26.571154  4603 leader_election.cc:304] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 874cc058dc204191b51fd0b57de88a88; no voters: 
I20260709 12:29:26.572315  4603 leader_election.cc:290] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:26.572800  4606 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:26.574203  4606 raft_consensus.cc:697] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 1 LEADER]: Becoming Leader. State: Replica: 874cc058dc204191b51fd0b57de88a88, State: Running, Role: LEADER
I20260709 12:29:26.574946  4606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874cc058dc204191b51fd0b57de88a88" member_type: VOTER }
I20260709 12:29:26.575467  4603 sys_catalog.cc:565] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:26.579887  4607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "874cc058dc204191b51fd0b57de88a88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874cc058dc204191b51fd0b57de88a88" member_type: VOTER } }
I20260709 12:29:26.580354  4608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874cc058dc204191b51fd0b57de88a88. Latest consensus state: current_term: 1 leader_uuid: "874cc058dc204191b51fd0b57de88a88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874cc058dc204191b51fd0b57de88a88" member_type: VOTER } }
I20260709 12:29:26.580780  4607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:26.581254  4608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:26.588289  4613 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:26.593497  4613 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:26.594674 19603 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:26.602576  4613 catalog_manager.cc:1383] Generated new cluster ID: d30c357ae02b4a03b72e4b9c58b7b3b4
I20260709 12:29:26.602892  4613 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:26.614807  4613 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:26.616147  4613 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:26.630136  4613 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88: Generated new TSK 0
I20260709 12:29:26.630779  4613 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:26.662087 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:26.668362  4624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:26.669703  4625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:26.673358 19603 server_base.cc:1061] running on GCE node
W20260709 12:29:26.674722  4627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:26.678925 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:26.679224 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:26.679380 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600166679367 us; error 0 us; skew 500 ppm
I20260709 12:29:26.679904 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:26.682220 19603 webserver.cc:533] Webserver started at http://127.19.36.193:37269/ using document root <none> and password file <none>
I20260709 12:29:26.682654 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:26.682803 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:26.683008 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:26.684136 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-0-root/instance:
uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.688800 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:29:26.692201  4632 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.693059 19603 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:26.693354 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-0-root
uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.693624 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:26.708871 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:26.710083 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:26.711620 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:26.713831 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:26.714046 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.714291 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:26.714485 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.756045 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.193:37869
I20260709 12:29:26.756170  4694 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.193:37869 every 8 connection(s)
I20260709 12:29:26.761157 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:26.768991  4699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:26.772424  4700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:26.773981  4702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:26.775368 19603 server_base.cc:1061] running on GCE node
I20260709 12:29:26.776367  4695 heartbeater.cc:344] Connected to a master server at 127.19.36.254:32959
I20260709 12:29:26.776610 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:29:26.776821  4695 heartbeater.cc:461] Registering TS with master...
W20260709 12:29:26.776924 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:26.777338 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600166777321 us; error 0 us; skew 500 ppm
I20260709 12:29:26.777940 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:26.777887  4695 heartbeater.cc:507] Master 127.19.36.254:32959 requested a full tablet report, sending...
I20260709 12:29:26.780679  4568 ts_manager.cc:194] Registered new tserver with Master: 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869)
I20260709 12:29:26.780829 19603 webserver.cc:533] Webserver started at http://127.19.36.194:40941/ using document root <none> and password file <none>
I20260709 12:29:26.781524 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:26.781734 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:26.782012 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:26.782769  4568 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47014
I20260709 12:29:26.783375 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-1-root/instance:
uuid: "6d0e2be318224ee5a8f54ea891b0856a"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.788154 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:29:26.791782  4707 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.792580 19603 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:29:26.792935 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-1-root
uuid: "6d0e2be318224ee5a8f54ea891b0856a"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.793226 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:26.817037 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:26.818310 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:26.819991 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:26.822445 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:26.822695 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.822968 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:26.823117 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.864394 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.194:44691
I20260709 12:29:26.864657  4769 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.194:44691 every 8 connection(s)
I20260709 12:29:26.869367 19603 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:26.876953  4773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:26.878612  4774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:26.880367 19603 server_base.cc:1061] running on GCE node
W20260709 12:29:26.881227  4776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:26.882231 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:26.882534 19603 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:26.882761 19603 hybrid_clock.cc:648] HybridClock initialized: now 1783600166882739 us; error 0 us; skew 500 ppm
I20260709 12:29:26.883440 19603 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:26.886413  4770 heartbeater.cc:344] Connected to a master server at 127.19.36.254:32959
I20260709 12:29:26.886505 19603 webserver.cc:533] Webserver started at http://127.19.36.195:42893/ using document root <none> and password file <none>
I20260709 12:29:26.887004  4770 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:26.887287 19603 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:26.887583 19603 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:26.887857 19603 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:26.887888  4770 heartbeater.cc:507] Master 127.19.36.254:32959 requested a full tablet report, sending...
I20260709 12:29:26.889097 19603 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-2-root/instance:
uuid: "5cbdbb7559fb49919a700494dc154588"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.889991  4568 ts_manager.cc:194] Registered new tserver with Master: 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:26.891444  4568 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47026
I20260709 12:29:26.893942 19603 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:29:26.897850  4781 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.898856 19603 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:29:26.899232 19603 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-2-root
uuid: "5cbdbb7559fb49919a700494dc154588"
format_stamp: "Formatted at 2026-07-09 12:29:26 on dist-test-slave-hfcn"
I20260709 12:29:26.899581 19603 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf7kfbm/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599599738032-19603-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:26.912914 19603 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:26.914132 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:26.915656 19603 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:26.917971 19603 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:26.918205 19603 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.918460 19603 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:26.918644 19603 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:26.958933 19603 rpc_server.cc:307] RPC server started. Bound to: 127.19.36.195:46201
I20260709 12:29:26.959046  4843 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.36.195:46201 every 8 connection(s)
I20260709 12:29:26.972879  4844 heartbeater.cc:344] Connected to a master server at 127.19.36.254:32959
I20260709 12:29:26.973412  4844 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:26.974166  4844 heartbeater.cc:507] Master 127.19.36.254:32959 requested a full tablet report, sending...
I20260709 12:29:26.976258  4568 ts_manager.cc:194] Registered new tserver with Master: 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:26.976838 19603 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014580724s
I20260709 12:29:26.977963  4568 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47030
I20260709 12:29:27.002945  4568 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47040:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:27.005708  4568 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:27.075261  4735 tablet_service.cc:1511] Processing CreateTablet for tablet 933310e093f24d6c87a460c3471eca97 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:27.076730  4732 tablet_service.cc:1511] Processing CreateTablet for tablet d7adec6a263b40ae8f500e6d4de59996 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:27.078466  4731 tablet_service.cc:1511] Processing CreateTablet for tablet 04f0cfa224354c31afe3140f261990d7 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:27.076815  4735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 933310e093f24d6c87a460c3471eca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.080133  4733 tablet_service.cc:1511] Processing CreateTablet for tablet ae78f7d9184346839a507b7bd4e2659a (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:27.080502  4734 tablet_service.cc:1511] Processing CreateTablet for tablet 4a3068fedf904cd3b43cbcd34a3e61aa (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:27.080982  4731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04f0cfa224354c31afe3140f261990d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.081889  4734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a3068fedf904cd3b43cbcd34a3e61aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.083712  4732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7adec6a263b40ae8f500e6d4de59996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.085284  4730 tablet_service.cc:1511] Processing CreateTablet for tablet 70f0d3ef958849f08eaf18856e6567e2 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:27.086938  4733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae78f7d9184346839a507b7bd4e2659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.089399  4730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70f0d3ef958849f08eaf18856e6567e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.097059  4729 tablet_service.cc:1511] Processing CreateTablet for tablet 748fe2ac99874ea9b112159a53cc3beb (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:27.098536  4729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748fe2ac99874ea9b112159a53cc3beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.103459  4728 tablet_service.cc:1511] Processing CreateTablet for tablet ddfff7a3d59a47de8bc9f6929041196a (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:27.104894  4728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfff7a3d59a47de8bc9f6929041196a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.108213  4808 tablet_service.cc:1511] Processing CreateTablet for tablet 4a3068fedf904cd3b43cbcd34a3e61aa (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:27.109632  4808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a3068fedf904cd3b43cbcd34a3e61aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.110144  4805 tablet_service.cc:1511] Processing CreateTablet for tablet 04f0cfa224354c31afe3140f261990d7 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:27.111521  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04f0cfa224354c31afe3140f261990d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.112340  4807 tablet_service.cc:1511] Processing CreateTablet for tablet ae78f7d9184346839a507b7bd4e2659a (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:27.113708  4807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae78f7d9184346839a507b7bd4e2659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.115607  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 70f0d3ef958849f08eaf18856e6567e2 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:27.116250  4858 tablet_bootstrap.cc:492] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.116219  4806 tablet_service.cc:1511] Processing CreateTablet for tablet d7adec6a263b40ae8f500e6d4de59996 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:27.117024  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70f0d3ef958849f08eaf18856e6567e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.120294  4803 tablet_service.cc:1511] Processing CreateTablet for tablet 748fe2ac99874ea9b112159a53cc3beb (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:27.121690  4803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748fe2ac99874ea9b112159a53cc3beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.121555  4802 tablet_service.cc:1511] Processing CreateTablet for tablet ddfff7a3d59a47de8bc9f6929041196a (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:27.123026  4858 tablet_bootstrap.cc:654] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.123564  4806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7adec6a263b40ae8f500e6d4de59996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.123410  4809 tablet_service.cc:1511] Processing CreateTablet for tablet 933310e093f24d6c87a460c3471eca97 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:27.124687  4809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 933310e093f24d6c87a460c3471eca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.128108  4801 tablet_service.cc:1511] Processing CreateTablet for tablet 197e1cac162e444ebbb81bd7f563902b (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:27.129560  4801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197e1cac162e444ebbb81bd7f563902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.133237  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfff7a3d59a47de8bc9f6929041196a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.133133  4731 tablet_service.cc:1511] Processing CreateTablet for tablet 197e1cac162e444ebbb81bd7f563902b (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:27.134600  4731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197e1cac162e444ebbb81bd7f563902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.180344  4860 tablet_bootstrap.cc:492] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.197893  4660 tablet_service.cc:1511] Processing CreateTablet for tablet 933310e093f24d6c87a460c3471eca97 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:27.200819  4659 tablet_service.cc:1511] Processing CreateTablet for tablet 4a3068fedf904cd3b43cbcd34a3e61aa (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:27.203542  4658 tablet_service.cc:1511] Processing CreateTablet for tablet ae78f7d9184346839a507b7bd4e2659a (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:27.206207  4657 tablet_service.cc:1511] Processing CreateTablet for tablet d7adec6a263b40ae8f500e6d4de59996 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:27.208806  4656 tablet_service.cc:1511] Processing CreateTablet for tablet 04f0cfa224354c31afe3140f261990d7 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:27.211642  4860 tablet_bootstrap.cc:654] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.211442  4655 tablet_service.cc:1511] Processing CreateTablet for tablet 70f0d3ef958849f08eaf18856e6567e2 (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:27.216013  4654 tablet_service.cc:1511] Processing CreateTablet for tablet 748fe2ac99874ea9b112159a53cc3beb (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:27.219875  4660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 933310e093f24d6c87a460c3471eca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.229221  4653 tablet_service.cc:1511] Processing CreateTablet for tablet ddfff7a3d59a47de8bc9f6929041196a (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:27.244837  4652 tablet_service.cc:1511] Processing CreateTablet for tablet 197e1cac162e444ebbb81bd7f563902b (DEFAULT_TABLE table=test-workload [id=0cd8e9d35a5d412084c57b4e5a67261b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:27.251130  4652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197e1cac162e444ebbb81bd7f563902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.255717  4654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748fe2ac99874ea9b112159a53cc3beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.259473  4659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a3068fedf904cd3b43cbcd34a3e61aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.262290  4656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04f0cfa224354c31afe3140f261990d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.263408  4655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70f0d3ef958849f08eaf18856e6567e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.264233  4653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfff7a3d59a47de8bc9f6929041196a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.267498  4658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae78f7d9184346839a507b7bd4e2659a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.271831  4657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7adec6a263b40ae8f500e6d4de59996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.332434  4862 tablet_bootstrap.cc:492] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.332809  4860 tablet_bootstrap.cc:492] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.333382  4860 ts_tablet_manager.cc:1403] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.153s	user 0.047s	sys 0.024s
I20260709 12:29:27.336400  4860 raft_consensus.cc:359] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.337175  4860 raft_consensus.cc:385] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.337476  4860 raft_consensus.cc:740] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.338254  4860 consensus_queue.cc:260] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.339522  4858 tablet_bootstrap.cc:492] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.340039  4858 ts_tablet_manager.cc:1403] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.224s	user 0.057s	sys 0.087s
I20260709 12:29:27.343116  4858 raft_consensus.cc:359] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.343822  4858 raft_consensus.cc:385] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.344118  4858 raft_consensus.cc:740] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.344892  4858 consensus_queue.cc:260] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.339025  4862 tablet_bootstrap.cc:654] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.353284  4858 ts_tablet_manager.cc:1434] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.013s	user 0.013s	sys 0.000s
I20260709 12:29:27.353621  4770 heartbeater.cc:499] Master 127.19.36.254:32959 was elected leader, sending a full tablet report...
I20260709 12:29:27.356727  4858 tablet_bootstrap.cc:492] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.360107  4860 ts_tablet_manager.cc:1434] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.004s
I20260709 12:29:27.360889  4844 heartbeater.cc:499] Master 127.19.36.254:32959 was elected leader, sending a full tablet report...
I20260709 12:29:27.364809  4862 tablet_bootstrap.cc:492] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.365339  4860 tablet_bootstrap.cc:492] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.365406  4862 ts_tablet_manager.cc:1403] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.006s
W20260709 12:29:27.368952  4771 tablet.cc:2406] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:29:27.368731  4862 raft_consensus.cc:359] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.369740  4862 raft_consensus.cc:385] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.370133  4862 raft_consensus.cc:740] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.371120  4862 consensus_queue.cc:260] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.372952  4860 tablet_bootstrap.cc:654] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.379876  4860 tablet_bootstrap.cc:492] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.380383  4860 ts_tablet_manager.cc:1403] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.001s
I20260709 12:29:27.381449  4862 ts_tablet_manager.cc:1434] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.007s
I20260709 12:29:27.382262  4695 heartbeater.cc:499] Master 127.19.36.254:32959 was elected leader, sending a full tablet report...
I20260709 12:29:27.382797  4862 tablet_bootstrap.cc:492] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.383299  4860 raft_consensus.cc:359] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.384063  4860 raft_consensus.cc:385] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.384366  4860 raft_consensus.cc:740] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.385239  4860 consensus_queue.cc:260] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.387456  4866 raft_consensus.cc:493] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.390054  4858 tablet_bootstrap.cc:654] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.389919  4866 raft_consensus.cc:515] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.388010  4860 ts_tablet_manager.cc:1434] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:29:27.391579  4860 tablet_bootstrap.cc:492] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.399502  4860 tablet_bootstrap.cc:654] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.399526  4866 leader_election.cc:290] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.402199  4862 tablet_bootstrap.cc:654] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.407505  4858 tablet_bootstrap.cc:492] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.408070  4858 ts_tablet_manager.cc:1403] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.052s	user 0.004s	sys 0.011s
I20260709 12:29:27.409343  4860 tablet_bootstrap.cc:492] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.409874  4860 ts_tablet_manager.cc:1403] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.008s
I20260709 12:29:27.412895  4860 raft_consensus.cc:359] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.413647  4860 raft_consensus.cc:385] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.413964  4860 raft_consensus.cc:740] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.414935  4860 consensus_queue.cc:260] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.417238  4860 ts_tablet_manager.cc:1434] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:27.418222  4860 tablet_bootstrap.cc:492] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.423230  4862 tablet_bootstrap.cc:492] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.423817  4862 ts_tablet_manager.cc:1403] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.004s
I20260709 12:29:27.424134  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.424862  4860 tablet_bootstrap.cc:654] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.410858  4858 raft_consensus.cc:359] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.425343  4858 raft_consensus.cc:385] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.425658  4858 raft_consensus.cc:740] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.426523  4858 consensus_queue.cc:260] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.426918  4862 raft_consensus.cc:359] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.427738  4862 raft_consensus.cc:385] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.428028  4862 raft_consensus.cc:740] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.428777  4858 ts_tablet_manager.cc:1434] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:29:27.429833  4858 tablet_bootstrap.cc:492] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.436153  4858 tablet_bootstrap.cc:654] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.437639  4860 tablet_bootstrap.cc:492] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.438190  4860 ts_tablet_manager.cc:1403] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 12:29:27.437970  4862 consensus_queue.cc:260] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
W20260709 12:29:27.440163  4636 leader_election.cc:343] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.441284  4860 raft_consensus.cc:359] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.442045  4860 raft_consensus.cc:385] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.442102  4862 ts_tablet_manager.cc:1434] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:29:27.442453  4860 raft_consensus.cc:740] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.443265  4862 tablet_bootstrap.cc:492] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.443475  4860 consensus_queue.cc:260] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.448135  4860 ts_tablet_manager.cc:1434] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:29:27.448561  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.449193  4860 tablet_bootstrap.cc:492] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
W20260709 12:29:27.454430  4636 leader_election.cc:343] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.454974  4636 leader_election.cc:304] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f; no voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:27.456568  4860 tablet_bootstrap.cc:654] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.460409  4862 tablet_bootstrap.cc:654] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.461766  4858 tablet_bootstrap.cc:492] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.462272  4858 ts_tablet_manager.cc:1403] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.018s
I20260709 12:29:27.464670  4858 raft_consensus.cc:359] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.465222  4858 raft_consensus.cc:385] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.465374  4866 raft_consensus.cc:2749] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.465545  4858 raft_consensus.cc:740] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.466374  4858 consensus_queue.cc:260] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.468927  4858 ts_tablet_manager.cc:1434] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:27.469619  4860 tablet_bootstrap.cc:492] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.469923  4858 tablet_bootstrap.cc:492] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.470160  4860 ts_tablet_manager.cc:1403] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 12:29:27.472735  4864 raft_consensus.cc:493] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.473233  4864 raft_consensus.cc:515] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.475703  4864 leader_election.cc:290] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.479734  4858 tablet_bootstrap.cc:654] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.482740  4863 raft_consensus.cc:493] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.483325  4863 raft_consensus.cc:515] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.485402  4862 tablet_bootstrap.cc:492] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.485285  4860 raft_consensus.cc:359] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.486018  4862 ts_tablet_manager.cc:1403] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.015s
I20260709 12:29:27.486042  4860 raft_consensus.cc:385] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.486907  4860 raft_consensus.cc:740] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.488139  4860 consensus_queue.cc:260] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.489600  4862 raft_consensus.cc:359] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.490572  4862 raft_consensus.cc:385] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.490971  4860 ts_tablet_manager.cc:1434] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.001s
I20260709 12:29:27.492436  4860 tablet_bootstrap.cc:492] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.492379  4863 leader_election.cc:290] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.502141  4860 tablet_bootstrap.cc:654] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.490933  4862 raft_consensus.cc:740] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.514071  4862 consensus_queue.cc:260] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.516431  4862 ts_tablet_manager.cc:1434] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.007s
I20260709 12:29:27.519445  4866 raft_consensus.cc:493] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.519711  4858 tablet_bootstrap.cc:492] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.520282  4858 ts_tablet_manager.cc:1403] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.000s
I20260709 12:29:27.519992  4866 raft_consensus.cc:515] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.522310  4866 leader_election.cc:290] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.524657  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.525362  4745 raft_consensus.cc:2468] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:27.526365  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.528173  4636 leader_election.cc:304] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
W20260709 12:29:27.529070  4636 leader_election.cc:343] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.530215  4862 tablet_bootstrap.cc:492] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.530390  4866 raft_consensus.cc:2804] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.530783  4866 raft_consensus.cc:493] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.531122  4866 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.547528  4860 tablet_bootstrap.cc:492] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.548138  4862 tablet_bootstrap.cc:654] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.548454  4860 ts_tablet_manager.cc:1403] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.056s	user 0.025s	sys 0.028s
I20260709 12:29:27.549367  4858 raft_consensus.cc:359] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.550207  4858 raft_consensus.cc:385] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.550508  4858 raft_consensus.cc:740] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.553543  4860 raft_consensus.cc:359] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.554683  4860 raft_consensus.cc:385] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.555035  4860 raft_consensus.cc:740] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.555320  4863 raft_consensus.cc:493] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.555866  4863 raft_consensus.cc:515] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.555936  4860 consensus_queue.cc:260] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.557264  4862 tablet_bootstrap.cc:492] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.557770  4862 ts_tablet_manager.cc:1403] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 12:29:27.557991  4863 leader_election.cc:290] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.558684  4860 ts_tablet_manager.cc:1434] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:29:27.559226  4866 raft_consensus.cc:515] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.559800  4860 tablet_bootstrap.cc:492] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.560354  4864 raft_consensus.cc:493] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.560846  4864 raft_consensus.cc:515] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.561926  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.562613  4745 raft_consensus.cc:2468] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:27.562793  4864 leader_election.cc:290] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.563696  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:27.564349  4744 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.564800  4670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.567560  4860 tablet_bootstrap.cc:654] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.570045  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:27.570191  4670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.570667  4819 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:27.572193  4711 leader_election.cc:343] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.572419  4744 raft_consensus.cc:2468] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:27.573725  4636 leader_election.cc:304] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:27.574345  4670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae78f7d9184346839a507b7bd4e2659a" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.574690  4884 raft_consensus.cc:2804] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:27.575675  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.560796  4862 raft_consensus.cc:359] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.576306  4862 raft_consensus.cc:385] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.576612  4862 raft_consensus.cc:740] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.577229  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae78f7d9184346839a507b7bd4e2659a" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.577555  4862 consensus_queue.cc:260] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
W20260709 12:29:27.577205  4711 leader_election.cc:343] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.582338  4819 raft_consensus.cc:2468] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:27.583634  4866 leader_election.cc:290] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.586959  4785 leader_election.cc:304] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:27.588138  4864 raft_consensus.cc:2804] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.588610  4864 raft_consensus.cc:493] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:27.588838  4616 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:27.589082  4864 raft_consensus.cc:3060] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.592626  4858 consensus_queue.cc:260] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.595026  4866 raft_consensus.cc:493] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.595449  4860 tablet_bootstrap.cc:492] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
W20260709 12:29:27.598518  4785 leader_election.cc:343] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.595687  4866 raft_consensus.cc:515] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.596200  4860 ts_tablet_manager.cc:1403] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.017s
W20260709 12:29:27.603018  4785 leader_election.cc:343] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.603322  4860 raft_consensus.cc:359] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
W20260709 12:29:27.604048  4785 leader_election.cc:343] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.604205  4860 raft_consensus.cc:385] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.604643  4860 raft_consensus.cc:740] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.598176  4864 raft_consensus.cc:515] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.605567  4860 consensus_queue.cc:260] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.604554  4785 leader_election.cc:304] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588; no voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:27.607054  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.607239  4864 leader_election.cc:290] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.607815  4818 raft_consensus.cc:2468] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:27.608245  4864 raft_consensus.cc:2749] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.608609  4884 raft_consensus.cc:697] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:27.609335  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.609589  4711 leader_election.cc:304] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:27.598379  4858 ts_tablet_manager.cc:1434] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 12:29:27.610754  4863 raft_consensus.cc:2804] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.611255  4863 raft_consensus.cc:493] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:27.611425  4636 leader_election.cc:343] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.611611  4863 raft_consensus.cc:3060] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.612321  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
W20260709 12:29:27.613878  4636 leader_election.cc:343] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.614318  4636 leader_election.cc:304] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f; no voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:27.615231  4866 leader_election.cc:290] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.615491  4888 raft_consensus.cc:2749] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.606921  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.613968  4858 tablet_bootstrap.cc:492] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.616621  4819 raft_consensus.cc:2393] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6d0e2be318224ee5a8f54ea891b0856a in current term 1: Already voted for candidate 5cbdbb7559fb49919a700494dc154588 in this term.
I20260709 12:29:27.619465  4711 leader_election.cc:304] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d0e2be318224ee5a8f54ea891b0856a; no voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588
I20260709 12:29:27.624286  4863 raft_consensus.cc:515] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.625378  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:27.625726  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:27.626317  4863 leader_election.cc:290] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.626566  4745 raft_consensus.cc:3060] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:27.627085  4785 leader_election.cc:343] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.609548  4884 consensus_queue.cc:237] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.631378  4862 ts_tablet_manager.cc:1434] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.015s
I20260709 12:29:27.632627  4862 tablet_bootstrap.cc:492] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.636001  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:27.638316  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:27.638952  4819 raft_consensus.cc:3060] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.641769  4745 raft_consensus.cc:2468] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:27.644316  4785 leader_election.cc:304] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:27.645152  4860 ts_tablet_manager.cc:1434] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.045s	user 0.019s	sys 0.011s
I20260709 12:29:27.646008  4858 tablet_bootstrap.cc:654] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.646428  4860 tablet_bootstrap.cc:492] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.643046  4862 tablet_bootstrap.cc:654] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.649267  4889 raft_consensus.cc:2804] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:29:27.651567  4711 leader_election.cc:343] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.651698  4568 catalog_manager.cc:5697] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 1 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.653568  4819 raft_consensus.cc:2468] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:27.655431  4711 leader_election.cc:304] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:27.656425  4863 raft_consensus.cc:2749] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.658667  4889 raft_consensus.cc:697] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:27.660235  4891 raft_consensus.cc:2804] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:27.659960  4889 consensus_queue.cc:237] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.662880  4860 tablet_bootstrap.cc:654] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.672155  4891 raft_consensus.cc:697] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:27.679874  4858 tablet_bootstrap.cc:492] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.680766  4858 ts_tablet_manager.cc:1403] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.005s
I20260709 12:29:27.680136  4891 consensus_queue.cc:237] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.685846  4858 raft_consensus.cc:359] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.694180  4858 raft_consensus.cc:385] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.694727  4858 raft_consensus.cc:740] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.695052  4862 tablet_bootstrap.cc:492] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.696808  4862 ts_tablet_manager.cc:1403] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.064s	user 0.026s	sys 0.025s
I20260709 12:29:27.697443  4860 tablet_bootstrap.cc:492] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.698228  4860 ts_tablet_manager.cc:1403] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.002s
I20260709 12:29:27.699276  4858 consensus_queue.cc:260] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.702040  4858 ts_tablet_manager.cc:1434] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:29:27.701632  4860 raft_consensus.cc:359] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.705241  4860 raft_consensus.cc:385] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.702209  4862 raft_consensus.cc:359] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.705636  4860 raft_consensus.cc:740] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.706020  4858 tablet_bootstrap.cc:492] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.705947  4567 catalog_manager.cc:5697] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.706670  4860 consensus_queue.cc:260] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.708988  4860 ts_tablet_manager.cc:1434] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 12:29:27.709172  4889 raft_consensus.cc:493] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.709641  4889 raft_consensus.cc:515] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.710059  4860 tablet_bootstrap.cc:492] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:27.706099  4862 raft_consensus.cc:385] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.711604  4862 raft_consensus.cc:740] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.711702  4889 leader_election.cc:290] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.712365  4862 consensus_queue.cc:260] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.713268  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748fe2ac99874ea9b112159a53cc3beb" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.713609  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748fe2ac99874ea9b112159a53cc3beb" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.714653  4862 ts_tablet_manager.cc:1434] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
W20260709 12:29:27.714965  4785 leader_election.cc:343] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.715736  4862 tablet_bootstrap.cc:492] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
W20260709 12:29:27.715700  4785 leader_election.cc:343] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.716343  4785 leader_election.cc:304] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588; no voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:27.717393  4889 raft_consensus.cc:2749] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.721493  4858 tablet_bootstrap.cc:654] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.724136  4860 tablet_bootstrap.cc:654] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.725031  4884 raft_consensus.cc:493] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.725685  4884 raft_consensus.cc:515] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.728256  4884 leader_election.cc:290] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.728597  4862 tablet_bootstrap.cc:654] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.728996  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
W20260709 12:29:27.731009  4636 leader_election.cc:343] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.737687  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.739359  4860 tablet_bootstrap.cc:492] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:27.739928  4860 ts_tablet_manager.cc:1403] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.011s
I20260709 12:29:27.740121  4858 tablet_bootstrap.cc:492] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.740635  4858 ts_tablet_manager.cc:1403] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.007s
I20260709 12:29:27.742664  4860 raft_consensus.cc:359] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.743383  4860 raft_consensus.cc:385] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.743645  4860 raft_consensus.cc:740] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.744313  4860 consensus_queue.cc:260] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.744421  4566 catalog_manager.cc:5697] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.746405  4860 ts_tablet_manager.cc:1434] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:27.747503  4889 raft_consensus.cc:493] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:27.747814  4636 leader_election.cc:343] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.748041  4889 raft_consensus.cc:515] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.748311  4636 leader_election.cc:304] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f; no voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:27.749446  4884 raft_consensus.cc:2749] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.751410  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.751837  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.752174  4669 raft_consensus.cc:2468] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:27.752557  4745 raft_consensus.cc:2468] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:27.753784  4785 leader_election.cc:304] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:27.754063  4889 leader_election.cc:290] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.744341  4858 raft_consensus.cc:359] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.754761  4890 raft_consensus.cc:2804] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.754791  4858 raft_consensus.cc:385] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.755393  4858 raft_consensus.cc:740] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.755339  4890 raft_consensus.cc:493] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.755959  4890 raft_consensus.cc:3060] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.756146  4862 tablet_bootstrap.cc:492] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.756736  4862 ts_tablet_manager.cc:1403] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.019s
I20260709 12:29:27.756266  4858 consensus_queue.cc:260] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.758685  4858 ts_tablet_manager.cc:1434] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260709 12:29:27.759229  4884 raft_consensus.cc:493] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.759706  4858 tablet_bootstrap.cc:492] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.759737  4884 raft_consensus.cc:515] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.762495  4884 leader_election.cc:290] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.760221  4862 raft_consensus.cc:359] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.763988  4862 raft_consensus.cc:385] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.764343  4862 raft_consensus.cc:740] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.763988  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae78f7d9184346839a507b7bd4e2659a" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.765136  4862 consensus_queue.cc:260] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.765970  4745 raft_consensus.cc:2468] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:27.766448  4858 tablet_bootstrap.cc:654] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.767300  4636 leader_election.cc:304] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:27.767613  4862 ts_tablet_manager.cc:1434] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:29:27.768399  4884 raft_consensus.cc:2804] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.768748  4884 raft_consensus.cc:493] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.769085  4884 raft_consensus.cc:3060] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.771075  4862 tablet_bootstrap.cc:492] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.765221  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae78f7d9184346839a507b7bd4e2659a" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.774230  4819 raft_consensus.cc:2468] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:27.780634  4862 tablet_bootstrap.cc:654] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.789357  4884 raft_consensus.cc:515] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.791720  4890 raft_consensus.cc:515] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.794967  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:27.795699  4669 raft_consensus.cc:3060] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.796751  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae78f7d9184346839a507b7bd4e2659a" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:27.797379  4745 raft_consensus.cc:3060] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.803560  4669 raft_consensus.cc:2468] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:27.804643  4745 raft_consensus.cc:2468] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:27.804899  4785 leader_election.cc:304] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:27.805819  4636 leader_election.cc:304] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:27.805788  4889 raft_consensus.cc:2804] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:27.806654  4888 raft_consensus.cc:2804] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:27.807123  4889 raft_consensus.cc:697] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:27.807152  4888 raft_consensus.cc:697] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:27.809847  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae78f7d9184346839a507b7bd4e2659a" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:27.809695  4888 consensus_queue.cc:237] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.810420  4819 raft_consensus.cc:3060] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.808032  4889 consensus_queue.cc:237] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.818100  4819 raft_consensus.cc:2468] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:27.819890  4884 leader_election.cc:290] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.824446  4890 leader_election.cc:290] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.828645  4568 catalog_manager.cc:5697] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:27.832501  4862 tablet_bootstrap.cc:492] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.833101  4862 ts_tablet_manager.cc:1403] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.062s	user 0.028s	sys 0.032s
I20260709 12:29:27.833599  4858 tablet_bootstrap.cc:492] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.834156  4858 ts_tablet_manager.cc:1403] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.075s	user 0.019s	sys 0.052s
I20260709 12:29:27.834079  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:27.834802  4745 raft_consensus.cc:3060] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.836023  4862 raft_consensus.cc:359] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.836773  4862 raft_consensus.cc:385] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.837114  4862 raft_consensus.cc:740] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.837258  4858 raft_consensus.cc:359] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.836869  4567 catalog_manager.cc:5697] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 1 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.838047  4858 raft_consensus.cc:385] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.838515  4858 raft_consensus.cc:740] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.838063  4862 consensus_queue.cc:260] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.839579  4858 consensus_queue.cc:260] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.840884  4862 ts_tablet_manager.cc:1434] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:27.841951  4862 tablet_bootstrap.cc:492] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.842418  4745 raft_consensus.cc:2468] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:27.845077  4858 ts_tablet_manager.cc:1434] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:29:27.846114  4858 tablet_bootstrap.cc:492] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.850517  4862 tablet_bootstrap.cc:654] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.852494  4858 tablet_bootstrap.cc:654] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.860719  4888 raft_consensus.cc:493] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.861274  4888 raft_consensus.cc:515] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.862179  4862 tablet_bootstrap.cc:492] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.862697  4862 ts_tablet_manager.cc:1403] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 12:29:27.863497  4888 leader_election.cc:290] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.864122  4858 tablet_bootstrap.cc:492] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.864605  4858 ts_tablet_manager.cc:1403] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:29:27.865505  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.865911  4891 raft_consensus.cc:493] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.865541  4862 raft_consensus.cc:359] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.866163  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.866420  4862 raft_consensus.cc:385] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.866420  4891 raft_consensus.cc:515] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.866930  4862 raft_consensus.cc:740] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.867049  4819 raft_consensus.cc:2393] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ab80e12f0ef4df38684a32e134d1e7f in current term 1: Already voted for candidate 6d0e2be318224ee5a8f54ea891b0856a in this term.
I20260709 12:29:27.867777  4862 consensus_queue.cc:260] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.868132  4636 leader_election.cc:304] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f; no voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:27.868854  4891 leader_election.cc:290] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.868976  4888 raft_consensus.cc:3060] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.869885  4862 ts_tablet_manager.cc:1434] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:27.870420  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748fe2ac99874ea9b112159a53cc3beb" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.870851  4862 tablet_bootstrap.cc:492] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:27.870664  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748fe2ac99874ea9b112159a53cc3beb" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.871029  4669 raft_consensus.cc:2468] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:27.871346  4819 raft_consensus.cc:2468] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:27.872606  4858 raft_consensus.cc:359] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.873270  4858 raft_consensus.cc:385] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.873525  4858 raft_consensus.cc:740] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.874202  4858 consensus_queue.cc:260] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.875677  4711 leader_election.cc:304] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:27.876048  4858 ts_tablet_manager.cc:1434] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:29:27.876528  4891 raft_consensus.cc:2804] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.876976  4891 raft_consensus.cc:493] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.877007  4858 tablet_bootstrap.cc:492] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:27.877548  4891 raft_consensus.cc:3060] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.883493  4862 tablet_bootstrap.cc:654] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.884006  4858 tablet_bootstrap.cc:654] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.886487  4891 raft_consensus.cc:515] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.889943  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748fe2ac99874ea9b112159a53cc3beb" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:27.890532  4669 raft_consensus.cc:3060] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.892006  4888 raft_consensus.cc:2749] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.899372  4669 raft_consensus.cc:2468] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:27.900753  4891 leader_election.cc:290] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.900751  4711 leader_election.cc:304] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:27.901262  4888 raft_consensus.cc:493] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.901137  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748fe2ac99874ea9b112159a53cc3beb" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:27.902184  4888 raft_consensus.cc:515] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.902359  4819 raft_consensus.cc:3060] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.902395  4896 raft_consensus.cc:2804] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:27.902359  4862 tablet_bootstrap.cc:492] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:27.903287  4896 raft_consensus.cc:697] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:27.903447  4862 ts_tablet_manager.cc:1403] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.009s
I20260709 12:29:27.904188  4896 consensus_queue.cc:237] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.905288  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.907040  4862 raft_consensus.cc:359] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.907780  4862 raft_consensus.cc:385] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:29:27.907785  4636 leader_election.cc:343] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.908135  4862 raft_consensus.cc:740] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.908278  4888 leader_election.cc:290] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.908901  4862 consensus_queue.cc:260] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.909873  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:27.910743  4819 raft_consensus.cc:2468] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:27.910709  4818 raft_consensus.cc:2468] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:27.913224  4862 ts_tablet_manager.cc:1434] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:29:27.913887  4858 tablet_bootstrap.cc:492] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:27.914258  4858 ts_tablet_manager.cc:1403] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.005s
I20260709 12:29:27.916347  4858 raft_consensus.cc:359] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.916863  4858 raft_consensus.cc:385] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.917019  4858 raft_consensus.cc:740] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.917516  4858 consensus_queue.cc:260] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.918962  4858 ts_tablet_manager.cc:1434] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:29:27.913277  4636 leader_election.cc:304] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:27.919972  4888 raft_consensus.cc:2804] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.920334  4888 raft_consensus.cc:493] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.920729  4888 raft_consensus.cc:3060] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.922576  4890 raft_consensus.cc:493] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.923130  4890 raft_consensus.cc:515] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.925261  4890 leader_election.cc:290] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.925370  4567 catalog_manager.cc:5697] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.927587  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:27.928289  4669 raft_consensus.cc:2468] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:27.928226  4888 raft_consensus.cc:515] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.929702  4785 leader_election.cc:304] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:27.930361  4888 leader_election.cc:290] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:27.927928  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:27.931056  4745 raft_consensus.cc:2468] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:27.932613  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:27.933073  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:27.933542  4745 raft_consensus.cc:3060] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.933708  4819 raft_consensus.cc:3060] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.933902  4890 raft_consensus.cc:2804] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.934360  4890 raft_consensus.cc:493] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.934708  4890 raft_consensus.cc:3060] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.938870  4819 raft_consensus.cc:2468] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:27.940106  4636 leader_election.cc:304] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:27.941094  4888 raft_consensus.cc:2804] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:27.941596  4888 raft_consensus.cc:697] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:27.941349  4745 raft_consensus.cc:2468] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:27.942452  4890 raft_consensus.cc:515] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.942517  4888 consensus_queue.cc:237] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.945235  4890 leader_election.cc:290] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:27.946305  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:27.946965  4669 raft_consensus.cc:3060] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.946946  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:27.948145  4745 raft_consensus.cc:3060] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.953051  4669 raft_consensus.cc:2468] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:27.954398  4785 leader_election.cc:304] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:27.955446  4890 raft_consensus.cc:2804] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:27.956727  4890 raft_consensus.cc:697] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:27.958038  4745 raft_consensus.cc:2468] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:27.957764  4890 consensus_queue.cc:237] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:27.960294  4567 catalog_manager.cc:5697] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 1 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.967026  4566 catalog_manager.cc:5697] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:28.010012  4888 consensus_queue.cc:1048] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.025943  4887 consensus_queue.cc:1048] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.039788  4887 raft_consensus.cc:493] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.040364  4887 raft_consensus.cc:515] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.042909  4887 leader_election.cc:290] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:28.043622  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:28.044067  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197e1cac162e444ebbb81bd7f563902b" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:28.044539  4745 raft_consensus.cc:2393] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ab80e12f0ef4df38684a32e134d1e7f in current term 1: Already voted for candidate 5cbdbb7559fb49919a700494dc154588 in this term.
I20260709 12:29:28.045876  4636 leader_election.cc:304] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f; no voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:28.046541  4887 raft_consensus.cc:3060] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.052314  4887 raft_consensus.cc:2749] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.118438  4896 consensus_queue.cc:1048] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.128955  4863 consensus_queue.cc:1048] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.149740  4890 consensus_queue.cc:1048] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.155215  4890 raft_consensus.cc:493] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.155745  4890 raft_consensus.cc:515] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.157881  4890 leader_election.cc:290] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869)
I20260709 12:29:28.159705  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:28.160455  4745 raft_consensus.cc:2468] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:28.162274  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:28.162873  4669 raft_consensus.cc:2468] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:28.163682  4785 leader_election.cc:304] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:28.164649  4890 raft_consensus.cc:2804] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.165020  4890 raft_consensus.cc:493] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.165362  4890 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.165655  4889 consensus_queue.cc:1048] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.174284  4890 raft_consensus.cc:515] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.176604  4890 leader_election.cc:290] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869)
I20260709 12:29:28.177465  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:28.177652  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:28.178113  4745 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.178429  4669 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.184422  4745 raft_consensus.cc:2468] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:28.185623  4669 raft_consensus.cc:2468] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:28.185961  4785 leader_election.cc:304] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:28.186882  4890 raft_consensus.cc:2804] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.187376  4890 raft_consensus.cc:697] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:28.188009  4890 consensus_queue.cc:237] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.194862  4567 catalog_manager.cc:5697] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:28.243806  4890 consensus_queue.cc:1048] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.256140  4864 consensus_queue.cc:1048] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.288656  4819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "933310e093f24d6c87a460c3471eca97"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
mode: GRACEFUL
new_leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:55178
I20260709 12:29:28.289255  4819 raft_consensus.cc:606] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Received request to transfer leadership to TS 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:28.294469  4819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
mode: GRACEFUL
new_leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:55178
I20260709 12:29:28.295045  4819 raft_consensus.cc:606] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Received request to transfer leadership to TS 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:28.310778  4745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
mode: GRACEFUL
new_leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:60074
I20260709 12:29:28.311353  4745 raft_consensus.cc:606] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Received request to transfer leadership to TS 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:28.324916  4669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
mode: GRACEFUL
new_leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:35650
I20260709 12:29:28.325479  4669 raft_consensus.cc:606] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Received request to transfer leadership to TS 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:28.330793  4669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
mode: GRACEFUL
new_leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:35650
I20260709 12:29:28.331447  4669 raft_consensus.cc:606] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Received request to transfer leadership to TS 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:28.334544  4863 consensus_queue.cc:1048] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.334681  4887 consensus_queue.cc:1048] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.342876  4819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
mode: GRACEFUL
new_leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:55178
I20260709 12:29:28.343552  4819 raft_consensus.cc:606] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Received request to transfer leadership to TS 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:28.348991  4863 consensus_queue.cc:1048] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.362648  4864 consensus_queue.cc:1048] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.365337  4888 consensus_queue.cc:1048] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.380272  4889 raft_consensus.cc:993] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588: : Instructing follower 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:28.380772  4889 raft_consensus.cc:1081] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Signalling peer 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:28.385149  4744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:60070
I20260709 12:29:28.386535  4744 raft_consensus.cc:493] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.386443  4890 consensus_queue.cc:1048] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.387230  4744 raft_consensus.cc:3060] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.394794  4744 raft_consensus.cc:515] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.397714  4670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:28.398337  4670 raft_consensus.cc:3060] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.399484  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfff7a3d59a47de8bc9f6929041196a" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:28.400213  4819 raft_consensus.cc:3055] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:28.400552  4819 raft_consensus.cc:740] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:28.401742  4744 leader_election.cc:290] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:28.401587  4819 consensus_queue.cc:260] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.402859  4819 raft_consensus.cc:3060] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.406236  4670 raft_consensus.cc:2468] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:28.407335  4711 leader_election.cc:304] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:28.408206  4863 raft_consensus.cc:2804] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:28.408641  4863 raft_consensus.cc:697] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:28.409464  4863 consensus_queue.cc:237] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.410079  4819 raft_consensus.cc:2468] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:28.416378  4568 catalog_manager.cc:5697] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 1 to 2, leader changed from 5cbdbb7559fb49919a700494dc154588 (127.19.36.195) to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.421198  4888 consensus_queue.cc:1048] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.431936  4887 consensus_queue.cc:1048] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.440817  4884 raft_consensus.cc:993] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f: : Instructing follower 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:28.441272  4887 raft_consensus.cc:1081] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Signalling peer 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:28.442984  4744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:60058
I20260709 12:29:28.443570  4744 raft_consensus.cc:493] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.443934  4744 raft_consensus.cc:3060] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.449290  4744 raft_consensus.cc:515] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.451035  4744 leader_election.cc:290] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:28.452169  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:28.452236  4670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70f0d3ef958849f08eaf18856e6567e2" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:28.452809  4819 raft_consensus.cc:3060] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.452996  4670 raft_consensus.cc:3055] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:28.453420  4670 raft_consensus.cc:740] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:28.454181  4670 consensus_queue.cc:260] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.455267  4670 raft_consensus.cc:3060] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.459074  4819 raft_consensus.cc:2468] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:28.460331  4711 leader_election.cc:304] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:28.460753  4670 raft_consensus.cc:2468] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:28.461248  4863 raft_consensus.cc:2804] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:28.461762  4863 raft_consensus.cc:697] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:28.462495  4863 consensus_queue.cc:237] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.468863  4566 catalog_manager.cc:5697] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 1 to 2, leader changed from 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193) to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.645041  4884 raft_consensus.cc:993] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f: : Instructing follower 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:28.645599  4887 raft_consensus.cc:1081] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Signalling peer 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:28.647326  4744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:60058
I20260709 12:29:28.647897  4744 raft_consensus.cc:493] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.648257  4744 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.653810  4744 raft_consensus.cc:515] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.655714  4744 leader_election.cc:290] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:28.656798  4670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:28.657485  4670 raft_consensus.cc:3055] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:28.656862  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:28.657790  4670 raft_consensus.cc:740] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:28.658665  4819 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.659154  4670 consensus_queue.cc:260] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.660437  4670 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.661903  4890 consensus_queue.cc:1048] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.666487  4819 raft_consensus.cc:2468] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:28.667179  4670 raft_consensus.cc:2468] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:28.667951  4711 leader_election.cc:304] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:28.668902  4863 raft_consensus.cc:2804] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:28.669454  4863 raft_consensus.cc:697] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:28.670365  4863 consensus_queue.cc:237] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.671010  4925 raft_consensus.cc:993] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588: : Instructing follower 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:28.671567  4925 raft_consensus.cc:1081] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Signalling peer 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:28.676839  4668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "933310e093f24d6c87a460c3471eca97"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:35644
I20260709 12:29:28.677783  4668 raft_consensus.cc:493] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.678226  4668 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.681726  4863 raft_consensus.cc:993] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a: : Instructing follower 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:28.682271  4891 raft_consensus.cc:1081] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Signalling peer 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:28.682149  4568 catalog_manager.cc:5697] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 1 to 2, leader changed from 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193) to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.684207  4669 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:35646
I20260709 12:29:28.684724  4669 raft_consensus.cc:493] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.685101  4669 raft_consensus.cc:3060] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.690359  4668 raft_consensus.cc:515] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.691076  4925 consensus_queue.cc:1048] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.693099  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:28.694362  4669 raft_consensus.cc:515] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.693701  4668 leader_election.cc:290] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:28.693912  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:28.696576  4669 leader_election.cc:290] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:28.697990  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:28.698652  4818 raft_consensus.cc:3060] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.705024  4818 raft_consensus.cc:2468] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:28.705469  4819 raft_consensus.cc:3055] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:28.705948  4819 raft_consensus.cc:740] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:28.706041  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7adec6a263b40ae8f500e6d4de59996" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:28.706125  4636 leader_election.cc:304] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:28.706914  4744 raft_consensus.cc:3055] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:28.707388  4744 raft_consensus.cc:740] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:28.706825  4819 consensus_queue.cc:260] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.707554  4887 raft_consensus.cc:2804] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:28.708153  4887 raft_consensus.cc:697] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:28.708393  4819 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.708220  4744 consensus_queue.cc:260] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.709028  4887 consensus_queue.cc:237] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.709631  4744 raft_consensus.cc:3060] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.716173  4819 raft_consensus.cc:2468] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:28.716436  4744 raft_consensus.cc:2468] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:28.717365  4636 leader_election.cc:304] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:28.718078  4887 raft_consensus.cc:2804] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:28.718559  4887 raft_consensus.cc:697] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:28.718603  4568 catalog_manager.cc:5697] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 1 to 2, leader changed from 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194) to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 2 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.719390  4887 consensus_queue.cc:237] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.730778  4568 catalog_manager.cc:5697] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 1 to 2, leader changed from 5cbdbb7559fb49919a700494dc154588 (127.19.36.195) to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 2 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.735394  4889 raft_consensus.cc:993] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588: : Instructing follower 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:28.735960  4889 raft_consensus.cc:1081] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Signalling peer 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:28.738174  4668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:35644
I20260709 12:29:28.738734  4668 raft_consensus.cc:493] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.739094  4668 raft_consensus.cc:3060] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.744714  4668 raft_consensus.cc:515] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.746423  4668 leader_election.cc:290] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:28.747293  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:28.747639  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3068fedf904cd3b43cbcd34a3e61aa" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:28.747928  4744 raft_consensus.cc:3060] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.748263  4819 raft_consensus.cc:3055] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:28.748559  4819 raft_consensus.cc:740] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:28.749188  4819 consensus_queue.cc:260] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.750182  4819 raft_consensus.cc:3060] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:28.753239  4744 raft_consensus.cc:2468] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:28.754345  4636 leader_election.cc:304] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:28.755280  4887 raft_consensus.cc:2804] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:28.755901  4819 raft_consensus.cc:2468] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:28.755940  4887 raft_consensus.cc:697] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:28.757681  4887 consensus_queue.cc:237] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:28.764535  4568 catalog_manager.cc:5697] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 1 to 2, leader changed from 5cbdbb7559fb49919a700494dc154588 (127.19.36.195) to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 2 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.902961  4668 raft_consensus.cc:1275] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.904234  4896 consensus_queue.cc:1048] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.912180  4819 raft_consensus.cc:1275] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.914327  4891 consensus_queue.cc:1048] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.942024  4819 raft_consensus.cc:1275] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.943327  4891 consensus_queue.cc:1048] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.951162  4668 raft_consensus.cc:1275] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.952675  4891 consensus_queue.cc:1048] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.178455  4744 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.186039  4668 raft_consensus.cc:1275] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.186826  4744 raft_consensus.cc:1275] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.187737  4863 consensus_queue.cc:1048] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.188441  4887 consensus_queue.cc:1048] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.190828  4744 raft_consensus.cc:1275] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.193921  4888 consensus_queue.cc:1048] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:29.194372  4819 raft_consensus.cc:1275] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.196004  4888 consensus_queue.cc:1048] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.212081  4818 raft_consensus.cc:1275] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.214444  4887 consensus_queue.cc:1048] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.216437  4817 raft_consensus.cc:1275] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.219605  4863 consensus_queue.cc:1048] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.232173  4818 raft_consensus.cc:1275] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.234089  4948 consensus_queue.cc:1048] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:29.251075  4744 raft_consensus.cc:1275] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:29.253091  4947 consensus_queue.cc:1048] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:31.379776  4566 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:31.382565  4566 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:31.435927  4652 tablet_service.cc:1511] Processing CreateTablet for tablet 51c65145402b4d15b96a7b47fba3d447 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:31.437528  4730 tablet_service.cc:1511] Processing CreateTablet for tablet 51c65145402b4d15b96a7b47fba3d447 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:31.438048  4652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c65145402b4d15b96a7b47fba3d447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.439242  4730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c65145402b4d15b96a7b47fba3d447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.443218  4728 tablet_service.cc:1511] Processing CreateTablet for tablet 2e5002fd95af4b95997c13feb68bbcba (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:31.443966  4654 tablet_service.cc:1511] Processing CreateTablet for tablet 2e5002fd95af4b95997c13feb68bbcba (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:31.444687  4728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e5002fd95af4b95997c13feb68bbcba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.445546  4654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e5002fd95af4b95997c13feb68bbcba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.449227  4809 tablet_service.cc:1511] Processing CreateTablet for tablet 51c65145402b4d15b96a7b47fba3d447 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:31.449470  4802 tablet_service.cc:1511] Processing CreateTablet for tablet 2e5002fd95af4b95997c13feb68bbcba (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:31.450977  4809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51c65145402b4d15b96a7b47fba3d447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.454947  4735 tablet_service.cc:1511] Processing CreateTablet for tablet 2b3afd1c223f42b1bda891c9dcfa5150 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:31.456631  4735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b3afd1c223f42b1bda891c9dcfa5150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.456336  4655 tablet_service.cc:1511] Processing CreateTablet for tablet 2b3afd1c223f42b1bda891c9dcfa5150 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:31.457759  4655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b3afd1c223f42b1bda891c9dcfa5150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.460474  4657 tablet_service.cc:1511] Processing CreateTablet for tablet f66dc3001f5149f683e43c2db87d9e0f (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:31.462019  4657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66dc3001f5149f683e43c2db87d9e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.460541  4729 tablet_service.cc:1511] Processing CreateTablet for tablet f66dc3001f5149f683e43c2db87d9e0f (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:31.467202  4729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66dc3001f5149f683e43c2db87d9e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.468851  4969 tablet_bootstrap.cc:492] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:31.473907  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e5002fd95af4b95997c13feb68bbcba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.477377  4969 tablet_bootstrap.cc:654] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.480650  4804 tablet_service.cc:1511] Processing CreateTablet for tablet 2b3afd1c223f42b1bda891c9dcfa5150 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:31.482144  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b3afd1c223f42b1bda891c9dcfa5150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.521843  4971 tablet_bootstrap.cc:492] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:31.531067  4969 tablet_bootstrap.cc:492] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:31.531872  4969 ts_tablet_manager.cc:1403] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.063s	user 0.020s	sys 0.023s
I20260709 12:29:31.533599  4805 tablet_service.cc:1511] Processing CreateTablet for tablet f66dc3001f5149f683e43c2db87d9e0f (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:31.535049  4805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66dc3001f5149f683e43c2db87d9e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.535566  4652 tablet_service.cc:1511] Processing CreateTablet for tablet b3553184055c4292ac3acc49631d9152 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:31.539134  4652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3553184055c4292ac3acc49631d9152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.538472  4969 raft_consensus.cc:359] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.539682  4969 raft_consensus.cc:385] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.540057  4969 raft_consensus.cc:740] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.540874  4969 consensus_queue.cc:260] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.543732  4969 ts_tablet_manager.cc:1434] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:29:31.554983  4730 tablet_service.cc:1511] Processing CreateTablet for tablet b3553184055c4292ac3acc49631d9152 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:31.556375  4730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3553184055c4292ac3acc49631d9152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.566757  4808 tablet_service.cc:1511] Processing CreateTablet for tablet b3553184055c4292ac3acc49631d9152 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:31.568279  4808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3553184055c4292ac3acc49631d9152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.574510  4972 tablet_bootstrap.cc:492] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:31.578653  4969 tablet_bootstrap.cc:492] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:31.578573  4658 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb30e39bc65458cbc6f1f4e87a9e6ab (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:31.580291  4658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb30e39bc65458cbc6f1f4e87a9e6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.586402  4969 tablet_bootstrap.cc:654] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.598652  4728 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb30e39bc65458cbc6f1f4e87a9e6ab (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:31.600196  4728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb30e39bc65458cbc6f1f4e87a9e6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.601778  4971 tablet_bootstrap.cc:654] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.611444  4972 tablet_bootstrap.cc:654] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.623914  4809 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb30e39bc65458cbc6f1f4e87a9e6ab (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:31.625324  4809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb30e39bc65458cbc6f1f4e87a9e6ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:31.629336  4616 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:31.638775  4657 tablet_service.cc:1511] Processing CreateTablet for tablet f36103d928c44b22bf506cce041a25a8 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:31.640228  4657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f36103d928c44b22bf506cce041a25a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.651412  4969 tablet_bootstrap.cc:492] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:31.651306  4730 tablet_service.cc:1511] Processing CreateTablet for tablet f36103d928c44b22bf506cce041a25a8 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:31.652164  4969 ts_tablet_manager.cc:1403] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.002s
I20260709 12:29:31.652858  4730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f36103d928c44b22bf506cce041a25a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.655380  4969 raft_consensus.cc:359] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.656265  4969 raft_consensus.cc:385] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.656600  4969 raft_consensus.cc:740] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.657544  4969 consensus_queue.cc:260] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.655380  4808 tablet_service.cc:1511] Processing CreateTablet for tablet f36103d928c44b22bf506cce041a25a8 (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:31.666069  4808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f36103d928c44b22bf506cce041a25a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.668499  4802 tablet_service.cc:1511] Processing CreateTablet for tablet 99be8cd85c6943998dcb402ec0bfc4de (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:31.669976  4802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99be8cd85c6943998dcb402ec0bfc4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.672232  4972 tablet_bootstrap.cc:492] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:31.672835  4972 ts_tablet_manager.cc:1403] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.099s	user 0.014s	sys 0.000s
I20260709 12:29:31.672681  4728 tablet_service.cc:1511] Processing CreateTablet for tablet 99be8cd85c6943998dcb402ec0bfc4de (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:31.674002  4728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99be8cd85c6943998dcb402ec0bfc4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.676070  4972 raft_consensus.cc:359] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.676945  4972 raft_consensus.cc:385] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.677340  4972 raft_consensus.cc:740] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.678188  4972 consensus_queue.cc:260] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.678368  4804 tablet_service.cc:1511] Processing CreateTablet for tablet f2200a06b9094b1e84a7fef9defaef4c (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:31.680884  4657 tablet_service.cc:1511] Processing CreateTablet for tablet f2200a06b9094b1e84a7fef9defaef4c (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:31.681329  4729 tablet_service.cc:1511] Processing CreateTablet for tablet f2200a06b9094b1e84a7fef9defaef4c (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:31.682229  4657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2200a06b9094b1e84a7fef9defaef4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.682643  4729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2200a06b9094b1e84a7fef9defaef4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.682539  4652 tablet_service.cc:1511] Processing CreateTablet for tablet 99be8cd85c6943998dcb402ec0bfc4de (DEFAULT_TABLE table=soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:31.683925  4652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99be8cd85c6943998dcb402ec0bfc4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.689143  4804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2200a06b9094b1e84a7fef9defaef4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.697141  4971 tablet_bootstrap.cc:492] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:31.697702  4971 ts_tablet_manager.cc:1403] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.176s	user 0.001s	sys 0.014s
I20260709 12:29:31.699090  4958 raft_consensus.cc:493] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.695340  4972 ts_tablet_manager.cc:1434] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:29:31.699695  4958 raft_consensus.cc:515] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.702046  4958 leader_election.cc:290] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:31.703115  4972 tablet_bootstrap.cc:492] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:31.710891  4969 ts_tablet_manager.cc:1434] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 12:29:31.714002  4972 tablet_bootstrap.cc:654] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.724063  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
W20260709 12:29:31.733855  4636 leader_election.cc:343] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:31.724030  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
W20260709 12:29:31.736630  4636 leader_election.cc:343] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:31.737105  4636 leader_election.cc:304] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f; no voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:31.719632  4969 tablet_bootstrap.cc:492] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:31.720073  4971 raft_consensus.cc:359] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.738361  4971 raft_consensus.cc:385] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.738484  4976 raft_consensus.cc:2749] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:31.738724  4971 raft_consensus.cc:740] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.739682  4971 consensus_queue.cc:260] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.742429  4971 ts_tablet_manager.cc:1434] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:29:31.743490  4971 tablet_bootstrap.cc:492] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:31.754719  4971 tablet_bootstrap.cc:654] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.758266  4969 tablet_bootstrap.cc:654] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.766439  4969 tablet_bootstrap.cc:492] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:31.766978  4969 ts_tablet_manager.cc:1403] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.003s
I20260709 12:29:31.767089  4972 tablet_bootstrap.cc:492] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:31.767534  4971 tablet_bootstrap.cc:492] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:31.767647  4972 ts_tablet_manager.cc:1403] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.010s
I20260709 12:29:31.768042  4971 ts_tablet_manager.cc:1403] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 12:29:31.770424  4971 raft_consensus.cc:359] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.770380  4969 raft_consensus.cc:359] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.771370  4971 raft_consensus.cc:385] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.770947  4972 raft_consensus.cc:359] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.771420  4969 raft_consensus.cc:385] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.771742  4971 raft_consensus.cc:740] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.771845  4972 raft_consensus.cc:385] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.771891  4969 raft_consensus.cc:740] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.772388  4972 raft_consensus.cc:740] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.772614  4971 consensus_queue.cc:260] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.772907  4969 consensus_queue.cc:260] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.773401  4972 consensus_queue.cc:260] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.774814  4971 ts_tablet_manager.cc:1434] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:31.776247  4971 tablet_bootstrap.cc:492] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:31.777226  4969 ts_tablet_manager.cc:1434] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:29:31.778322  4969 tablet_bootstrap.cc:492] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:31.783500  4971 tablet_bootstrap.cc:654] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.785274  4969 tablet_bootstrap.cc:654] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.795385  4972 ts_tablet_manager.cc:1434] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 12:29:31.803757  4971 tablet_bootstrap.cc:492] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:31.804562  4971 ts_tablet_manager.cc:1403] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.017s
I20260709 12:29:31.805212  4972 tablet_bootstrap.cc:492] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:31.808645  4971 raft_consensus.cc:359] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.809624  4971 raft_consensus.cc:385] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.810009  4971 raft_consensus.cc:740] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.810947  4971 consensus_queue.cc:260] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.811813  4972 tablet_bootstrap.cc:654] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.816656  4971 ts_tablet_manager.cc:1434] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:29:31.817649  4971 tablet_bootstrap.cc:492] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:31.820272  4969 tablet_bootstrap.cc:492] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:31.820837  4969 ts_tablet_manager.cc:1403] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.017s
I20260709 12:29:31.823374  4969 raft_consensus.cc:359] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.824049  4971 tablet_bootstrap.cc:654] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.824127  4969 raft_consensus.cc:385] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.825995  4969 raft_consensus.cc:740] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.826819  4972 tablet_bootstrap.cc:492] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:31.826795  4969 consensus_queue.cc:260] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.827458  4972 ts_tablet_manager.cc:1403] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.002s
I20260709 12:29:31.829398  4969 ts_tablet_manager.cc:1434] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:31.830452  4969 tablet_bootstrap.cc:492] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:31.831208  4971 tablet_bootstrap.cc:492] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:31.831717  4971 ts_tablet_manager.cc:1403] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.014s	user 0.000s	sys 0.012s
I20260709 12:29:31.831684  4972 raft_consensus.cc:359] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.832540  4972 raft_consensus.cc:385] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.832845  4972 raft_consensus.cc:740] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.833647  4972 consensus_queue.cc:260] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.836190  4972 ts_tablet_manager.cc:1434] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:29:31.837203  4972 tablet_bootstrap.cc:492] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:31.837436  4971 raft_consensus.cc:359] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.837901  4969 tablet_bootstrap.cc:654] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.838230  4971 raft_consensus.cc:385] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.838579  4971 raft_consensus.cc:740] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.841032  4971 consensus_queue.cc:260] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.845860  4972 tablet_bootstrap.cc:654] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.845969  4971 ts_tablet_manager.cc:1434] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.006s
I20260709 12:29:31.847288  4971 tablet_bootstrap.cc:492] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:31.853113  4969 tablet_bootstrap.cc:492] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:31.857209  4971 tablet_bootstrap.cc:654] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.858144  4976 raft_consensus.cc:493] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.858839  4976 raft_consensus.cc:515] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.861665  4976 leader_election.cc:290] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:31.864005  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3afd1c223f42b1bda891c9dcfa5150" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:31.864928  4745 raft_consensus.cc:2468] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:31.865064  4925 raft_consensus.cc:493] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.865782  4925 raft_consensus.cc:515] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.867264  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3afd1c223f42b1bda891c9dcfa5150" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:31.868444  4925 leader_election.cc:290] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
W20260709 12:29:31.869132  4636 leader_election.cc:343] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:31.869915  4971 tablet_bootstrap.cc:492] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:31.870143  4636 leader_election.cc:304] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 5cbdbb7559fb49919a700494dc154588
I20260709 12:29:31.870963  4969 ts_tablet_manager.cc:1403] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.001s
I20260709 12:29:31.870471  4971 ts_tablet_manager.cc:1403] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.000s
I20260709 12:29:31.870515  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:31.873660  4976 raft_consensus.cc:2804] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:31.874171  4976 raft_consensus.cc:493] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.874518  4976 raft_consensus.cc:3060] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.873273  4668 raft_consensus.cc:2468] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:31.875763  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:31.880234  4745 raft_consensus.cc:2468] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:31.883857  4785 leader_election.cc:304] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:31.884537  4972 tablet_bootstrap.cc:492] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:31.885099  4972 ts_tablet_manager.cc:1403] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.048s	user 0.023s	sys 0.012s
I20260709 12:29:31.886250  4925 raft_consensus.cc:2804] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:31.887010  4925 raft_consensus.cc:493] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.887488  4925 raft_consensus.cc:3060] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.887897  4969 raft_consensus.cc:359] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.888667  4969 raft_consensus.cc:385] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.888967  4969 raft_consensus.cc:740] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.889525  4957 raft_consensus.cc:493] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.889813  4969 consensus_queue.cc:260] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.890056  4957 raft_consensus.cc:515] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.890859  4976 raft_consensus.cc:515] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.892544  4957 leader_election.cc:290] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:31.894439  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:31.895330  4668 raft_consensus.cc:2468] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:31.892855  4976 leader_election.cc:290] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:31.896512  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3afd1c223f42b1bda891c9dcfa5150" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:31.897275  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:31.897420  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b3afd1c223f42b1bda891c9dcfa5150" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:31.897950  4745 raft_consensus.cc:3060] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.897810  4971 raft_consensus.cc:359] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.898713  4971 raft_consensus.cc:385] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.899073  4971 raft_consensus.cc:740] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.899976  4971 consensus_queue.cc:260] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.900157  4925 raft_consensus.cc:515] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.903652  4925 leader_election.cc:290] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:31.906116  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:31.906339  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:31.906898  4668 raft_consensus.cc:3060] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.907040  4744 raft_consensus.cc:3060] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.907462  4711 leader_election.cc:304] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:31.908373  4957 raft_consensus.cc:2804] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:31.908812  4957 raft_consensus.cc:493] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.909390  4957 raft_consensus.cc:3060] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.911093  4745 raft_consensus.cc:2468] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:31.912634  4636 leader_election.cc:304] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:31.913888  4958 raft_consensus.cc:2804] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:31.914881  4976 raft_consensus.cc:493] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.916299  4976 raft_consensus.cc:515] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.917480  4971 ts_tablet_manager.cc:1434] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.004s
I20260709 12:29:31.918534  4976 leader_election.cc:290] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:31.918998  4971 tablet_bootstrap.cc:492] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:31.919404  4958 raft_consensus.cc:697] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:31.920545  4958 consensus_queue.cc:237] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.923513  4744 raft_consensus.cc:2468] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:31.926609  4972 raft_consensus.cc:359] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.927419  4972 raft_consensus.cc:385] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.927474  4668 raft_consensus.cc:2468] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:31.927770  4972 raft_consensus.cc:740] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.928243  4976 raft_consensus.cc:493] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.928575  4785 leader_election.cc:304] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
W20260709 12:29:31.929924  4711 leader_election.cc:343] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:31.933084  4925 raft_consensus.cc:2804] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:31.928656  4972 consensus_queue.cc:260] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.932014  4976 raft_consensus.cc:515] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.941416  4976 leader_election.cc:290] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:31.935639  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:31.937076  4819 raft_consensus.cc:3060] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.947362  4818 raft_consensus.cc:2468] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:31.948078  4969 ts_tablet_manager.cc:1434] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.002s
I20260709 12:29:31.938951  4971 tablet_bootstrap.cc:654] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.938913  4957 raft_consensus.cc:515] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.942752  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66dc3001f5149f683e43c2db87d9e0f" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:31.934855  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:31.951313  4745 raft_consensus.cc:2468] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:31.951831  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66dc3001f5149f683e43c2db87d9e0f" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:31.952589  4818 raft_consensus.cc:2468] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:31.952994  4636 leader_election.cc:304] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:31.954109  4636 leader_election.cc:304] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:31.949654  4969 tablet_bootstrap.cc:492] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:31.957500  4989 raft_consensus.cc:2804] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:31.959860  4993 raft_consensus.cc:493] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.959977  4987 raft_consensus.cc:493] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.960539  4987 raft_consensus.cc:515] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.962611  4987 leader_election.cc:290] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:31.957994  4989 raft_consensus.cc:493] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.960403  4993 raft_consensus.cc:515] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.956713  4958 raft_consensus.cc:2804] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:31.959055  4925 raft_consensus.cc:697] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:31.966629  4957 leader_election.cc:290] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
W20260709 12:29:31.966738  4636 leader_election.cc:343] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:31.969121  4565 catalog_manager.cc:5697] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 1 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:31.970134  4819 raft_consensus.cc:2468] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:31.964465  4989 raft_consensus.cc:3060] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.973083  4925 consensus_queue.cc:237] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.973560  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:31.970804  4958 raft_consensus.cc:493] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.976464  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:31.977134  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
W20260709 12:29:31.978330  4711 leader_election.cc:343] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:31.982156  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:31.984704  4958 raft_consensus.cc:3060] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.989912  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:31.990873  4668 raft_consensus.cc:3060] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:31.998664  4711 leader_election.cc:343] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:31.983433  4972 ts_tablet_manager.cc:1434] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.095s	user 0.014s	sys 0.001s
I20260709 12:29:31.999104  4989 raft_consensus.cc:515] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:31.989902  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:32.014467  4971 tablet_bootstrap.cc:492] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:32.015100  4971 ts_tablet_manager.cc:1403] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.096s	user 0.018s	sys 0.028s
I20260709 12:29:32.015719  4972 tablet_bootstrap.cc:492] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:32.016206  4969 tablet_bootstrap.cc:654] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.018357  4971 raft_consensus.cc:359] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.019110  4971 raft_consensus.cc:385] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.019526  4971 raft_consensus.cc:740] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
W20260709 12:29:32.020159  4785 leader_election.cc:343] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:32.031401  4669 raft_consensus.cc:2393] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cbdbb7559fb49919a700494dc154588 in current term 1: Already voted for candidate 5ab80e12f0ef4df38684a32e134d1e7f in this term.
I20260709 12:29:31.999593  4711 leader_election.cc:304] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d0e2be318224ee5a8f54ea891b0856a; no voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588
I20260709 12:29:32.033324  4785 leader_election.cc:304] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588; no voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:32.035337  4991 raft_consensus.cc:3060] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.035905  4957 raft_consensus.cc:2749] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:32.021469  4958 raft_consensus.cc:515] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
W20260709 12:29:32.037541  4711 leader_election.cc:343] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:32.040228  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66dc3001f5149f683e43c2db87d9e0f" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:32.040870  4745 raft_consensus.cc:3060] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.022897  4668 raft_consensus.cc:2468] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:32.040452  4972 tablet_bootstrap.cc:654] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.045193  4711 leader_election.cc:304] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 5cbdbb7559fb49919a700494dc154588
I20260709 12:29:32.045998  4957 raft_consensus.cc:2804] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.025943  4989 leader_election.cc:290] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.046468  4957 raft_consensus.cc:697] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:32.030153  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:32.047483  4957 consensus_queue.cc:237] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.048223  4958 leader_election.cc:290] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.030391  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.050081  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66dc3001f5149f683e43c2db87d9e0f" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.050459  4745 raft_consensus.cc:2468] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:32.051748  4636 leader_election.cc:304] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.050750  4818 raft_consensus.cc:3060] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.052716  4958 raft_consensus.cc:2804] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.053337  4958 raft_consensus.cc:697] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:32.020262  4971 consensus_queue.cc:260] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.054417  4744 raft_consensus.cc:3060] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.052258  4991 raft_consensus.cc:2749] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:32.064112  4744 raft_consensus.cc:2468] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:32.068315  4972 tablet_bootstrap.cc:492] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:32.068893  4972 ts_tablet_manager.cc:1403] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.009s
I20260709 12:29:32.069710  4971 ts_tablet_manager.cc:1434] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 12:29:32.071623  4971 tablet_bootstrap.cc:492] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:32.073045  4972 raft_consensus.cc:359] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.001816  4993 leader_election.cc:290] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:32.073809  4972 raft_consensus.cc:385] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.073803  4818 raft_consensus.cc:2468] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:32.054416  4958 consensus_queue.cc:237] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.080884  4972 raft_consensus.cc:740] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.087683  4819 raft_consensus.cc:2468] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
W20260709 12:29:32.088653  4711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.36.193:37869, user_credentials={real_user=slave}} blocked reactor thread for 43560.9us
I20260709 12:29:32.089085  4971 tablet_bootstrap.cc:654] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.085716  4972 consensus_queue.cc:260] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.093189  4636 leader_election.cc:304] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.094146  4976 raft_consensus.cc:2804] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.094760  4976 raft_consensus.cc:697] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:32.095791  4976 consensus_queue.cc:237] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.105201  4972 ts_tablet_manager.cc:1434] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:29:32.104876  4568 catalog_manager.cc:5697] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:32.106431  4972 tablet_bootstrap.cc:492] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:32.114030  4972 tablet_bootstrap.cc:654] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.115483  4971 tablet_bootstrap.cc:492] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:32.116057  4971 ts_tablet_manager.cc:1403] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.045s	user 0.001s	sys 0.013s
I20260709 12:29:32.117623  4567 catalog_manager.cc:5697] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.119040  4971 raft_consensus.cc:359] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.119902  4971 raft_consensus.cc:385] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.120193  4971 raft_consensus.cc:740] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.120971  4971 consensus_queue.cc:260] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.123291  4971 ts_tablet_manager.cc:1434] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:32.124553  4971 tablet_bootstrap.cc:492] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:32.131379  4971 tablet_bootstrap.cc:654] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.137473  4971 tablet_bootstrap.cc:492] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:32.138110  4971 ts_tablet_manager.cc:1403] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.005s
I20260709 12:29:32.139536  4969 tablet_bootstrap.cc:492] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:32.140097  4969 ts_tablet_manager.cc:1403] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.191s	user 0.011s	sys 0.017s
I20260709 12:29:32.141428  4971 raft_consensus.cc:359] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.142244  4971 raft_consensus.cc:385] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.142553  4971 raft_consensus.cc:740] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.143267  4969 raft_consensus.cc:359] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.143373  4971 consensus_queue.cc:260] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.144181  4969 raft_consensus.cc:385] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.144519  4969 raft_consensus.cc:740] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.145253  4969 consensus_queue.cc:260] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.146328  4971 ts_tablet_manager.cc:1434] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:32.147480  4971 tablet_bootstrap.cc:492] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a: Bootstrap starting.
I20260709 12:29:32.147984  4969 ts_tablet_manager.cc:1434] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:29:32.149039  4969 tablet_bootstrap.cc:492] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:32.154896  4971 tablet_bootstrap.cc:654] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.158667  4969 tablet_bootstrap.cc:654] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.164531  4567 catalog_manager.cc:5697] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 1 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.165858  4972 tablet_bootstrap.cc:492] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:32.165971  4567 catalog_manager.cc:5697] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 1 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.166746  4972 ts_tablet_manager.cc:1403] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.018s
I20260709 12:29:32.169888  4971 tablet_bootstrap.cc:492] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a: No bootstrap required, opened a new log
I20260709 12:29:32.170460  4971 ts_tablet_manager.cc:1403] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.010s
I20260709 12:29:32.170164  4972 raft_consensus.cc:359] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.170933  4972 raft_consensus.cc:385] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.171311  4972 raft_consensus.cc:740] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.172106  4972 consensus_queue.cc:260] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.173576  4971 raft_consensus.cc:359] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.174340  4971 raft_consensus.cc:385] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.174458  4972 ts_tablet_manager.cc:1434] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:32.174638  4971 raft_consensus.cc:740] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.175679  4972 tablet_bootstrap.cc:492] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:32.175494  4971 consensus_queue.cc:260] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.178395  4971 ts_tablet_manager.cc:1434] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.006s
I20260709 12:29:32.182677  4972 tablet_bootstrap.cc:654] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.189023  4969 tablet_bootstrap.cc:492] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:32.189541  4969 ts_tablet_manager.cc:1403] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.005s
I20260709 12:29:32.192584  4969 raft_consensus.cc:359] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.193367  4969 raft_consensus.cc:385] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.193651  4969 raft_consensus.cc:740] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.194378  4969 consensus_queue.cc:260] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.197049  4969 ts_tablet_manager.cc:1434] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:32.198185  4972 tablet_bootstrap.cc:492] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:32.198768  4972 ts_tablet_manager.cc:1403] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.004s
I20260709 12:29:32.201967  4972 raft_consensus.cc:359] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.202943  4972 raft_consensus.cc:385] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.203264  4972 raft_consensus.cc:740] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.204026  4972 consensus_queue.cc:260] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.206235  4972 ts_tablet_manager.cc:1434] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:29:32.208403  4969 tablet_bootstrap.cc:492] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:32.211263  4925 raft_consensus.cc:493] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.211791  4925 raft_consensus.cc:515] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.213841  4969 tablet_bootstrap.cc:654] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.213908  4925 leader_election.cc:290] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:32.215013  4972 tablet_bootstrap.cc:492] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:32.215368  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:32.216096  4744 raft_consensus.cc:2468] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:32.215643  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:32.217439  4668 raft_consensus.cc:2468] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 0.
I20260709 12:29:32.217471  4785 leader_election.cc:304] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.218444  4925 raft_consensus.cc:2804] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:32.218815  4925 raft_consensus.cc:493] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.219151  4925 raft_consensus.cc:3060] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.225596  4972 tablet_bootstrap.cc:654] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.226107  4925 raft_consensus.cc:515] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.228171  4925 leader_election.cc:290] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:32.230042  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:32.230170  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:32.230749  4668 raft_consensus.cc:3060] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.230893  4744 raft_consensus.cc:3060] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.232012  4969 tablet_bootstrap.cc:492] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:32.232323  4972 tablet_bootstrap.cc:492] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:32.232578  4969 ts_tablet_manager.cc:1403] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.008s
I20260709 12:29:32.232834  4972 ts_tablet_manager.cc:1403] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260709 12:29:32.235816  4972 raft_consensus.cc:359] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.236505  4972 raft_consensus.cc:385] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.236771  4972 raft_consensus.cc:740] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.237509  4972 consensus_queue.cc:260] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.239598  4744 raft_consensus.cc:2468] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:32.240085  4972 ts_tablet_manager.cc:1434] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:32.240001  4969 raft_consensus.cc:359] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.241043  4969 raft_consensus.cc:385] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.241264  4972 tablet_bootstrap.cc:492] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588: Bootstrap starting.
I20260709 12:29:32.241143  4785 leader_election.cc:304] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.241372  4969 raft_consensus.cc:740] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.239930  4668 raft_consensus.cc:2468] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 1.
I20260709 12:29:32.242592  4925 raft_consensus.cc:2804] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.242401  4969 consensus_queue.cc:260] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.243206  4925 raft_consensus.cc:697] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:32.244166  4925 consensus_queue.cc:237] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.245314  4969 ts_tablet_manager.cc:1434] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:29:32.246390  4969 tablet_bootstrap.cc:492] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f: Bootstrap starting.
I20260709 12:29:32.250748  4972 tablet_bootstrap.cc:654] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.253994  4969 tablet_bootstrap.cc:654] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.262602  4972 tablet_bootstrap.cc:492] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588: No bootstrap required, opened a new log
I20260709 12:29:32.262789  4969 tablet_bootstrap.cc:492] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f: No bootstrap required, opened a new log
I20260709 12:29:32.263128  4972 ts_tablet_manager.cc:1403] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.003s
I20260709 12:29:32.263574  4969 ts_tablet_manager.cc:1403] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.012s
I20260709 12:29:32.265758  4957 raft_consensus.cc:493] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.266345  4957 raft_consensus.cc:515] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.266455  4972 raft_consensus.cc:359] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.267181  4972 raft_consensus.cc:385] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.267611  4972 raft_consensus.cc:740] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.268584  4957 leader_election.cc:290] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.268718  4972 consensus_queue.cc:260] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.268944  4567 catalog_manager.cc:5697] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 1 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:32.270581  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99be8cd85c6943998dcb402ec0bfc4de" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:32.270828  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99be8cd85c6943998dcb402ec0bfc4de" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:32.271282  4972 ts_tablet_manager.cc:1434] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:32.271585  4819 raft_consensus.cc:2468] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:32.272632  4711 leader_election.cc:304] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.268627  4969 raft_consensus.cc:359] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.273284  4969 raft_consensus.cc:385] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.273483  4957 raft_consensus.cc:2804] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:32.274010  4957 raft_consensus.cc:493] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.273622  4969 raft_consensus.cc:740] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.274384  4957 raft_consensus.cc:3060] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.275049  4969 consensus_queue.cc:260] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.275982  4668 raft_consensus.cc:2468] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:32.277602  4969 ts_tablet_manager.cc:1434] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:29:32.282838  4957 raft_consensus.cc:515] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.286044  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99be8cd85c6943998dcb402ec0bfc4de" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.286049  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99be8cd85c6943998dcb402ec0bfc4de" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:32.285101  4957 leader_election.cc:290] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.286882  4819 raft_consensus.cc:3060] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.286967  4668 raft_consensus.cc:3060] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.295416  4668 raft_consensus.cc:2468] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:32.296531  4711 leader_election.cc:304] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.297336  4957 raft_consensus.cc:2804] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.297590  4819 raft_consensus.cc:2468] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:32.298054  4957 raft_consensus.cc:697] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:32.299424  4957 consensus_queue.cc:237] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.307107  4568 catalog_manager.cc:5697] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.352721  4958 consensus_queue.cc:1048] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.364991  4958 consensus_queue.cc:1048] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.425429  4958 raft_consensus.cc:493] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.425935  4958 raft_consensus.cc:515] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.427798  4958 leader_election.cc:290] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.428558  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a" is_pre_election: true
I20260709 12:29:32.428669  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:32.429358  4744 raft_consensus.cc:2468] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:32.429365  4819 raft_consensus.cc:2468] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 0.
I20260709 12:29:32.430716  4636 leader_election.cc:304] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.431408  4958 raft_consensus.cc:2804] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:32.431843  4958 raft_consensus.cc:493] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.432147  4958 raft_consensus.cc:3060] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.432277  4925 consensus_queue.cc:1048] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:32.438848  4958 raft_consensus.cc:515] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.443773  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:32.443954  4958 leader_election.cc:290] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.444497  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.459848  4744 raft_consensus.cc:3060] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.460296  4818 raft_consensus.cc:3060] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.445703  4925 consensus_queue.cc:1048] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:29:32.468384  4818 raft_consensus.cc:2468] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:32.468717  4744 raft_consensus.cc:2468] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 1.
I20260709 12:29:32.470352  4636 leader_election.cc:304] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.471216  4976 raft_consensus.cc:2804] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.471767  4976 raft_consensus.cc:697] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:32.473896  4976 consensus_queue.cc:237] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.475739  4818 raft_consensus.cc:3060] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.476454  4957 consensus_queue.cc:1048] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:32.486343  4567 catalog_manager.cc:5697] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 1 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.488075  4994 consensus_queue.cc:1048] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.559394  4976 consensus_queue.cc:1048] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.570827  5026 raft_consensus.cc:493] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.571342  4989 consensus_queue.cc:1048] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.571478  5026 raft_consensus.cc:515] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.573858  5026 leader_election.cc:290] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.575426  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" is_pre_election: true
I20260709 12:29:32.576172  4668 raft_consensus.cc:2468] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:32.576488  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588" is_pre_election: true
I20260709 12:29:32.577240  4818 raft_consensus.cc:2468] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 0.
I20260709 12:29:32.577383  4711 leader_election.cc:304] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.578334  5026 raft_consensus.cc:2804] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:32.578729  5026 raft_consensus.cc:493] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.579092  5026 raft_consensus.cc:3060] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.588878  5026 raft_consensus.cc:515] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.591003  5026 leader_election.cc:290] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.591892  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:32.592070  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.592720  4668 raft_consensus.cc:3060] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.592811  4818 raft_consensus.cc:3060] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.598719  4818 raft_consensus.cc:2468] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:32.598847  4668 raft_consensus.cc:2468] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 1.
I20260709 12:29:32.599706  4711 leader_election.cc:304] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.600452  5026 raft_consensus.cc:2804] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.601046  5026 raft_consensus.cc:697] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:32.601899  5026 consensus_queue.cc:237] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.610205  4568 catalog_manager.cc:5697] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 1 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.619139  4989 consensus_queue.cc:1048] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.632423  4958 consensus_queue.cc:1048] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.649487  4818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
mode: GRACEFUL
new_leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:55178
I20260709 12:29:32.650167  4818 raft_consensus.cc:606] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Received request to transfer leadership to TS 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:32.657195  4745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
mode: GRACEFUL
new_leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:60074
I20260709 12:29:32.657894  4745 raft_consensus.cc:606] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Received request to transfer leadership to TS 5ab80e12f0ef4df38684a32e134d1e7f
I20260709 12:29:32.669450  4668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3553184055c4292ac3acc49631d9152"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
mode: GRACEFUL
new_leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:35650
I20260709 12:29:32.670006  4668 raft_consensus.cc:606] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Received request to transfer leadership to TS 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:32.675045  4668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
mode: GRACEFUL
new_leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:35650
I20260709 12:29:32.675665  4668 raft_consensus.cc:606] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Received request to transfer leadership to TS 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:32.681170  4818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
mode: GRACEFUL
new_leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:55178
I20260709 12:29:32.681818  4818 raft_consensus.cc:606] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Received request to transfer leadership to TS 6d0e2be318224ee5a8f54ea891b0856a
I20260709 12:29:32.690478  4745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
mode: GRACEFUL
new_leader_uuid: "5cbdbb7559fb49919a700494dc154588"
 from {username='slave'} at 127.0.0.1:60074
I20260709 12:29:32.691107  4745 raft_consensus.cc:606] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Received request to transfer leadership to TS 5cbdbb7559fb49919a700494dc154588
I20260709 12:29:32.744175  4994 consensus_queue.cc:1048] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.760555  5034 consensus_queue.cc:1048] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.793365  4993 consensus_queue.cc:1048] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:32.807077  4993 consensus_queue.cc:1048] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:32.816432  4925 raft_consensus.cc:993] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: : Instructing follower 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:32.816970  4993 raft_consensus.cc:1081] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Signalling peer 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:32.818548  4745 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:60070
I20260709 12:29:32.819116  4745 raft_consensus.cc:493] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:32.819567  4745 raft_consensus.cc:3060] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.825109  4745 raft_consensus.cc:515] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.827158  4745 leader_election.cc:290] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.827876  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:32.828544  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36103d928c44b22bf506cce041a25a8" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.828553  4668 raft_consensus.cc:3060] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.829224  4818 raft_consensus.cc:3055] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:32.829726  4818 raft_consensus.cc:740] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:32.830554  4818 consensus_queue.cc:260] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.831791  4818 raft_consensus.cc:3060] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.835541  4668 raft_consensus.cc:2468] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:32.836529  4711 leader_election.cc:304] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.837177  5034 raft_consensus.cc:2804] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:32.837572  5034 raft_consensus.cc:697] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:32.838080  4818 raft_consensus.cc:2468] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:32.838367  5034 consensus_queue.cc:237] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.845711  4567 catalog_manager.cc:5697] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 1 to 2, leader changed from 5cbdbb7559fb49919a700494dc154588 (127.19.36.195) to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.888185  4925 raft_consensus.cc:993] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: : Instructing follower 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:32.888657  4993 raft_consensus.cc:1081] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Signalling peer 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:32.890067  4668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:35644
I20260709 12:29:32.890529  4668 raft_consensus.cc:493] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:32.890805  4668 raft_consensus.cc:3060] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.895859  4668 raft_consensus.cc:515] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.897552  4668 leader_election.cc:290] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.898283  4745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:32.898296  4958 consensus_queue.cc:1048] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:32.898671  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51c65145402b4d15b96a7b47fba3d447" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.899154  4745 raft_consensus.cc:3060] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.899384  4818 raft_consensus.cc:3055] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:32.899842  4818 raft_consensus.cc:740] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:32.900585  4818 consensus_queue.cc:260] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.901747  4818 raft_consensus.cc:3060] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.906214  4745 raft_consensus.cc:2468] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:32.907657  4636 leader_election.cc:304] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.909039  4958 raft_consensus.cc:2804] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:32.909570  4958 raft_consensus.cc:697] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:32.910418  4958 consensus_queue.cc:237] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.907785  4976 raft_consensus.cc:993] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: : Instructing follower 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:32.911329  4818 raft_consensus.cc:2468] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:32.911592  4988 raft_consensus.cc:1081] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Signalling peer 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:32.915107  4745 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:60058
I20260709 12:29:32.915612  4745 raft_consensus.cc:493] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:32.915951  4745 raft_consensus.cc:3060] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.916476  4976 consensus_queue.cc:1048] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.923123  4745 raft_consensus.cc:515] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.924268  4567 catalog_manager.cc:5697] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 1 to 2, leader changed from 5cbdbb7559fb49919a700494dc154588 (127.19.36.195) to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 2 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.928184  4745 leader_election.cc:290] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:32.931567  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:32.932440  4819 raft_consensus.cc:3060] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.934048  4668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb30e39bc65458cbc6f1f4e87a9e6ab" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:32.934607  4668 raft_consensus.cc:3055] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:32.934864  4668 raft_consensus.cc:740] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:32.935570  4668 consensus_queue.cc:260] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.936795  4668 raft_consensus.cc:3060] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.941115  4819 raft_consensus.cc:2468] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:32.942380  4711 leader_election.cc:304] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:32.943251  5034 raft_consensus.cc:2804] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:32.943745  5034 raft_consensus.cc:697] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:32.943928  4668 raft_consensus.cc:2468] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:32.944695  5034 consensus_queue.cc:237] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:32.953760  4567 catalog_manager.cc:5697] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 1 to 2, leader changed from 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193) to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:33.009693  4989 raft_consensus.cc:993] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: : Instructing follower 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:33.009922  4994 raft_consensus.cc:993] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: : Instructing follower 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:33.010211  4988 raft_consensus.cc:1081] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Signalling peer 6d0e2be318224ee5a8f54ea891b0856a to start an election
I20260709 12:29:33.010478  5034 raft_consensus.cc:1081] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Signalling peer 5ab80e12f0ef4df38684a32e134d1e7f to start an election
I20260709 12:29:33.012053  4745 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3553184055c4292ac3acc49631d9152"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
 from {username='slave'} at 127.0.0.1:60058
I20260709 12:29:33.012061  4668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
 from {username='slave'} at 127.0.0.1:35646
I20260709 12:29:33.012705  4745 raft_consensus.cc:493] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:33.013151  4745 raft_consensus.cc:3060] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.013170  4668 raft_consensus.cc:493] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:33.014008  4668 raft_consensus.cc:3060] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.014974  5034 consensus_queue.cc:1048] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:33.020591  4668 raft_consensus.cc:515] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.020747  4745 raft_consensus.cc:515] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.023351  4669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:33.023953  4669 raft_consensus.cc:3055] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:33.024183  5034 raft_consensus.cc:993] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: : Instructing follower 5cbdbb7559fb49919a700494dc154588 to start an election
I20260709 12:29:33.024299  4669 raft_consensus.cc:740] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:33.024803  5034 raft_consensus.cc:1081] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Signalling peer 5cbdbb7559fb49919a700494dc154588 to start an election
I20260709 12:29:33.025141  4819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3553184055c4292ac3acc49631d9152" candidate_uuid: "6d0e2be318224ee5a8f54ea891b0856a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:33.025740  4819 raft_consensus.cc:3060] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.025280  4669 consensus_queue.cc:260] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.026521  4669 raft_consensus.cc:3060] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.028733  4668 leader_election.cc:290] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:33.029341  4818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cbdbb7559fb49919a700494dc154588"
I20260709 12:29:33.029644  4745 leader_election.cc:290] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 5cbdbb7559fb49919a700494dc154588 (127.19.36.195:46201)
I20260709 12:29:33.029978  4818 raft_consensus.cc:3060] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.034806  4669 raft_consensus.cc:2468] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:33.035715  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e5002fd95af4b95997c13feb68bbcba" candidate_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:33.034950  4819 raft_consensus.cc:2468] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0e2be318224ee5a8f54ea891b0856a in term 2.
I20260709 12:29:33.036346  4744 raft_consensus.cc:3055] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:33.036773  4744 raft_consensus.cc:740] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:33.037195  4818 raft_consensus.cc:2468] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:33.037675  4744 consensus_queue.cc:260] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.038571  4636 leader_election.cc:304] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:33.038708  4744 raft_consensus.cc:3060] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.040225  4989 raft_consensus.cc:2804] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:33.040699  4989 raft_consensus.cc:697] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Becoming Leader. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:33.041527  4989 consensus_queue.cc:237] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.041846  4818 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
 from {username='slave'} at 127.0.0.1:55176
I20260709 12:29:33.042577  4818 raft_consensus.cc:493] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:33.042994  4818 raft_consensus.cc:3060] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.047629  4744 raft_consensus.cc:2468] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab80e12f0ef4df38684a32e134d1e7f in term 2.
I20260709 12:29:33.048213  4711 leader_election.cc:304] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:33.050323  4818 raft_consensus.cc:515] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.050817  4994 raft_consensus.cc:2804] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:33.051581  4994 raft_consensus.cc:697] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Becoming Leader. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:33.053368  4670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:33.053989  4670 raft_consensus.cc:3060] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.052520  4994 consensus_queue.cc:237] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.056313  4957 consensus_queue.cc:1048] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:33.057658  4818 leader_election.cc:290] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 2 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:33.058979  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2200a06b9094b1e84a7fef9defaef4c" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:33.059617  4744 raft_consensus.cc:3055] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:33.059919  4744 raft_consensus.cc:740] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:33.060395  4565 catalog_manager.cc:5697] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f reported cstate change: term changed from 1 to 2, leader changed from 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194) to 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193). New cstate: current_term: 2 leader_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:33.060667  4744 consensus_queue.cc:260] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.061832  4744 raft_consensus.cc:3060] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:33.064257  4670 raft_consensus.cc:2468] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 2.
I20260709 12:29:33.066079  4785 leader_election.cc:304] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab80e12f0ef4df38684a32e134d1e7f, 5cbdbb7559fb49919a700494dc154588; no voters: 
I20260709 12:29:33.066692  4669 raft_consensus.cc:1240] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Rejecting Update request from peer 6d0e2be318224ee5a8f54ea891b0856a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:29:33.066918  4993 raft_consensus.cc:2804] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:33.067555  4993 raft_consensus.cc:697] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 2 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:33.068385  4993 consensus_queue.cc:237] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:33.072149  4744 raft_consensus.cc:2468] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 2.
I20260709 12:29:33.074885  4567 catalog_manager.cc:5697] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a reported cstate change: term changed from 1 to 2, leader changed from 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193) to 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194). New cstate: current_term: 2 leader_uuid: "6d0e2be318224ee5a8f54ea891b0856a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:33.076980  4568 catalog_manager.cc:5697] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 1 to 2, leader changed from 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194) to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 2 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:33.224026  4818 raft_consensus.cc:1275] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.225528  4957 consensus_queue.cc:1048] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.235582  4669 raft_consensus.cc:1275] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.236894  4994 consensus_queue.cc:1048] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.329960  4818 raft_consensus.cc:1275] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.331296  4994 consensus_queue.cc:1048] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.339921  4669 raft_consensus.cc:1275] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.341583  5026 consensus_queue.cc:1048] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.449474  4818 raft_consensus.cc:1275] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.450625  4989 consensus_queue.cc:1048] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.457280  4744 raft_consensus.cc:1275] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Refusing update from remote peer 5cbdbb7559fb49919a700494dc154588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.458367  4745 raft_consensus.cc:1275] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.458940  4993 consensus_queue.cc:1048] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.459960  4989 consensus_queue.cc:1048] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.469214  4669 raft_consensus.cc:1275] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Refusing update from remote peer 5cbdbb7559fb49919a700494dc154588: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.470790  4925 consensus_queue.cc:1048] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:33.487535  4669 raft_consensus.cc:1275] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.488997  4818 raft_consensus.cc:1275] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 6d0e2be318224ee5a8f54ea891b0856a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.489256  5026 consensus_queue.cc:1048] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.491398  5026 consensus_queue.cc:1048] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.492672  4818 raft_consensus.cc:1275] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.494046  4988 consensus_queue.cc:1048] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.509320  4745 raft_consensus.cc:1275] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Refusing update from remote peer 5ab80e12f0ef4df38684a32e134d1e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.511489  4976 consensus_queue.cc:1048] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.694458  4565 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:29:35.719347  4565 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
W20260709 12:29:35.898936  5070 log.cc:927] Time spent T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:29:35.899335 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:29:35.899233  5069 log.cc:927] Time spent T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:29:35.905647  5068 log.cc:927] Time spent T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: Append to log took a long time: real 0.067s	user 0.005s	sys 0.000s
I20260709 12:29:35.912408  5076 tablet.cc:1724] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:35.900868  5071 log.cc:927] Time spent T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:29:35.923712  5059 log.cc:927] Time spent T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 12:29:35.922897  5072 log.cc:927] Time spent T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:29:35.980141  4636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:44691, user_credentials={real_user=slave}} blocked reactor thread for 89877.5us
I20260709 12:29:35.869462  5073 tablet.cc:1724] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:36.025115  5083 log.cc:927] Time spent T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.002s
W20260709 12:29:36.051268  5074 log.cc:927] Time spent T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:29:36.083772  5100 tablet.cc:1724] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:36.119805  5093 log.cc:927] Time spent T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 12:29:36.129792  5105 tablet.cc:1724] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.134529  5108 tablet.cc:1724] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:36.148334  5096 log.cc:927] Time spent T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:29:36.164494  4711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.193:37869, user_credentials={real_user=slave}} blocked reactor thread for 136867us
I20260709 12:29:36.165926  5114 tablet.cc:1724] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.171280  4744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7"
dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
mode: GRACEFUL
new_leader_uuid: "5cbdbb7559fb49919a700494dc154588"
 from {username='slave'} at 127.0.0.1:35376
I20260709 12:29:36.171911  4744 raft_consensus.cc:606] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Received request to transfer leadership to TS 5cbdbb7559fb49919a700494dc154588
I20260709 12:29:36.179935  5106 tablet.cc:1724] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.188733  5110 raft_consensus.cc:993] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: : Instructing follower 5cbdbb7559fb49919a700494dc154588 to start an election
I20260709 12:29:36.189481  5110 raft_consensus.cc:1081] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Signalling peer 5cbdbb7559fb49919a700494dc154588 to start an election
I20260709 12:29:36.195088  5112 tablet.cc:1724] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:36.202841  4785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:44691, user_credentials={real_user=slave}} blocked reactor thread for 46012.2us
I20260709 12:29:36.142868  5111 tablet.cc:1724] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.256325  5073 tablet.cc:1724] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:36.291749  4785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.194:44691, user_credentials={real_user=slave}} blocked reactor thread for 87861.3us
W20260709 12:29:36.299476  4711 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.36.195:46201, user_credentials={real_user=slave}} blocked reactor thread for 123445us
I20260709 12:29:36.223788  5116 tablet.cc:1724] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.315564  5121 tablet.cc:1724] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.315920  5076 tablet.cc:1724] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.325021  5111 tablet.cc:1724] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.338752  5105 tablet.cc:1724] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.340853  5121 tablet.cc:1724] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.340803  5111 tablet.cc:1724] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.367604  4816 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
 from {username='slave'} at 127.0.0.1:55176
I20260709 12:29:36.368304  4816 raft_consensus.cc:493] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:36.368724  4816 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.368762  5111 tablet.cc:1724] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.377154  5076 tablet.cc:1724] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.378754  5073 tablet.cc:1724] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.383363  5106 tablet.cc:1724] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.384486  4816 raft_consensus.cc:515] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:36.363274  5123 tablet.cc:1724] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.408104  5112 tablet.cc:1724] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.380805  5116 tablet.cc:1724] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.419728  5100 tablet.cc:1724] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.413324  5114 tablet.cc:1724] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.430126  4667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "933310e093f24d6c87a460c3471eca97"
dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
mode: GRACEFUL
new_leader_uuid: "5cbdbb7559fb49919a700494dc154588"
 from {username='slave'} at 127.0.0.1:35880
I20260709 12:29:36.430955  4667 raft_consensus.cc:606] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Received request to transfer leadership to TS 5cbdbb7559fb49919a700494dc154588
I20260709 12:29:36.433439 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:36.434132 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:36.435087  5108 tablet.cc:1724] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:36.440884  4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:36.442800  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04f0cfa224354c31afe3140f261990d7" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:36.445361  4667 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.445513  4744 raft_consensus.cc:3055] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:36.445962  4744 raft_consensus.cc:740] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d0e2be318224ee5a8f54ea891b0856a, State: Running, Role: LEADER
I20260709 12:29:36.446792  4744 consensus_queue.cc:260] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:36.447976  4744 raft_consensus.cc:3060] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.435911  4816 leader_election.cc:290] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 3 election: Requested vote from peers 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869), 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691)
I20260709 12:29:36.461244  4667 raft_consensus.cc:2468] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 3.
I20260709 12:29:36.468504  4744 raft_consensus.cc:2468] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 3.
I20260709 12:29:36.471372  4785 leader_election.cc:304] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:36.472568  5119 raft_consensus.cc:2804] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:36.473178  5119 raft_consensus.cc:697] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 3 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:36.474135  5119 consensus_queue.cc:237] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:36.488809  4568 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88: soft_deleted_table [id=1807167faa714e379c5cf69b5bf37fa5] alter complete (version 1)
I20260709 12:29:36.500576  4565 catalog_manager.cc:5697] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 2 to 3, leader changed from 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194) to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:36.585223  5097 raft_consensus.cc:993] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f: : Instructing follower 5cbdbb7559fb49919a700494dc154588 to start an election
I20260709 12:29:36.585875  5090 raft_consensus.cc:1081] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Signalling peer 5cbdbb7559fb49919a700494dc154588 to start an election
I20260709 12:29:36.587579  4816 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "933310e093f24d6c87a460c3471eca97"
dest_uuid: "5cbdbb7559fb49919a700494dc154588"
 from {username='slave'} at 127.0.0.1:55160
I20260709 12:29:36.588212  4816 raft_consensus.cc:493] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:36.588593  4816 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.596830  4816 raft_consensus.cc:515] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:36.599339  4816 leader_election.cc:290] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 3 election: Requested vote from peers 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691), 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869)
I20260709 12:29:36.600070  4744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6d0e2be318224ee5a8f54ea891b0856a"
I20260709 12:29:36.600783  4744 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.600860  4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "933310e093f24d6c87a460c3471eca97" candidate_uuid: "5cbdbb7559fb49919a700494dc154588" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5ab80e12f0ef4df38684a32e134d1e7f"
I20260709 12:29:36.601505  4667 raft_consensus.cc:3055] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:36.601837  4667 raft_consensus.cc:740] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5ab80e12f0ef4df38684a32e134d1e7f, State: Running, Role: LEADER
I20260709 12:29:36.602658  4667 consensus_queue.cc:260] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:36.604113  4667 raft_consensus.cc:3060] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:36.609133  4744 raft_consensus.cc:2468] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 3.
I20260709 12:29:36.610345  4785 leader_election.cc:304] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbdbb7559fb49919a700494dc154588, 6d0e2be318224ee5a8f54ea891b0856a; no voters: 
I20260709 12:29:36.611310  5119 raft_consensus.cc:2804] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:36.612345  4667 raft_consensus.cc:2468] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbdbb7559fb49919a700494dc154588 in term 3.
I20260709 12:29:36.612588  5119 raft_consensus.cc:697] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 3 LEADER]: Becoming Leader. State: Replica: 5cbdbb7559fb49919a700494dc154588, State: Running, Role: LEADER
I20260709 12:29:36.613857  5119 consensus_queue.cc:237] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } }
I20260709 12:29:36.623553  4568 catalog_manager.cc:5697] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 reported cstate change: term changed from 2 to 3, leader changed from 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193) to 5cbdbb7559fb49919a700494dc154588 (127.19.36.195). New cstate: current_term: 3 leader_uuid: "5cbdbb7559fb49919a700494dc154588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbdbb7559fb49919a700494dc154588" member_type: VOTER last_known_addr { host: "127.19.36.195" port: 46201 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:36.896255  4670 raft_consensus.cc:1275] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Refusing update from remote peer 5cbdbb7559fb49919a700494dc154588: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:36.897457  5119 consensus_queue.cc:1048] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:36.907930  4742 raft_consensus.cc:1275] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Refusing update from remote peer 5cbdbb7559fb49919a700494dc154588: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:36.911603  5119 consensus_queue.cc:1048] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.109961  4670 raft_consensus.cc:1275] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Refusing update from remote peer 5cbdbb7559fb49919a700494dc154588: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.111203  5119 consensus_queue.cc:1048] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab80e12f0ef4df38684a32e134d1e7f" member_type: VOTER last_known_addr { host: "127.19.36.193" port: 37869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.120003  4742 raft_consensus.cc:1275] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Refusing update from remote peer 5cbdbb7559fb49919a700494dc154588: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:37.122116  5132 consensus_queue.cc:1048] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0e2be318224ee5a8f54ea891b0856a" member_type: VOTER last_known_addr { host: "127.19.36.194" port: 44691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:37.435171 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:37.442538 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:37.443053 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:38.444193 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:38.449509 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:38.449995 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:39.452528 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:39.466658 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:39.467504 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:40.468746 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:40.473735 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:40.474121 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:41.475284 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:41.482283 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:41.482759 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:42.483837 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:42.489184 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:42.489673 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:43.490767 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:43.496238 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:43.496716 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:44.497668 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:44.502872 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:44.503368 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:45.504504 19603 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:45.509456 19603 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:45.509891 19603 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:46.567921 19603 tablet_server.cc:179] TabletServer@127.19.36.193:0 shutting down...
I20260709 12:29:46.611094 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:46.611989 19603 tablet_replica.cc:333] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.612963 19603 raft_consensus.cc:2243] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.613737 19603 raft_consensus.cc:2272] T f2200a06b9094b1e84a7fef9defaef4c P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.616387 19603 tablet_replica.cc:333] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.616977 19603 raft_consensus.cc:2243] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.617509 19603 raft_consensus.cc:2272] T f36103d928c44b22bf506cce041a25a8 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.620100 19603 tablet_replica.cc:333] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.620783 19603 raft_consensus.cc:2243] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.621361 19603 raft_consensus.cc:2272] T b3553184055c4292ac3acc49631d9152 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.623863 19603 tablet_replica.cc:333] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.624526 19603 raft_consensus.cc:2243] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.625499 19603 raft_consensus.cc:2272] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.628438 19603 tablet_replica.cc:333] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.629069 19603 raft_consensus.cc:2243] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.630151 19603 raft_consensus.cc:2272] T f66dc3001f5149f683e43c2db87d9e0f P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.632642 19603 tablet_replica.cc:333] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.633289 19603 raft_consensus.cc:2243] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.633857 19603 raft_consensus.cc:2272] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.636283 19603 tablet_replica.cc:333] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.636854 19603 raft_consensus.cc:2243] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.637354 19603 raft_consensus.cc:2272] T 70f0d3ef958849f08eaf18856e6567e2 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.640007 19603 tablet_replica.cc:333] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.640568 19603 raft_consensus.cc:2243] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.641450 19603 raft_consensus.cc:2272] T 51c65145402b4d15b96a7b47fba3d447 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.643985 19603 tablet_replica.cc:333] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.644608 19603 raft_consensus.cc:2243] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.645087 19603 raft_consensus.cc:2272] T 748fe2ac99874ea9b112159a53cc3beb P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.647424 19603 tablet_replica.cc:333] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.647996 19603 raft_consensus.cc:2243] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.648887 19603 raft_consensus.cc:2272] T d7adec6a263b40ae8f500e6d4de59996 P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.651093 19603 tablet_replica.cc:333] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.651669 19603 raft_consensus.cc:2243] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.652614 19603 raft_consensus.cc:2272] T 2e5002fd95af4b95997c13feb68bbcba P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.654927 19603 tablet_replica.cc:333] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.655570 19603 raft_consensus.cc:2243] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.656154 19603 raft_consensus.cc:2272] T 99be8cd85c6943998dcb402ec0bfc4de P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.658689 19603 tablet_replica.cc:333] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.659377 19603 raft_consensus.cc:2243] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.660602 19603 raft_consensus.cc:2272] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.663653 19603 tablet_replica.cc:333] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.664420 19603 raft_consensus.cc:2243] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.665413 19603 raft_consensus.cc:2272] T ae78f7d9184346839a507b7bd4e2659a P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.667693 19603 tablet_replica.cc:333] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.668179 19603 raft_consensus.cc:2243] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.668690 19603 raft_consensus.cc:2272] T 197e1cac162e444ebbb81bd7f563902b P 5ab80e12f0ef4df38684a32e134d1e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.670907 19603 tablet_replica.cc:333] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.671423 19603 raft_consensus.cc:2243] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.671830 19603 raft_consensus.cc:2272] T ddfff7a3d59a47de8bc9f6929041196a P 5ab80e12f0ef4df38684a32e134d1e7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.674014 19603 tablet_replica.cc:333] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.674523 19603 raft_consensus.cc:2243] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.675025 19603 raft_consensus.cc:2272] T 933310e093f24d6c87a460c3471eca97 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.677390 19603 tablet_replica.cc:333] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f: stopping tablet replica
I20260709 12:29:46.677888 19603 raft_consensus.cc:2243] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.678390 19603 raft_consensus.cc:2272] T 04f0cfa224354c31afe3140f261990d7 P 5ab80e12f0ef4df38684a32e134d1e7f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:46.704172  4711 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 19 similar messages]
W20260709 12:29:46.707881  4711 consensus_peers.cc:597] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a -> Peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Couldn't send request to peer 5ab80e12f0ef4df38684a32e134d1e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.726285 19603 tablet_server.cc:196] TabletServer@127.19.36.193:0 shutdown complete.
W20260709 12:29:46.765252  4785 consensus_peers.cc:597] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 -> Peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Couldn't send request to peer 5ab80e12f0ef4df38684a32e134d1e7f. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:37869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.775513 19603 tablet_server.cc:179] TabletServer@127.19.36.194:0 shutting down...
W20260709 12:29:46.790887  4785 consensus_peers.cc:597] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 -> Peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Couldn't send request to peer 6d0e2be318224ee5a8f54ea891b0856a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.809219 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:46.810077 19603 tablet_replica.cc:333] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.810843 19603 raft_consensus.cc:2243] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.811544 19603 raft_consensus.cc:2272] T f2200a06b9094b1e84a7fef9defaef4c P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.814043 19603 tablet_replica.cc:333] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.814762 19603 raft_consensus.cc:2243] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.815987 19603 raft_consensus.cc:2272] T f36103d928c44b22bf506cce041a25a8 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:46.817647  4785 consensus_peers.cc:597] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 -> Peer 6d0e2be318224ee5a8f54ea891b0856a (127.19.36.194:44691): Couldn't send request to peer 6d0e2be318224ee5a8f54ea891b0856a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.819003 19603 tablet_replica.cc:333] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.819907 19603 raft_consensus.cc:2243] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.820905 19603 raft_consensus.cc:2272] T b3553184055c4292ac3acc49631d9152 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:46.822916  4711 consensus_peers.cc:597] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a -> Peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Couldn't send request to peer 5ab80e12f0ef4df38684a32e134d1e7f. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:37869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.823575 19603 tablet_replica.cc:333] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.824334 19603 raft_consensus.cc:2243] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.824998 19603 raft_consensus.cc:2272] T 2b3afd1c223f42b1bda891c9dcfa5150 P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.827764 19603 tablet_replica.cc:333] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.828410 19603 raft_consensus.cc:2243] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.829010 19603 raft_consensus.cc:2272] T f66dc3001f5149f683e43c2db87d9e0f P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.832059 19603 tablet_replica.cc:333] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.832705 19603 raft_consensus.cc:2243] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.833602 19603 raft_consensus.cc:2272] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.835820 19603 tablet_replica.cc:333] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.836421 19603 raft_consensus.cc:2243] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.836957 19603 raft_consensus.cc:2272] T 51c65145402b4d15b96a7b47fba3d447 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.839373 19603 tablet_replica.cc:333] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.839937 19603 raft_consensus.cc:2243] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.840866 19603 raft_consensus.cc:2272] T 70f0d3ef958849f08eaf18856e6567e2 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.843418 19603 tablet_replica.cc:333] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.844002 19603 raft_consensus.cc:2243] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.844616 19603 raft_consensus.cc:2272] T 933310e093f24d6c87a460c3471eca97 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:46.846550  4785 consensus_peers.cc:597] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 -> Peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Couldn't send request to peer 5ab80e12f0ef4df38684a32e134d1e7f. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:37869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.847373 19603 tablet_replica.cc:333] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.848254 19603 raft_consensus.cc:2243] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.849373 19603 raft_consensus.cc:2272] T 748fe2ac99874ea9b112159a53cc3beb P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:46.851367  4711 consensus_peers.cc:597] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a -> Peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Couldn't send request to peer 5ab80e12f0ef4df38684a32e134d1e7f. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:37869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.852499 19603 tablet_replica.cc:333] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.853318 19603 raft_consensus.cc:2243] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.853801 19603 raft_consensus.cc:2272] T 2e5002fd95af4b95997c13feb68bbcba P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.856200 19603 tablet_replica.cc:333] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.856768 19603 raft_consensus.cc:2243] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.857717 19603 raft_consensus.cc:2272] T 99be8cd85c6943998dcb402ec0bfc4de P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.860273 19603 tablet_replica.cc:333] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.860862 19603 raft_consensus.cc:2243] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.861397 19603 raft_consensus.cc:2272] T 4a3068fedf904cd3b43cbcd34a3e61aa P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.863690 19603 tablet_replica.cc:333] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.864234 19603 raft_consensus.cc:2243] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.864678 19603 raft_consensus.cc:2272] T ae78f7d9184346839a507b7bd4e2659a P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.866822 19603 tablet_replica.cc:333] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.867405 19603 raft_consensus.cc:2243] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.867857 19603 raft_consensus.cc:2272] T d7adec6a263b40ae8f500e6d4de59996 P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.870013 19603 tablet_replica.cc:333] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.870538 19603 raft_consensus.cc:2243] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.871423 19603 raft_consensus.cc:2272] T ddfff7a3d59a47de8bc9f6929041196a P 6d0e2be318224ee5a8f54ea891b0856a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.873575 19603 tablet_replica.cc:333] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.874104 19603 raft_consensus.cc:2243] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.874547 19603 raft_consensus.cc:2272] T 197e1cac162e444ebbb81bd7f563902b P 6d0e2be318224ee5a8f54ea891b0856a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.876828 19603 tablet_replica.cc:333] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a: stopping tablet replica
I20260709 12:29:46.877437 19603 raft_consensus.cc:2243] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.877883 19603 raft_consensus.cc:2272] T 04f0cfa224354c31afe3140f261990d7 P 6d0e2be318224ee5a8f54ea891b0856a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.922319 19603 tablet_server.cc:196] TabletServer@127.19.36.194:0 shutdown complete.
W20260709 12:29:46.956776  4785 consensus_peers.cc:597] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 -> Peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Couldn't send request to peer 5ab80e12f0ef4df38684a32e134d1e7f. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:37869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:46.967661 19603 tablet_server.cc:179] TabletServer@127.19.36.195:0 shutting down...
W20260709 12:29:46.986346  4785 consensus_peers.cc:597] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 -> Peer 5ab80e12f0ef4df38684a32e134d1e7f (127.19.36.193:37869): Couldn't send request to peer 5ab80e12f0ef4df38684a32e134d1e7f. Status: Network error: Client connection negotiation failed: client connection to 127.19.36.193:37869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:47.002007 19603 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:47.002748 19603 tablet_replica.cc:333] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.003552 19603 raft_consensus.cc:2243] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.004518 19603 raft_consensus.cc:2272] T f2200a06b9094b1e84a7fef9defaef4c P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.006924 19603 tablet_replica.cc:333] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.007602 19603 raft_consensus.cc:2243] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.008153 19603 raft_consensus.cc:2272] T f36103d928c44b22bf506cce041a25a8 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.010392 19603 tablet_replica.cc:333] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.011016 19603 raft_consensus.cc:2243] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.011600 19603 raft_consensus.cc:2272] T 2e5002fd95af4b95997c13feb68bbcba P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.013733 19603 tablet_replica.cc:333] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.014292 19603 raft_consensus.cc:2243] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.014797 19603 raft_consensus.cc:2272] T 2b3afd1c223f42b1bda891c9dcfa5150 P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.017050 19603 tablet_replica.cc:333] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.017688 19603 raft_consensus.cc:2243] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.018187 19603 raft_consensus.cc:2272] T b3553184055c4292ac3acc49631d9152 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.020483 19603 tablet_replica.cc:333] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.021062 19603 raft_consensus.cc:2243] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.021627 19603 raft_consensus.cc:2272] T f66dc3001f5149f683e43c2db87d9e0f P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.023836 19603 tablet_replica.cc:333] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.024425 19603 raft_consensus.cc:2243] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.024947 19603 raft_consensus.cc:2272] T 5bb30e39bc65458cbc6f1f4e87a9e6ab P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.027041 19603 tablet_replica.cc:333] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.027630 19603 raft_consensus.cc:2243] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.028090 19603 raft_consensus.cc:2272] T 70f0d3ef958849f08eaf18856e6567e2 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.030215 19603 tablet_replica.cc:333] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.030820 19603 raft_consensus.cc:2243] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.031319 19603 raft_consensus.cc:2272] T 51c65145402b4d15b96a7b47fba3d447 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.033419 19603 tablet_replica.cc:333] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.034003 19603 raft_consensus.cc:2243] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.034852 19603 raft_consensus.cc:2272] T 933310e093f24d6c87a460c3471eca97 P 5cbdbb7559fb49919a700494dc154588 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.036900 19603 tablet_replica.cc:333] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.037457 19603 raft_consensus.cc:2243] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.038314 19603 raft_consensus.cc:2272] T 04f0cfa224354c31afe3140f261990d7 P 5cbdbb7559fb49919a700494dc154588 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.040474 19603 tablet_replica.cc:333] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.041014 19603 raft_consensus.cc:2243] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.041465 19603 raft_consensus.cc:2272] T ae78f7d9184346839a507b7bd4e2659a P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.043605 19603 tablet_replica.cc:333] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.044142 19603 raft_consensus.cc:2243] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.044612 19603 raft_consensus.cc:2272] T 99be8cd85c6943998dcb402ec0bfc4de P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.046667 19603 tablet_replica.cc:333] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.047174 19603 raft_consensus.cc:2243] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.047626 19603 raft_consensus.cc:2272] T 4a3068fedf904cd3b43cbcd34a3e61aa P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.049660 19603 tablet_replica.cc:333] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.050161 19603 raft_consensus.cc:2243] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.050619 19603 raft_consensus.cc:2272] T d7adec6a263b40ae8f500e6d4de59996 P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.052691 19603 tablet_replica.cc:333] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.053158 19603 raft_consensus.cc:2243] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.053630 19603 raft_consensus.cc:2272] T ddfff7a3d59a47de8bc9f6929041196a P 5cbdbb7559fb49919a700494dc154588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.055701 19603 tablet_replica.cc:333] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.056182 19603 raft_consensus.cc:2243] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.057061 19603 raft_consensus.cc:2272] T 197e1cac162e444ebbb81bd7f563902b P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.059041 19603 tablet_replica.cc:333] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588: stopping tablet replica
I20260709 12:29:47.059594 19603 raft_consensus.cc:2243] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:47.060035 19603 raft_consensus.cc:2272] T 748fe2ac99874ea9b112159a53cc3beb P 5cbdbb7559fb49919a700494dc154588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.101922 19603 tablet_server.cc:196] TabletServer@127.19.36.195:0 shutdown complete.
I20260709 12:29:47.139803 19603 master.cc:562] Master@127.19.36.254:32959 shutting down...
I20260709 12:29:47.165297 19603 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:47.165853 19603 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:47.166152 19603 tablet_replica.cc:333] T 00000000000000000000000000000000 P 874cc058dc204191b51fd0b57de88a88: stopping tablet replica
I20260709 12:29:47.189160 19603 master.cc:584] Master@127.19.36.254:32959 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20815 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (42204 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (587338 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 12:29:47.248327 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2262 messages since previous log ~32 seconds ago
I20260709 12:29:47.248577 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2262 messages since previous log ~32 seconds ago
I20260709 12:29:47.248767 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2261 messages since previous log ~32 seconds ago
I20260709 12:29:47.248982 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:29:47.249183 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:29:47.249393 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 1 messages since previous log ~221 seconds ago
I20260709 12:29:47.249605 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 4 messages since previous log ~191 seconds ago
I20260709 12:29:47.249876 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~0 seconds ago
I20260709 12:29:47.250186 19603 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
